Improving Economic Research

Garrett C. Christensen and Edward Miguel have a new paper ($). They conclude

There are many potential avenues for promoting the adoption of new and arguably preferable practices, such as the data sharing, disclosure and pre-registration approaches described at length in this article. One issue that this article does not directly address is how to most effectively – and rapidly – shift professional norms and practices within the economics research community. Shifts in graduate training curricula, journal standards (such as the Transparency and Openness Promotion Guidelines), and research funder policies might also contribute to the faster adoption of new practices, but their relative importance remains an open question. The study of how social norms among economists have shifted, and continue to evolve, in this area is an exciting social science research topic in its own right, and one that we hope is also the object of greater scholarly inquiry in the coming years.

I have been thinking quite a bit about this recently. To make a long story short:

1. Economic phenomena are rife with causal density. Theories make predictions assuming “other things equal,” but other things are never equal.

2. When I was a student, the solution was thought to be multiple regression analysis. You entered a bunch of variables into an estimated equation, and in doing so you “controlled for” those variables and thereby created conditions of “other things equal.” However, in 1978, Edward Leamer pointed out that actual practice diverges from theory. The researcher typically undertakes a lot of exploratory data analysis before reporting a final result. This process of exploratory analysis creates a bias toward finding the result desired by the researcher, rather than achieving a scientific ideal of objectivity.

3. In recent decades, the approach has shifted toward “natural experiments” and laboratory experiments. These suffer from other problems. The experimental population may not be representative. Even if this problem is not present, studies that offer definitive results are more likely to be published but consequently less likely to be replicated.

I agree with Christensen and Miguel that the norms and incentives within the economics profession are the key. For a long time, both the norms and the incentives have pulled researchers in the direction of getting certain types of results, which has pulled them away from the direction of following robust methods. That culture is very difficult to change.

Note that I recently discovered the web site The Replication Network.

Update on the Long-term Stories

Over a decade ago, I suggested following five long-term stories:

productivity; cognitive neuroscience, solar power, cancer therapy, and mainstream media meltdown

It looks like mainstream media meltdown has proceeded quite far since then. People are just as happy to get their fake news from Facebook or Drudge as they are from the NYT or CNN.

Productivity is not such a clear story. If you include unsubsidized health insurance, college tuition, and housing in San Francisco in your cost-of-living index, it is obvious that real wages and productivity are going nowhere. Note that these are all sectors in which public policy subsidizes demand and restricts supply.

If you restrict yourself to consumer durables and non-durables, productivity growth might look decent. If you look at things from the perspective of the autodidact, productivity growth is tremendous. In the last ten years, the amount that you can learn on YouTube has exploded. Whether you need to fix your toilet, learn to play “Crossroads” on guitar, or study advanced academic subjects, it is all there.

I still think that the productivity story is important. The challenge is figuring out how best to follow it.

Cognitive neuroscience seems to me to be less exciting than I expected a decade ago. The Google trend for “cognitive neuroscience” seems to me to be flat/declining.

In 2005, the U.S. Department of Energy predicted that solar power would be price competitive by now. That was too optimistic. However, we can at least say that solar costs are moving in the right direction. And there are still plenty of optimistic articles out there. Six months ago, Peter Diamindis wrote,

for those who want to do the calculations, at a 30 percent annual growth rate, it looks like this: In five years, we go from 0.4 percent to 1.5 percent. In 10 years, we’re at 5.5 percent. In 15 years, we’re at 20 percent, and in 21 years, we’re at 98 percent.

That is, if the share of solar power in total energy production grows at 30 percent per year for a long time, it takes about 20 years to take over. Of course, if it “only” grows at 10 percent per year, it takes much, much longer.

Cancer therapy is proving to be a tough slog. But I think that one can say that the cure rate is increasing, and the death rate is declining. One can argue that, while cancer is still obviously very difficult and very important, some attention is now shifting to the diseases of the brain, such as Alzheimer’s.

The Psychology of Politics

Maria Konnikova surveys some of the literature.

Lord and his colleagues asked people to read a series of studies that seemed to either support or reject the idea that capital punishment deters crime. The participants, it turned out, rated studies confirming their original beliefs as more methodologically rigorous—and those that went against them as shoddy.

This and other studies serve to highlight confirmation bias, which helps to reinforce tribalism in politics.

I would like to point out that this form of confirmation bias is a very important problem in academia. That is why I think that studies should be evaluated methodologically before the results are known. A referee should be asked whether the study is capable of producing results that influence someone to change their minds. Could the results turn out to be against your prior beliefs? If so, would that influence your prior beliefs?

William Easterly on Technocratic Elites and Democracy

He writes,

The problem occurs when some people turn out not to share those enlightened values and insist on challenging them. Technocrats, in these situations, don’t know what to say because they can’t rely on evidence to make their case. So when technocrats are all we have to defend democracy, fights over fundamental values become embarrassingly one-sided.

The piece is mostly how the elites are right in their outlook but weak in their messaging. I disagree with a lot of it. In the past, Easterly has pointed out how elites are wrong in their outlook, in that they over-estimate the value of centralized control.

In this essay, Easterly interprets the Trump election as a revolt against elite values of equality and inclusion. However, he could have interpreted as a revolt against elite arrogance. I think that there is at least as much to be said for the latter as for the former.

Timothy Taylor on Christmas Trees

He wrote,

One artificial tree used for one year has greater environmental impact than one natural tree. However, an artificial tree can also be re-used over a number of years. Thus, there is some crossover point, if the artificial tree is used for long enough, that its environmental effect is less than an annual series of trees. For example, the ellipsos study finds that an artificial tree would need to be used for 20 years before its greenhouse gas effects would be less than those of an annual series of natural trees. The PE Americas study offers a wide range of scenarios, and summarizes, but here is the situation “for the base case when individual car transport distance for tree purchase is 2.5 miles each way. Because the natural tree provides an environmental benefit in terms of Global Warming Potential when landfilled, and Eutrophication Potential when composted or incinerated, there is no number of years one can keep an artificial tree in order to match the natural tree impacts in these cases. … For all other scenarios, the artificial tree has less impact provided it is kept and reused for a minimum between 2 and 9 years, depending upon the environmental indicator chosen.”

I think that it is hopeless to try to do these sorts of estimates. For one thing, note that if everyone bought artificial trees, then there would be fewer Christmas tree farms.

Are there artificial trees that can pass a Turing test? Do they smell like a tree? Do they have interesting asymmetries? If not, then it would seem as though artificial trees would be less fun to decorate.

WaPo Watch, Week 4

I was away most of last week, so I did not see much of the actual newspaper. Two stories stood out in my mind.

First, there was a story claiming that Trump’s Cabinet choices were selected in part on the basis of how they look on television.

First off, consider the double standard. Did the Post go back to previous Presidents and find officials whose looks were off-putting? Who were the bad-looking people that President Obama appointed to top spots in his Administration?

A much more interesting and balanced take on the Trump team comes from Ray Dalio (pointer from Tyler Cowen.) An excerpt:

the people he chose are bold and hell-bent on playing hardball to make big changes happen in economics and in foreign policy (as well as other areas such as education, environmental policies, etc.). They also have different temperaments and different views that will have to be resolved.

I think this is much more important than their looks. Note that President Obama’s most important domestic initiative, the Affordable Care Act, was overseen by Kathleen Sebelius, a career politician who clearly was not appointed for her management skills. She was nominally in charge of the infamous Obamacare web site.

The second story that struck me was the one about the Obama Administration’s decision to abstain on the UN Security Council resolution that caused an outcry in Israel. What struck me was that the lead story was completely free of editorializing, even though the Post‘s editorial page decried the decision. This made me want to go back and give more bias points to the story that the Post wrote about Mr. Trump’s phone call with the President of Taiwan. There, the editorializing dominated the front page.

Some Holiday Cheer

From Peter Gray,

Another reason for the increased ease of Self-Directed Education lies in technology. Today, anyone with a computer and Internet connection can access essentially all the world’s information. Self-directed learners who want to pursue almost any subject can find articles, videos, discussion groups, and even online courses devoted to it. They can gain information and share thoughts with experts and novices alike, throughout the world, who have interests akin to theirs. Students in standard schools must study just what the school dictates, in just the ways that the school decides; but self-directed learners can find subjects and means of study that match their own particular interests and styles of learning. Self-directed learners are not held back by the slow pace of a school course, nor are they rushed ahead when they want more time to think about and delve deeply into any given aspect of the interest they’re pursuing.

The future belongs to the auto-didacts.

Crony Non-profitism

Jeff Bergner writes,

Like the Sierra Club and other environmental groups, the NRDC receives grants from the federal government both directly and indirectly. In addition, it participates in a process referred to as “sue and settle.” It brings lawsuits against the EPA on issues where EPA officials actually have no objection to being sued. EPA and NRDC officials then “settle” these suits, sometimes almost immediately and together, in the form of “consent decrees,” write new regulations ostensibly to settle the lawsuits. On many occasions, the EPA also pays the legal fees of the NRDC and similar groups as part of its settlement.

He gives many other examples.

What Are Americans Moving Less?

Timothy Taylor writes,

between the 2002-3 and 2015-16 measurements, the share of moves that were 50 miles or less rose from 32.3% of all moves to 42.3% of all moves. The main offsetting decline was in moves of between 200 to 499 miles, which fell from 20.7% of all moves back in 2002-3 to 13.8% in 2015-16.

Those strike me as large changes. Taylor discusses many possible explanations. One story that he does not consider is the possibility that employment opportunities and housing construction have become disconnected. In cities where they might be job opportunities, such as San Francisco, housing is not allowed to be built. In places where housing is allowed to be built, there are not as many job opportunities.