Raj Chetty on Empiricism Without Theory

The talk is here. Pointer from Tyler Cowen.

Broadly speaking, Chetty makes two points. One is that behavioral economics has inspired empirical analysis that can be useful for policy. The other is that we do not have to care about theory. Although theory might guide us to try certain empirical studies and might explain why a policy will work, all we need is the empiricism to know that a policy will work.

I found this view at best shallow and at worst not persuasive. Take one of his examples, in which a study guided by the theory of Loss Aversion found that classroom results improve more when incentives are framed in terms of what teachers will lose if results are bad than in terms of bonuses they will earn if results are good. My thoughts:

1. Given that the Null Hypothesis tends to be more robust than studies that purport to show significant effects from educational interventions, one ought to be pretty cautious about this.

2. A non-empirical a priorist of a Misesian bent, without knowing any behavioral economics, would recommend a market-provided school over a government-provided school. Among other advantages, the market will tend to punish poor performance, as markets tend to do. So in this example, without doing any empirical work at all, one can arrive at a recommendation that would be at least as effective as anything that Chetty might propose. While I am no Misesian, I do find the generic arguments against central planning more compelling than just about any empirical finding suggesting nudging opportunities.

3. I think that those who advocate behavioral economics would do well to also acknowledge behavioral politics. If Chetty understood how teachers unions operate where I live, he might be feel less a bit less excited about the opportunities for reform using his findings.

Related: Noah Smith on how the profession has been moving left.

Economics has become much more empirical, and that has made it much harder to wave away the possibility of market inefficiencies.

Pointer from Mark Thoma.

6 thoughts on “Raj Chetty on Empiricism Without Theory

  1. “all we need is the empiricism to know that a policy will work. ” Because the policy and it’s consequences will be exactly as same as the experiment? Except by definition and design (and indeed why we experiment) it is not the experiment.

    Since when is it left to acknowledge market inefficiencies? Left is to assume that the voting market doesn’t have any.

    • In addition to a basic conceptual problem (you need something like a theory to tell you what an empirical result means – even if you want to subject your priors to empirical veto or modification) there is a practical confusion. Say this or that policy tests out well in a few randomized trials. What happens when the policy is applied on a system-wide basis? Does it continue to work, or is there a hormesis effect, and you actually make things worse?

      And once you make a policy universal how do you get anywhere close to a Null Hypothesis in order to evaluate it? If outcomes go negative how do you then tease out the now-universal policy to see if it’s become the problem? This would become a particularly thorny problem if you want to “test” a social policy over say twenty years, and by then, as Arnold says, behavioral politics, AKA politics comes fully into play.

  2. “Economics has become much more empirical, and that has made it much harder to wave away the possibility of market inefficiencies.”

    Whoops, I think he meant to say,
    “Economics has become much more empirical, and that has made it much harder to wave away the very high probability of failed government programs.”

    There, fixed it.

    • I would think that the empiricism holds out, in the minds of those like Smith, the promise of true but non-dogmatic knowledge. Thereby it offers the possibility of control beneficently exercised. What’s a little freedom when we can improve some outcomes eight or nine percent at a small confidence interval (allegedly)? That’s the temptation.

  3. I must be missing something:

    “Behavioral economic theory obviates the need for economic theory”

    What?!

    • OK, I see, was reading without full attention.

      This argument seems to rely on the premise that we can know exactly when the theory has been perfected, and now accounts for all the things that matter.

Comments are closed.