Robert Doar writes,
Momentum for evidence-based policymaking is building at all levels of government, from federal legislation funding rigorous evaluations to the bipartisan Commission on Evidence-Based Policymaking to counties looking to make funding decisions based on results.
I am afraid that my reaction is to be cynical. When you make funding decisions for programs based on evidence, what will change will be the reported evidence, not the programs.
During the Vietnam War, Secretary of Defense Robert McNamara was famous for demanding statistical evidence that strategies were working. He got what he was asking for, but the statistical evidence did not capture what was really happening.
To cite another example, the Stiglitz-Orszag paper on Fannie Mae and Freddie Mac appeared to be evidence-based. Recall that they wrote,
This analysis shows that, based on historical data, the probability of a shock as severe as embodied in the risk-based capital standard is substantially less than one in 500,000 – and may be smaller than one in three million. Given the low probability of the stress test shock occurring, and assuming that Fannie Mae and Freddie Mac hold sufficient capital to withstand that shock, the exposure of the government to the risk that the GSEs will become insolvent appears quite low.
Within any organization, including a profit-seeking business, one has to be cynical about “evidence.” Show me a CEO who always believes every report he or she receives from middle management, and I will show you a company that is at high risk for going bankrupt very soon. I have never been a large-company CEO, but if I were I would make a point of setting up internal checks and balances so that I did not have to rely on any one set of carefully crafted reports.
You are entitled to ask, “How can you be against evidence? Evidence is bound to make policies better than if evidence is ignored.”
My response is that I am afraid that evidence will be distorted to make spending programs and regulations appear better than they really are. I will take public choice theory over misleading evidence, any day.