Category Archives: Mark Thoma is Indispensable

What is the Meaning of Credibility?

Posted on by

A recent survey of leading economists, called the IGM forum, asked two questions about CBO forecasts.

Question A: Forecasting the effects of complex legislative actions is hard, so even competent, non-ideological and non-partisan projections could differ substantially from outcomes.

Question B: Adjusting for legal restrictions on what the CBO can assume about future legislation and events, the CBO has historically issued credible forecasts of the effects of both Democratic and Republican legislative proposals.

The answers were overwhelmingly affirmative for both. I have been following the IGM forum for years, and you rarely see such a strong consensus.

What does the term “credible” mean?

Does the affirmative answer to question B mean that the forecasts are accurate enough that policy makers should take them seriously? John Whitehead seems to think so. Pointer from Mark Thoma.

Or does the affirmative answer to question A mean that the forecasts are not accurate enough to reliably guide policy? Russ Roberts and I would tend to think so.

Anyone, including Russ or me, who criticizes economic methods faces the following argument.

1. Policy has to be based on some model and some forecast.

2. A formal model or a statistical forecast is more rigorous than intuition/opinion.

3. Therefore, the best approach is to use formal models and statistical forecasts.

I think that the problem comes in the way that one interprets point (1). Consider two possibilities:

1a. Policy has to be based on a “model” and a “forecast” which rule out any empirical analysis of how policy is formulated and implemented. Also, the “model” and the “forecast” can ignore the possible evolutionary responses of decentralized activity, including possible emergent market solutions to the problem that the policy is intended to solve, as well as innovations responding to the policy that mitigate its effects or that produce unintended adverse consequences.

1b. Policy has to be based on a “model” and a “forecast” which do take into account empirical public policy and dynamic market responses to the original problem and to the proposed solution.

If we interpret point 1 as “1b,” then I accept the logic that a model is better than no model and a forecast is better than no forecast.

If we interpret point 1 as “1a”, then the argument is a swindle. Models that ignore empirical public policy and dynamic market responses are not necessarily better than intuition/opinion, and they should not be regarded as credible.

Taking into account these requirements for credibility, CBO forecasts are not credible. Using them may very well do more harm than good.

Justin Fox Inadvertently Makes the Case Against Empiricism

Posted on by

On the question of whether Federal workers are overpaid relative to private sector workers, He writes,

The Federal Salary Council, a government advisory body composed of labor experts and government-employee representatives, regularly finds that federal employees make about a third less than people doing similar work in the private sector. The conservative American Enterprise Institute and Heritage Foundation, on the other hand, have estimated that federal employees make 14 percent and 22 percent more, respectively, than comparable private-sector workers.

Pointer from Mark Thoma. My comments:

1. The empirical estimates are supposed to “control for” the many factors that could affect salaries: benefit packages, education level of workers, other measures of skill, etc. But obviously, there is no clear and unambiguous choice of how to control for these factors, or else everyone would get the same estimate. Ed Leamer hit the profession over the head with this 35 years ago.

2. Could you have predicted ahead of time which organization’s “research” would find a result favorable to Federal workers and which organization would find unfavorable results? Of course you could. So how do you sustain the belief that normative economics and positive economics are distinct from one another, that economic research cleanly separates facts from values?

3. A number of us have observed that the rate of exit from the public sector to the private sector is not terribly high, and that the ratio of applicants to vacancies in public sector jobs is not terribly low. If public sector pay were too low, you would expect government agencies to be rife with unfilled positions, due to high exit and low entry.

4. Point (3) is an example of what Noah Smith would dismiss as “casual intuition.” But in this instance, I would argue that casual intuition has a higher signal-to-noise ratio than does formal empiricism.

Economists, Empiricism, Humility, etc.

Posted on by

Peter Dorman writes,

what passes for empiricism in economics at present is often deficient in an empiricist, self-critical spirit and methodology. At the same time, the debates over topics like the minimum wage, the effects of charter schools on educational outcomes and the like are on a vastly higher plane when they are about data sets and analytical assumptions than the certitude of my unquestioned beliefs against the certitude of yours. It’s also a cheap and not altogether forthcoming dodge to respond to econometric disputes with a flip “There is never a clean empirical test that ultimately settles these issues.” (Roberts) That’s a epistemology.

Pointer from Mark Thoma.

Adam Ozimek writes,

Calls for skepticism of empirical economists also need to be matched with “compared to what?” Often those arguing for more humility about empiricism aren’t actually embracing humility, but instead are making space for their own narratives that are no less humble. For example, Russ says he doesn’t know how many jobs NAFTA has created or destroyed because “thousands and thousands of jobs are created every month and it is very difficult, perhaps impossible to know which ones are related to NAFTA.” Certainly, humility with regard to this question is useful. But then at the end of that paragraph Russ tells us he believes “trade neither destroyed nor created jobs on net.” Zero is not the same as “I don’t know,” nor is it necessarily any more humble than some specific estimate with wide confidence intervals.

…Economists do disagree on whether the direct effect of immigrants on native wages is a small positive or small negative. But they agree it is small. It’s easy to take this conclusion for granted as somehow common sense. But the truth is that, in the absence of the empiricism, the claim that immigration has held back wages by 20% for everyone would be much harder to argue against.

Noah Smith adds,

Theories can be wrong, stylized facts can be illusions, and empirical studies can lack external validity. But where does casual intuition even come from? It comes from a mix of half-remembered theory, half-remembered stylized facts, received wisdom, personal anecdotal experience, and political ideology. In other words, it’s a combination of A) low-quality, adulterated versions of the other approaches, and B) motivated reasoning.

If we care about accurate predictions, motivated reasoning is our enemy. And why use low-quality, adulterated versions of theory and empirics when you can use the real things?

Pointers from Tyler Cowen, who adds

A lot of the bias in empirical methods comes simply from which questions are asked/answered.

I say “amen” to that. For example, in the health care policy debate, the empiricists at CBO are telling us how many people would “lose” their health insurance under the GOP proposal. If you want to, you can question the CBO’s empirical estimates (their forecasts for Obamacare were, in the words of Avik Roy, “way off”). But that is not where I think the debate should go.

Instead, I think we ought to be talking about the real meaning of “insurance” in the context of health care. I think we ought to talk about the pros and cons of individuals having less “coverage” and making their own decisions about medical procedures with high costs and low benefits vs. having more “coverage” but subject to restrictions placed on them by bureaucrats. I think we ought to be talking about the issue that Timothy Taylor raised the other day, namely, should we be spending less on medical services and more on other things that are conducive to better health. (Note that Taylor’s post is grounded in formal empiricism.)

Perhaps we ought to be listening to Dierdre McCloskey’s view that economics is a discipline that uses rhetoric. I think that Russ Roberts and I would complain about the pseudo-scientific rhetoric that gets used.

Noah Smith’s use of the rhetorical phrase “the real things” is an example of the sort of language that lacks humility. It implies that there is a great distance between casual observation combined with theoretical introspection on the one hand and formal empirical work on the other, with the implication that the latter dominates the former. I would say that in some cases it is the former that is unreliable and in other cases it is the latter.

I hope that we can all agree that a lack of humility consists of pretending to know something for certain when it is in fact doubtful. We can then argue about what sort of approaches to economics are conducive to humility or a lack thereof.

Again, I have a longer essay on this topic, but it will not appear until this summer.

Japan and the Consensual Hallucination Hypothesis

Posted on by

Kevin Drum writes,

Japan’s deflation has persisted even in the face of massive BOJ efforts that, according to conventional economics, should have restored normal levels of inflation.

Pointer from Mark Thoma. BOJ = Bank of Japan, their central bank.

We have all been taught that money and inflation are tightly linked. Those of you have read Specialization and Trade know that one of my heresies is to deny that this is true under normal circumstances (large government deficits that can only be financed by printing money are the exception; to get hyperinflation, you need the fiscal driver of money creation). I say that money and “the overall price level” are a consensual hallucination. They are embedded in cultural norms and expectations.

The consensual hallucination hypothesis (which was held by the late Fischer Black) is consistent with the Japanese experience.

The Deplorables Heuristic

Posted on by

Chris Dillow writes,
I was being tribal: I didn’t want to be part of a tribe that had a disproportionate number of people I despised. I was using a form of the social proof rule of thumb. I was allowing the numbers of others making their choices to guide mine. The fact that decent people tended to favour remain (with of course counter-examples on both sides) strengthened [m]y support for the cause.

Pointer from Mark Thoma.

The heuristic that Dillow followed was this: he saw many Brexit supporters as racists, therefore he would not support Brexit.

On Facebook, one of my friends posted that although she wanted to attend the anti-Trump march, she was troubled by some of the positions espoused by leaders of the march. So, although I assume that she broadly sympathizes with the marchers, she was having doubts because of this particular heuristic.

For any cause, there are some supporters who are deplorable. I am sure that Chris Dillow could find some prominent Remainers for whom he has animosity, although they are not as numerable as those on the Leaver side.

I think that a heuristic that says “Do not associate with a political cause if you find a fair number of its supporters deplorable” would leave you unwilling to support any political cause.

And that might not be a bad thing.

Taking Macroeconomics Backward Through Regression

Posted on by

Olivier Blanchard recently wrote that there ought to be two classes of macroeconomic models.

Theory models, aimed at clarifying theoretical issues within a general equilibrium setting. Models in this class should build on a core analytical frame and have a tight theoretical structure. They should be used to think, for example, about the effects of higher required capital ratios for banks, or the effects of public debt management, or the effects of particular forms of unconventional monetary policy. The core frame should be one that is widely accepted as a starting point and that can accommodate additional distortions. In short, it should facilitate the debate among macro theorists.

Policy models, aimed at analyzing actual macroeconomic policy issues. Models in this class should fit the main characteristics of the data, including dynamics, and allow for policy analysis and counterfactuals. They should be used to think, for example, about the quantitative effects of a slowdown in China on the United States, or the effects of a US fiscal expansion on emerging markets.

In response, Simon Wren-Lewis rejoiced,

Ever since I started blogging I have written posts … to try and convince fellow macroeconomists that Structural Econometric Models (SEMs), with their ad hoc blend of theory and data fitting, were not some old fashioned dinosaur, but a perfectly viable way to do macroeconomics and macroeconomic policy.

Pointers from Mark Thoma.

For why Blanchard and Wren-Lewis are wrong, see my essay Macroeconometrics: the Science of Hubris. If the profession follows their advice, macroeconomics will be regressing in every sense of the word.

The Economist as Entrepreneur

Posted on by

Beatrice Cherrier comments on Esther Duflo’s AEA lecture, “The economist as plumber.”

She wanted economists to reconceive economic agents, policy-makers and bureaucrats as bounded “humans” embedded in wider power structures and cultures, and to realize that thinking goods ideas is not enough to improve the latter’s welfare. “Incentive architecture” is thus needed, and economics expertise is especially relevant because it deals with behavioral, incentive and market equilibrium issues. The recent success of (some) “nudge” has given some salience to benefits of crafting incentives carefully, for instance by fixing regulations to prevent firms from exploiting loopholes. Plumbing was also beneficial for economics as science, she continued, as it helped generate counterfactuals by randomizing on entire markets. Plumbing also shines the spotlight on issues theorists had previously ignored, like how important the default scenario is. Economics as plumbing requires a more pragmatic and experimental mindset, she concluded, as it requires them to make decision without having a full knowledge of the system to be tinkered (“tinkering” was one of the keywords of the speech).

I recommend the whole post. Pointer from Mark Thoma.

When I think of trial-and-error tinkering to try to solve a problem, I think of an entrepreneur.

In fact, we might be better off thinking of policy-oriented economists as state-backed entrepreneurs. That is, while ordinary entrepreneurs have to convince investors to back their ideas and convince customers to pay for their offerings, state-backed entrepreneurs have to convince politicians to back their ideas.

I am not very enthusiastic about state-backed entrepreneurship. I believe that the market does a more rigorous job of experimentation, evaluation, and evolution.

I am glad to see alternatives to the image of the economist as a white-coated scientist. However, I do not think it should be replaced by an image of droopy-pantsed plumber. I am afraid that a more accurate image is of an entrepreneur who is trying to short-cut the market by pitching business ideas to a political audience. And we should be cognizant that politics is an inferior arena for trying out entrepreneurial schemes.

Daniel Little on the Evolution of Disciplines

Posted on by

He writes,

academic disciplines are in fact highly contingent in their development, and that there is no reason to expect convergence around a single “best” version of the discipline. The history of disciplines should better be understood in analogy to the brachiation and differentiation associated with the evolution of species and sub-species over time — lots of contingency, with a consequent specialization of the intermediate results to the demands of a particular point in time. This implies that a discipline like sociology or political science could have developed very differently, with substantially different ideas about research questions and methods.

I take the view that economics evolved in the wrong direction in the United States, particularly following the second World War, as it followed down the path laid out by Paul Samuelson. Some of my thoughts on this are expressed in Specialization and Trade. More thoughts are on the way in a long essay.

Economists and Mr. Trump

Posted on by

Justin Wolfers writes (not Justin Fox, as I mis-typed earlier) that at the recent American Economics Association meetings,

Over three days of intense discussions, I didn’t encounter a single economist who expressed optimism that Mr. Trump’s administration would be good for the economy. The optimists were those who thought Mr. Trump would not have the energy to actually implement his agenda; the pessimists’ thoughts veered toward disaster.

Pointer from Mark Thoma.

It is possible that they are correct. However, I doubt it. While I disagree with Mr. Trump on immigration and trade, and I condemn his interventions with individual business decisions, I think that these will cause relatively little harm. This harm could be more than offset by reining in regulations, replacing Obamacare, and/or tax reform.

What is true is that Mr. Trump and the professoriate have an adversarial relationship. Mr. Obama takes his world view from the faculty lounge of the sociology department, and he very much respected academic credentials. Mr. Trump is the opposite.

I think that credentialed economists deserve a bit more respect than what we receive from Mr. Trump, but much less than what many American Economics Association members seem to think we are entitled to. I think that Justin Wolfers’ colleagues are fantasizing about a scenario in which Mr. Trump causes an economic disaster so that the status of academic economists shoots up. But I do not think see this as a very likely scenario.

On the generic topic of academic expertise in government, Tyler Cowen writes,

when it comes to the nuts and bolts of governance, typically I would prefer to be ruled by the Harvard faculty, even recognizing the biases of experts. They understand the importance of applying expertise to complex problems, and they realize many issues do not respond well to common-sense fixes. The citizenry usually cannot make good decisions, or for that matter expert appointments, when technocracy is required.

I tend to focus on what I call the knowledge-power discrepancy. Joe Citizen may have less knowledge than Professor Jones, but Professor Jones could be more dangerous. That is because Professor Jones may over-estimate his suitability for telling other people what to do.

Compared with academics, business executives and military leaders have more experience with the challenge of implementing ideas. A good business executive would not take it for granted that a web site is going to work. A good general would emphasize all of the difficulties and risks of trying to shape the Middle East.

Noah Smith on Higher Education Policy

Posted on by

He writes,

As long as the number of available college spots remains roughly fixed, reducing the price of college will have only a very modest effect in creating broad-based economic opportunity.

My recommended solution is to focus on increasing the number of college spots available. Those could be four-year university slots, or vocational education — a mix of both would probably be best. But the key is that supply should go up.

Pointer from Mark Thoma.

Of course, from an economic point of view, Smith’s point is spot on. However, the Kling Theory of Public Choice is that public policy will always choose to subsidize demand and restrict supply. That is what is most in the interest of incumbent suppliers, who are the drivers of public policy.