There were 77 countries under consideration, and 49 of them were in recession in 2009. Economists – as reflected in the averages published in a report called Consensus Forecasts – had not called a single one of these recessions by April 2008.
… Making up for lost time and satisfying the premise of an old joke, by September of 2009, the year in which the recessions actually occurred, the consensus predicted 54 out of 49 of them – that is, five more than there were. And, as an encore, there were 15 recessions in 2012. None were foreseen in the spring of 2011 and only two were predicted by September 2011.
He cites research from Prakash Loungani and Hites Amir. Some comments:
1. This underlines the fact that macroeconomists are using equations that are not verified empirically.
2. Perhaps the “target the forecast” mantra of market monetarists would not work as robustly as they might hope.
3. According to the NBER, the U.S. was already well into a recession by April 2008 and already out of recession by September of 2009.
“Perhaps the “target the forecast” mantra of market monetarists would not work as robustly as they might hope.”
I think the Market Monetarists would add that the central bank has the capability to, “make the forecast come true, on average,” through level-targeting.
That is, the central bank can keep trying to adjust NGDP to make it match a pre-determined arbitrary path if it over or under-shoots, but that the central bankers can merely use the futures market as a best-guess tool to try and figure out the interest rate (or amount of quantitative easing) that makes the market believe they are choosing the optimal policy to accomplish that goal.
OMG, that’s 0 for 64. “Dismal” doesn’t do it justice. I’d have to go with “abysmally, incredibly, conspiracy-fodder awful”
Has this lack of forecasting success caused the community of economists to step back from reliance on models? There should be some point where, if the underlying principles are understood, there should be SOME ability to forecast the short term future. A lack of predictive value should trigger a re-evaluation of methodology and the ‘science’ itself.
Interestingly, from the post it appears that this was an average of many models. Well, if some people have it right, but others don’t, the average will not be the correct thing to do. Question: was there any breakdown by type of economist, i.e. by school of thought?
Watching the way a discipline deals with the failure of its models to reliably and accurately forecast reality is a very revealing exercise.
Do they admit they have a problem and stop giving confident forecasts. Do they desperately try to salvage the system with fudge-factors and just-so excuses? Do they accurately give very large error bars that make their predictions sound silly, “GDP growth will be 2%, plus or minus 3%.” Do they go on with business as usual? Do they aggressively defend the models and bully critics? How hard is it to persuade people to abandon the old ways and develop and adopt new theories?
One wonders about the intellectual history in Physics and Biology.
But what about Macroeconomics or Climate Change?
With global warming, practically all the computer simulation forecasts had today’s actual temperatures far below their confidence intervals. So, one would expect the rational response to be to admit that there is some common error, and try to identify and correct it. But I’ve heard practically nothing about this despite many years of below-forecast temperature data.
Estimate of the risks of climate disruption are extremely dependent on even very small changes in the ‘climate sensitivity’ factor. So, whether one is alarmed or indifferent to CO2 emissions depends utterly on any news of a likely updated estimate. And yet the issue is mostly ignored.
I don’t know that this is sufficient evidence to condemn macroeconomic models, or arguably even evidence against them at all. I expect Arnold and many of the readers believe that even the smartest participants in the financial sector do not have sufficient models to beat the stock market better than luck alone. That doesn’t mean their models have no basis in reality, it may mean the price changes they hoped to benefit from are priced into the market immediately.
Central bank policy is not determined by market prices (not until Scott Sumner gets made Fed chair), but a related effect can happen. When there are recessions the central bank can predict, maybe those are recessions they can act to stop with the correct monetary policy. And then like the immediately priced-in stock value changes, the evidence that they made a good prediction is destroyed by the prediction existing. Maybe the only recessions remaining are ones they couldn’t predict.
I don’t think it’s clear what’s going on here, although I do have to admit the quoted prediction records sound pretty dismal.
Recessions happen in the USA on presidential election cycles with about a 5% error.
“Target the forecast” means just what it says. If you were targeting, say, *NGDP a year in the future* you would have to be concerned about our inability to forecast it. But Market Monetarists say, “Don’t target NGDP-one-year; target *the present market forecast of NGDP-one-year*.” *That* is something we can gauge quite reliably (and we could do even better with a subsidized futures market).