Here is the 30-minute version of the 2009 New Yorker video interview with Bob Shiller and Nassim Taleb. (Tyler linked to a four-minute segment a few days ago). I want to talk about the difference between Shiller’s and Taleb’s views of inefficient markets.

When I teach regression in statistics, I show what I call the Pythagorean relationship, which describes what computer programs report as the analysis of variance. You are trying to predict a variable, Y, and the predicted values along the regression line are called Y-hat. I draw a right triangle with the standard deviation of Y-hat on one side, the standard error of the regression on another side, and the standard deviation of Y on the hypotenuse. The Pythagorean Theorem then gives you the analysis of variance.

Anyway, a lesson of this is that in an efficient prediction, the variance of your prediction will be less than the variance of the variable that you predict. Mathematically, this is because one side of a right triangle is always shorter than the hypotenuse. Intuitively, if your predictions vary by more than the variable you are trying to predict, then you can do better by toning down your predictions and moving them closer to the mean of the variable.

Shiller’s insight was to apply this idea to asset prices. In some sense, the stock price is a prediction of discounted future dividends, which I will refer to as average realized dividends. In that case, if the stock market is efficient, then the variance of stock prices should be less than the variance of average realized dividends. In fact, it is easy to see that the variance of stock prices is much higher than that of average realized dividends.

What this says, and what Fama and French later confirmed, is that you can make money by betting on mean reversion in stock prices. To do so, you assume use historical average dividends as a proxy for average realized dividends going forward. If you follow a strategy of buying when prices are low relative to historical average dividends and selling when prices are high relative to historical average dividends, then it seems that you will earn an above-normal profit.

Taleb would not bet on mean reversion. Instead, he would load up on out-of-the-money options. That way, you are betting on Black Swans.

Taleb’s point of view gets back to my criticism of Shiller’s work. From Taleb’s point of view, Shiller is like the turkey, who every day notices that the farmer is feeding him and taking care of him. The turkey does not realize that Thanksgiving is coming, and this will change the farmer’s behavior. Similarly, the markets appear to be mean-reverting, but what Shiller does not know is that a Black Swan event could come along.

For example, suppose that bond market investors have a probability p of a Black Swan, meaning that the U.S. government runs out of other options and monetizes a lot of its debt, leading to hyperinflation and making long-term bonds effectively worthless. For simplicity, suppose that this Black Swan either will or will not occur on January 1, 2020. With that simple assumption, on January 1, 2020, the true value of a long-term bond will be either 100 or 0. Whichever it turns out to be, when Shiller does his analysis in 2025, he will find that the variance of the “correct” bond price is zero. Since the price of bonds between now and 2020 is a predictor of the “correct” future bond price, to be an efficient predictor its variance can be no larger than zero.

However, between now and January 1, 2020, the bond price will vary as bond market investors’ estimate of p varies. Thus, the variance of bond prices will not be zero.

I take the view that this possibility of a Black Swan (aka, the peso problem) precludes the use of realized data to construct a “variance bound.” Only in a world where you can rule out Black Swans can you be certain that Shiller has found a market anomaly.

Although I lean toward Taleb, I consider that Shiller may be right. In any case, it is worth contemplating the tension between the two.