Doubting the Null Hypothesis

From a chapter of the 2017 Economic Report of the President.

Zimmerman (2014) compares students whose GPAs are either just above or just below the threshold for admission to Florida International University, a four-year school with the lowest admissions standards in the Florida State University System. This study finds that “marginal students” who are admitted to the school experience sizable earnings gains over those who just miss the cutoff and are thus unlikely to attend any four-year college, translating into meaningful returns net of costs and especially high returns for low-income students. Using a similar methodology, Ost, Pan, and Webber (2016) study the benefit of completing college among low-performing students whose GPAs are close to the cutoff for dismissal at 13 public universities in Ohio. They find substantial earnings benefits for those who just pass the cutoff and complete their degree.

My thoughts:

1. Usually, the economic report of the President comes out in early February. Since the Obama Administration will still be in office in January, I would have expected to see the report come out then. I guess having it come out now makes it easier for the Council members to clip back into academia the second semester, if that is what they want.

2. The second result may not really support the authors’ policy view. If the only difference between Alice and Bob is that Alice just barely got the sheepskin, then Alice did not learn more than Bob. That suggests that Alice’s earnings advantage was due solely to signaling.

3. Remember that regardless of economic theory, government policy tends toward subsidizing demand and restricting supply. Subsidizing demand and restricting supply raises price, not quantity. If you really want to produce a large increase in college attendance and completion, you have to do something to increase supply. The only supply-side policies discusses in the chapter are regulations intended to identify and punish low-quality institutions.

7 thoughts on “Doubting the Null Hypothesis

  1. “regulations intended to identify and punish low-quality institutions.”

    Which I would assume means making life hard on for-profit startups, aka, restricting supply.

    • Part of the value of the degree is that it is scarce. Also, wouldn’t a curious person wonder if the GPA was a poor predictor and the admission standards should be improved? Odd to selection bias the result by picking people who finish the degree. The implication that these results somehow prove that more money should be funneled to university fails on multiple grounds.

      • This is a really good point about choosing the students who finish the degree. The 4 year graduation rate is only 26%:

        http://colleges.usnews.rankingsandreviews.com/best-colleges/fiu-9635

        Also, it seems pretty clear to me that signaling is a major factor here. For instance, when you learn someone went to say, Cal Tech, you would assume that their IQ likely lies in a narrow, but high, distribution. If you hear someone went to a large state school, the distribution of where their IQ likely lies would have to be wider, stretching further down the scale (but also overlapping with Cal Tech on the high side). The same principle must be true of employers who compare college graduates to high school graduates.

        • Yep. Seems like a huge sleight-of-hand in these results. What about the 75% of marginal students who are admitted but never earn a degree, wasting a lot of time and running up debt? How do *they* compare to those who weren’t admitted?

  2. The Ost paper estimates a whopping 4.5% earnings difference around the cutoff for Ohio’s most marginal public college students, and has unimpressive R-squared numbers.

    That is, small enough at an important discontinuity that it is probably hard to say that’s just sheepskin effect rather than being correlated with actual productivity from an employer’s perspective.

    I think “substantial earning benefit” is a, um, optimistic characterization of those results.

  3. Y’know, the environment’s shifted. Used to be, back about 1950, maybe 5% of each cohort coming of age went to college, mostly affluent males and one could argue it was a status thing — of COURSE your Director of Hiring must have a degree in history from Princeton! And all those college grads got good grades, unless they turned to beatniks or drunks, of course they did!

    Mid1960’s, when I was in school, 10% of those cohorts were going to college, maybe two guys for each girl, and the additional education seemed to be work related — the country needed engineers to build weapons that would hold back the Russians and accountants to make ever larger corporations more profitable and better trained bureaucrats to staff government offices. And the economy was growing, we needed those graduates, as any idiot could see!

    And now … we’ve got 30-40 % of those youngsters in college — which might be about as many as can actually gain from increased education in abstract studies. And there are more females than males, which … I’ll handwave and mumble something about social and developmental factors. And we’ve had a really crappy, slow growing economy since about 2008 or maybe 2001, and American employers are known for outsourcing jobs, and manufacturing employment is shrinking and print journalism is going downhill and people are mumbling about Uber driving taxi companies under and Google producing autonomous driving systems which will eliminate a couple of million truckers’ jobs and school systems are laying off teachers as fast as they can and the overall economy isn’t going up more than a percent or two each year and …

    Y’know. I don’t think these wonderful comparisons of students and educational outcomes in the 2010’s and the 1970’s really hold up. Maybe Bryan Caplan’s onto something with his screaming about education being nothing more than advertising one’s social status, but … I think that says more about Caplan and what he chooses to look at than overall reality.

Comments are closed.