The Null Hypothesis in Health Insurance

is that, in the United States, better health insurance produces no difference in health outcomes. Recently, for example, Katharine Baicker, et al, found

This randomized, controlled study [in Oregon] showed that Medicaid coverage generated no significant improvements in measured physical health outcomes in the first 2 years, but it did increase use of health care services, raise rates of diabetes detection and management, lower rates of depression, and reduce financial strain.

Pointer from, well, everyone. All I can say is that this is really separating what David Brooks calls the “detached” from the “engaged.” The latter are making an all-out effort at what I call trying to close minds on your own side.

Somewhat detached commentary includes

Tyler Cowen, Ray Fisman, and Reihan Salam.

Robin Hanson has an even stronger version of the null hypothesis. His version says that differences in health care spending produce no difference in health care outcomes. He and I disagree about how to characterize this result. Let me try to explain how we differ. Let us stipulate that:

1. Some medical procedures improve health, but not in a way that shows up in statistics. For example, if you get your broken arm fixed, you are much better off than not getting it fixed, but this will probably not show up in measured statistics of health outcomes, including longevity.

2. Some medical procedures are a waste (futile care, unwanted care, treatments of non-existent ailments, treatments that do not work, and so on).

3. Some medical procedures have an adverse effect on health.

4. Some medical procedures improve health outcomes, but only with a low probability (e.g., precautionary screening).

5. Some medical procedures definitely improve health outcomes in a measurable way.

Note also, that most studies of medical spending are not controlled experiments. In observational studies, including cross-country comparisons, the results tend to be dominated by a 6th factor, namely that health outcomes are determined much more by individual genes and behavior than by medical intervention.

Robin and I agree that (5) is true. The question becomes, how does (5) wash out in the statistics on differences in spending? His view is that there has to be enough (3) to offset the (5). My view is that it is mostly that (1), (2), and (4) serve to dilute (5). If I am correct, then researchers should find some quantitative differences in health outcomes, but these differences will not be statistically significant. Out of (bad) habit, they will report this as “no difference in outcomes.” This makes it sound as if they have proven the null hypothesis, when they have merely failed to reject it.

Of course, in a large study (as this was), there may not be much difference between failing to reject the null and proving it. The confidence interval around zero could be small (if someone has access to the paper, you can let me know).

2 thoughts on “The Null Hypothesis in Health Insurance

  1. If the null hypothesis is no improvement in health outcomes, isn’t a finding that depression rates dropped grounds to reject the null hypothesis? On the one hand kind of nitpicky, but on the other hand mental health is important!

Comments are closed.