We have a moral revulsion to doing controlled experiments on small groups of people. Yet we have no problem conducting uncontrolled experiments at large scale.
Consider the California governor’s order to close public parks and beaches. We are unlikely to learn anything from this experiment.
A controlled experiment would start with two sets of healthy young people. Some would stand outside in a setting where there is an infected person. Others would stand outside where there is not an infected person. Still others would stand inside where there is an infected person. And others would stand inside where there is not an infected person. We would compare outcomes.
Many people would find this idea repulsive. Yet we would learn something from it.
Years ago, I noticed the same thing about education. I happened to be seated at dinner next to a high-ranking career official in the Department of Education. I asked why educators rarely used controlled experiments to determine the efficacy of alternative methods. He said, “Would you want your child to be the subject of an experiment?” My jaw dropped. “They do it all the time!” was my reply. “They just don’t do it rigorously/”
Schools were often introducing new variations in curriculum. Like the decision to close the beaches, these were arbitrary changes based on intuition, with no plan to learn whether or not they were effective. Tampering, as W. Edwards Deming would call it.
Markets allow society to test the behavior of entrepreneurs. The social mechanisms for testing the behavior of politicians do not work nearly as well.
All it takes to believe that testing is immoral is to believe that your preferred policy is obviously the best policy. In that case testing amounts to knowingly subjecting some people to a worse outcome.
I’m pretty sure most people do believe their preferred policies are obviously the best. There are very few Arnold Klings (or Scott Alexanders) out there who can live with uncertainty.
Well stated.
I’ll add that it has never occurred to most folks that all of life is experiments. Their mindset is ‘just do what works’ or ‘sounds good’ or ‘what the experts tell us’.
The high-ranking education official has never viewed macro actions as an experiment.
A laconic essay about social self-deception!
Spectator sports & reality TV are other instances of self-deception about experiments. We don’t countenance voluntary participation in controlled social-science experiments involving risk of serious injury, but many people enjoy spectator sports and reality TV partly to observe volunteers (athletes, ordinary people on TV) engage in competition or conflict under pressure, and with risk of injury or humiliation, in compartmentalized, rule-bound settings in the wild.
A mass experiment doesn’t change the participants relative position. If it fails, it drags everyone down absolutely, but preserves the relative positions. A micro experiment that fails damages both the absolute and relative position of those that participate (or are forced to participate).
As such, wherever relative position is important (and it is extremely important in many many cases…..), it is far safer to be part of a mass experiment than a micro experiment, far easier to support a mass experiment than a micro experiment.
Doesn’t take a moral/ immoral lens to understand the behavioural incentive to prefer mass experiments, only takes a lens that understands people have a rational sensitivity to relative position.