I am particularly interested in the IHME project because it includes state-by-state forecasts. For Minnesota, it currently projects 1,039 deaths. This is in marked contrast with the model on which Minnesota’s governor claims to have relied when he issued the state’s shutdown order. That model, according to the governor, projected more than 70,000 deaths in the state unless people were ordered to stay home and businesses were closed. He hasn’t said, to my knowledge, how much good his stay-home order will do, according to the same model.
I have a distrust of computer models that goes back a long time. Coincidentally, today the latest issue of EconJournalWatch is out, including my piece on why Edward Leamer, a skeptic of econometric models, deserves the Nobel Prize. I’ll say more about that in a post later this week.
It amazes me that people trust computer models as much as they do. Have they forgotten how computer models performed in rating mortgage securities prior to the financial crisis of 2008?
I would prefer to base decisions on experiments, not on models. For example, on the utility of masks, I think that the best evidence we have is the quasi-experiment of Hong Kong and Taiwan vs. Europe and the U.S. But I would be happy to see an experiment in which similar communities adopt different approaches, with one of approach being the lockdown and another approach being allowing people to go to work and school wearing masks.
As you know, I would like to see experiments performed to see how the virus spreads (testing the “doorknob effect,” for example), And a careful study of a random sample of the population to try to get a handle of how many people have the virus currently and how many people have had the virus.
You want experiments in the chaos to sort out the sustainable.
You have a model, the Monte Carlo model. It is the model that models experiments. If you want the Monte Carlo model to merge completely with experiments then you need betting pits. Easy to do.
Let us model the Covid trials as a horse race, make some bets. Libertarians of this philosophy will claim fair betting pushes the winner closer to the finish line. Get us through this sooner.
We have an index of covid trials already.
A futures bet is a future promise to buy or sell some quantity of dosage predicted by an individual trial. The inflationary effect of the new liquidity market will cause all prior purchases go up, first. It is the first one there effect, mostly buyers of dosages. As results come in the futures market will adjust dosage demand to shape supply and demand that they match distribution costs.
What’s wrong with computer models? https://en.wikipedia.org/wiki/The_Limits_to_Growth
All kidding aside, there is a new Santa Fe Institute presentation by multiple modeling teams discussing the second wave… highly recommend.
https://youtu.be/HDwydir9s_A
Finally got my Covid results back EARLY, only seven days after testing.
All models are worthless at a minimum because the testing data is incomplete and lagged and will be for some time. Meaning it’s garbage in, garbage out.
Well yea computer models are only as good as they inputted by human, etc. etc.
The reality of COVID-19: It is making everybody an armchair statistician and it feels like every nation and 50 states are running separate first year Stat class experiments. Literally we don’t have a control subject so every state/nation does not have a data on null hypothesis here. So this is not study education studies, etc. And everybody is looking for answers TODAY!
And considering the spread of COVID-19 is like pinkeye at Elmentary School playground computer/exponential models will vary greatly with R spreads at .7, 1.5 or 3, all of which we are seen the last 5 months.
It’s probably too late to get any actionable intelligence – verified insights that would be usable in time to adjust course enough to make a big difference, before this event naturally plays itself out.
That’s infuriating and lamentable, but it means that the most productive use of our efforts are to:
1. figure out how to get things back to normal, as safely and as soon as possible,
2. How to best unwind all the extraordinary government interventions, especially credit allocation, also as safely and quickly as possible, to make sure the cure is not worse than the disease,
3. Make sure we’re ready for the next one, and
4. Hold people accountable, clean house where needed, and collect “truth and reconciliation” style testimony elsewhere.
So we need an overall Renormalization Strategy to restore health to the system alongside the populace, and which consists of a Reintegration Strategy, an Exit Strategy, a new Epidemic Strategy, and a State Capacity Strategy.
As part of the new Epidemic Strategy, certainly, we absolutely must do large, rigorous, studies to obtain replicated answers to all of the scandalously open questions anyone would have figured would have been disposed of definitively long before the year 2020.
We will have time to do all that in the years to come, to mourn our losses, to build our stockpiles, and to ensure better domestic security of essentials.
But, it takes a long time to build support for any effort. There is not enough time to learn more about the virus quickly enough to do any good. But there is perhaps only just enough time to get a critical mass of people onboard for a return to normalcy before bad decisions cement in place as the new social structure and become permanently irreversible.
Trying to do experiments or even clinical trials in the middle of an epidemics is a fool’s errand. There is no way to get good data in the middle of an epidemic. Maybe the U.S. should have spend more money studying aerosol physics contamination transfer, the benefits of masks, etc during the good times but since no cares during the good times, no good data exists.
The key fact: doubling time is only a few days.
If you’ve already got a lot of cases, then that’s just too quick. By the time you study and establish the new facts, it’s too late to use them to do any good. You’ve placed your bets, now you have to live with the results of your blind gamble.
Either what you decided to do was good enough, and the doubling will slow a lot and keep going down soon (some guess within just a few weeks) or it wasn’t, and if so, most folks are going to get infected not much longer after that. By the time you know for sure one way or the other, not much you can do would help you avoid that outcome.
Also, some strategies are incompatible, and the experiments can’t be run so long as the tested zones are not isolated from each other, with people unable to cross boundaries. With countries that’s not unusual. Between states in the US it is almost unprecedented.
Rather than models or experiments, I would use an analogy to battles.
Soldiers are trained to operate within the “fog of war”; you don’t know what’s going on and taking time and resources to find out must be approached with great caution; you may need the time and resources much more than you need the info. In a crisis, a mediocre plan vigorously implemented is often better than a perfect plan implemented slowly. Plans must adjust quickly as new information comes in, which it will.
In general, the intellectual class (of which I am a somewhat lapsed member) isn’t trained to work this way. Getting more information is our reflexive response and our raison d’etre. But that’s not how crises work. Let’s be honest- if we had the best reasonably practical model, it’s not likely to give us (the general public) any more actionable information than “stay home and wash your hands”.
Everyone keeps trying to find the quick and easy way out of the pandemic. What everyone should have realized a couple of weeks ago is that there is no easy way out of a diseases where the people are contagious before they are symptomatic.
What everyone needs to realize is that everyone needs to do their maximum to limit exposure and limit the spread. Masks, testings, or experimental drugs are not going to save the day and allow everyone to get back to going to brunch. However, Americas like Italians and the Spanish are too lazy, too self-centered, and generally too stupid to do what it takes. And several 100K of Americans are gong to have to die because we have those traits.
Actually it’s easy to think of a number of things that might work. Variolation of everyone under 60 (except those with disqualifying conditions) might have worked, and might still be a good idea. Building as many ventilators as humanly possible is an obvious choice. People are working on drugs and vaccines right now, and good luck to them. But picking one (or more) and doing it is the important part.
Although I am stuck at home and my hands have been washed, so I suppose arguing on the internet isn’t hurting in this case. But it certainly isn’t helping.
You may want to look at what is happening in NYC before you say that it is OK to kill a large number of people below the age of 60 (and probably kill virtually everyone is a nursing home) just so a few people can go back to drinking in a bar or eating brunch.
https://slate.com/technology/2020/04/coronavirus-new-york-er-doctors-log.html
There is no easy way out. That is the only message that we should be hearing from very politician, community leader, etc. There is no clever idea that is going to solve this. The only thing that saves the economy and gets everyone back to some level of normally is maximum effort of everyone.
There is a reasonable case that variolation would kill very few people; fewer than the alternative. This virus isn’t often fatal to the young and healthy and there are clever things we could do to make it less fatal. “Maximum level of everyone” sounds like a bad idea under the circumstances – “stay home and wash your hands” is probably better advice.
OTOH, I think “coordinated effort” would be a great idea. Any plan plus resources and commitment is likely to work out better than every plan pursued haphazardly all at once.
I have an idea for a fairly simple experiment. I’m not going to do it myself, and I don’t know what the point of proposing it here is, except that the squirrels in my yard are tired of me yelling at them about the CDC.
Find some non-toxic (and not foul-tasting), easily traceable liquid. Have subjects swish it around like mouthwash, spit it out, then either fake a sneeze or trigger a sneeze. Aim at concentric circles at different distances. Do the test with no cover at all, a cheesecloth mask, and a tea-towel mask. (I saw somewhere that tea towels were good for this.)
At 0.5, 1, and 2 meters, what’s the reduction in liquid spread versus no cover. If it’s 5%, then R-naught isn’t going to budge. If it’s 30%, that’s going to “flatten the curve”. Intuitively, I don’t see why it wouldn’t be much *more* than 30%.
If cheese cloth works as well as tea towels, we can do this with bandanas and all play cowboy together — then America will really be Great Again.
God bless you and the mask you danced in with, Mr. Kling!
Now, back to the squirrels… “FauCHEEEEEEEE!”
Maybe I’m punchy, but I laughed out loud.
Perhaps somebody already made a similar measurement.
https://academic.oup.com/annweh/article/54/7/789/202744
“Results obtained in the study show that common fabric materials may provide marginal protection against nanoparticles including those in the size ranges of virus-containing particles in exhaled breath.” That is from a study that says “40–90% instantaneous penetration”.
I don’t know why there is a wide range but between 40% and 90% I will just say it’s about 65%, so a 35% reduction. So if the person coughing is wearing a mask and the person breathing is wearing a mask the total penetration is 0.65 times 0.65 which is 0.42. This means R nought is reduced by 58%. In reality we will get even lower R noughts because if a person coughs he will get ostracized.
Why on earth are there workplace and other “public” water taps that are not automatic and why did they not all get changed 30 years ago or whenever it was that per capita wealth was high enough that the cost was insignificant? There is perhaps a majority that just don’t care. If your hands are dirty you don’t notice dirt. May be it’s optimal for people to have dirty hands and more herd immunity for all the previous infections but for some reason it is no longer optimal for SARS-Cov-2?
Arnold,
You should be reading Greg Cochran who will probably drive you nuts but is usually right about everything he thinks about:
https://westhunt.wordpress.com/2020/03/31/no-salvation-in-the-denominator/
This is not a question of ‘models’ but how epidemiology works.
John Hinderaker says “patients for whom a ventilator is indicated have a 75% chance of survival if they get a ventilator, but only 25% if they don’t.”
Andrew Cuomo says close to the opposite: “If you go on a ventilator there is only a 20% chance you will come off it.”
What else does the governor say? “The first order of business is to save lives, period. Whatever it costs.”
Now if I understand the man’s thinking on this, we could make the cost infinite, fully aware that we are, at the same instant, writing off four-fifths of that. We should knowingly set fire to 80% of the infinite amount of resources we’ve chosen to devote to the goal of saving not very many people’s lives. That’s the endgame, isn’t it? What we call “victory” isn’t all that much to show for our sacrifice. Where we end up is close to where we would have landed anyway if we’d never lit our hair on fire. Which isn’t easy to justify to the people on whom we inflict this suffering. We do this harm to them for 20% of 75% of no reason at all. We shorten their lives so as not to shorten the lives of others. We place this burden on them so as to relieve some fraction of a burden on some fraction of as many other people.
I noticed being confused about your ‘anti-model’ stance recently and then again reading this post just now.
Models that aren’t based on or validated by experiments are untrustworthy. A good model should (at least) roughly fit previous experiments (and other pertinent data). A great model should successfully predict future experiments (to some degree).
But experiments are not an alternative to models – for one, they require models to exist at all, to even be considered or imagined as worth performing.
I agree with your general sentiment that most “computer models” are not very good, but I don’t think that has much to do (necessarily) with them being computer models versus just bad models.
But I’m also less sure that they’re ‘bad’ models versus ‘not really-hard science’ models, and even this is a drastic over-simplification of exactly how few ‘really-hard science’ models there are. Quantum mechanics is incredibly precise, but only for extremely simple systems. Newtonian mechanics and the theories of Relativity are very precise too – so much so that discrepancies between model predictions and experimental results are often clues that our data about the systems under study is incomplete or incorrect. But every ‘softer’ science is attempting to study more and more complicated systems. In the ‘softest’ sciences, those studying people, the subjects of the models and experiments are potentially capable of altering their behavior based on the consequences of those sciences (e.g. their published works)!
I think computer models are useful to the extent that they serve as good intuition pumps – and their goodness should be measured based on how well we are able to confirm that specific phenomena observed in the output of those computer models matches what we observe in our studies of and experiments on the corresponding real world systems in which we’re interested.
Of course we should experiment – and especially about ‘micro models’ like ‘How well do masks work to mitigate transmission?’. Ideally, the results of those experiments would be incorporated into the computer models too.
I think the ‘danger’ in using computer models is in expecting their output to be precise. I wouldn’t and don’t expect the 70,000 deaths predicted by the model used by the governor of Minnesota to be accurate – just suggestive.
Similarly, I think there’s lots of interesting dynamics that could be studied in the form of computer models of your ‘patterns of sustainable specialization and trade’ model (or family of models). I wouldn’t expect those models to be able to accurately predict the world economy, or even a small and extremely isolated economy – human beings are too complex! But I would expect those models to demonstrate ‘micro phenomena’ for which we could, possibly, develop models that are accurate enough to predict real world experiments or observations.
Maybe a better title, and tag line for this idea, would be something like ‘Macro models vs micro experiments’.