1. I remain a vaccine skeptic. Consider these two recent reports.
Out of 170 adult volunteers in the nearly 44,000-subject trial who developed Covid-19 with at least one symptom, 162 received a placebo, while eight got the vaccine, Pfizer and BioNTech said.
Ninety-five people in a 30,000-subject study developed Covid-19 with symptoms; of those, 90 had received a placebo and only five Moderna’s vaccine.
OK. Assume half received the placebo (Does anyone know the actual percent that received the placebo?). So with no vaccine, 162 out of 22,000 got the disease in the Pfizer study. That is less than 0.75 percent. 90 out of 15,000 got the disease in the Moderna study. That is 0.3 percent.
Extrapolate that to the entire population. Of 330 million people, if 0.5 percent get the disease, that would be 1.65 million people. If the fatality rate is 0.5 percent, then that would mean just over 8000 deaths, which is about one week’s worth in reality. If the whole country were like the sample that received the placebo, this disease would never have made it into the public consciousness.
Another way to look at it: The Pfizer study followed participants for more than three months, starting in late July. During that time, more than 6 million new U.S. cases were reported, or about 2 percent of the entire population (the percentage would be even higher if you exclude children from the numerator and the denominator). So more than twice as many people got it in the general population as got it in the placebo sample studied by Pfizer.
Still another way to look at it: the number of cases that emerged in the placebo population was less than what can emerge from a single super-spreader event. Apparently, there were zero super-spreader events in either study. So these studies tells us nothing about the ability of the vaccine to work against a super-spreader. I also suspect that they tell us almost nothing about the ability of the vaccine to work for vulnerable populations.
Maybe the vaccines are 90 percent effective, in which case it is easy to recommend them. Maybe they are 0 percent effective, in which case it is easy to dismiss them. But what if in reality they are 50-70 percent effective? That would create a dilemma. From a central planner perspective, you want everyone to take a vaccine even if it is only 50 percent effective, because that would dramatically slow the spread of the virus. But meanwhile, a lot of individuals who got the vaccine will still get sick and die. That would put the agencies in charge in an awkward position, without any credibility left to deal with the next pandemic.
2. While I am being contrarian, let me go after the “keep R below 1” theory. That is the theory that if we can get the reproduction rate below 1 and keep it there, we can eradicate the disease. Ergo, even a mostly-ineffective intervention, such as an inaccurate test, or an unreliable vaccine, or a mostly-useless lockdown, if it brings R below 1, can achieve eradication.
My problem with “keep R below 1” is that it is a representative-agent model. That is, it treats everyone the same, with identical probability of getting or spreading the disease. But in fact people differ greatly in terms of vulnerability and in terms of propensity to spread the disease. Inferences that “scientists” draw from the representative-agent model are generally bogus. I don’t trust anyone who would make policy based on a representative-agent model, and that includes anyone who uses the “keep R below 1” theory.
3. I am still not impressed with “the science.” They (scientists) are still arguing over the extent of asymptomatic transmission. They are still arguing about the effectiveness of masks. Katya Simon, to whom Tyler Cowen provides a link, writes,
Implement indoor mask mandates for public spaces. Outdoor mask mandates are ridiculous. COVID19 does not appear to transmit outdoors. Enjoy our great outdoors!
Heather Heying and Bret Weinstein say the same thing. But Cambridge Massachusetts, which is where a lot of leading-edge biological research takes place, is a paragon of outdoor mask-wearing.
They are still arguing about the effectiveness of lockdowns.
They are still arguing about how long immunity lasts.
They are still arguing about long-term effects.
I believe that they are still arguing about the mechanism by which the virus causes illness.
All of this reinforces my doubts that a vaccine will prove as effective in practice as it has in trials.
Note that many of the issues about which there are arguments could be clarified, if not completely settled, by careful controlled experiments. As I pointed out more than 6 months ago, experiments would be really useful, but the people you would count on to do them are not willing to do so.
Experimental results are signal. Pronouncements that are not based on experiments are noise. Don’t tell me to “listen to the science” when what I am being asked to listen to is noise.
4. So where are we today? As of the other day, the average daily death totals were higher than at any time since early May. (Tyler Cowen shows a chart.) Unless you are more impressed than I am about the vaccine test results, it is appropriate to be a virus pessimist right now.
I think that there is at least a 25 percent chance that we will be as fearful of the virus a year from now as we are now. And if our fears have declined, this may be due mostly to a change in reporting about the virus. Perhaps someone with congestive heart failure who dies with the virus will no longer be counted as a virus death. Perhaps the press will no longer report cases of long-term damage from the virus. Should such a change in reporting take place, a cynic might call it the “Biden effect.”
The people who have enough trust in the scientists to participate in a vaccine trail are not much like a random sample of the general population. The are the very people who are the most trusting of those scientists.
That means they are also much more likely to be taking their advise on social distancing, masks, and attending large gatherings where super spreader events could occur. It would be very weird if the placebo group didn’t have substantially fewer cases than the general population.
Exactly. I came down here to post this but you beat me to it. Instead of being contrarian just to be contrarian, a little thinking helps.
Arnold, you be you. I’ll be getting a vaccine.
I didn’t get the impression Arnold was saying he wouldn’t get the vaccine nor that he would tell other people not to.
He expressed doubt about how effective it might be in certain situations relative to the headline number.
Isn’t this effect what the placebo group is controlling for?
24 minutes into this causal inference podcast by epidemiologists,, they discuss the ways in which the efficacy could be overstated from the vaccine trials:
https://casualinfer.libsyn.com/happy-anniversary-to-us-season-2-episode-1
They doubt the 90% reported by Pfizer will hold, but still find the news encouraging. Part if the issue is that people who are infected between doses (it requires two) or within 7 days of the last dose drop out of the study, which may explain the lowish infection rate. They discuss possible sources of bias in this type of trial.
They also discuss comparing the infection rate to an expected rate. The study was international, so the US infection rate isn’t quite right.
I don’t think it is warranted to be so pessimistic on the vaccines because the control group had fewer incidences than the general population. The trial participants were predominantly selected from populations at higher risk of catching the disease without other interventions. This group was made up of healthcare workers and other people more likely to understand basics of disease transmission and more likely to engage in non-pharmaceutical interventions to reduce their risk. The Moderna vaccine trial design information can be found here:
https://clinicaltrials.gov/ct2/show/NCT04470427
The people who think covid-19 is a hoax didn’t bother enrolling in the vaccine study at all. So it is not unreasonable to expect a lower incidence of disease in the control group because of population sampling biases.
With regard to your comment about there being no cases of super spreader events in the study population, I am very confused. Why couldn’t all of the cases in the study population be from super spreading events? The study design cannot rule one way or another on this question. Remember, the participants in these studies are not living together or even in the same metro area as most other participants.
The number of participants in the control groups for each study is known, see here:
Moderna has a 1:1 ratio, see page 7
https://www.google.com/url?q=https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf&sa=U&ved=2ahUKEwjN2bz03Z3tAhUoQzABHcH8DikQFjAEegQICBAB&usg=AOvVaw0-2FQho9l6ZxudGUTPwDJF
Pfizer also has a 1:1 ratio, see page 14
https://www.google.com/url?q=https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf&sa=U&ved=2ahUKEwiE7ea14J3tAhUgSzABHTgaA1oQFjADegQIBxAB&usg=AOvVaw3b1_1iWOe-lu6diswrWy5J
I generally agree that it would be useful to have challenge data for these vaccines and for other interventions in the pandemic. And I agree that it is still within the realm of possibility that the vaccines will be less effective than currently expected because of the nonrepresentative nature of the study participants. However, I see no reason to expect these vaccines to be useless.
Agree, and nice rebuttal. This is an attempt to take down any vaccine trial ever, not a Covid vaccine trial specifically. The objections don’t really make sense and seem statistically ill-considered. It is not guaranteed that the effectiveness will be 95%, but it’s all but guaranteed the effectiveness is quite high.
The key is the randomness in placebo vs vaccine. As long as you take a large enough sample and apply it randomly at 1:1, you can very clearly take away a large difference in contracting Covid in placebo vs. vaccine as an effective vaccine. Doesn’t mean the number is perfect, definitely means it is effective.
Nobody believes Covid-19 is a “hoax”, but many believe that its risks and impact have been overblown. Authorities have shut down schools, unemployed millions, told everyone they shouldn’t see their families at the holidays, implemented stay in place orders, forced everyone to wear face coverings everywhere, and other actions. These may indeed be very disproportionate to the risk. Arnold even lays out some of the ways in which he thinks the risks have been overblown by the narrative.
I’m on record as saying it doesn’t matter if the vaccine works or not. What matters is that we have an excuse to “declare victory” and be allowed by our betters to start living our lives again. Even if the vaccine was nothing but a placebo, if you just change how many cycles your PCR test does and how you recognize deaths you could essentially make COVID disappear for all intents and purposes.
I take Arnold to mean that the vaccine might as well be the famous “tiger rock” from the Simpsons. Whether we worry about the disease or not will be based on the medias reporting of it, not its reality.
Actually, many people believe it is a hoax. On one of my ICUs I have a pt with a Covid diagnosis who is refusing Covid meds since Covid isn’t really. She is angry at my team because they cant figure out wha tis really wrong with her. Was have another who thinks it is a hoax, but is at least willing to take meds. My favorite is the guy who thinks his IV meds have chips in them so the government can track him.
Steve
My model of the disease is that viral load matters. If the population tested generally avoided coming into contact with a high viral load, then the vaccine was tested in circumstances where its effectiveness was not so much needed. If it is not so effective when people encounter higher viral loads, the 90 percent estimates will be way overstated.
>—“My model of the disease is that viral load matters. ”
This is a perfectly reasonable model. Even so, shouldn’t we expect that the vaccine will very substantially raise the viral load that most people can tolerate without serious illness and therefore make a huge positive difference in reducing the severity of the pandemic?
Yes. Viral load mattering is fine, but it misunderstands randomness. Yes, the sample may be more risk averse than the larger population (though I doubt it, people that are in vaccine trials are generally quite bold), but within sample you would not see that difference in the number of people contracting vaccine without effectiveness.
I think you are right, and people are missing your point. But I think people are missing the point, because they might be thinking there is some non-linear magical effect going on. Meaning you can’t simply stack the effectiveness of the vaccine on top of all other variables.
So imagine if wearing a mask is 50% effective, wearing a mask plus vaccine is 95% but no mask and vaccine is only 30% effective 🙂 That kind of thing can happen if getting sick is a non-linear function of the viral load or if the efficacy of the vaccine is non-linear w.r.t viral load, or a combination of both. Or if our population sample, oversamples a hidden variable that is correlated with the effectiveness of the drug. Let’s say they just happen to all be white people, and the vaccine might be totally ineffective for Indian people.
I don’t think it’s easy to correct for all of these.
But we can look at phase 3 trials of other vaccines, and see how they correlate with eventual efficacy of the same vaccines. You can then come up with a empirical correction factor. My feeling is that if the efficacy of vaccines in reality were always substantially lower in practice than in phase 3 trials, then there would either be an official name for this correction factor, or the trial procedures would be amended to address them.
I just don’t think the hoards of PhDs at any of these institutes are dumber than I am, or to be frank, any of us, speculating here.
Any evidence for this? The whole idea of science is to test ideas against actual data not what people find reasonable.
I think you missed the point of Arnold’s post, the results indicate that the samples used for the trials are different from the general population. If we take your assumptions about the vaccine groups to be roughly correct the proper interpretation of the results would not be ‘vaccine is 90% effective’ but ‘vaccine plus mask wearing plus distancing is 90% effective’. This leaves us with two awkward conclusions.
1. We don’t know the effectiveness of the vaccine without distancing and masks
And
2. The general public does not expect a functioning vaccine to mean ‘take vaccine but also still wear masks and distance’. Nearly every commenter talks about life going ‘back to normal’ after they get a vaccine.
Still do not understand the point. Even if a serious viral load overcomes a vaccine (doesn’t really make sense as the vaccine must be stimulating the body’s immune response), there is some effectiveness for people that still take some precautions.
Given that, there will be less virus to transmit as more people are vaccinated, which means less virus to those at-risk. The viral load of everyone will decrease, and the virus will diminish. More people are vaccinated, virtuous cycle of less load, less virus.
It doesn’t have to be as effective as stated, but there is no way it’s not effective.
About my point where the vaccine is stimulating the body’s immune response, the idea of a low viral load not infecting someone is that it stimulates the body’s response enough to fight it off. The virus is already clearly doing that.
I didn’t get that from Arnold’s post.
For one, is there any reason to believe that the trial group and placebo group acted differently? Even if you believe that people who volunteer for clinical trials are different then the general population, there is no reason to believe that having then taken that group and randomly sorted them in two that there would be behavioral differences.
My guess is that a certain percentage of volunteers in the vaccine arm can tell from reactions that they did NOT get the placebo. So, if anything, the vaccine arm is likely to take more risks than the placebo arm.
I was a bit confused by your analysis and would love to see an elaboration of point 2 (“the keep R below 1 theory”). Obviously, the modeling of everyone being identical is a simplification, and real life is more complicated with some people being more likely to spread and others being less likely. However, the way I imaging R as that it is weighted average of all the individual R’s. If you reduce the weighted average well below 1, why wouldn’t the virus go away?
Also, as a helpful contrast, you could compare the covid vaccines to the measles and polio vaccines. The measles vaccine when first introduced had an effectiveness no more than .93. Similarly when first introduced, the polio vaccine had an effectiveness ranging from .7 to .94, depending on the strain. Despite the less than perfect protection, both diseases were eradicated.
What is different about Covid, as compared to measles or polio, that a vaccine of .9 to .95 effectiveness wouldn’t eradicate the virus?
“is a paragon of outdoor mask-wearing.”
We have spent the last several months in crowded outdoor spaces. Not necessarily BLM protest level packed together, but very often not much less. We never got it despite being within six feet of thousands of people. The urge to implement useless face burkhas outdoors is another sign of the satanic nature of leftists.
Meanwhile, they just started forcing toddlers at daycare to wear masks all day. I gave up and brought my daughter home.
The study protocol, latest amendment, is a 1:1 randomization ratio between study arms; stratified by risk.
speaking of asymptomatic transmission…
This Wuhan screening study in Nature of the entire population of 6 million is fascinating..
https://www.nature.com/articles/s41467-020-19802-w
Probably the vaccines will work. After all, vaccines are being developed in China, the UK, the US, Germany and elsewhere. Sinopharm (China) says they have dosed 1 million times with their vaccine and it is safe.
All these people are deluded? True, all the aforementioned have a stake in vaccines working, but still…no intelligent whistleblowers yet who say vaccines likely not effective.
I think we can go back to normal, and good. COVID-19 will never go away, some vulnerable will always die.
I wonder about the minks dying. All of them? Did enough survive to create a resistant stock?
And if COVID-19 is natural, why did it evolve to simultaneously infect many species, such as dogs, cats, man, minks and a couple other species, such as marmots?
It is possible that the whole country *is* (roughly) the same as the sample that received the plaecbo – but other things are different:
1) the season (time of year)
2) The criteria for a “case”. According to WaPo, the Pfizer study defined a case as “someone with a positive test result and at least one symptom”. So that cuts out the craziest of false positives – and all asymptomatic folks.
According to WaPo, the Pfizer study defined a case as “someone with a positive test result and at least one symptom”.
WHAT?!?!?!
You don’t need “at least one symptom” to be included in the case numbers that are reported every day. And a lot of research says you don’t need any symptoms to be a spreader. When most people hear 95% effective, they think the person getting the vaccine has a 95% chance of being protected from ever being a “case”, from ever having to worry about infecting somebody, from ever being denied access to someplace because of a positive COVID test.
Here’s the link (paywalled): https://www.washingtonpost.com/health/2020/11/09/pfizer-coronavirus-vaccine-effective/
IIRC, one of the other vaccine trials requires two symptoms? (I now see Josh M mentions this further down)
—–
I’ll also add that even [a positive COVID test plus a symptom or two] isn’t a great definition for a case, given the high false positive rate of tests (at least for PCR tests) *and* the overlap in symptoms between COVID and other things:
https://www.cdc.gov/flu/symptoms/flu-vs-covid19.htm
You would need to compare infection rates for the study to the infection rate for general population during the same time/region. Regions flare up, then die down. Selecting a region needs to be approved, subjects enrolled, etc., there is significant lag. If spot was picked in May, by the time the study starts enrolling subjects in June, dosing in July, and finally immunity a month after first dose, how widespread was the virus and how fast was it spreading in the region? I would say your concerns are valid, but I think the comparisons are wrong.
As I understand it there is an underreported benefit of the vaccines.
The trials counted the number of people that got SICK in the placebo vs vaccine groups.
They do NOT report the SEVERITY of the sickness. As I understand it, even if you do get sick after having the vaccine, the severity is likely to be much lower since your body already has some training on how to handle this particular biological attack. Incomplete or partial training is better than no training at all.
So the 90% to 95% “effectiveness” is actually a more conservative number than it appears to be.
The other key fact is that around 15K people were given the vaccines with no major issues reported. So the downside of getting the vaccine is quite low. Compare this to the downside of not getting the vaccine and getting Covid. It seems like some small percentage (2 or 3 percent?) of unvaccinated infected people will suffer severe effects from Covid.
Low downside, big potential upside. I would take the vaccine right now.
Behold, we know not anything;
I can but trust that good shall fall
At last—far off—at last, to all,
And every winter change to spring.
-Tennyson
The disparity between cases in the placebo group and cases in the population at large is due to different definitions of “case.” To be a case in the vaccine trials, you need to test positive and also have two or more symptoms. To be a case in the population wide statistics you need only test positive — asymptomatic, false positive, doesn’t matter.
WaPo (paywalled, also posted above) says only 1 symptom for Pfizer trial: https://www.washingtonpost.com/health/2020/11/09/pfizer-coronavirus-vaccine-effective/
but I definitely recall hearing 2+ symptoms were required in other vaccine trials.
I’m assuming this inconsistency amongst trials makes analysis harder – as with states setting their own (slightly?) different ‘case’ definitions when reporting numbers. But maybe trial settings are different enough – e.g. due to timing factors like MB described – that this is just a drop in the bucket?
(In other words, maybe data collection is never all on the same page, in the real world?)
I am not a statistics guy, so I always wind up throwing up my hands when approaching evaluation methodologies, but my question to Arnold would be this: what would the trial results have to be to give you some comfort?
We just got gifted two viable vaccines with a third, from AstraZeneca, a few weeks behind.
If this was 1950, we would be celebrating with ticker tape parades in every major city.
Time to mellow out and get some perspective. Even at 70% efficacy, we are well on our way.
Happy Thanksgiving!
You are assuming that all participants got the vaccine on day 1 of the trial. Looking at Pfizer’s press releases, for example only 29k were enrolled as of September 14, and less than 10k in mid August, the graph looks fairly linear. So the total “enrolled patient time” is probably half of what you are assuming, so having half the cases is….to be expected. Of course summer was a fairly low period, so it’s not a perfect analysis, but it eats into your pessimistic argument quite a bit.
Arnold, I recommend you to look into the nature of these mRNA vaccines. This type of vaccine is completely new and untested. You’re in for a questionable stuff.
Just to clarify: the vaccine from AstraZeneca is not based on new mRNA techniques. The AZ approach has been used safely and effectively in vaccines for TB, malaria, Mers and Ebola.
I think this is a straw man of ‘keep R <1.' I think everyone is well aware of the heterogeneity in R, and when someone says keep R <1, it's assumed they mean keep the mean of R below 1. You could argue that many preferred interventions only push down R for segments of the population for which it's already low, while leaving superspreaders unaffected, thus having little effect on the mean, but I think it's well-established that superspreader events are a big part of if not most of the problem. Your critique has nothing on efforts to shut down bars, indoor dining, or orthodox weddings, for example; right or wrong, those policies are aimed squarely at stopping super-spreading type situations.
OK. Assume half received the placebo (Does anyone know the actual percent that received the placebo?)
It is 1:1, randomised. Fairly easy to find their method (Google pfizer vaccine protocols).
So with no vaccine, 162 out of 22,000 got the disease in the Pfizer study. That is less than 0.75 percent. 90 out of 15,000 got the disease in the Moderna study. That is 0.3 percent.
1) likely vaccine volunteers may be more conscientious than average and have a lower chance of getting infected. 2) Not all volunteers would be dosed at the start of the three month trial period, likely it is a rolling effort. 3) Possibly some people are getting infected, but may be excluded (infected betweeen 1st and 2nd dose).
With the data the way it is, and if the safety results are ok, it makes sense to dose a larger section of the population (what’s the alternative?). Given the accelerating nature of the pandemic in the US and parts of Europe, if the vaccines are effective, their effects will be pretty obvious in the next couple of months.
“So more than twice as many people got it in the general population as got it in the placebo sample studied by Pfizer.”
Or the tests applied to the general population give many false positives.