Testing a vaccine

A follow-up/clarification to my earlier post:

I believe in what I call the Avalon-Hill model of how the virus affects people. That is, it depends on a combination of viral load and patient vulnerability. Accordingly, I would like to see a vaccine tested on various combinations of these factors. That means that the experimenter should control the viral load rather than leave it to chance in the context of selection bias (people who volunteer for the trial may be behaving in ways that reduce their probability of being exposed to high viral load).

In principle, that means assigning a high viral load to some high-risk subjects in both the control group and the placebo group. That could discomfit the experimenter, not to mention the experimental subjects.

But if you don’t do that, what have you learned? If the most severe cases in the real world come from people exposed to high viral loads, and almost no one in your trial was exposed to a high viral load, then you have at best shown that the vaccine is effective under circumstances where it is least needed.

14 thoughts on “Testing a vaccine

  1. Who, if anyone, would volunteer to take a high viral load?

    And whoever they were, why would we think they were anything like random sample of people.

    • Volunteer? The world is well provided with prisons and concentration camps.

      But that just makes your second point even stronger.

  2. It is interesting that bioethicists have decided that the safety of a few thousand people who would willingly be exposed to the virus in a challenge trial is more important than the safety of the millions who are unwillingly exposed to the virus. I get that we don’t fully understand the long term impacts of the virus (waivers could include worst-case scenarios to cover that), but that also applies for those who unwillingly contract it as well, which is thousands per day as this drags on.

  3. The question is really about the marginal value of this information, and the marginal cost of getting it. From a medical standpoint, the most interesting questions are “How much does this reduce the spread of the disease?” and “How much does this reduce the severity of the disease?” (immunity counts!). The control-experiment test they’re doing with the vaccine is probably the best design for a real-world answer for the second question. The first question is harder to answer, but deliberately exposing subjects doesn’t really answer the question of how much they would then spread. So while the additional information would certainly be good to have (how much does it protect against what kind of exposure, etc.), it’s not clear what we would then use that information for: The decision of which vaccines to use can be made pretty effectively with the case rate from the existing experiment (and additional data as time passes, but that’s unlikely to change the answer much). So, even assuming you trust medical experimenters to spend lives and QALYs wisely, how much is that extra data worth?

  4. I worry that your arguments would have squashed the development of polio and MMR vaccines.

    Your model is overly simple. We know something about how viruses and the immune system interact–in particular, how the immune system learns from exposure to the disease. We know how to artificially trigger that response in certain cases. This model works well enough to have enabled the development of vaccines against other diseases.

    Beyond the study you site, we have incontrovertible data that the new vaccines trigger immune system responses at the cellular level. (At least in the vast majority of cases. And we also know that the placebos never do.) Think of the broad population testing data as due-diligence icing on top of that cake.

    Note that the low level of exposure in the population test is the one that occurred under (or just after) oppressive and expensive measures to curb the spread of the virus.

  5. These posts seem to be answering the wrong question. The relevant question is: “Will the vaccine significantly reduce the number of serious cases at a population level?”

    It may well be true that we currently lack evidence to show that the vaccines will be highly effective in high risk persons who receive high viral loads [something about which I’m agnostic]. But even if that’s the case, the current evidence does suggest that the vaccine is likely to be effective in the vast majority of cases. And that should result in a diminished spread, with fewer cases of high-risk persons receiving high viral loads.

    In other words, even if the vaccine doesn’t protect high risk persons from high viral loads, we should still be quite optimistic if the vaccine it makes it less likely that high risks persons will receive high viral loads to begin with.

    • Well said. It’s remarkable how often blog commenters don’t grasp a key feature of infectious diseases.

  6. I think there’s a strong case to be made for testing different viral loads on willing adult participants who are in the least risk camp (e.g. college students).

    As for testing high viral loads on vulnerable patients (who may only be given a placebo), other than perhaps patients who already have at least one terminal illness I don’t think that would be ethical even if they agreed to it.

  7. If the rate of death / degree of sickness is linear w.r.t the viral load, then this is not an issue. Even if it’s a nonlinear function then we’d still be approximately correct.

    It would only be an issue if it’s a very sharply nonlinear function. Meaning you’d almost always not get sick with a small dose under a threshold, but then almost always get sick past a certain threshold.

    Yes, if that ends up being the case, then the numbers might only show that the vaccine is 95% effective when you’re within a certain threshold.

    I don’t know what reason have to think that the viral load in this very non-linear way. Are other viruses known to behave like that? I don’t know I’m asking.

  8. I suspect virus load is fairly random, and thus don’t think that it’s a particularly crucual confounder to control for. Specifically, conditional on being infected, I don’t think that someone who was ‘extra careful’ is going to be much less likely to get a high load infection than a less careful person, especially at a level that would be necessary to overwhelm a primed immune system. I don’t think this critique of the vaccine is very compelling.

  9. Two things:

    1. The rate of number of infections in the placebo arm was slow-growing because the tests got off the ground in the U.S. in the late summer-early fall lull in the number of infections. By late October, the growth rate in placebo arm infections was rising quickly.

    2. Some volunteers in the vaccine arm can tell they got the vaccine rather than the placebo due to physical reactions such as pain near the injection site. That would suggest that the average behavior within the vaccine arm would be a little riskier than among the placebo arm, so that would bias the reported efficacy down somewhat.

    • I came here from Marginal Revolution. I must say that the quality of comments here is far higher than at Mr Cowen’s playpen.

Comments are closed.