Thoughts on heterogeneity

Tyler Cowen asks why numbers imply spread rates and death rates that are so difficult to reconcile across regions and countries.

People are feeding their elegant dashboards, nifty charts, and fancy computer models with worthless numbers. Nobody seems to want to listen to me on that. But it would not surprise me to find that all of the heterogeneity that cannot be explained by demographics and differences in treatment quality is simply an artifact of the way that numbers are collected.

Only fools claim to know precisely the true spread rates or the true death rates. We don’t even have decent ballpark estimates.

If we were to obtain data that were good enough to infer true spread rates and death rates, and these rates turn out to differ greatly across regions, then I would speculate on a combination of two factors. First, different variants of the virus, which spread and kill at different rates. Second, a highly skewed spreading phenomenon. That is, instead of every infected person proceeding to infect exactly 2.2 other people, you have a few infected persons infecting dozens of others, and most infected people infecting no one else. Put those two factors together, and you will get heterogeneity. But I emphasize that this is purely speculative. Don’t take this idea and run with it. Stop guessing. Get some facts first.

I wish someone at the CDC would take and run with the idea of obtaining scientific data, rather than guessing using the numbers that are being collected. In a scientific study, the investigator chooses who gets tested for the virus, and when the tests are conducted. The study uses the same type of test kit on every subject, preferably a test kit with a low rate of false positives and false negatives. Tests are conducted by carefully trained workers who follow very standard procedures. Before we test a large sample of people, we administer two tests to 100 people and count the number of times that we get different results on the two tests. If it is large, then we need to figure out how many tests we need to do on one person to get a reliable result.

Of the many problems with numbers as collected and reported, consider the issue of time lag. Suppose that two regions each test 1000 infected people on day 1. Region A reads and records the results a few hours later. Region B reads and records the results a week later. Suppose that the one-week spread rate is 100 percent per week, and each region then tests 1000 new infected people. Suppose that the death rate is 1 percent, and death occurs near the end of the week.

After day 8, each region has 2000 cases and 10 deaths. But region A, which reads the results quickly, will report that cases are doubling weekly and the death rate is 10/2000, or 0.5 percent. Region B, which reads the results slowly, will still report 1,000 cases, with a death rate of 1.0 percent.

Another problem is that there is very large variation in the ratio of tests to infected people, not only across regions but over time within a region. As you ramp up testing, you increase the reported spread rate and lower the reported death rate.

Almost all health agencies have chosen not to monitor this crisis scientifically. I wish I could change that.

23 thoughts on “Thoughts on heterogeneity

  1. We only have to look at the start of the spread in New York City to see that some people can infect 11 others.

    On March 3, a second case was confirmed, a lawyer in his 50s who lives in New Rochelle, Westchester County, immediately north of New York City, and works in Midtown Manhattan… He was admitted to a hospital in Westchester on February 27 and diagnosed with pneumonia, and released from isolation after testing negative for the flu.

    On March 4, the number of cases in New York increased to 11 as nine people linked to the lawyer tested as positive, including his wife, a son, a daughter, a neighbor, and a friend and his family.

    • This is microeconomics, not macroeconomics. Predictive models are only useful for planning. R0 discussions are useful for short term planning but each new symptomatic person represent an independent Seed that has germinated and must be Isolated to prevent it from taking Root.

      • I am fairly certain that we could engineer situations where one person could infect way more than a mere 9 people. We have at least one recorded case of a person infecting 9 people, but I doubt that is the upper limit.

        I am now imagining a crowded subway system, with an infected person riding in the same car all day to stay out of the cold, as crushing crowds of commuters go in and out of the train on their way.

    • Well, I won’t criticize this thinking too much, but you are missing the point of R0. You don’t know that the lawyer directed infected those other 9 people- they almost surely infected each other to some degree. There was a patient zero somewhere in the Fall or Summer of 2019, and you can say he infected every victim on the planet using your logic.

  2. Testing kit shortage, they are rationed in LA

    The only stat that seems to be holding is the death rate for symptomatic patients at about 1.5%. So I focus on that and that should be dropping down by half as we treat the symptoms with better cocktails. But it is not dropping. If that death rate does not drop data will get worse, not better as the hospitals get overwhelmed.

    • Table 4 Summary of hospitalized cases of COVID-19 reported in Canada released yesterday shows 32% of Hospitalizations require ICU care and 15% require ventilation.

      This only represents 213 hospitalizations since 35% of the case data does not include hospitalization information. This emphasizes the flaws/failures in the Canadian system; basic information technology requirements and scaling do not seem to be part of the extensive pandemic planning that occurred since SARS in 2003. Data collection should have been automated and real-time.

    • Be a little careful with your definition of that stat; you are describing the death rate of diagnosed symptomatic patients. We have no idea how many people have symptoms but don’t get diagnosed for whatever reason. Maybe they symptoms are mild and they don’t connect them to the virus. Maybe they get a false negative after being tested. Maybe they die in a car crash on the way to the hospital. Whatever the case, it is critical to remember that we are not testing everyone, even if they have symptoms of some sort, and that the tests are not perfectly accurate.

  3. “We only have to look at the start of the spread in New York City to see that some people can infect 11 others.”

    On Day 1, 1 person tests positive for CV19. Also on Day 1, 99 other people are not tested because they are asymptomatic but they are also infected with CV19. On Day 2, 100 other people test positive for CV19. Did 1 person infect 100 others or did 100 people infect 1 other person each?

    We have NO idea how many people have already recovered from CV19 that thought they had the seasonal flu or they had such minor symptoms they didn’t know they were sick at all. CV19 had well over a month to spread in the US from the time between the first case diagnosed in Wuhan and the first case diagnosed in Washington. If CV19 is even half as contagious as the “experts” say do you think only a hundred thousand people have been infected and recovered is it more like millions?

    • The Canadian cases have focused on three waves: 1. Wuhan, 2. Iran/SE-Asia, and 3. EU/US. They have taken a strategy of focusing on symptomatic people connected to Travel. If asymptomatic transmission was prevalent then this strategy should have failed spectacularly. Ontario’s Dr. David Williams said that since March 13th, about 1 million travellers returned to Ontario (normal travel + March Break + returning Snow Birds). Skate to where the puck is going. Wuhan is ancient history.

      • Ontario daily #COVID2019 Emergency Alert since March 27 This phone alert started two weeks too late but everyday is a new day 1 in this battle. This fight is about Communication as much as it is about Strategy and Execution. Many provinces have a 811 number for their TeleHealth system. Ontario has a 1-800 number. This was a mistake. 811 to communicate with knowledgeable nurses and 911 to get emergency help are basic tools everyone should know. The Province also quickly changed to providing a TeleMedicine billing code so that family doctors can communicate remotely with their patients. Telling people to stay at home when they are sick is easy; keeping them from panicking when their symptoms make them feel like they’ve been hit by a truck is tougher.

    • The number of reported US cases has been increasing by an order of magnitude every 10 days or so. It’s plenty contagious; it’s just still rare. Exponential growth is gradual, then sudden.

  4. Science is great for knowing things but when it comes to crisis response, you usually don’t want science. You don’t run a clinical trial during a mass casualty response.

    When you are trying to contain an outbreak, you don’t randomly distribute test kits – you do contact tracing. But that means you can’t really use that data as if it were a sample. And accusing people of distributing the kits for maximum effectiveness in the short term is … silly. That’s like getting upset because firefighters put the water on the fire rather than randomly distributing it and collecting the data.

    You also, ideally, don’t develop tests to have an unbiased error model. Instead, you develop them for the immediate purpose – either with very low false negatives (to trip quarantines), or very low false positives (if treatment is potentially deadly), or whatever you can manage (to get a test out the door). But you can’t use the same test for everything equally well.

    Our data is also bad because if tests are rare, you don’t bother testing dead bodies, but if you are afraid of contagion, the first thing you do is limit contact with bodies – bury them fast, keep people away. So you immediately begin undercounting the fatalities, which gives people a false sense of security (and changing the standard tends to freak people out because the story gets worse, so you end up in a bind).

    This is all without people intentionally hiding anything, to prevent panic or avoid punishment/blame.

  5. Italy had 1/3 of ICU beds compared per capita to US and Germany. UK has 1/2 Italy so watch out there, but still lovin their rationed NHS health care.

    • Worth noting that NY, CA and WA states have pretty low beds per 1000 people too. It might not be a coincidence that problems are hitting those places harder in the US.

  6. Only two more of the eight days necessary to get back my Covid test.

    Good news is that the antibiotic they gave me worked very well. I suspect that I caught the same cold my family had but in me it resulted in a bad sinus infection that kept me sicker longer. In normal times I would have gotten an antibiotic earlier.

  7. What of the interesting results of the increase in testing is that the NYC Police Department is responsbile for more than 0.5% of all the positive results in the U.S. Somehow I doubt if all of the positives come from their normal jobs. I assume that the law enforcement officers, as a group, are ignoring the recommendations about distancing.

    • Bingo. A metermaid all got in my face yesterday which led me to think why are you an essential worker AND why are you not father than six feet away. You don’t need to be screaming at me spittle everywhere one foot away to issue me a parking ticket. And to make it worse you know damn well had I spit in his face like he spit in mine I would have been arrested for assault as many in the news have been recently for daring to have acid reflux while talking to a cop.

      Cops, like healthcare workers, are ignoring the recommendations like normal after all rules are for thee, not me. Waters wet, may they all die.

  8. “health agencies”

    Wait, government agencies playing politics over the job they are nominally empowered to do …COLOR ME SHOCKED! Shocked I tell you.

  9. It doesn’t seem clear that determining the spread and death rates has been a priority. In this country testing has not been a priority, and it is not even clear that all diagnoses — including those cases ending in death — have been supported by tests. Medical personnel have been fully absorbed with saving patients, not data collection.

  10. I’ve heard of a semirandom test of 650 or so Colorado residents to determine the incidence of coronavirus in the general population. It had zero positives. Coronavirus is still rare and will probably continue to be rare for at least two weeks. If present trends continue, it will hit the inflection point (50% prevalence) toward late April/early May.

Comments are closed.