Science: method vs. sayings

Emily Oster writes,

Back in spring 2020, many of us (here’s my particular take) argued that the only way to really track case rates (and serious illness rates and so on) was to engage in a program of random testing. Some countries, like the U.K., did that. But the U.S. did not. And we still do not have such a program. And while our tracking of the pandemic has improved tremendously with better testing and better data reporting, this would still be helpful.

Consider this. Right now South Dakota has low case rates relative to most places with its vaccination rates. Is this because of natural immunity from very, very high infection rates in the winter? Or is it, more plausibly, a lack of testing? We have no idea, since without random sample testing, we cannot know.

Rigorous thinkers, like Oster, naturally employ a scientific mindset. Apparently, many people trained in epidemiology don’t. Epidemiological “science” seems more like a set of sayings that students learn. They learn to scold people based on those sayings.

Even when they do studies, epidemiologists do not seem to think like scientists. Hence you have many irreproducible and unreliable results.

11 thoughts on “Science: method vs. sayings

  1. Since the CDC just announced that what had been the “gold standard” test generates too many false positives ( https://www.cdc.gov/csels/dls/locs/2021/07-21-2021-lab-alert-Changes_CDC_RT-PCR_SARS-CoV-2_Testing_1.html ) perhaps not having potentially bad information to “track” wasn’t all that huge a problem. One wonders what good all that tracking did in the UK. Admittedly they did s fine job surveying that produced a lot of data but it would be interesting to see if that information was ever actually used for any worthwhile purpose other than blah blah blah reports. Oster wants data to dish dirt on Kristi Noem. Would anyone else have a better use for it?

    • Economist Tim Harford discussed this on “More or Less” last year and the conclusion was that test and tracking didn’t work at all in the U.K.

      I’m skeptical that it worked well in South Korea, essentially an island since it borders North Korea, but journalists among others claim it did because Covid deaths were much lower than in most of the West. Yet Japan did very little test and tracking but also had comparatively low Covid deaths. Both Japan and South Korea were among the lowest testers per capita in the OECD, both have low obesity/healthier populations and neither is counting a Covid death nearly as liberally as Western countries.

  2. If we had better tracking, we would know the timing of breakthrough cases. For example, is there a specific date AFTER the second jab that breakthrough infections become viable? That would seem to be a better way to time the booster than a guesstimation of ‘9 months.’ We would know more concretely (scientifically?) an indicator of vaccine effectiveness. We would also have data on how age, gender, ethnic (etc) differences impact vaccine efficacy.

  3. This is a bit old, but as of May 2021, 1 in 6 epidemiologists were still unwilling to bring in their own mail without precautions and 1 in 4 were unwilling to gather with friends outside. A whopping 9 in 10 were unwilling to go on a date with a stranger. As this was mid-May, a large number of them likely had the opportunity to have been either partially or fully vaccinated at this point.

    If I wasn’t told in advance what group of people was being surveyed, I might have guessed neurotics rather than scientists.

    https://www.nytimes.com/2021/05/12/upshot/covid-epidemiologists.html

  4. Do you think this applies to the epidemiology paper out of Carnegie Mellon showing that PhDs are the most vaccine hesitant educational group? They have a very large sample, with over 10000 PhDs surveyed, but they didn’t obtain/present much information about their specific characteristics (field, age, occupational status, etc.) so it is hard to know why they are so hesitant.

  5. The biggest error is that testing for the presence of a virus is different then testing for disease and infection and cause of death. The “elite” are adamant about testing for viral presence, and for their good reason. As long as you test for SARS-COV-2 germs you will find them.

    For the past week I have been visiting parts of the USA that are ignoring Covid – where 99% DO NOT wear masks and life is 99% normal (this includes going unmasked to the DMV). These are places that report high Covid cases (according to the experts), but the people don’t care.

    Thing is, places with elaborate Covid mitigations and hand-wringing report just as high Covid case rates.

    Covid hysteria is no different then religious hysteria. We all sin and live in a fallen world. Pastors and their followers can, with great fervor, denounce sin & lust and demonstrate that fervor by constantly chastising sinful behavior and blaming influences that tolerate sin. For those who adopt this puritanical perspective, the more fervent one is the better ones standing amongst the puritans. The feedback loop encourages hypersensitivity to sin which becomes intolerance for humanity itself

    And yet, it turns out there is no correlation between the # of puritans in ones community and the incidence of sinful conduct. But there is a correlation between the # of puritans and the level of misery, despair and anxiety.

  6. Americans, especially in red states, won’t tolerate systematic testing for fear it would be used to enable forced vaccinations and even abductions such as those now happening in the thousands in Australia.

  7. South Dakota has had almost 15% of the population test positive, and they clearly only test people with symptoms who show up at hospitals and clinics. There are probably two factors more important here than testing rates- South Dakota is further north where the Summer wave last year came later- merged mostly into the Fall/Winter wave, so check back in October; second it might be that almost 100% of the state is either vaccinated or recovered from the disease, but, again, check back in October.

    As long as the country continues to test at these levels, we will probably continue to get the season waves of “new cases”. If we tested for flu or rhinovirus infections this way, it would probably look very similar. The key to ending the panic is going to be to end the testing. I don’t think we are smart enough any longer to do this- we are ruled by idiots.

  8. CDC reports covid hospitalization data per capita by state and county ( see: https://covid.cdc.gov/covid-data-tracker/#datatracker-home ). What would a survey tell you that hospitalizations per capita would not? Weekly state profile reports are here: https://healthdata.gov/browse?tags=covid-19-spr
    What information is lacking for Oster to answer her question? The only advantage to survey data is it is never used as raw data but instead must be normalized with a lot of assumptions that are more or less arbitrary and designed to support a desired outcome ( see adjustments to NASA climate reporting and the US census). Oster’s article might as well be entitled “Desperately seeking data that can be diddled to confirm my biases.”

  9. I especially enjoy the spectacle of economists teaming up with epidemiologists to make forecasts.

    Can errors cancel out?

  10. What is missing is good language to describe risk, and discussion about what various data means.
    Oster seems to have missed data from:
    Data Provided by
    White House COVID-19 Team,
    Joint Coordination Cell, Data Strategy and Execution Workgroup
    Dataset Owner -HHS Office of the Chief Data Officer

    (HT anonymous 3:56)
    Like for Florida. https://healthdata.gov/Community/COVID-19-State-Profile-Report-Florida/ht94-9tjc
    With many nice Florida charts from the Pop-Out.
    Including page 7, with 4 charts. 3 are mostly red (Danger! Danger! Run, Will Robinson, Run!) 1) New Cases, 2) Test positives, 3) Hospital admissions.
    But then, one chart is mostly white, “no data” 4) Deaths.

    Out of 20,000,000 people, how many have to die for there to be some “crisis”. This is NOT a “data” question, this is a values question.
    If “one death is too many”, then there are lots of crises. Especially cars, and gang murders (in Chicago; not sure about Florida).
    If it takes 1% per year, 200,000 out of 20m, Florida is not in crisis.
    And it is deaths, more than hospitalizations, that is the key fear.
    Tho hospitalizations are far far more important than cases.

    What about 200/day? Deaths. 2k/day hospitalizations?
    20/day?

    ~p15 there is the whole USA deaths, with no data for Florida but the following category divisions for Deaths per 100k:
    0
    0.1-0.9
    1.0-1.9
    2.0-4.9
    5.0-9.9
    10.0-14.9
    15.0 or more
    No data

    Why these divisions? And isn’t weekly better than daily? Is monthly too little granularity? What decisions are made by who with this data?

    There’s not been enough discussion about the meta-values and what different data “mean”.

    Emily points out, as could Arnold, that many questions were known months ago that good data could answer – but the data provided is not good for answering those questions.

    Covid deaths from those vaccinated versus deaths from those not vaccinated is one piece of data that I didn’t see tracked, that should be tracked.
    Weekly totals by 3 gross age groups would also be useful:
    over 65 (70? 60?)
    younger than 18 (in K-12).
    Other adults.

    At some point the “risk” will be “low enough” to end mask mandates, and maybe end preferential treatment for those with (experimental) vaccines.
    Lots to talk about over merely “not enough data”.

Comments are closed.