Null hypothesis watch

Kevin Mahnken reports,

Thirteen-year-olds saw unprecedented declines in both reading and math between 2012 and 2020, according to scores released this morning from the National Assessment of Educational Progress (NAEP). Consistent with several years of previous data, the results point to a clear and widening cleavage between America’s highest- and lowest-performing students and raise urgent questions about how to reverse prolonged academic stagnation.

…NCES Commissioner Peggy Carr told reporters that 13-year-olds had never before seen declines on the assessment, and the results were so startling that she had her staff double-check the results.

…when average scores for most students were stagnant, scores for the lowest-performing students were down; when scores for most students were down, scores for the lowest-performing plummeted.

Read the whole thing. Pointer from Tyler Cowen, who points out that the results are from before the virus closed schools.

The fact that the United States has much higher health care spending than other countries but no higher life expectancy is frequently talked about in left-wing circles. But the fact that the more we spend on K-12 education the less we get in terms of better test scores is never mentioned. Conventional wisdom is that we need to spend less on (private-sector providers of) health care and more on (government-run) schools. Even the Niskanen Center paper on “cost-disease socialism,” while it has an entire 5-page section decrying the bloated expense of higher education, only mentions K-12 education in a couple of relatively innocuous paragraphs.

Perhaps the strongest indictment of K-12 education is the movement to get rid of SAT scores as a requirement for college applications. Would this idea have gotten anywhere if test scores for minorities were improving rather than getting worse?

The Null Hypothesis says that we cannot get better results by increasing spending. But it also says that we could spend less and get the same results.

38 thoughts on “Null hypothesis watch

  1. As a high school social studies teacher during this period I would have predicted this trend based on anecdotal experience. I saw two things operating in concert to explain lower student performance. I saw a consistent decline in the quality of home life for our poorest students, drugs (biggest issue), transience, single parent families, parental incarceration all seemed to get worst as the 2010’s progressed. These conditions were further exasperated by my school district’s watering down of rigor and education standards in favor of “mass customized learning”, this oxymoronic policy claimed that you could give a 13 year old a laptop, some learning standards and then they could pursue what they wanted to learn on their own learning timeline. This utopian fantasy of how education could work never harmed our highest achieving students because they had the means, intellect, and parental supports to navigate most any education “system”, but boy did it damage our lowest achievers, and it gave license to our worst teachers to double down on their careless and lazy practices. In fact when our districts test scores inevitably went down, our districts response was that we aren’t worried about those metrics, we think the most important measure of student success is how much “hope” students feel. To measure this “hope” the same administrators that oversaw the lowering of academic standards and performance sent out a crude survey to students, and declared that nearly two thirds of our students feel “hopeful” and this was cause for celebration. Of course these same administrators never surveyed another school district with real learning standards to see if those students were more or less hopeful. So declining conditions for our least advantaged students combined with low expectation curriculum makes me think we are lucky our declines weren’t worse.

    • Thank you. This is really good. Loved the sentences on the “utopian fantasy.”

      Hope that the “education realist” decides to chime in as well.

      • I hope the education realist would say we had a lot of good stuff 40 years ago. Home economics, early access to training for the trades, and tracked classes for students who want to go to college. Sadly things have become so polarized that even suggesting tracked classes is the end of the conversation. Instead we were told to individualize for struggling students and students who are excelling. So in a 60 minute period I had to teach economics to a group of kids with a 5th grade reading level and arithmetic skills and a group of kids who were actually ready to learn high school economics. The results, both groups got less from me, and my curriculum got watered down. Ohh they also said I should reach personal finance too because we don’t have home economics. The solutions are there, and I think the pendulum will swing back to some of our tried and true methods, in the meantime we are failing students, and the test scores are but one measure of our failures.

  2. Does anybody know any high-IQ couples with a lot of kids?

    Who made having a large family a goal? Who made family life central to life, rather than careers?

    Then…duh.

      • Here’s an example of the president of the school we want to send our kid too.

        “X holds a degree in Construction engineering from Midwest State U. He owns his own construction company. He is involved in the local parish and has four boys. He enjoys wood working, hunting, and beekeeping.”

        They all read a lot like that. 3+ kids (one has twelve kids).

        These people exist, but they all live in the exurbs and go to church.

        • Some Mormons too.

          Although the definition of “large family” has been downsized.

          In the 1950s-60s Los Angeles, any typical family could have four kids, including mine.

          The Catholic families routinely had more.

      • Correct. However, don’t neglect the high IQ South Asian Americans. The Indian Americans like to reproduce too and their politics are generally of the *extreme* woke variety. That probably eliminates any conservative political upside.

        • How would you rate the based to cringe levels of various Asian subgroups?

          I’d say East Asians seem to me the most based, with Chinese the most extreme. Indians seem the most cringe, maybe more than white libs. At least the women.

          People ask why Virginia Asians will seem to be going for Terri, but the answer is that they are Indians and not Koreans. Answer may be different in the voting booth vs survey though.

          • In terms of wokeness, easily the South Asians are the most cringe worthy. It isn’t even close at least not over here in the Dallas Fort Worth (DFW) area.

            Now to me, the most interesting question is why does this happen to be the case? The best that I can come up with is this:

            1) genetic ideological taste buds towards wokeness a la “The Righteous Mind” and inbreeding between the upper castes that prevents ideological diversity.

            2) the extreme elitism of the Brahmins, etc. They seem to be accustomed to being able to boss everyone around and aren’t used to similar IQ folks with differing opinions that aren’t as easily swayed by bad arguments. It appears to drive them completely bonkers and results with them calling for speech restrictions, censorship, etc.

            “Check your Brahmin privilege” would be an interesting meme.

            Any thoughts? What have I gotten wrong?

          • I think that when your society is uniformly high IQ and homogenous it selects for things that it doesn’t in societies with huge disparities in background. Homogenous high IQ societies tend towards high trust solutions with uniform rules.

            But if you’re some tiny elite swimming in a sea of low functioning people you barely consider human and have been genetically separate from for like thousands of years, you’re not going to select for high trust solutions based on uniform rules. That’s probably rational, given the circumstances.

            The Indians caste system is probably the most rigid in all of human history. It’s unique how separate they managed to keep groups living right next to each other, even Razib seems fascinated by it.

            Also, keep in mind that in India something like affirmative action is just an expected part of life. They are used to it, and it’s the price the tiny elite pays to keep their heads. Meritocracy requires relatively broadly distributed merit to have enough support from society, places like China have that but India doesn’t.

            When you have Chinese trying to share a society with this mindset it gets ugly, Amy Chua wrote a whole book about it and I saw it up close between the Chinese Malays and Malaysian Malays at my college.

        • Are they though? During Trumps time all I saw was pro-Trump from them. Some friends in Texas are all pro guns too. Maybe it differs by state?

  3. To eliminate or narrow the decline, here are some possible solutions that I’ve heard being discussed:

    1) eliminate AP courses and gifted programs
    2) mandatory applied CRT training for all students
    3) SEL programs
    4) prosecute concerned parents as domestic terrorists to end open dialogue

    Did I miss anything?

    • “Re-norm” the tests. Questions that tend to be missed the most often and which are especially missed in racially disproportionate ways are presumptively ‘problematic’ and should be temporarily removed pending 25 years of study. Keep eliminating questions in descending order of difficulty until the scores come out politically acceptable again.

      • The problem is that the less ‘g’ loaded you make the test the more of an advantage that is to Asian study grinds. Gap Widens.

      • It’s difficult to imagine a test that someone with IQ 75 could pass with ease. It certainly wouldn’t involve reading or writing. Maybe sitting in a certain zone for a period of time?

        • That’s true, but my guess is that they find ways – maybe via some kind of official diagnosis – to exclude sub-75’s from the tests or, if tested, from the stats for ‘normal’ kids.

          At just above that level, well, seems to me that over the last generation or two there’s a been a decentralized Manhattan Project of a sort to discover tests, classes, and processes that those kids *can* pass, or (what is quite different) for which they can be just barely officially declared to have passed and thus granted some kind of credential and socially promoted to be made someone else’s problem, all with the minimum number of egregious public scandals about kids getting through despite illiteracy and being constantly AWOL.

          Bottom line is, I just don’t see how any institution these days can withstand the corrupting pressure to do whatever it takes to make the numbers come out politically correct, by hook or by crook. Or by coming up with some bogus rationalization to just giving up on the testing altogether. The easiest way to fix embarrassing results is to have no more results.

          • The solution is so easy it is already being implemented- just give them a handicap rating for any test.

  4. I had a talk with an aunt recently that drove home the “you are conservative about what you know best and liberal about what you don’t know”.

    She lives in NYC and is a lifelong Democrat and Trump hater. But she despises De Blasio and defund the police. And she doesn’t like woke-ism or socialism (she still calls everyone she doesn’t like racist, but she doesn’t like other people calling her racist).

    So I mentioned I was a bit sad that one of our good friends with kids our kids age have decided to move to Texas. The nail in the coffin for them was the way our blue state treated COVID, especially in the schools. Their red county had masks forced on them against their will, and of course there is the Critical Race Theory and Transgender nonsense they don’t like.

    I didn’t bring up any of that though, just mentioned Texas, and she was immediately nonplussed. “They have terrible schools down there.” I’m not sure what piece of evidence she had to make this claim, but she made it. I think explaining the Null Hypothesis to her would have flown over her head.

    The Northeast is a case study in our of control school spending. Between when I was a kid and today budgets and property taxes exploded. Many don’t realize that property taxes are double in the northeast compared to the rest of the country (that’s just the rate, not even counting the higher assessed values) mostly to pay for school budgets that are double or triple the rest of the country. For awhile SALT disguised some of this, but now SALT is gone.

    I expect the goal of liberal America is to remake the rest of the country like the northeast. With everyone paying 2%+ property taxes and private schooling options being unable to operate cheaply because the public schools offer such lavish compensation its hard to compete.

    • Please tell your aunt that she is absolutely correct about Texas being awful. Also, remind her to please pass that message on to all of her blue friends throughout NY and beyond.

      Kind regards,

      Texas

      • I’m from Kansas and so many people complain about the weather.

        My standard response: “Small price to pay to keep the refugees from NY and CA away.”

        Keep flying over fly over country my dudes. Don’t stop.

  5. Instead of talking about how spending doesn’t work, can we start talking about what does work?

    Focus entirely on programs that work. Then figure out how much they cost. It is possible we spend too much on what does not work, but too little on what does work. Arguing about the size of the budget is a distraction. Funds aren’t an input to education, they are a means of financing inputs.

    Finally, let’s investigate the track record of Catholic schools in the Northeast in the 19th century. The Irish and Italians came over here poor, unschooled, from rural areas to big cities, and alienated from the broad culture of their new country. Their neighborhoods were filthy, crime-ridden, and diseased, and family life could be troubled. But the harsh discipline of the Catholic schools turned them into productive citizens who joined the mainstream in less than a century. And those schools had small budgets.

    • It rather looks as though what has worked can’t be bought and doesn’t scale well. Also, what worked in the 19th century probably wouldn’t work now–the society is too different.

    • You’re never going to get people to agree on what is actually working, especially in this day and age.

      Standardized testing would be ideal, but as other commenters have pointed out, there are active efforts to corrupt them or discontinue their use altogether.

      Two things are needed in schools right now: Discipline and accountability.

      Kids need to understand that there are consequences for disruptive behavior. They need to be punished when they engage in it. When the best you can do to discipline unruly children is to suspend them for a week, you have effectively zero ability to make kids behave. The kids view that as a reward, not a punishment.

      As for accountability, ineffective teachers need to be firable. Why teacher’s unions are a thing I will never understand. They exist only to perpetuate and advance mediocrity in all its forms in the public schools. You can’t even reward effective teachers because of pay scales and other asinine union rules designed to protect the mediocre.

      Electing the right board members is nice, but the best they can do is fire the superintendent, and replace him with a similarly sleazy political hack who is good at lying during the interview with the board. How many PHD’s in Educational Administration with 15 years experience in the public schools hold even centrist political views? Not many, I’d imagine.

      As much as I wish it weren’t so, we’d be better off burning the system down and starting over. All of it is a wretched hive of scum and villainy, to paraphrase Obi Wan.

  6. I have two kids in elementary school in the dmv and the math instruction is appalling. A concept is introduced but there is zero repetition, no homework. On the rare occasions there is homework there are maybe 5 problems. Worksheets have been replaced by apps on their school provided device which are worthless and often have incomprehensible instructions written by the lowest bidder at some coding sweatshop abroad. Most of their math work involves drawing a bunch of boxes and lines (not geometry but on how to figure out basic arithmetic). My kids don’t even have the basics of carrying and borrowing for addition and subtraction down. Forget multiplication, division, etc. I am told that this is new math and it is pedagogically sound. If the goal is to create an innumerate generation then they nailed it. Anyhow, much to my kids’ disdain, Daddy math is filling the void.

  7. Although it’s a commonplace these days to note that more spending doesn’t necessarily get you better results, the possibility that less spending could get you better results is rarely considered. Yet, when we compare spending levels around the world, we see that lower spending countries frequently outperform the big spenders. Looking at this data: https://nces.ed.gov/programs/coe/indicator/cmd
    we see for example that Finland, frequently considered to have the best education outcomes, only spends at close to the OECD average.

    But even more interesting, Ireland, reduced per student spending over recent years and still outperform ed the US by a wide margin (admittedly not a high bar) but also achieved much more of so-called “equity” that the petty despots and tyrants administering US school districts so ardently claim they desire. Ireland:

    “Ireland ranks 4th out of 36 OECD countries and 3rd out of 27 EU countries for reading literacy
    Ireland ranks 8th out of 77 countries/regions involved in PISA 2018 for reading literacy[1]
    in reading, Ireland has significantly fewer low-performing students (11.8% below level 2) and significantly more high performers (12.1% at levels 5 and 6) than the OECD average
    PISA results show the difference in performance between schools in Ireland is lower than the OECD
    in Ireland, the difference between schools in student performance in reading literacy is less than half of what it is, on average, across OECD countries[2]
    post-primary schools in Ireland can therefore be considered relatively equitable, as well as having above average performance in the three assessment domains.”

    https://www.gov.ie/en/press-release/f6e114-major-international-study-finds-irelands-students-among-top-performe/

    So there is a “spend less, do better” example.

    The standard objection to such international comparisons is that US teachers are “underpaid” and could earn much more if they selfishly decided to take their talents elsewhere but instead nobly and selflessly devote themselves to the children’s welfare.

    That might even be true but it doesn’t necessarily follow that even more spending is the answer. You could spend much less on school district administration, even eliminate the districts and allow schools to operate independently, and use some of the savings to throw more money at teachers. Eliminating Federal interference in local school administration would also be a net winner. Finally, the money saved by eliminating the US Department of Education could be used to finance the recently enacted payoffs to parents program but that program could be modified to serve as a “fund the child, not the school” education program. Cowen linked to something about the UK academy system which wouldn’t be a bad model.

    At any rate tolerating the status quo is not only immoral but radically nihilist. A truly conservative approach to the disgusting state of the US education system is radical reform.

  8. The motivation behind the push for removing SAT/ACT/GRE from standardized testing is to change the racial composition of the students. Some racial groups are intended to benefit at the expense of others.

    A reasonable view is regular people should have more autonomy and legal rights to pursue the education + career track of their choice and the administration staff at universities should have less authority in dictating who does and who does not have the right to pursue particular career pathways.

    Kling has repeatedly advocated for a random lottery admission system for college, which does not use SAT/ACT scores. That is equivalent to the more dramatic phrasing that it abolishes SAT/ACT based admissions processes. Kling’s logic is that college courses should be rigorous and challenging but merely registering for classes should be less of a barrier.

    • I don’t think we need a pure lottery. You could limit a lottery to students who meet minimum standards for test scores and high school grades. The point is to get rid of the admissions “process.” But I do think that *if* you could have rigorous grading standards for college courses then you could put the burden on potential students to decide whether or not they are qualified.

      • What do you think of community colleges with rigorous calculus + chemistry + physics courses? That is what you describe: rigorous grading + content but open admission.

  9. The Coleman Report in 1966 said we’d get little result from more spending. And that was reaffirmed by a 2016 50-yr anniversary symposium on the Report. Family background and neighborhood has the most impact on school success. But there’s no profit in that. And as is the rage now, “parents shouldn’t be involved in their child’s education” except as passives and patsies for the “educators.”

    “Altogether, expenditures and facilities have much smaller associations with secondary and postsecondary outcomes than many scholars and policy advocates assume. The overall conclusion of the Coleman Report—that family background is far and away the most important determinant of educational achievement and attainment—is as convincing today as it was fifty years ago.”
    https://conversableeconomist.blogspot.com/2016/10/the-coleman-report-n-equal-educational.html

  10. I think all the evidence indicates we, as a species, are just getting dumber generation after generation.

  11. Every pathology Charlie described has been true for decades, yet scores increased through most of the period or at least maintained. We never had a halcyon era of vocational training, and no one can seriously think that Home Ec and Shop were the secret to a purposeful young adulthood, can they?

    White kids scores stayed the same, so it’s kind of hard to take “r’s” complaints seriously. We’ve had ineffective teachers for decades and in fact teachers are *more* qualified now than they were in the 70s and 80s when scores saw their biggest increase. I don’t know when people will accept that Catholic schools cherrypick and their success stories aren’t evidence. My god, it’s been 30 years. Get a different story.

    While I agree that discipline in out of control schools would make life better for low income kids would be a good thing, there’s no way that would increase NAEP scores, since we won’t be allowed to kick those kids out, and they’ll be learning less since they aren’t in school. If you’re holding out a lot of hope for the well-behaved kids to learn a lot more in better disciplined classrooms, don’t. It will make their lives better, even if they don’t get smarter, and that’s enough.

    Also true that some of the score drop is due to dysgenic causes, but that too was true in the past, so that means that in the past all those terrible schools you all despise so much–and the recent past, ten, twenty, thirty years, ,were doing really well in teaching the kids that you all are now thinking of as the stupid ones.

    But in fact, as Tom Loveless notes, the most likely explanation, particularly in math, is the change about ten years ago towards more abstraction in math, more non-fiction, and more writing than reading. While the Common Core pushed more abstraction early, Loveless established that these changes happened everywhere (including outside US) with or without Common Core.

    The thinking went like this: we teach too much basic math in elementary schools, it’s too easy (far more elementary kids got advanced or proficient than do high school kids). So then they have to absorb it all in eighth grade, and for some it’s too late. So we’ll push abstraction and complex thinking down earlier in the curriculum, that way algebra and more complex math won’t be a complete shock.

    It goes without saying this didn’t work. What isn’t proven yet, but that Loveless (and I) think is likely is that the push of abstraction younger not only didn’t help, but actively hurt low achieving kids, while actually helping bright kids (whose NAEP scores improved slightly). I wrote about this on my blog, but Arnold’s blog treats comments with links as spam.

    On the topic of money: once again, when will people learn? We don’t spend increasing funds on the “normal” kid. Teachers’ salaries haven’t dramatically increased. One major reason for increased teachers is increased charters. For some reason, conservatives forget that giving something away for free–like private school–creates more of it. So sure, charters are popular, particularly popular with parents who pay for mid-tier privates and realize they can get the same population selection for free. Charters decimated the private school market, moving a lot of private school students onto the government dime. More teachers pensions, more government costs, but oh, by golly, those parents can “choose” to get education on the government dime and you all will celebrate it while simultaneously bitching about the cost of education.

    But the real increases are in immigration–another thing Arnold celebrates, if at least most of the commenters don’t–where the kids are dumped on the school system for increased services, and ELL–all to lower test scores, and then in special ed.

    Meanwhile, instead of trying to reduce special ed costs or trying to fchange the law so states don’t have to educate immigrants they had no choice in accepting, conservatives are busy demanding that we keep increasing the number of private schools for free government dollars.

    By the way, the left doesn’t think “kids aren’t catching up, so it’s time to eliminate tests”. The left has always hated tests. They have been trying to do away with them for at least 20 years.

    This all came about because for a few months last June, the center and right were both stupid enough to buy into the George Floyd hype. That’s all they needed.

    • –“Meanwhile, instead of trying to reduce special ed costs or trying to fchange the law so states don’t have to educate immigrants they had no choice in accepting, conservatives are busy demanding that we keep increasing the number of private schools for free government dollars.”–

      Interesting ideas.

      I agree that charter schools per se aren’t the answer. Yet at the same time, getting rid of public education spending is probably impossible. I’d rather let parents have the option to have their kid’s per capita share of spending put in their bank account for home schooling purposes. You’re married with three kids? You can ship your kids to the local school system, to a private school, or just take the $40,000 of funding and you get to raise the kids on your own. Of course, so that this doesn’t turn into a single mom subsidy, single parents will only have the option to use their funding as a voucher, though that the voucher can be used at any school regardless of accreditation.

Comments are closed.