But it’s Los Angeles that’s shown the biggest progress across the board. White kids have improved by 21 points in math and 10 in reading. Black kids have improved by 20 points in math and 16 in reading. That’s the best progress among black kids among all five cities, and close to the best progress for white kids too.
I’m showing you progress from 2003 to 2017.
Thanks to a commenter for the pointer. Drum takes it as given that the improvement in test scores reflects some improvement in quality of teaching. But this runs counter to the Null Hypothesis, which is that educational interventions do not matter.
Applying the null hypothesis, I would bet that the difference in test scores between 2003 and 2017 is due to a quirk of some sort. Were “white” and “black” always defined the same way? Did the rules of who was “eligible” to take the tests change? Etc.
Sorry to be so cynical, but I would want to see more data before I would treat the improvement as real.
More likely, as Los Angeles has gentrified and home prices have soared, low test scorers have moved out of town. LA has had the largest gentrification in the U.S. since 2000 according to various surveys, e.g. https://www.cnbc.com/2018/03/06/la-and-the-other-top-10-most-gentrified-zip-codes-in-the-us.html.
This is likely correct. A rule of thumb in educational research is that if there’s any possibility at all that an effect was caused by selection bias, the effect was almost certainly caused by selection bias.
https://fredrikdeboer.com/2017/03/29/why-selection-bias-is-the-most-powerful-force-in-education/
Fredrik deBoer tiptoes up to the edge of political correctness. He tells us that we shouldn’t use the fact that, for example, the students at suburban schools learn more than the students in city schools to say that suburban schools are better than city schools. Rather, “Students from lower socioeconomic backgrounds have consistently lower performance on a broad variety of metrics…” The suburban schools look better because their students are inevitably going to do better.
But then he pulls back. The reason city kids do worse is “the burdens of racial discrimination and poverty.”
Yeah. In deBoer’s next windmill joust, he has contracted to write a book where he will attempt to argue that intelligence is substantially genetic, with a twist. He is not allowed to imply that now-discredited scientists who thought intelligence was substantially genetic were anything less than completely wrong.
https://fredrikdeboer.com/2018/03/30/my-book-is-in-fact-an-anti-race-science-book/
Yes. I would not be surprised to see changes in test scores in Compton, Inglewood, etc negatively correlate with those of LAUSD over the same time period.
I’ll add something else. It’s a well know effect that as white DNA % goes up amongst those that identify as black, test scores go up. Obama is probably a famous example. He identifies as black but is the child of a white woman and a high status high IQ African black. When you look at high end black achievement from American blacks it’s loaded with people who have light skin tone. Average “blacks” in America are actually 20% white.
I think as we get more mixed race kids that identify as brown but have lots of white DNA various “gaps” may appear to close but mostly due to identification error.
Also there will be a huge market for studies that confirm gap closure and this is just one of the things that will be ignored to produce the desired results.
It’s a well know effect that…
It’s not well known to me, could you cite some source(s)?
I’m a math teacher and I basically agree with your “null hypothesis” with one caveat.
So I don’t seem to have much effect on whether my students learn math concepts. I show them a new idea: Some instantly understand, some never get it no matter how much work we do.
However, the amount of topics I cover with them must matter, right? So for instance there are some teachers who simply don’t cover that many topics throughout the year. Even if the teaching itself doesn’t much matter, I hope that the fact that I cover all of the material does.
Or is your null hypothesis more about “learning to think” or “improving at math generally” on which count I would agree.
I also think that teachers who create fear in their students are actually pretty effective (I am absolutely not one of these teachers, for better or worse). I notice that if the kids are scared of failing they tend to sort of “turn on” their brains more.
Anyway, I love your blog– this is my first time posting I believe. Thank you for all you do.
I’ve only done a little teaching, but my experience matches yours. I’d add that most students are going to get a number of teachers of varying quality; this will tend to average out differences. In general, it seems that the range of difference in ability* from student to student is larger than the range of difference in effectiveness from school to school (and much of the interschool difference is probably more to do with how disruptive the kids are than whether the teachers are good).
*”Ability” here means the ability to succeed in school.
One might distinguish between a ‘sharp null hypothesis’, which is that each and every education intervention has no effect, and the null hypothesis that education interventions have an average treatment effect of zero, but with some noise. We should probably expect outliers in both directions due purely to chance.
Myriad demographic changes in the LA school district over the 2003-2017 period examined in detail here: https://gse.berkeley.edu/sites/default/files/users/bruce-fuller/Berkeley-UW-LAUSD-June2016.pdf would appear to confirm your null hypothesis. In particular increase in the educational attainment level of Hispanic mothers and outmigration of Blacks. Nevertheless, pages 14-16 examine the increase in students attending charter schools over the same period. I am not willing to rule out the influence of charter schools. And even if the null prevails at the macro level, no amount of metrics are going to account for the benefits realized by families that have some autonomy in making educational decisions. Overall NAEP scores may not have changed but yet public schools have succeeded in producing countless school shooters. Its time to pull the plug on traditional public schools.
Mandated testing began in about 2002 under the NCLB. Frum has no before and after comparison.
What has improved is the ability of the students to sit quietly and take a test to get more federal money. We should test the utility of keeping quiet for an hour and taking a test, maybe good maybe bad,who knows.
I was at a PTA meeting in Arlington and it really surprised to hear the principal explain how some kids’ scores were excluded from reporting. I got the basic impression that juking the stats was official policy.
As (mostly) resident of SoCal the last 38 years, I do find the change in crime and education amazing because nobody in 1992 during Rodney King riots was predicting either would happen. Seriously, there was feeling LA crapped in 1992 – 1994 after riots and earthquakes. While I do agree with some of the gentrification arguments there are some holes in it. (In terms I sold on any one theory of lower crime although I mostly moved by later family formation/acceptance of birth control/abortion that started in the 1970s.)
1) In terms of crime, the big drop happened from 1990 – 1996 before gentrification really started making an impact. (Check out SoCal home prices were down during that period.) I wish people note this when claiming gentrification solved crime and education would deal with this. Also the other issue of gentrification forgets that the heavy crime African-American areas, Compton & Inglewood, had mostly Hispanic-Americans and some Asian-Americans, not white citizens, move in. (Early 1990s was our first wave of ‘white flight’ if you know our history.)
2) I wish people supporting gentrification would argue that it changes the current local behavior as well. Compton and Oakland homeowners in 1997 were suddenly sitting valuable land and that changed behavior of the young people. I live in area heavy with working class Hispanic-Americans and I have seen changes here the last 12 years. There is a virtuous circle working here.
3) I tend to think when the crime wave was finally apparent in late 1990s, the schools improved a lot just because they were dealing with less crime. Again a virtuous circle issue.
4) I don’t think the categories changed here and I am not sure why that is an issue. I accept this does not disprove the null hypothesis but it is apparent something changed in LA county driving up these scores. (It is probably 10 things creating a circular function.)
In terms of changing behavior, the biggest reason why the US birth took another step downward the last 9 years is because of urban Hispanic- & African-Americans are having significant less children. Our 2017 birth was the lowest in the last 60 years except for the huge drop in the mid-1970s.