In the course of a year, urban charter schools increase student achievement 4X as much as increasing public funding by 10%.
The effect sizes are small, but statistically significant. How strongly this contradicts the null hypothesis I leave up to you to decide.
I see some dimensional analysis issues.
It is certainly close and it is probably a wise course for Blue State urban cities for 7 – 12 grades. I still think there is a huge self-selection bias here although I suspect some of it is being ‘special’ going to charter school gets better student studies and behavior.
Likely more selection bias than anything, tending to reinforce the null hypothesis.
Just as Bryan Caplan’s hypothesis that one’s genes determine what he is like as an adult has a discontinuity at the low end — namely, unless he was raised by wolves — so too the null hyptothesis likely has a discontinuity at the low end — namely, unless one was educated alongside wolves.
The null hypothesis — and the fact that pretty much any clever model of schooling seems to arrive at the same results — means that kids will get out of school what kids can get out of school, so long as the environment is not disruptive or violent. Charter schools almost certainly clip that tail off the distribution.
You do realize nobody is actually raised by wolves, right? And figuratively speaking, we all are. Now pretend everyone decided parenting didn’t matter. Suddenly we’d be raised by wolves who also didn’t even care about trying.
We simply don’t know enough to be able to test outcomes properly, let alone understand the inputs. But we should not assume that causal density implies that nothing matters.
Arnold, it would be helpful to be reminded, even many times, how you like to phrase the “null hypothesis”. Most especially here, but also in your prior 12 points.
0 Hyp: “No politically feasible increase in edu spending will significantly increase test scores of students” (your phrase would be??)
I think a small increase in test scores at the same or lower cost is a BIG win, because of happier parents, plus the potential future benefits of learning what works, best, for which students.
It’s just THE null hypothesis. Measured outcome differences are the result of random statistical noise.
Arnold’s version is just that the null hypothesis will always remain unfalsified with enough data points.
The “Null Hypothesis” hypothesis helps concentrate the mind. But is it always true?
I wonder about this. Probably there are interventions that work, aren’t there?
It’s just that “politically palatable treatments that we are willing to pay for, with inputs that we can buy, in an incentive structure that more or less stays the same, in which most students cannot be harshly disciplined or expelled, and everyone goes home at the end of the day to their home and neighborhood environment” is subject to the Null Hypothesis.
I suspect that if boot camp didn’t accomplish anything, soldiers-to-be wouldn’t be sent to boot camp. And sometimes they are draftees. Or you have universal conscription.
Maybe the difference is between “education” and “training.”
Eric Greitens wrote in resilience that “education changes what you know–training changes what you can do.” I paraphrase freely.
The null hypothesis is the baseline in inferential statistics. It’s possible the null hypothesis is more likely than any alternative, given a certain “dissection” or view of a problem.
When the null hypothesis dominates, this is a signal to reevaluate the meta and make sure you are taking a good approach or asking a good question.
The comparison in the article is somewhat sketchy. It compares mild spending increases with charter schools in New Orleans – a famously broken school district that was basically closed and restarted with charter schools after Katrina.
This is really the best case for charter schools – clearly broken public schools and a requirement to reform the whole system, so charter schools can’t attract parents by just skimming the cream. I’m a fan of charter schools, but this comparison shows just how radical the required changes will be to see their effects work.
This story is less at odds with the null hypothesis – seen as the inability to get _reproducible_ results – than it seems at first.
New Orleans schools were charterized in the wake of Katrina. They also received a very large injection of federal cash. They also experienced a very very large exodus of students as the population of the city dropped 50% (eventually recovering to 80% of previous) or so, probably preferentially losing lower income families.
Singling out any one of the numerous major changes that New Orleans went through as ‘the reason’ for school improvement is just lying with some subtlety.
To overcome the Null Hypothesis, we must have a plan that can possibly work.
First, stop using IQ tests. Next use technology to free up resources and reduce costs. Finally, figure out how to apply the resources to human capital value added.
California school spending has increased a lot in the past three years. Sacramento must quietly concur with the null hypothesis since no metric of student performance was required pre- or post the new money. If education leaders thought there was anything scalable they certainly would have required it: this is California where everything that can be centrally controlled, is.