surely students who took statistics with you know statistics better than students who didn’t take statistics at all, right? The null hypothesis taken to the extreme would suggest that having taken your- or any- statistics course, should provide no benefit in understanding statistics at all.
If we define the intervention as “took a statistics course” vs. “did not take a statistics course,” then I believe that the intervention worked for some students. However, if we define it as “took a statistics course with Dr. Kling” vs. studied the material some other way, then I would not bet against the null hypothesis.
The same commenter writes,
Arnold, are you of the belief that educational outcomes would be the same if we got rid of schools altogether? That almost seems to be what you’re implying here.
Again, this raises the question of what is the relevant experiment. (I have often not been clear on this.) For example, if you want to compare home schooling with standard schooling, I would not be willing to bet against the null hypothesis. But if you compare standard schooling with no schooling, you are talking about something else entirely.
Learning from other humans is an essential trait of human nature. To hold dogmatically to the null hypothesis, one would have to suggest that the amount and type of learning that children undertake is a function only of their individual characteristics and of the culture in which they are embedded. It does not depend on the way that the institutions of schooling are structured.
The institutional structure does affect resource allocation with respect to teaching and learning. I would speculate that our school system probably makes more efficient use of resources than would a system in which schools did not exist. I would speculate that it makes less efficient use of resources than would a system of vouchers and competition rather than government-managed schools.
Would I go far as to say that the only difference that schools make is in resource allocation, not in outcomes? I doubt that such an extreme position is warranted. But statistically, educational interventions tend to affect resource allocation much more than outcomes. For educational interventions within roughly the current institutional setting, the null hypothesis is not an iron law, but it is an empirical regularity.
What stands out to me is just how little variation we see between schooling options. Public schools are all run on the same basic plan. Catholic schools are too, but with stricter discipline. Private schools aren’t much different, but with a wealthier clientele.
Only a few niche alternatives, such as Montessori and Waldorf, offer something truly different, and they obviously attract unusual families.
In terms of missed academic opportunity, smart kids don’t get education that helps them reach their potential, average kids don’t start the technical training that would be more useful then high school oriented towards college, and poor kids have to deal with a lot of discipline problems that should simply be expelled.
Beyond that it’s mostly missallocation of resources. Not just teacher salaries, but a lot of what we call real estate is just public school tuition.
The biggest issue with public schools though isn’t academics, but ideology. They certainly are learning a worldview from the system.
I should note the same ideology would be taught at most private schools.
average kids don’t start the technical training that would be more useful then high school oriented towards college,
Why are training people for jobs that have lower wages than 40 years ago?
Also why train people for jobs that will be PSST in 15 years?
Why don’t companies, like in Germany, participate in hiring people with the right technical training?
Vocational and technical is right now a conservative hobby horse unless companies are actually going to hire these workers and help them build careers.
People in trades are making good money relative to both the unskilled and people getting lib arts BAs from low tier schools (or in about half of cases, dropping out halfway).
You can’t make the mediocre iq into engineers or computer programmers.
One of the tragedies of public, compulsory education, besides that it is and always will be a political prize, is that being government, i.e., “official”, many come to believe that what is taught in the schools is all that one needs to know to get along in life. There is some randomness at the margins, but on the whole, not only the topics, but also the often misleading, simplified content is thought to be all that is needed.
That is because people really believe the null hypothesis, but will never admit it.
Without a doubt you can tell which people did and did not take college stats. People who did not take stats are not as careful on understanding results. The basic items I remember were:
1) If you don’t see the difference in the averages, there probably is not any correlation/causation.
2) Correlation does not mean causation but it could mean something. (The more I review stats the more I understand many correlation/causation end up being circular references.)
3) If you can not reject the null hypothesis, that still is a valid result. (I literally had this conversation with a large customer four months ago.)
4) Beware of other variables.
Take for the Bojas analysis on the Cuban refugees on Miama non-HS graduate wages study a few years heralding how much immigrants hurt citizen wages.
1) The study did not include any group other than male non-HS graduate wages so the drop in male non-HS wages seemed to be limited in scope. Did women wages or other HS/college graduate go up? Also the drop in wages of $.25 to $.50/hour was fairly limited here so I did not see a rejection of null hypothesis.
2) The worst part of the Bojas study was over the US non-HS graduate wages were following in the early 1980s so the Miami results were more dictated by historical labor markets than the Cuban refugees.
3) In my view the study did not prove the drop IMO but it is a point to analyze if the deportation of law abiding long term Illegal Immigrants is going to increase the Rust WWC wages. I don’t see this happening partially based on this study.
4) In terms of falling non-HS graduate wages, probably the biggest reason of the fall during the early 1980s or after 1974 – 1994 was both the retirement/death of Depression era workers (where non-HS graduates were more common) and economy over-expanding inflation of high job growth of Baby Boomers and women in the work force.
I had always implicitly understood your null hypothesis as an “at any plausible current margin” hypothesis. Interpretive charity!
Also, I am surprised you answered the statistics question without citing http://xkcd.com/552.
Right. And really the problem is arguing against the constant stream of proposals that will spend lots more money. The problem is not arguing against proposals to cut spending and schooling down to zero.
The problem we are dealing with is people trying to make a name for themselves, or virtue signalling, by proposing new interventions which will cost a lot more. The null hypothesis says that, probably, most of the new expenditure would be totally wasted.
Another concept worth assessing in the null hypothesis frame is “diminishing marginal returns.” Doing more of the same, or spending more money on the the same general approaches, probably won’t make any difference, because we’re probably already saturated with what “more money” can buy in education in terms of producing measurable improvements in life outcomes.
“But if you compare standard schooling with no schooling, you are talking about something else entirely.”
Something like ‘unschooling’, perhaps? My understanding is that those kids do fine in higher education:
http://www.smithsonianmag.com/smart-news/unschooled-kids-have-few-problems-once-they-hit-college-180952613/
What happens when you let kids direct their own education in an all-you-can-eat-for free information environment? The numbers are small at this point, but the answer so far seems to be that they do well.
I was basically unschooled. The first time I set foot in a classroom was my first day of college. I got degrees in math and computer science. When I actually got into a real school I was always surprised by how many things I considered common knowledge were new concepts to my classmates. However, I was always very interested in learning things, so I would do things like read the encyclopedia and I taught myself first semester calculus to prepare for college.
I don’t know if I would have more knowledge had I gone through more standard schooling, but I doubt it.
Why do unschoolers bother with college?
In most cases I would assume it’s for the credential. For most professions, if you lack the degree it doesn’t matter if you have the knowledge.
That is exactly it. I knew that getting a job would be much easier if I had the sheepskin.
In most cases, I never found lectures to be all that useful. Most learning I did was during homework.
The null hypothesis looks right if the trials are input A and just more or less of A. But my experience with the education of students on the autism spectrum is that the null hypothesis can’t be even evaluated if the trial is standard special education versus a substantially different pedagogical approach such as using Rapid Prompt Method (RPM). There are areas of education where thinking the null hypothesis is correct can shutdown valuable innovation. There may prove a level of RPM above which “more” does not get a positive response from certain students. A question I asked her months ago was would schools and taxpayers be excited or unhappy if they discovered that spending 100% on special education gave much better results? I think most would be unhappy to learn of such a development.
My null hypothesis says that spending more inflation adjusted on schooling than was spent in 1960 has no positive effect on education.
Also that only a tiny fraction (perhaps .1%) of schools are good or bad schools. The rest are judged good or bad have good and bad students. And so parents should relax should only try to avoid the worst .1% of schools and not pay more for a home in a “good” school district.
Also my mother has told me that her grand parents went to school for 1 years, yet they where educated. They were not superstitious as many in their era. They could read and write and do arithmetic. They read books. They owned and ran a barber shop. they speculated in real-estate (but lost most but not all of it in the great depression) and sent their son to Brown University. School is not only place people learn.
Perhaps 3 years of formal schooling would be enough for most people. Never the less in the modern world we would need something for them to do so maybe keeping most in school to 16 is a good idea, not for education but to keep them out of trouble. The Amish drop out early.
For a modern example, the guy who invented this car: http://www.commutercars.com/ claims to have dropped out of school at 14.
Also, if we cannot teach children more and it looks like we cannot, we should focus on teaching the most valuable stuff.
For educational interventions within roughly the current institutional setting, the null hypothesis is not an iron law, but it is an empirical regularity.
Arnold is not talking about swapping an American school for a school in Kolkata.
I am. The world is flat.
I view it…or “filter” it…as took a Statistics Course with Dr. Kling vs some panel in DC intervened to try to improve Dr. Kling’s results. Stated that way, I think even the panel in DC secretly believes it.
Here’s a way of affecting outcomes for the better. Have parents pay fees.
The first difference this makes is that the kids turn up. They attend more often when there’s a price tag attached. It’s obvious, but I’m still going to spell it out: Parents who are aware of the money they’re spending will make a point of having their kids turn up.
This really works, and we already established that it works in 1890, one year before the British government got involved for the worse.
Literacy was more than 98% in England and Wales when education was a competitive, diverse, voluntary, thriving free market in 1890. Then in 1891 education was nationalised, but in 1890 we have it on record in the Parliamentary Debates that it is “regular fee payers who are the regular attendees, and it is very often the free children who are the most irregular.”
That was a British MP speaking. And what this means is that in the face of the evidence, despite being told what would happen, the government went ahead anyway and destroyed what the British people had built for themselves.
There were Anglican schools and non-conformist schools. There were Catholic schools and Wesleyan schools. New schools opened every day. The different churches and societies built five thousand new schools in the 1870s.
Poor kids, for free, could go to the same schools as the kids whose parents paid fees. The poorest kids didn’t always show up, like that MP pointed out in 1890. But the British government chose to do the opposite of what would have been smart or effective. After 1891, all kids would be like poor kids, and the education system would be uncoupled from logic and rationality. The government would create a commons, and it would be a tragedy.
The simple, effective, logical mechanism of even a token fee was destroyed for all those parents who opted for “free” education. And “free” is magic. Everything turns to shit.
The incentive used to be, before 1891, that British parents were involved in their kids getting an education. Then the British government involved itself, pushed the parents out, and destroyed the mechanism that would have created a better outcome. And to start with it wasn’t more expensive in terms of the figures being spent. The British government was spending the same sums the British people would have been spending voluntarily, based on 1870s and 1880s trends.
But the parents weren’t engaged anymore, so that all went to seed. Because of course the second, third and fourth differences have to do with schools being run for the benefit of unaccountable government employees, where the costs are camouflaged, and everyone lives at everyone else’s expense.
Here’s a way of affecting outcomes for the better. Have parents pay fees.
And if the parents don’t? I assume the kids don’t go to school. And, gee, those turn out to be the ones whose parents didn’t pay previously.
I think you have to really be careful about the direction of causality here. To some extent, the fee paying makes the parents more diligent about monitoring their children and their children’s schools. But I think much more important is that the parents who cared (and who had kids who would do well in school) were the parents who cared enough to pay for their kid’s schooling.
Don’t make the mistake of thinking that being on the basketball team makes you tall.
“The church schools had originally offered a free education to all, but fees were introduced when teachers noted that they improved attendance. The British and Foreign Society introduced them in 1816, the National Society in 1828, the Congregationalists in 1848, and the Wesleyans in 1854. The Clerical Superintendent of the National Society told the 1834 Parliamentary Committee that his Society had been reluctant to introduce fees; but parents, when they paid for education, valued it more and they transmitted that increased valuation to their children who worked harder and played truant less.”
So that was how the experiment played out. None of the parents paid previously.
It’s ancient history now. We have the results, and it’s on the record, and everyone’s paying the price.
Let’s see. The National Society, which had been running free schools, starts to require payment. This means that the Society now has more money, which it can use in various ways, including to pay its employees. A few years later, the Clerical Superintendent of the Society testifies that the fee is making students “work[] harder and play[] truant less.”
Hmmm. It might be the truth.
And, of course, it doesn’t answer the question, “How many more young people would have gone to school if there had been no fee?” Perhaps that doesn’t matter. Perhaps they wouldn’t get anything worthwhile from school anyhow.
Which raises the further questions: How high should the fee be? And what should be done with the children whose parents don’t pay it?
“How many more young people would have gone to school if there had been no fee?” Not more. Fewer. Fewer did, before the fee. The fee increased attendance.
The fee increased attendance.
Wow! A situation where people buy more when the price goes up. Not quite water flowing uphill–but it suggests to me that a lot more is going on here than a fee increase. Perhaps an increasing middle class and a more favorable attitude to schooling.
Sorry, but your “null hypothesis” seems to be “if we don’t change anything that matters, then the outcomes aren’t very different”.
That’s not really much of a null hypothesis.
Perhaps you can be clearer as to what your null hypothesis is?
No, that is exactly the definition of a null hypothesis!
The main point of it, btw, is that there are all these studies that maybe show statistically significant results, but they would not if they ran the studies correctly.
Arnold is almost losing sight of the fact that at base, it is just a null hypothesis, not a theory or opinion. The theory that it is robust or opinions on certain interventions are ancillary to the main point.
I have taken so much statistics, but this might be the only time it has ever been useful!
There is a framing problem here.
It seems that most of the discussion about the effects of schooling, school resource allocations, and so forth, all more or less presume that the main beneficiary of an education is the student.
I doubt the student is the only, or even primary beneficiary. I think our society presses so hard to teach the topics it does because the health of the political economy as a whole is driven by having a supply of people competent in the things the society needs. High school dropouts are a problem, not really as a humanitarian issue (though that is of course a real thing), but because they very often end up being a drag on the political economy, and huge loss of potential contribution to it.
What is more, while the null hypothesis is at least cautionary about the effects of resource allocation on things like student test results or incomes, we ought to wonder deeply if it applies to social functioning. That is, are the real measures (be they of effect or just correlation): rates of incarceration, rates of being the victims of crime, rates of being unable to earn some kind of living – all distinct from degrees and test scores?
The heart of those issues is poor parenting. Schools aren’t parents.
Walter E. Williams once suggested the production function is “multiplicative.”
Educational Outcomes = Home x School.
I can see how in some cases that would be true, especially with respect to issues of
1. Respect for authority
2. Ability to control one’s attention and follow directions
http://www.jewishworldreview.com/cols/williams072998.html
These are two good paragraphs from Williams’ piece:
“Children who tend to do well in school generally come from two-parent families. Their parents ensure that homework is done and educational material is in the home. They’re lectured about school behavior and punished when necessary. Parents make sure they attend school every day and on time. Parents respond to school notices and grades. And, they make sure their kids go to bed at a reasonable hour.
A good home environment is vital to getting a good education. The measures that parents take to create a good home environment cannot be done by government or educators. If what only parents can do is not done, education results will always be disappointing.”
BTW, Williams was raised by a single mother in the projects of Philadelphia, a story you can read in his _Up from the projects_.
In his memoirs he writes that most of the families in the projects were two-parent families, and as an absent-father child he was in the minority.
His mother did not have much money, but frequently took him and siblings to free cultural resources, not just the library but the Museum of Art.
The details fade in my mind. His memoir is a quick read.
He also asserts that jobs were easy to get and many of his peers had two or three jobs, from which they learned much more than they learned in their classes, especially if they were indifferent and unmotivated students.