I do think that those who are skeptical of our ability to meaningfully measure school performance are expressing a level of data and empirical skepticism that is not applied elsewhere.
You should read the whole post. I’ve seen excerpts from Tyler Cowen and Don Boudreaux, and neither their excerpt nor mine really conveys what Ozimek is complaining about.
My own view is that judging an organization from the outside is hazardous. I remember when Harvard’s David Cutler was touting pay-for-quality as the solution for compensating doctors, and I considered the notion absurd. If people in Washington know what individual doctors should be doing and how they should be paid, then they must also know what individual middle managers throughout the business world should be doing and how they should be paid.
Organizational outcomes should not be judged by statisticians running regressions. They should be judged by consumers voting with their dollars (using vouchers would count as voting with their own dollars).
Individual performance in the context of an organization should not be judged by Harvard economists. It should be judged by their managers, who know the context in which they work.
The problems with education and medical care is that we insulate consumers from paying with their own money for those services. This imposes a socialist calculation problem, with adverse consequences in both areas.
But we accept that schools judge, rank and grade students and parents daily. I agree that it is a debate about “within the flaws of the calculation problem, what are the distant second-bests?” Exit is pretty powerful although they are doing their best to dissemble/destroy and subvert recent advances in that area, for instance by creating focus schools like those that are STEM and Arts focused which I suspect is a fraud. If you made all students learn the same tripe so that you could get an apples-to-apples comparison of schools you would lose the flexibility that individual students would require for true learning. But if you are feeding them all the same tripe and while you pretend you are STEM focused you are probably just trying to increase your switching costs while building in an excuse why you should be allowed to be bad at Arts.
Ozimek means that contrarian detractors are being too skeptical. But the other way to interpret this assertion, assuming it’s true, is to say that people in those ‘elsewhere’ fields aren’t being nearly skeptical enough!
Indeed, Ozimek appears to be asserting that we should expect the same level of precision in our ability to judge these subjects as outsiders. So be skeptical, or don’t be, but be consistent.
One could argue that there are different levels of precision involved in each case, but I haven’t seen that argument made. I’m glad Ozimek is pushing the discussion in this direction.
I guess I don’t understand the point he is trying to make at all. For example, earlier he says,
“How acceptable is a black-white pay gap versus a black-white test score gap? You hear that schools can’t be held responsible for fixing this, but I’m guessing most who say that wouldn’t accept a firm who tried to say it’s not their problem to fix.”
Huh? Of course it is acceptable to any right-thinking person — that pay gap obviously reflects performance which is probably a reflection of skills and abilities (which in turn is a reflection of IQ and other character traits.) None of this is rocket science. I would be happy to agree with that firm that it is not their problem to fix black-white pay gaps just like I don’t think it is society’s problem to “fix” black-white test scores (which is impossible since it reflects underlying genetic reality.)
It is hard to judge an organization, period. Very successful organizations can seem dysfunctional from the inside, organizations run as well as possible can seem like failures from the outside.
Leaving the judgment to consumers (or parents, whatever the case may be) is something I agree with but it needs defending. In order to defend it, we must be able to judge both organization *and* consumers, to an extent. Furthermore, we must judge the judges. There’s something inescapably circular about the whole thing, but the bottom line is that a particular understanding of organizations, consumers, and judges, and what we are capable of understanding about any of them from a given vantage point, is required to defend any perspective on the evaluation of any of them as well as who should be left to do the evaluating.
If you talk to parents, they usually have a very strong opinion about what schools they want their children in. They fight tooth and nail trying to get their kids into charter programs.
Which is why I find it odd to understand opposition to school vouchers. My best try at an explanation is that the scale is too small. A local public school often has around 1000 students in it. Any form of school choice–vouchers or otherwise–means that either students have to commute further, or the schools will have to be smaller. Still, that doesn’t explain why people oppose the idea even in principle.
No, the problem is consumers can only judge the superficial aspects to arrive at conclusions cost equals quality so the only solution is more must be spent and this occurs regardless of who is paying, so the only limit is what they can afford collectively, whether through insurance or through settlement.
Nonsense. I make and have made thousands of judgements about products and services and am in a much better position to judge their appropriateness than any outside party. Of course, I pay attention to recommendations and reviews and factor that information into my decisions.
Why is it hard to judge schools? That seems like a very strange claim to me.
The standardized tests I have looked at–IB, AP, SAT, GRE–are all very good. I consider it a good sign if a school helps students at a school improve on those tests over time, and a bad sign if they don’t.
Likewise for teachers. If everyone at a school fails AP U.S. History for 5 years in a row, then that’s a bad teacher. You don’t have to look inside the classroom to know this.