The WSJ, had an article in the print edition on November 27 that I cannot find on line (their search function is not helpful). The print article was called ‘Adversity’ Has Big Effect on SAT Scores. What I can find online instead is this:What Happens if SAT Scores Consider Aversity? Find Your School.

Anyway, the WSJ uses a Georgetown education researcher’s regression equation relating SAT scores to “adversity scores” to make inferences such as

Top public magnet schools performed exceptionally well in adjusted SAT scores, meaning their scores jump when adversity is accounted for.

To see why this is not a valid inference, suppose that there were two students of identical backgrounds but different ability levels. Presumably, the magnet school would select the student with higher ability, leaving the other student to attend a regular school. The more able student would get a higher SAT score, but that would say nothing about the magnet school’s “performance.”

I sent a letter to the editor of the WSJ about this, but they did not print it. But I hope that someone there gets the message that this was statistical malpractice.