I know that I should be accustomed to news media misuse of statistics by now. But somehow that doesn't help.
"Charter schools fare worse than traditional", reads today's page 5A Birmingham News headline on an Associated Press story about a recently released study comparing charter schools to traditional public schools. Here's the lead paragraph in its entirety: "Fourth-graders in traditional public schools are doing better in both reading and math than students in charter schools, the government says in a report fueling fresh debate over school choice."
You would think the AP reporter would be a little more careful about disguising his agenda. The shame of it is that there is nothing wrong with writing a news story on this report, and it could have been done in a useful way. Instead, it's been hijacked and turned into propaganda.
First and foremost, nowhere in the article does the study explain whether the study compared charter schools to the specific traditional schools they were intended to replace. Surely the proper test for whether a charter school is worthwhile is whether the charter school is doing a better job than the traditional public school its students would be attending if it weren't for the charter school. In other words, it's simply not meaningful to compare a charter school's performance with some kind of national average performance for traditional public school, at least not for the purpose of determining whether or not the charter schools are a good use of public resources. But as the study comparies 150 charter schools to 6,764 public schools, it appears that this may not have been the purpose of the study. But that's the way the AP presented it.
Second, the lead paragraph says this study fuels "fresh debate over school choice". I suppose charter schools are a part of the school choice movement, but they are a small and unique part. School choice is about the money following the student to the student's choice of school, be it public or private. Even if this study proved what the AP wants it to, it says nothing about the merits of using public money to send students to non-charter private schools, or even about giving students free choice among those "traditional public schools" the AP is so excited about.
Finally, as anyone with a basic knowledge of charter schools knows, charter schools vary widely in philosophy, quality, track record, experience, etc. There are some terrific charter schools in the United States; there are also some terrible ones. The whole idea of a charter school is to use public money but not impose any particular standards. Free money; no rules (to speak of, anyway). (It's a mark of how bad public schools had become that an idea like this got off the ground in the first place. Imagine the government contracting for almost any other function on this basis!) The point is that it's not very helpful to look at average performance of all charter schools; surely the more relevant data would be the performance of the more successful charter schools. Those would be the ones with ideas worth imitating.
A carefully written story would have made it clear that charter schools as a whole, so far at least, don't appear to be raising students' performance to the "national average" of traditional public school performance. This does raise questions about whether "no standards at all" is really appropriate for charter schools, which are after all funded by taxpayers. But it doesn't say much of anything about whether "charter schools" are "worse" than "traditional public schools", as the headline and story lead imply.