The Educational Testing Service recently released a study comparing the academic performance of students in public and private schools, “Comparing Private and Public Schools Using Hierarchical Linear Modeling.” It is based on analysis of 2003 NAEP test data of 4th and 8th grade students in math and reading.

Authors Henry Braun and Frank Jenkins concluded that, “In grades 4 and 8 for both reading and mathematics, students in private schools achieved at higher levels than students in public schools. The average difference in school means ranged from almost 8 points for grade 4 mathematics, to about 18 points for grade 8 reading. The average differences were all statistically significant.

"Adjusting the comparisons for student characteristics resulted in reductions in all four average differences of approximately 11 to 14 points. Based on adjusted school means, the average for public schools was significantly higher than the average for private schools for grade 4 mathematics, while the average for private schools was significantly higher than the average for public schools for grade 8 reading. The average differences in adjusted school means for both grade 4 reading and grade 8 mathematics were not significantly different from zero.”

One might presume that as soon as the ETS study hit the streets, voucher advocates immediately tried to discredit it. And as usual, they called on the go-to guy for vouchers, Harvard's Paul Peterson. He and Elena Llaudet totally redesigned the work of Braun and Jenkins and, guess what? They came up with opposite conclusions in the paper, “On the Public-Private School Achievement Debate.” http://www.ksg.harvard.edu/pepg/PDF/Papers/PEPG06-02-PetersonLlaudet.pdf

In that paper, Peterson and Llaudet eliminated the variables that related to the demographics of the students, substituted less meaningful variables and naturally came up with different results.

P&L claimed that they substituted, “better measures,” of student characteristics, but in fact, they removed most characteristics that would identify students as either low income or disadvantaged. In what some people would think to be outrageous, P&L removed the use of Title I status, free/reduced lunch status, limited-English proficiency, the existence of an Individual Educational Plan to identify disadvantaged kids. They also discarded absenteeism as an indicator of poverty, disregarded the number of books in the home or the existence of a computer as an indicator of middle class status. They replaced those variables with the students' estimate of the parents' education, as if these kids would know. Worse, they used the region of the country where the school is located as an indicator of income level, which causes all children in both private and public schools in the same city to be considered undifferentiated as to demographics. And they used how much English was spoken in the home as described by the children to substitute for school-identified limited English status.

Without any adjustments for poverty, limited-English and kids with disabilities, private schools' test scores were higher in this study. But when these were adjusted for their presence in school children, the reverse was generally true in the Braun and Jenkins study. But because P&L threw out most all of the variables that related to poverty and whether the child was disadvantaged, they returned the analysis to one with almost no adjustment for those characteristics. Some may believe that these are extreme measures to fix the results and discredit the Braun and Jenkins work. Thus, it is hard to believe that most people will take the Peterson study seriously, but rather see it as another biased attempt to promote privatization.

The bottom line, as Braun and Jenkins state, "The presumed panacea of private-style organizational models—the private-school advantage—is not supported by this comprehensive dataset.”

Dr. Solomon is a retired University of Kentucky Professor and can be reached at [email protected]