Maybe We Don’t Need School Ratings

The following post was written for a feature called “Yes, but…” – and published at Thoughts on Public Education in December 2011.  John Fensterwald asked how California should measure school quality if not with the narrow, testing-based Academic Performance Index (API).  I was one of five respondents, along with state Senator Darrell Steinberg, former State Superintendent for Public Instruction Bill Honig, and education policy experts Fred Jones and Jeff Camp.  Please click here to see more of their biographical information and their responses, along with my contribution and the subsequent comments and discussions. I’m pleased to see that, a few years down the line, California has moved in the direction I’d hoped for, with the implementation of the Local Control Accountability Plan.

Imagine for a moment that California used letter grades rather than the Academic Performance Index to rate schools. If I were a parent whose child attended a high school with a “D” on its state report card, I would be gravely concerned that this school would fail to provide my student with the skills to succeed in college, and a college education is vital to my child’s future. If I had a choice, I would certainly want to move my child to an “A” school. I know these report cards aren’t perfect, but there must be a world of difference between the “D” and the “A” rankings, right? And if the “A” school was also listed among Newsweek’s Best High Schools, so much the better, I’m sure.

Wrong. The “D” school is better.

Or to be more precise, the “D” school is better if the measure of quality is college preparation. Don’t believe me? Take a look at this study – “College- and Career-Ready: Using Outcomes Data to Hold High Schools Accountable for Student Success” – from Florida. Writer Chad Aldeman sums it up this way:

“While [the ‘D’ school] got dismal marks from state and federal accountability schemes, it was actually quite successful in a number of important ways. It graduated a higher percentage of its students than [the “A” school] and sent almost the same percentage of its graduates off to college. Once they arrived on college campuses, [the ‘D’ school] graduates earned higher grades and fewer of them failed remedial, not-for-credit math and English courses than their [‘A’ school] peers.

“In other words, D-rated [High School] was arguably doing a better job at achieving the ultimate goal of high school: preparing students to succeed in college and careers. But because Florida’s accountability systems didn’t measure college and career success in 2006, nobody knew.”

The study concludes, as you might anticipate, with a call for more data going into accountability systems, and it’s hard to argue with that. But the catch is that any rating or ranking is going to miss something, and is going to create simplistic lists of winners and losers out of what should be a more complex view of school quality.

It is time to distinguish between having data and claiming to know what it means. If we were conducting chemical experiments, it might be different. With schools, we are “measuring” extended periods of highly complex interactions among hundreds or thousands of people (different combinations of people every year), operating under different combinations of influences, and we have yet to agree as a state or society about the outcomes that matter most in that complex setting.

Ultimately, I would argue that the state should be in the business of providing resources and guidelines, and leaving the final assessments of quality and success to professional and local agencies. These agencies must ensure transparency and protect the interests of all stakeholders. They should be comfortable examining widely varying types of data and appreciating the value of each. Their judgments and conclusions would be informed by data and observations, but expressed in words – reports that don’t hide behind the false certainty or pseudo-objectivity of final scores, points, grades, or gold stars.

California high schools already engage in an accreditation process similar to that description, carried out by the Western Association of Schools and Colleges. Why not make it more meaningful, but less intensive, and expand the approach to other levels?

If our citizenry can’t handle that shift, then we have a goal for our educational system, not to produce citizens, media, and political leaders who would prefer to have a meaningless “A” or “D” slapped on a school, rather than understand and express the complex realities of school quality.