Yesterday, the Center for American Progress (CAP) released a new report, Return on Educational Investment: A district-by-district evaluation of U.S. educational productivity. The report examined the productivity of more than 9,000 school districts across 45 states based on three different return-on-investment (ROI) measures.

The measures were designed to account for cost-of-living differences, as well as differences in student populations such as number of students in poverty, number of special education students, and and number of ELL (English Language Learner) students. CAP also developed an interactive map and Web tool where you can compare your district’s productivity to similar districts in your state.

Key Findings

Cost of inefficiency

• Low productivity costs the nation’s school system as much as $175 billion a year, or about 1 percent of GDP.
• In more than half the states there was no clear relationship between spending and achievement.
• Many school districts could boost student achievement without increased spending.
  • For example, the report argued that an Arizona school district could see up to a 36 percent increase in achievement if it increased its efficiency from the lowest level to the highest, all else being equal.

Productivity by District Type

• Productivity varies greatly within states. For example, the range of spending for California school districts that ranked in the top third in achievement varied by nearly $8000 per student.
• Over 400 districts nationwide were rated highly inefficient on all three measures.
  • These districts enroll about a million students combined.
• Students from disadvantaged backgrounds are more likely to be enrolled in highly inefficient districts.
• Some urban districts are far more productive than others.

Causes of Inefficiency

• Highly productive districts are focused on improving student outcomes.
• The least productive districts spend an extra 3 percentage points of their budgets on administration, operations, and other non-instructional expenditures.
• States and districts fail to evaluate the productivity of schools and districts.

CAP used the best statistically methods possible to make apples-to-apples comparisons between districts, even though it is unlikely they were able to fully account for those differences. CAP says as much in its report. But the three productivity measures it calculates are a good start for districts and tax payers within those districts to evaluate whether they are getting enough bang for their buck.

Of course there are many ways CAP could have looked at the data, but there are two ways I think CAP could have (and in my mind, should have) included as outcome measures: student scale scores (instead of percent proficient) and high school graduation rates. These two ways would have provided a different perspective and maybe even a more accurate picture of the outcomes districts are producing.

I’m not arguing that results would be better or worse using these two measures, but I think both are important measures of student outcomes that districts should examine.

First off, using student scale scores (or raw scores) instead of simply the percent of students proficient provides more information about the quality of the schools’ output. Using such a binary system as percent proficient to evaluate the productivity of districts can mask important outcomes.

For example, two districts with similar students and similar cost-of-living both spend $8,000 per student. In both districts 100 percent of the students are proficient, but in the first district the average student scale score is just above the proficiency cut score, while in the second district the average scale score is at the Advanced level. The second district obviously has better outcomes than the first district, yet under CAP’s methodology they are both considered to have the same achievement levels.

But I feel even more strongly that high school graduation rates should have been included as an outcome. CAP based its outcomes on 4^th, 8^th and high school test scores, which are important measures to examine. But wouldn’t a district’s true productivity be based on how well its students are prepared for the real world once they leave high school?

A measure of how prepared students are for college or the workforce on the day they receive they leave high school would be the perfect outcome measure for our schools. When measuring a district’s outcomes, does it really matter if 4^th graders are low performing if they wind up graduating high school college- and career-ready? No, as an outcomes measure, what is important is the final output, not how they got there. Graduation rates are not a perfect measure of how prepared students are for the real world but it is the best measure we have to measure the final outcome of our students.

I know that one report cannot answer all questions, so hopefully CAP will produce a second report on educational productivity that looks at other outcomes measures such as scale scores and graduation rates. Even if they don’t, districts should examine these outcomes measures themselves to gain a better perspective on how much bang they get for their buck. – Jim Hull

For more information on school funding check out the Center’s Money Matters and Cutting to the Bone reports.