Comparing states to other countries: A fair comparison?

Can U.S. compete if only 32 percent of students are proficient in math? This was the headline over at last week in reference to a new report that compared each state’s math and reading performances to that of 65 other countries. Researchers at Harvard’s Program on Education Policy and Governance made the comparison by comparing the results of the 2007 8th grade NAEP assessments, which all states participated in, to the 2009 PISA results that 15-year olds (mostly high school sophomores) in the U.S. and 64 other countries participated in. Although the same students did not take both assessments and the assessments were taken in different years, the researchers claim that such a comparison is a fair representation of the Class of 2011 math and reading performance.

I’ll admit I’m not an assessment expert or a statistician, but I am a little skeptical of how fair that comparison is. For one, reliably comparing different assessments is quite difficult. These researchers took a fairly simplistic approach that just looked at the percent of 8th graders achieving at NAEP’s Proficient level (32 percent) in math and then looked to see at what score did the top-performing 32 percent of U.S. students score at on the 2009 PISA test. Then they used that PISA score to represent achieving ‘proficiency’ on NAEP for other countries. The same comparison was done for reading.

A similar study being conducted by NCES, scheduled to be released next year, takes a more comprehensive approach. In this study, they will compare NAEP to TIMSS results by having a sample of 8th grade students in each state take both the NAEP and TIMSS assessment in math and science. This will likely provide a more accurate comparison of how states truly compare to other countries, since they are using results from two assessments taken by the same students in the same year, unlike the Harvard study.

I could go on about the possible limitations of making such linkages, but such arguments would be fairly technical and wouldn’t change the main finding from the Harvard study: too few U.S. students are proficient in math. As I stated in The Proficiency Debate: A guide to NAEP achievement levels, NAEP’s “proficient” level is a fairly high, although reachable, standard, and is not the same as being “on grade level.” With that being said, the data makes it quite obvious that a significant number of countries have a far greater proportion of their students obtaining higher level skills than the U.S. and that in some states very few student are acquiring these knowledge and skills. As the Harvard report states, if all states were able to increase the proportion of students obtaining these skills to that of the proportion of Canada, the U.S. could increase its GDP by nearly $1 trillion per year over the next 80 years. Sounds like a great argument for states to increase their investment in education instead of cutting funding. This is a more effective alternative to cutting our nation’s deficit than simply cutting spending and raising taxes.– Jim Hull

Comments (1)

  1. Pingback: The EDifier - Center for Public Education

Leave a comment

Your email address will not be published.