Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

January 30, 2013

Misconceptions about international assessments

I had the pleasure of spending time with local school board members at NSBA’s annual Federal Relations Network (FRN) meeting here in DC. It has always been my favorite conference to attend as hundreds of school board members from around the country prepare to meet with their members of Congress on behalf of our public schools.

I enjoy the conference so much because the attendees are so passionate and knowledgeable about education policy and research. However, I noticed that some attendees — like the general public — can have misconceptions when it comes to some issues from time to time, which is understandable since most school board members are not trained educators or researchers. One of those misconceptions relates to international assessments which could impact how well our students are prepared for the global labor market.

All U.S. students are tested while other countries only test their high performing students

There seems to be a widely held belief that international comparisons are unfair to American schools because we educate and test all our students while other countries educate and/or test only their best students. As I wrote in the report More than a horse race: A guide to international assessments this is simply not true. Every industrialized country has now followed our lead and provides an education to all students, including students with disabilities and non-native language speakers, just as our schools have done for decades. Furthermore, as a condition for participating in international assessments each country must test a nationally representative sample of all students. If a country does not meet the sampling requirements then the country’s results are not compared to other countries. While some researchers question the sampling of some countries in some years, there is no evidence that other countries are gaming the system by only testing higher performing students.

Unfortunately, this is not what school board members heard from an expert in one of the general sessions. To the contrary, they were told that Taiwan (aka Chinese Taipei) tested their 7th graders to determine if they would go onto vocational schools or academic schools. It was strongly implied that those students who went onto academic schools in the 8th grade were the only students tested on TIMSS the 4th and 8th grade international math and science assessment.

Again, this is not true for a couple of reasons. For one, students in Taiwan are not tested until after the 9th grade—not the 7th grade– to determine if they will go onto vocational or academic schools. So there is no sorting of students into vocational and academic tracks until after the 9th grade. Even then, 95 percent of 9th graders go onto the 10th grade in either of these schools—not too shabby. Second, even if 8th graders were split between vocational and academic schools under the sampling rules for the international assessments students from both types of schools would have to be tested. So just because students don’t attend a certain type of school doesn’t means that they are excluded from international assessments.

Why is this important?

School board members are not alone. Media, parents, teachers, administrators, business leaders, and as we see even other education researchers can have wrong ideas about international comparisons.

So why is it important? Well, if we as country believe that our students are performing better than they actually are compared to students in other countries we can become complacent. We may not have the urgency we need to ensure our students are prepared for life in the 21st Century. As CPE’s report on a 21st Century Education found, U.S. students need more skills and knowledge, especially in math and science, to adequately compete for good jobs in the international labor market.

Ensuring that international comparisons are properly understood is vitally important for school board members, teachers, administrators, and other policymakers so they know where our students really stand so they have the information they need to make decisions on how to best get our students to where they need to be to compete in the 21st Century. – Jim Hull






3 Responses to “Misconceptions about international assessments”

  1. Hi Jim –

    Thanks for this. I was at FRN.

    Does this apply to Shanghai also? I have seen a couple reports that they only test the well off kids and do not test the millions of poor kids.

  2. Jim Hull says:

    Great question Lawrence. I have heard rumblings about Shanghai’s sample but have not seen any evidence they only tested their well of kids. With that said however, one should not take Shanghai’s success as proof China as a whole is outperforming the U.S. If the results were based on a sample of all students in the country, including students in the much poorer sections, I am quite certain their performance would fall dramatically. Shanghai is home to many of China’s well-off population so it is not likely indicative of the performance of the country as a whole. Although Shangai’s population is quite large and will produce a number of graduates that will compete with U.S. students in the global labor market, so their success should not be dismissed either.

  3. [...] Do other countries only test their best students making international comparisons unfair to the U.S.? Not so says Jim Hull senior policy analyst at NSBA’s Center for Public Education who has written on this subject in the CPE’s guide to international assessments. He found that while at one time it was true that other countries educated and assessed only their best students, over the past couple decades all developed countries have followed the U.S. lead and now educate and assess all students. Yet, there is a misconception that this is still the case which can have negative consequences on our students in the long-run says Hull. You can read more about this misconception of international assessments here. [...]

Leave a Reply


RSS Feed