Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

December 2, 2013

10 questions to understanding PISA results

The big day is almost upon us. Tomorrow the results from the 2012 Program for International Student Assessment (PISA) will be released. The rhetoric pertaining to the quality of our public schools is certainly going to be amplified tomorrow, with critics lamenting how the results show our public schools are in dire straits while others will argue the results are meaningless. To help you understand what the PISA results actually signify, the Center for Public Education has answered 10 key questions about what PISA actually measures and what the results mean for our public schools.

1. What is PISA?

The Program for International Student Assessment (PISA) is an assessment of reading, math, and science literacy given every three years to 15-year-old students in public and private schools in about 65 countries. The international institution Organisation for Economic Cooperation and Development (OECD) coordinates the development and administration of PISA worldwide while the U.S. Department of Education’s National Center for Education Statistics (NCES) conducts the assessments in the U.S.

Unlike most state assessments that measure how much knowledge a student has acquired, PISA is designed to measure how well students can apply their knowledge to real-world situations. To measure such skills, the test items on PISA are primarily “constructed response,” meaning the test-taker has to write their answers to the questions, and there are few multiple-choice items. U.S. students typically do not perform as well on open-ended, constructed response items. This is one reason many states are adopting new standards, including the new Common Core State Standards, which are intended to emphasize how well students can solve problems and think critically based on the concepts, topics and procedures they have learned.

2. Why are PISA results important?

PISA is one of the few tools we have to compare the outcomes of high school students internationally.  PISA provides valuable information on how prepared high school students are for postsecondary success whether in the workplace, career training, or higher education.

3. Is the U.S. ranking on PISA negatively impacted because unlike other countries the U.S. educates and tests all its students?

No, this used to be true several decades ago, but is no longer the case. Every industrialized country now educates all their students, including language minority, special needs and low-performing students. Every country that participates in PISA must adhere to strict sampling rules to ensure the country’s results are nationally representative of all 15-year-old students. Indeed, the decision to test secondary students at age 15 was made in part because young people at that age are still subject to compulsory schooling laws in most participating nations, which provides more assurance that PISA will capture the broadest sample.

4. Where does the U.S. really rank on PISA?

In 2009, 30 countries had higher mathematics scores than the U.S. but just 23 of these countries significantly outperformed the U.S. Because only a sample of each nation’s students participate in PISA, much like political polls, each country’s score has a margin of error. This means that the score is actually an estimate of how the country would perform if every 15-year-old took PISA. In science, 21 countries had higher scores than the U.S., but only 18 scored significantly higher; in reading, while 16 countries scored higher, just nine countries significantly outperformed the U.S.

OECD reports statistically significant differences in performance between nations, which is a more accurate way to look at PISA rankings than a straight listing of average scores.

5. Does PISA measure the effectiveness of public school systems?

Not completely, for three reasons: 1) PISA results are representative of the performance of all 15-year-olds in participating countries including those  attending private schools; 2) PISA makes no attempt to isolate schools from outside factors such as poverty or high proportions of non-native language speakers that may have an impact on  performance —such factors are important to include in the mix when evaluating the effectiveness of each country’s schools; and 3) No single measure can incorporate every outcome we expect from our public schools. To gain a better perspective of the overall effectiveness of educational systems, you should consider multiple measures. NSBA’s Center for Public Education’s Data First Data Center is a good resource to get you started when examining public schools in the U.S.

6. How does the U.S. stack up on other international measures?

The U.S. fares much better on other international assessments.  U.S. 4th and 8th graders performed among the top 10 countries in both math and science on the most recent Trends in Mathematics and Science Study, which was administered to more than 60 countries (TIMSS, 2011). Moreover, only four countries outperformed U.S. 4th graders in reading on the 2011 Progress on International Reading Literacy Study (PIRLS). Finally, U.S. students led the world in civics in 1999, the last year the CivEd was given. As of 2009, the nation’s 15-year-old students did not compare as well on PISA, especially in math and science. However, the U.S. performed better in reading by scoring among the “top 10.”

7. Has the U.S. shown improvement on PISA?

The U.S. saw a slight improvement in math scores between 2006 and 2009. It wouldn’t be surprising if such gains continued in 2012 as U.S. high school students continue to take more rigorous math courses. It is important to point out that the U.S. has demonstrated improvements on other measures since PISA was first given in 2000. U.S. 4th and 8th graders made among the greatest gains in math between 1995 and 2011 on TIMSS. The U.S. also made dramatic gains in on-time graduation rates by improving from 67 percent in 2000 to 75 percent in 2010 according to Education Week. Even on the National Assessment of Education Progress (NAEP), U.S. 4th and 8th graders have shown significant progress between 2000 and 2013, although high school students are not showing the same gains. The lack of progress on PISA appears to be the exception rather than the rule in terms of international comparisons.

8. How should the results be used?

We need to get beyond seeing PISA as a horse race by fixating on whether the U.S. finishes in win, place, or show. Instead, we need to see PISA results as an opportunity to assess if best practices in teaching and learning in other countries can also work for secondary schools here in the U.S. For example, we should  look at how much time other countries give teachers for professional development, how much they pay their teachers, how much time teachers spend in the classroom, how much flexibility exists at the local level, how special needs students are taught, and how much time students spend in school. Answers to these and others questions could be instructive for U.S. educators and policymakers. While PISA gives us an opportunity to learn from other countries it is important to keep in mind that just because a high-performing or high-gaining country does something does not mean it will work in U.S. schools.

9. Does poverty affect the U.S. performance on PISA more than in other countries?

Many analysts observe that poverty has a greater impact on student performance in the U.S. than elsewhere. For one thing, the U.S. has the highest child poverty rates among industrialized countries. For another, students in the U.S. who live in poverty tend to have less access to resources that research consistently shows impact student achievement, including highly effective teachers, access to rigorous curriculum, and high quality pre-k programs. Yet, poverty is just one of several factors that affect the standing of the U.S. In comparing the performance of top students around the world—where poverty is likely less of a factor—America’s top students still do not compare well to their peers in other countries. For example, in 2009 19 countries’ top students (scoring in the top 10 percent) outperformed the U.S.’s top students in science on PISA.

10. Are PISA results a precursor of America’s future economic competitiveness?

Our high school graduates’ preparation for postsecondary success certainly has some impact on the future economic competitiveness of the U.S. However, as stated in question 5, PISA is just one measure of high school students’ college and career readiness. In addition, many factors besides K-12 schooling contribute to the economic competitiveness of the U.S. and every other country, including, for example, a country’s monetary and fiscal policies. But for a country to maximize its economic output it needs a well-educated society which would lead to lower unemployment rates and less demand for government services. Stanford University Economist Eric Hanushek estimates that if the U.S. had scored 50 points higher on PISA in 2000 by 2015 GDP would be 4.5 percent higher than currently projected. Such an increase is the equivalent to the total expenditures on U.S. K-12 schools in 2015. Keep in mind, however, this does not mean that if the U.S. doesn’t improve on PISA that GDP will decline when our current high school graduates enter the workforce. However, it does show that education does affect future economic outcomes.






October 25, 2013

Good news but more work to be done

For nearly two decades researchers have attempted to link results from the domestic National Assessment of Educational Progress (NAEP) to the results from the international assessment Trends in International Mathematics and Science Study (TIMSS), to give states an opportunity to compare their math and science achievement levels to other countries across the world. Yesterday, researchers at the National Center for Education Statistics released the latest attempt to make such a link. This report provides not only the most current but also the most accurate measures of how each state’s performance would stack up internationally.  As Jim stated in his summary yesterday, the results give many states a reason to pat themselves on the back.

 A deeper look into the results shows that some states posted some of the world’s best scores. In math, Massachusetts, Minnesota, Vermont, New Jersey and New Hampshire scored higher than all but 5 of 47 countries. Science scores were similarly impressive; 9 of the 14 top scorers were U.S. states, with Massachusetts, Vermont and New Hampshire finishing in 2nd, 4th and 5th place respectively.

While these performances are certainly worth celebrating, other data points from the study warrant serious concern. Of the 36 states that scored above the international average in math, most of them, according to an EdWeek article, scored at the Intermediate proficiency level (There are 4 levels total: Advanced, High, Intermediate, and Low). In other words, the average 8th grade student in most states is not at the more advanced levels that Korea, Singapore, Japan and other countries are. If U.S. students hope to compete in a global environment, scoring above the average isn’t enough when other countries continue to achieve exemplary marks.

The data also show that many states contain vast disparities between high and low performers. Texas, for example, scored above the international average in math, but 30% of its 8th graders scored at the Low level or worse. And as a country, the rates aren’t much better; 32% of U.S. 8th graders performed at the Low level or lower and 69% were scored at the Intermediate level or below. To make matters worse, the aforementioned Edweek article states these figures haven’t changed much since 2007.

Moving forward, educators in most states should celebrate their ability to beat international averages while recognizing that above mediocre isn’t enough. Globalization requires more of our students reach the High and Advanced levels on international exams. –Jordan Belton






October 8, 2013

You don’t need to go to Harvard to benefit from college

Some parents and policymakers are uncomfortable with the recent emphasis in education policy on college readiness, which is typified by the adoption of the Common Core State Standards in 45 states. Reason likely being, the term college-ready is synonymous with preparing students for four-year postsecondary institutions like Harvard or their state’s flagship university. As the rhetoric behind the push to graduate all students college-ready typically revolves around graduating more students with a four-year degree, it is no wonder there is such apprehension.

It is absolutely true that not all students are meant or want to go to Harvard or any other four-year institution. Yet, it is clear most of today’s students will need education beyond high school to obtain a job to sustain a middle-class lifestyle. However, college-readiness goes beyond preparing all students to earn a four-year degree; it’s about preparing students to earn two-year degrees as well. A new report brings to light this fact by finding that community college graduates earn on-average $259,000 more over their working lifetime than those who only earn a high school degree. As a result, community college graduates pay $67,000 more in taxes. Moreover, community college graduates are less likely to need government assistance– such as unemployment benefits– as they are less likely to be unemployed as well.

Drawing attention to the success of community college graduates is critically important to promoting the fact that college-readiness is not just about preparing students for Harvard. The importance of the affordability and the technical training provided by our nation’s community colleges should not be overlooked. However, as my previous report on international college attainment rates showed, the U.S. does a decent job preparing students to earn a four-year degree but falls well short when it comes to the attainment of two-year degrees. If we as a nation focus on significantly improving the college attainment rates for our two-year colleges, the U.S. would not only once again be among the global leaders in college-attainment but it would also have a significant impact on the nation’s economy as well. – Jim Hull






September 10, 2013

New CPE report examines what’s wrong with current teacher PD offerings and how to fix it

CPE_AAG_inlineGraphic_HalfwidthMy first introduction to professional development as a young teacher was underwhelming to say the least.  When I first entered the profession, I was excited to meet other teachers and get down to the business of having lengthy, intellectual conversations about how to best teach our subjects.  Surely, we would debate which concepts to teach, which texts to use, and how we could get our students active and intellectually engaged.

Much to my dismay, in my first years of teaching, there was not one day carved into the expansive school year where I was able to meet with fellow teachers to really get down to the nitty-gritty questions of teaching.  What I did experience, though, was time set aside for “professional development” that did little to inform or improve my own teaching.

For example, when I started my first year of teaching, I sat through an entire day’s all-staff session devoted to the book Who Moved My Cheese?  The book chronicled how different mice responded to the cheese in their maze being moved somewhere else.  As valuable planning time before the first day of school ticked away, the entire staff analyzed which mouse they were in the book.  While I did learn that my department head was a “Sniff,” and another new teacher was a “Scurry,” I certainly didn’t learn anything that made me a better teacher.

Unfortunately, this session was just a taste of the prevalent realities of professional development in American schools, and as time went by, I had to realize that my job was not a place where I would learn to be a better teacher—I had to seek that out myself. I would have conversations after school with fellow teachers about teaching, or pay out of my own pocket to attend “real” professional development sessions that addressed real concerns of teaching my content, but these were isolated experiences, and they didn’t occur in a regular fashion as part of my work.

In all my years of teaching, I can’t think of one professional development experience sponsored by my school or school district that really addressed core issues of my instruction.  I’d like to think that my own experiences are an anomaly in a sea of schools with profound, helpful professional development, but unfortunately, this isn’t true … as you will learn in my report and the latest edition to the CPE library, Teaching the Teachers: Effective Professional Development in an Era of High Stakes Accountability.

-Allison Gulamhussein






August 16, 2013

CTE’s central role in the common core

Earlier this month I had the pleasure of speaking with Oklahoma educators at the state’s summer Career and Technical Education (CTE) conference.  I was asked to be part of a panel addressing the question, how to implement the common core into CTE.  My message was simple: the question is backwards because the common core cannot be implemented well without CTE.

Here’s why. The aim of the common core standards is college and career-readiness, not college or careers.  We’ve actually been doing the latter for a long time. Traditionally, high school students elected to either prepare for one path or the other. But as many studies have pointed out (including CPE’s Defining a 21st Century Education) in order to be successful after high school all new graduates need high-level knowledge like that formerly reserved for college-intending students even if they are more interested in jumpstarting their careers than attending a four-year college.

We also know that for most new jobs, a high school diploma alone will not be sufficient; rather they will demand some kind of postsecondary training or certification. In addition, individuals who don’t immediately seek more education after high school will likely need to get back into the system at some point during their working life, as various occupations disappear.  So we need to make sure graduates are prepared for an uncertain future and can continue education and training as they need it.

But college preparation is just one side of the college-career equation. Students who enroll in four-year colleges and universities will work someday.  All young people need to develop the skills valued in the workplace, for example, the ability to apply what they learn, connect information from across disciplines to solve problems, and read and interpret complex informational texts and documents.  Students and employers can’t assume that traditional academic study will teach these abilities.

The common core standards recognize that there is a lot of overlap between the knowledge and skills needed for college and those needed for good jobs. For many students, this likely means higher expectations in terms of academic content. But the common core also differs from subject matter as usual where the CTE field has a head start.  This is especially so in the emphasis on mathematical practices; more data, probability & statistics than in traditional college prep math; reading and writing informational texts; and specific reading and writing standards for science and technical subjects.

These are all innovations that aren’t seen in current state standards, but ones that I applaud.  I’m not alone.  David Conley and his team at the University of Oregon surveyed close to 2,000 postsecondary instructors about the relevance of the common core standards. About half of the respondents taught CTE courses in two-year institutions. The vast majority of instructors rated ELA for non-literary reading and writing very high. A large majority of the CTE group gave mathematical practices the highest importance rating.

These skill-based standards command different, more hands-on instructional approaches. CTE educators have a real advantage in this area compared to their core subject area peers. CTE programs are by definition applied. In this way, they have a lot to contribute to the combined efforts of high school faculty to negotiate the shift to new instruction.

Make no mistake, the common core standards cannot be the sole responsibility of math and English teachers. That’s too much of a burden on two disciplines. It won’t be done well. And it would ignore the valuable resources in other subject areas that should be brought to the table, including CTE.






« Newer PostsOlder Posts »
RSS Feed