Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

April 3, 2014

U.S. students score well on first PISA problem solving exam

Earlier this week, the Organization for Economic Cooperation and Development (OECD) released a report on the first ever Program for International Student Assessment (PISA) problem solving exam. 15-year-olds in the U.S. who took the exam scored above average but also had scores that were significantly lower than those of 10 of the other 44 countries and economies participating in the exam. Students in the U.S. performed on par with 15-year-olds in England, France, the Netherlands, Italy, Germany, and Norway, but still lagged behind highest-scoring Singapore and Korea, as well as students in several other nations.

American students’ scores on the problem solving portion of the PISA exam were quite a bit higher than their scores on the math, reading, and science portions of the exam, possibly implying that students in the U.S. are better at applying what they’ve learned to real life situations than they are at performing strictly academic tasks. That said, the nations that excelled on the problem solving portion of the exam, such as Korea and Singapore, also do well on the traditional academic sections of the PISA exam.

Right now these findings don’t mean much, as this is the first time a problem solving portion of the test has been administered and the sample size was very small (less than 1,300 students in the U.S. took the problem solving exam). However, it is good to see assessments moving beyond only measuring math and reading proficiency and at least attempting to measure deeper learning skills that are also needed to solve real-world problems.  At this point in time we can’t say how well an assessment like this truly predicts problems solving ability, but it seems like a positive development that PISA is acknowledging that problem solving skills will be important for many of these students in their future jobs. At the very least, these results provide an interesting cross-sectional picture of problem solving skills throughout the world. If you’d like to try your hand at some of the problems, sample questions from the 2012 PISA problem solving exam can be found here and here.

Filed under: Assessments,International Comparisons — Patricia Campbell @ 4:08 pm





March 31, 2014

Common Core standards undergoes field testing

Field testing for the Common Core-aligned Smarter Balanced Assessment Consortium (SBAC) and Partnership for Assessment of Readiness for College and Careers (PARCC) tests is now underway in many states and will continue to be implemented in others over the course of the next several weeks. More than four million students in grades 3-11 will be helping field test the new math and English language arts assessments. These field tests are an opportunity for students to experience the testing environment and get a sense of what they will be expected to know and be able to do for future Common Core-aligned assessments, but with no stakes attached at this time.

Doing field tests an entire year before these assessments will be used for any type of evaluation helps to ensure that the new Common Core-aligned assessments are reliable, valid, and fair for all students taking them, and gives SBAC and PARCC time to adjust both content or structural issues that might pop up during field testing. This trial run also gives teachers and schools a chance to practice administering the test and an opportunity to work out any technical or procedural problems before the assessments begin next year. Additionally, the field tests will introduce students to a type of assessment that is different from what many are used to: one that emphasizes critical thinking and problem solving and focuses less on memorization and simply filling in the correct bubble.

Of course, these assessments are not perfect, but that is all the more reason that this “test of the test” is important. Much of the controversy surrounding the implementation of the Common Core States Standards has involved how assessment would work and if it would be any different from the testing most states require now. This trial run allows thousands of teachers and millions of students across the country to become accustomed to the new system in what is essentially a “no stakes” testing environment. Will there be glitches? Of course – but the field tests allow time for these issues to be worked out before the actual test is administered next year. It might be an imperfect solution, but it is certainly a step in the right direction to ensure that CCSS-aligned assessments are the best that they can be before they are administered in a high stakes environment.

The Alliance for Excellent Education has a helpful Common Core Field Test Q&A available with more information. For more on the Common Core State Standards, visit CPE’s Common Core Resource page.

-Patricia Campbell






March 7, 2014

Big changes coming for the SAT – What do they really mean?

Big changes coming for the SAT – What do they really mean?

On Wednesday, the College Board announced a major overhaul of the SAT in what will be the second revision of the college entrance exam in less than ten years. Substantial changes include:

  • The test will again be scored out of 1600, and the penalty for guessing will be eliminated
  • Some of the more obscure vocabulary words are being thrown out and replaced with words that are commonly used in the academic and professional worlds
  • The essay portion of the test will now be optional and source-based, and students choosing to complete it will have 50 minutes, rather than 25, to do so
  • Math questions will focus on three main areas: problem solving and data analysis, algebra, and real-world math related to the design, technology, and engineering fields

Perhaps the most substantial change is that the new test will be closely aligned with what high schools are teaching.  It will require students to analyze nonfiction texts, build an argument using evidence, and apply math concepts to real life situations; all skills that are emphasized in the Common Core State Standards. The alignment between the new SAT and the CCSS is not surprising, as David Coleman, a key architect of the Common Core, now serves as President of the College Board. The goal for the redesign was to create an SAT that is more transparent, focused, and closely tied to the work that students do in school every day. The College Board believes the test should move toward evidence-based thinking and reinforcing the skills that students should have already learned in high school, and move away from the need for test taking tips, tricks, and strategies that make the test prep industry so profitable and allow affluent students whose families can afford expensive tutors and intense coaching to “game” to SAT. The College Board is also partnering with Kahn Academy to offer free online test preparation materials in an attempt to level the playing field for SAT-takers and curb exorbitant spending on test prep.

While the College Board’s goal of reducing inequality is certainly admirable, we have to ask – how much will these changes really matter? The SAT is becoming less and less relevant in college admissions decisions now that over 800 colleges and universities have “test optional” admissions policies.  Even among students who are still required to submit test scores for college admissions, the SAT is declining in popularity. For the last two years more students have chosen the ACT over the SAT for their college admissions test (although this could change now that both tests focus on what students have learned in school).  I am also relatively unconvinced that changing the test will rein in the culture of test prep hysteria among parents. This new SAT might be more difficult to “teach to” but that’s not going to stop affluent parents from purchasing every book, tutor, or service that might help their children gain an edge. Changing the test is not going to kill the test prep industry, as the College Board seems to hope it might.

The bright spot seems to be that the test is moving toward aligning with what students are actually learning in school. Since high school grades are routinely given more weight in college admissions, it just makes sense to test students on material that matches up with what they have learned, rather than arcane words they may never see again after SAT day. This realignment and the availability of free online prep materials are steps in the right direction, even if they don’t substantially change the culture of college test preparation. — Patricia Cambell

Filed under: Assessments,college,Common Core,High school — Patricia Campbell @ 1:14 pm





December 3, 2013

Disappointing results from latest international assessment

Results from the 2012 Program for International Assessment (PISA) were released today that compared the reading, mathematics, and science literacy of 15-year olds in 65 countries including the United States. Unfortunately, the overall results were not positive for our nation’s schools. In fact, the U.S. failed to improve on any of the three subjects tested since 2000- the first year PISA was administered. Due to this lack of improvement a greater number of countries outperformed the U.S. in 2012 than did in 2009—the last year PISA was administered—in all three subject areas. In particular, in mathematics the U.S. was significantly outperformed by 29 countries in 2012 compared to 24 countries in 2009. Even in reading where the U.S. has compared much more favorably, U.S. 15-year olds were outperformed by 19 countries in 2012 compared to just 9 countries in 2009.

What the results indicate is that while the U.S. performance remains relatively unchanged, other countries are leapfrogging over the U.S. by making significant gains in reading, mathematics, and science just between 2009 and 2012. These include countries such the United Kingdom, Ireland, Canada, Poland, and Australia which all have among the highest child poverty rates in the world and all these countries outperformed the U.S. in mathematics.  Certainly, poverty impacts student achievement but the U.S. can learn from these countries on how to more successfully educate poor students. One bright point from the PISA results for the U.S. is that the achievement gap between high and low-socioeconomic status (SES) did narrow slightly between 2009 and 2012. However, even if every other country had a similar SES rate as the U.S. the U.S. performance would actually drop slightly while the performance of many other countries would actually improve. This provides evidence that the mediocre U.S. performance is not simply due to demographics.

While the PISA results are disappointing they are the exception rather than the rule when it comes measuring U.S. performance. On other international assessments such as TIMSS and PIRLS the U.S. has made significant progress over the past decade or so. In fact, in math the U.S. is among the world leaders in gains between 1995 and 2011. The U.S. has also made significant gains on domestic assessments such as NAEP. And the U.S. estimated on-time graduation rate has improved from 67 percent in 2000 to 74 percent in 2010—which is nearly at an all-time high. This makes the lack of improvement on PISA all that more surprising. We need to take a deeper look into PISA data to find out why the U.S. is making such gains on other indicators that are not showing up in PISA. Without knowing the answer to this question limits our ability to use the PISA results to improve our schools.

The Findings

Mathematics Literacy

  • The U.S. score of 481 was significantly lower than the international average* of 496.
  • The U.S. was outperformed by 29 of 64 countries**.
    • Shanghai-China was the highest performing country (613) followed by Singapore (573), Hong-Kong-China (561), Chinese Taipei (560), and Korea (554).
    • The U.S. performed similarly to 9 countries including Norway, Italy, Russia, and Hungry.
    • The U.S. performed significantly better than 26 countries such as Israel (466), Greece (453), Mexico (413), and Brazil (391).
  • Scores for the U.S. have not improved.
    • Scores for the U.S. were similar between 2009 and 2012 as well as between 2000 and 2012.
    • Twenty-nine countries outperformed the U.S. in 2012 compared to 24 countries in 2009.
      • In 2009 Poland, Austria, Ireland, Czech Republic, and United Kingdom performed similarly to the U.S. but outperformed the U.S. in 2012.
  • The U.S. has fewer advanced students and more low performing students than most countries.
    • A smaller percentage of U.S. students (9 percent) scored within the top two PISA achievement levels than the international average (13 percent).
    • Twenty-seven countries had a higher percentage of high performing students. Shanghai-China led the world with more than half (55 percent) reaching these advanced levels followed by Singapore (40 percent), Chinese Taipei (37 percent), Hong Kong-China (34 percent), and Korea (31 percent).
    • The U.S. also had a larger proportion of low-performing students**(26 percent) than the international average (23 percent) and 29 counties had a lower percentage of low-performing students than the U.S.

Science Literacy

  • The U.S. did not score significantly different from the international average of 501.
  • The U.S. was outperformed by 22 of 64 other countries.
    • Shanghai-China was the highest performing country (580) followed by Hong-Kong-China (555), Singapore (551), Japan (547), and Finland (545).
    • The U.S. performed similarly to 13 countries including France, Italy, Norway, and Croatia.
    • The U.S. performed significantly better than 29 countries such as Russia (486), Sweden (485), Mexico (415), and Brazil (405).
  • Scores for the U.S. have not improved.
    • Scores for the U.S. were basically unchanged between 2009 and 2012.
    • The 2012 scores were also similar to the scores in 2000.
    • Twenty-two countries outperformed the U.S. in 2012 compared to 18 countries in 2009.
      • In 2009 Poland, Ireland, and the Czech Republic performed similarly to the U.S. but outperformed the U.S. in 2012.
  • The U.S. has fewer advanced students and more low performing students than most countries.
    • Seven percent of U.S. students scored within the top two PISA achievement levels which is similar to the international average.
    • Seventeen countries had a higher percentage of high performing students than the U.S. Shanghai-China led the world with 27 percent of students reaching these advanced levels followed by Singapore (23 percent), Japan (18 percent), and Finland (17 percent).
    • Twenty-one countries had a lower percentage of low-performing students than the U.S. However, the U.S. had a similar proportion of low-performing students (18 percent) than the international average.

Reading Literacy

  • The U.S. did not score significantly different from the international average of 496.
  • The U.S. was outperformed by 19 of 64 other countries.
    • Just like in mathematics and science Shanghai-China was the highest performing country (570) followed by Hong-Kong-China (545), Singapore (542), Japan (538), and Korea (536).
    • The U.S. performed similarly to 12 countries including France, Italy, United Kingdom, and Israel.
    • The U.S. performed significantly better than 34 countries such as Russia (475), Greece (477), Mexico (424), and Brazil (410).
  • Scores for the U.S. have not improved.
    • Scores for the U.S. were basically unchanged between 2009 and 2012.
    • The 2012 scores were also similar to the scores in 2000.
    • Ten more countries outperformed the U.S. in 2012 than in 2009.
      • In 2009 Poland, Ireland, Estonia, Switzerland, and Germany performed similarly to the U.S. but outperformed the U.S. in 2012.
  • The U.S. has fewer advanced students and more low performing students than most countries.
    • Eight percent of U.S. students scored within the top two PISA achievement levels which is similar to the international average.
    • Fourteen countries had a significantly greater share of high performers with Shanghai-China leading the world with 25 percent followed by Singapore (21 percent), and Japan (18 percent).
    • The U.S. also had a similar proportion of low-performing students (17 percent) than the international average although 14 countries had a higher percentage.

Demographics

  • The U.S. is not uniquely diverse.
    • The U.S. has about the same proportion of ‘disadvantaged’ students as the international average.
    • The U.S. has the 6th largest share of immigrant students.
    • When controlling for the socioeconomic status (SES) of students across countries the U.S. ranking would actually decline compared to other countries.

For more information about PISA and other international assessments of student achievement check out the Center’s More than a horse race: A guide to international tests of student achievement.

 

* The OECD average is used at the international average
** OECD used the term education systems instead of countries.
*** Students who scored below the 2nd PISA achievement level.






December 2, 2013

10 questions to understanding PISA results

The big day is almost upon us. Tomorrow the results from the 2012 Program for International Student Assessment (PISA) will be released. The rhetoric pertaining to the quality of our public schools is certainly going to be amplified tomorrow, with critics lamenting how the results show our public schools are in dire straits while others will argue the results are meaningless. To help you understand what the PISA results actually signify, the Center for Public Education has answered 10 key questions about what PISA actually measures and what the results mean for our public schools.

1. What is PISA?

The Program for International Student Assessment (PISA) is an assessment of reading, math, and science literacy given every three years to 15-year-old students in public and private schools in about 65 countries. The international institution Organisation for Economic Cooperation and Development (OECD) coordinates the development and administration of PISA worldwide while the U.S. Department of Education’s National Center for Education Statistics (NCES) conducts the assessments in the U.S.

Unlike most state assessments that measure how much knowledge a student has acquired, PISA is designed to measure how well students can apply their knowledge to real-world situations. To measure such skills, the test items on PISA are primarily “constructed response,” meaning the test-taker has to write their answers to the questions, and there are few multiple-choice items. U.S. students typically do not perform as well on open-ended, constructed response items. This is one reason many states are adopting new standards, including the new Common Core State Standards, which are intended to emphasize how well students can solve problems and think critically based on the concepts, topics and procedures they have learned.

2. Why are PISA results important?

PISA is one of the few tools we have to compare the outcomes of high school students internationally.  PISA provides valuable information on how prepared high school students are for postsecondary success whether in the workplace, career training, or higher education.

3. Is the U.S. ranking on PISA negatively impacted because unlike other countries the U.S. educates and tests all its students?

No, this used to be true several decades ago, but is no longer the case. Every industrialized country now educates all their students, including language minority, special needs and low-performing students. Every country that participates in PISA must adhere to strict sampling rules to ensure the country’s results are nationally representative of all 15-year-old students. Indeed, the decision to test secondary students at age 15 was made in part because young people at that age are still subject to compulsory schooling laws in most participating nations, which provides more assurance that PISA will capture the broadest sample.

4. Where does the U.S. really rank on PISA?

In 2009, 30 countries had higher mathematics scores than the U.S. but just 23 of these countries significantly outperformed the U.S. Because only a sample of each nation’s students participate in PISA, much like political polls, each country’s score has a margin of error. This means that the score is actually an estimate of how the country would perform if every 15-year-old took PISA. In science, 21 countries had higher scores than the U.S., but only 18 scored significantly higher; in reading, while 16 countries scored higher, just nine countries significantly outperformed the U.S.

OECD reports statistically significant differences in performance between nations, which is a more accurate way to look at PISA rankings than a straight listing of average scores.

5. Does PISA measure the effectiveness of public school systems?

Not completely, for three reasons: 1) PISA results are representative of the performance of all 15-year-olds in participating countries including those  attending private schools; 2) PISA makes no attempt to isolate schools from outside factors such as poverty or high proportions of non-native language speakers that may have an impact on  performance —such factors are important to include in the mix when evaluating the effectiveness of each country’s schools; and 3) No single measure can incorporate every outcome we expect from our public schools. To gain a better perspective of the overall effectiveness of educational systems, you should consider multiple measures. NSBA’s Center for Public Education’s Data First Data Center is a good resource to get you started when examining public schools in the U.S.

6. How does the U.S. stack up on other international measures?

The U.S. fares much better on other international assessments.  U.S. 4th and 8th graders performed among the top 10 countries in both math and science on the most recent Trends in Mathematics and Science Study, which was administered to more than 60 countries (TIMSS, 2011). Moreover, only four countries outperformed U.S. 4th graders in reading on the 2011 Progress on International Reading Literacy Study (PIRLS). Finally, U.S. students led the world in civics in 1999, the last year the CivEd was given. As of 2009, the nation’s 15-year-old students did not compare as well on PISA, especially in math and science. However, the U.S. performed better in reading by scoring among the “top 10.”

7. Has the U.S. shown improvement on PISA?

The U.S. saw a slight improvement in math scores between 2006 and 2009. It wouldn’t be surprising if such gains continued in 2012 as U.S. high school students continue to take more rigorous math courses. It is important to point out that the U.S. has demonstrated improvements on other measures since PISA was first given in 2000. U.S. 4th and 8th graders made among the greatest gains in math between 1995 and 2011 on TIMSS. The U.S. also made dramatic gains in on-time graduation rates by improving from 67 percent in 2000 to 75 percent in 2010 according to Education Week. Even on the National Assessment of Education Progress (NAEP), U.S. 4th and 8th graders have shown significant progress between 2000 and 2013, although high school students are not showing the same gains. The lack of progress on PISA appears to be the exception rather than the rule in terms of international comparisons.

8. How should the results be used?

We need to get beyond seeing PISA as a horse race by fixating on whether the U.S. finishes in win, place, or show. Instead, we need to see PISA results as an opportunity to assess if best practices in teaching and learning in other countries can also work for secondary schools here in the U.S. For example, we should  look at how much time other countries give teachers for professional development, how much they pay their teachers, how much time teachers spend in the classroom, how much flexibility exists at the local level, how special needs students are taught, and how much time students spend in school. Answers to these and others questions could be instructive for U.S. educators and policymakers. While PISA gives us an opportunity to learn from other countries it is important to keep in mind that just because a high-performing or high-gaining country does something does not mean it will work in U.S. schools.

9. Does poverty affect the U.S. performance on PISA more than in other countries?

Many analysts observe that poverty has a greater impact on student performance in the U.S. than elsewhere. For one thing, the U.S. has the highest child poverty rates among industrialized countries. For another, students in the U.S. who live in poverty tend to have less access to resources that research consistently shows impact student achievement, including highly effective teachers, access to rigorous curriculum, and high quality pre-k programs. Yet, poverty is just one of several factors that affect the standing of the U.S. In comparing the performance of top students around the world—where poverty is likely less of a factor—America’s top students still do not compare well to their peers in other countries. For example, in 2009 19 countries’ top students (scoring in the top 10 percent) outperformed the U.S.’s top students in science on PISA.

10. Are PISA results a precursor of America’s future economic competitiveness?

Our high school graduates’ preparation for postsecondary success certainly has some impact on the future economic competitiveness of the U.S. However, as stated in question 5, PISA is just one measure of high school students’ college and career readiness. In addition, many factors besides K-12 schooling contribute to the economic competitiveness of the U.S. and every other country, including, for example, a country’s monetary and fiscal policies. But for a country to maximize its economic output it needs a well-educated society which would lead to lower unemployment rates and less demand for government services. Stanford University Economist Eric Hanushek estimates that if the U.S. had scored 50 points higher on PISA in 2000 by 2015 GDP would be 4.5 percent higher than currently projected. Such an increase is the equivalent to the total expenditures on U.S. K-12 schools in 2015. Keep in mind, however, this does not mean that if the U.S. doesn’t improve on PISA that GDP will decline when our current high school graduates enter the workforce. However, it does show that education does affect future economic outcomes.






Older Posts »
RSS Feed