Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

December 7, 2016

PISA scores remain stagnant for U.S. students

The results of the latest PISA or the Program for International Student Assessment are in and as usual, we have an interpretation of the highlights for you.

If you recall, PISA is designed to assess not just students’ academic knowledge but their application of that knowledge and is administered to 15-year-olds across the globe every three years by the U.S. Department of Education’s National Center for Education Statistics (NCES) in coordination with the Paris-based Organisation for Economic Cooperation and Development (OECD). Each iteration of the PISA has a different focus and the 2015 version honed in on science, though it also tested math and reading proficiency among the roughly half-million teens who participated in this round. So, how did American students stack up?

In short, our performance was average in reading and science and below average in math, compared to the 35 other OECD member countries.  Specifically, the U.S. ranked 19th in science, 20th in reading and 31st in math. But PISA was administered in countries beyond OECD members and among that total group of 70 countries and education systems (some regions of China are assessed as separate systems), U.S. teens ranked 25th in science, 22nd in reading, and 40th in math.  Since 2012, scores were basically the same in science and reading, but dropped 11 points in math.

PISA Science

Before you get too upset over our less-than-stellar performance, though, there are a few things to take into account.  First, scores overall have fluctuated in all three subjects.  Some of the top performers such as South Korea and Finland have seen 20-30 point drops in math test scores from 2003 to 2015 at the same time that the U.S. saw a 13 point drop.  Are half of the countries really declining in performance, or could it be a change in the test, or a change in how the test corresponds with what and how material is taught in schools?

Second, the U.S. has seen a large set of reforms over the last several years, which have disrupted the education system.  Like many systems, a disruption may cause a temporary drop in performance, but eventually stabilize.  Many teachers are still adjusting to teaching the Common Core Standards and/or Next Generation Science Standards; the 2008 recession caused shocks in funding levels that we’re still recovering from; many school systems received waivers from No Child Left Behind which substantially change state- and school-level policies.  And, in case you want to blame Common Core for lower math scores, keep in mind that not all test-takers live in states that have adopted the Common Core, and even if they do, some have only learned under the new standards for a year or two.  Andreas Schleicher, who oversees the PISA test for the OECD, predicts that the Common Core Standards will eventually yield positive results for the U.S., but that we must be patient.

Demographics

Student scores are correlated to some degree with student poverty and the concentration of poverty in some schools.  Students from disadvantaged backgrounds are 2.5 times more likely to perform poorly than advantaged students.  Schools with fewer than 25 percent of students who are eligible for free or reduced price lunch (about half of all students nationwide are eligible) would be 2nd in science, 1st in reading, and 11th in math out of all 70 countries.  At the other end of the spectrum, schools with at least 75 percent of students who are eligible for free or reduced price lunch, 44th in science, 42nd in reading, and 47th in math.  Compared only to OECD countries, high-poverty schools would only beat four countries in science, four countries in reading, and five in math.

Score differences for different races in the U.S. show similar disparities.

How individual student groups would rank compared to the 70 education systems tested:

Science Reading Math
White 5th 4th 20th
Black 49th 44th 51st
Hispanic 40th 37th 44th
Asian 8th 2nd 20th
Mixed Race 19th 20th 38th

 

Equity

Despite the disparities in opportunity for low-income students, the number of low-income students who performed better than expected increased by 12 percentage points since 2006, to 32 percent.  The amount of variation attributable to poverty decreased from 17 percent in 2006 to 11 percent in 2015, meaning that poverty became less of a determining factor in how a student performed.

Funding

America is one of the largest spenders on education, as we should be, given our high per capita income.  Many have bemoaned that we should be outscoring other nations based on our higher spending levels, but the reality is that high levels of childhood poverty and inequitable spending often counteract the amount of money put into the system.  For more info on this, see our previous blogpost.






November 2, 2016

Thoughts on nuance and variance

As we approach the 2016 general election, I’ve heard public officials, family, and friends make very clear statements regarding which side of the aisle they support.  Yet, I find it hard to believe that the average American falls in line 100% with either political party, or supports every word and tenet of a particular public policy.  We are nuanced people.  Very few issues are as black-and-white as we’d like them to be.  Here’s a guide for things to consider when considering your stance on a particular issue, candidate, or political party, put in the context of educational issues.

  1. Most issues have an “it depends” clause.

With the onslaught of information available today, it makes sense that we want answers that are black-and-white.  The reality, though, is that there’s gray area for most policies and practices.  We also have to balance our ideological values with evidence.  Charter school proponents may believe in free-market values and choice to improve public schools through vouchers and charter schools, but I haven’t seen widespread evidence that choice in and of itself actually improves academic achievement or long-term outcomes in significant ways.  Yes, there are individual students who have benefited, but there are also individual students who have lost out.  Charter school opponents claim that taking away publicly-elected oversight through school boards is detrimental to the public’s ability to provide free and quality education to all.  Yet, the reality is that some public schools have dismal records, and charter or private schools have sometimes had success with the same students.  We have to acknowledge that we all want good things for our kids, and then use the evidence to figure out what that looks like without demonizing the other side.

  1. Most policies rely heavily on the quality of their implementation to be successful.

Common Core seems to be a prime example of this.  Two-thirds of Americans are in support of some sort of common standards across the country.  Yet, barely half of Americans are in support of Common Core.  Support for both questions have dwindled significantly from about 90% of public support in 2012.  Even presidential candidate Hillary Clinton has called the roll-out of Common Core “disastrous,” despite supporting them overall.

CommonCore

Source: http://educationnext.org/ten-year-trends-in-public-opinion-from-ednext-poll-2016-survey/

They were implemented quickly in many states, often without the curriculum materials or professional development to help teachers succeed in teaching the new standards.  While support for Common Core seems to be leveling off with teachers, who are most familiar with them, several states have repealed or are considering repealing the Common Core.  The new state standards that have been written in South Carolina and Indiana are extremely similar to the Common Core, which means that it may not be the concept or content that people disagree with so much as how they were implemented and the ensuing political backlash.

 

  1. Statistics usually tell us about an average (the typical student) but variance is also important.

Charter schools are a prime example of this.  On average, they have similar student achievement outcomes as traditional public schools.  But, there are schools that outperform their counterparts and schools that woefully underperform.  We have to think about those schools, too.

This is also clear in school segregation.  The average black student in the U.S. attends a school that is 49% black, 28% white, 17% Latino, 4% Asian, and 3% “Other,” but that doesn’t mean that every black student has this experience.  At the edges of the spectrum, however, 13% of U.S. public schools are over 90% black and Latino, while 33% of schools are less than 10% black and Latino.  To understand the reality, we need to look at the variety of students’ experiences (known in statistic-speak as “variance”) not just the average.

  1. There’s always room for improvement. “Fixing” a policy may mean making adjustments, not abandoning it altogether.

Student assessments under No Child Left Behind (2001) resulted in the narrowing of curriculum.  But, we also learned more about disadvantaged student groups and have continued closing the achievement gap for students of color.  Should we throw out testing altogether? Some would say yes, but most Americans say no.  Graduation rates, college enrollment, and achievement scores have all increased since NCLB passed in 2001.  What we can do is improve on student assessments.  Adjusting consequences for students, teachers, and schools could result in less narrowing of curriculum and subjects taught.  Involving more well-rounded tests that encourage creative and critical thinking would help teachers emphasize these skills in class.  Continued improvement in data use can help teachers and school administrators adjust their practices and policies to see continued student growth.  States have the power to make some of these changes under the new Every Student Succeeds Act without dismantling gains made under No Child Left Behind.






August 19, 2016

Too many tests in your district? You may need Assessment 101

2016-08-18_16-45-43Across the country, educators, parents, and students are expressing concerns about the amount of tests being administered in schools. To address this, we partnered with the education non-profit, Achieve, to support school board members and their districts in moving toward more coherent and streamlined assessment systems.

Recognizing the critical role school boards play in influencing systemic changes at the local level, CPE co-authored a training module for school boards based on Achieve’s student assessment inventory tool.

While the suite of resources is currently available for free here and here, CPE will continue its involvement by facilitating in-person and online training of the Assessment 101 process to participating school districts in three partner states, with the hope that these will lead to measurable reductions in testing there and in other districts who learn from and adopt this approach.






April 28, 2016

12th graders’ math scores drop, reading flatlines

And just when we had allowed ourselves to get giddy over record-shattering high school graduation rates.

NAEP, also known as the Nation’s Report card, released the results of its 2015 assessment of high school seniors’ math and reading proficiency. Like their 4th and 8th grade schoolmates, whose 2015 scores were published last fall, the nation’s 12th-graders either made no progress or dropped a few points, especially in mathematics. Worse, scores for the lowest performers fell the most in both subjects.

Let’s start with reading. The overall score was 1 point lower on the NAEP scale from two years ago, which is not a statistically significant change. However, 12th graders are performing 5 points lower compared to their peers in 1992, the first year the main-NAEP reading assessment was administered.

There was no noticeable change since 2013 in the scores of any racial/ethnic group, or in the achievement gaps between them.

Indeed, the biggest change was at the bottom. In just the last two years, the proportion of students who did not even read at the basic level grew, from 25 to 28 percent.  What this means in more tangible terms is that this group of soon-to-be-graduates cannot recognize the main purpose of expository text; cannot recognize the main purpose of an argument; and cannot explain a character’s action from a story description.

The math picture isn’t any rosier. The overall math score fell a significant 3 points on the NAEP scale. While this is still 2 points higher than in 2005 – the first administration of the test’s new math framework – it does represent a reversal after years of steady gains. As with reading, the math scores were relatively flat for every racial/ethnic group compared to 2013. One happy exception: scores for English language learners rose by 4 points.

Math also saw an increase of the wrong kind. A whopping 38 percent of high school seniors did not perform at the basic level in 2015, an increase of 3 points over 2013. This is troubling on its own merits. It is truly baffling when considering that 90 percent of seniors reported having taken Algebra II or a higher math course in high school.  We should see this group of low performers shrinking, not growing larger.

Of high interest to education policymakers and parents is the degree to which 12th graders are prepared for college work. Beginning in 2008, the National Assessment Governing Board, which oversees NAEP, commissioned several studies linking NAEP performance levels to college readiness. Based on the analysis, just slightly more than a third of seniors in 2015 scored at a level showing they had the knowledge and skills needed to succeed in freshmen courses. But ready or not, two-thirds of them will be bound for two- and four-year colleges the October following graduation.

Why is this happening? Many advocates have been quick to point to policies like Common Core, too much testing, not enough testing, or whatever other bee sticks in their bonnets. But as I have written elsewhere, there is not enough information at this point to lay the blame on any one of these, although they surely warrant watching. Likewise, some observers have noted the increase in childhood poverty, which also deserves attention.

I think another explanation might be found in one of our great successes. High school graduation rates have exploded in just the last 10 years. In 2013, 81 percent of all high school students graduated within four years. We know from research that failing grades are high risk factors for students. Up until recently, these low performers would have dropped out before showing up in the NAEP data as seniors. The fact that they are still in school is a good thing, but it may also be dragging 12th grade scores down.

The truth is, it’s too soon for us to know for sure why this happened. But there are enough questions that schools should be examining to get us back on the right track.

  • Do the high-level courses students are taking in larger numbers actually represent high-level content?
  • Do schools have enough counselors and other trained professionals to not just make sure students stay in school, but have the support they need to perform academically?
  • Are teachers also supported as they implement higher standards in their classrooms?
  • Finally, are federal, state and local policymakers providing the resources high schools need to assure every student graduates ready to succeed in college, careers and life?
Filed under: Assessments,CPE,High school,NAEP,Reading,Testing — Tags: , — Patte Barth @ 10:52 am





February 3, 2016

PARCC test results lower for computer-based tests

In school year 2014-2015, students took the Partnership for Assessment of Readiness for College and Careers (PARCC) exam on a pilot basis. The PARCC exam was created to be in alignment with the Common Core Standards and is among the few standardized assessment measures of how well school districts are teaching higher-level competencies.

On February 3, Education Week reported in an article that the results for students who took the computer-based version of the exam were significantly lower than the results for students who took a traditional pencil and paper version. While the article states that the PARCC organization does not have a response or clear answer on why this occurred, I will offer my own explanation based on my experience as a teacher of students who took this exam last year.

I taught high school History, and the largest discrepancy in the results between students who took the computer versus paper exam was at the high school level. This is my theory for the discrepancy. Throughout students’ academic careers we teachers teach them to “mark-up” the text. This means that as they read books, articles, poems, and primary sources etc. students should have a pen/pencil and highlighter in their hand. There are many acronyms for how students should “mark-up” their text. One is HACC- Highlight, Annotate, Circle unknown words, Comment. There are many others but the idea is the same. Students are taught to summarize each paragraph in the margins and make note of key words. This helps students to stay engaged with the reading, find main ideas, and critically think about what they are reading. It also makes it easier to go back and skim the text for the main ideas and remember what they read without re-reading.

Generally students are forced to mark-up/annotate the text in this way but, honestly, I still do this! And, I would bet that many adults do too. If you need to read a long article at work, many people print it out and read it with a pen in hand. It makes it easier to focus on what you are reading. Now imagine that someone is going to test you on that article. You will be even more anxious to read the article carefully and write notes for yourself in the margins.

The point is that students are taught to do this when reading, especially when reading passages for exams when there will be questions based on the passage. My own students had this drilled into them throughout the high school years when I knew and taught them. Sometime last year the teachers learned that our school would be giving the pilot version of the PARCC exam to our students. During a teacher professional development day we were asked to go online to the PARCC website and learn about the test and take a practice exam. I encourage you to go online and take it for yourself — this exam is hard! We were asked to analyze the questions and think about ways we could change our own in-class exams to better align with PARCC. We were told that it would soon replace our state’s standardized exam.

One of the first things we all noticed was how long the reading passages are for the ELA portion of the test. It took a long time to read through them and we all struggled to read it on a computer screen. I really wanted to have a printed version to write my notes down! It was long and detailed and I felt as though by the time I saw the questions I would have to re-read the whole passage to find the answer (or find the section where I could infer an answer). I knew the students would struggle with this and anticipated lower scores on this exam than the state test. I was thankful that their scores wouldn’t actually count this year. But what happens when this becomes a high-stakes test?

As I anticipated, the scores for students who took the computer-based exams were far lower than those who took a traditional paper test. The Illinois State Board of Education found that, across all grades, 50% of students scored proficient of the paper-based PARCC exam compared to only 32% of students who took the exam online. In Baltimore County, students who took the paper test scored almost 14 points higher than students of similar demographics who took the test on the computer.

The low scores on the test are a different story. Organizations will need to analyze the results of this major pilot test and determine its validity. Students and teachers, if it becomes mandatory, will have to adjust to better learn the standards and testing format associated with this test. The bigger story is that there are significant hardships that come with taking a computer-based test.

My main concern is the reading passages. I don’t believe teachers should abandon the “mark it up” technique to bend to computer-based testing because learning how to annotate a text is valuable throughout people’s lives. I saw the students struggle to stare at the computer screen and focus on the words. Many used their finger on the screen to follow along with what they were reading. It was clearly frustrating for them not to be able to underline and make notes like they were used to doing.

Other concerns are that this test is online. It requires access to the internet, a multitude of computers for students to test, and students and teacher who are technologically savvy. When my school gave the test, it took several days and a lot of scheduling and disruption to get all students to take the test given our limited number of computers. Certain rooms of the building have less reliable internet connection than others and some students lost connection while testing. Sometimes the system didn’t accept the student login or wouldn’t change to the next page. There were no PARCC IT professionals in the building to fix these issues. Instead, teachers who didn’t know the system any better than the students tried to help.

Not all students were ultimately able to take or finish the exam because of these issues. Thankfully their results didn’t matter for their graduation! There are also equity concerns between students who are familiar with computers and typing and those who do not have much exposure to technology. As a teacher in an urban school I can tell you that was not uncommon to see students typing essays on their phones because they didn’t have a computer.

As a whole, I’m not surprised by the discrepancy in test scores and I imagine that other teachers are not either. The Education Week article quotes the PARCC’s Chief of Assessment in saying “There is some evidence that, in part, the [score] differences we’re seeing may be explained by students’ familiarity with the computer-delivery system.” This vague statement only hits the tip of the iceberg. I encourage those analyzing the cause of the discrepancy to talk to teachers and students. Also, ask yourselves how well you would do taking an exam completely online, particularly when there are long reading passages. –Breanna Higgins

Filed under: Accountability,Assessments,Common Core,High school,Testing — Tags: , , — Breanna Higgins @ 4:27 pm





Older Posts »
RSS Feed