Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

October 28, 2016

NAEP science scores reveal progress at lower grades, stagnation at 12th-grade

The National Assessment Governing Board released its 2015 Science scores from the National Assessment of Educational Progress for fourth-, eighth- and 12th-graders.  The results were positive overall, with achievement gaps narrowing and scores improving for almost all student groups in fourth and eighth-grade students.  Twelfth-grade scores remained stagnant across all groups.

The tests assess students’ ability to identify and use science principles, use scientific inquiry, and use technological design in physical science, life science, and Earth and space science.  Student responses are a combination of written (including multiple choice and open-ended questions) and interactive computer and hands-on tasks.

Of great concern, however, are the persistent gaps between students of different races, genders, and education status (English Learners and Special Education students).  While these gaps are narrowing, we have to figure out how to provide greater opportunities to all students.

NAEP_Science_Gaps

http://nationsreportcard.gov/science_2015/files/overview.pdf

 

Some states are improving at faster rates than others (only noted if the state had a statistically significant change):

NAEP_Science_States

Source: www.nationsreportcard.gov

 

It seems that the gains made in fourth and eighth-grade scores erode by 12th-grade.  Some of this may be due to lack of access to engaging classes and curriculum that could draw students into STEM fields.

NAEP_Science_Teens

Source: http://www.amgeninspires.com/students-on-stem/

2015 NAEP Science scores show similar trends as math and reading tests, which emphasizes the question: How do we move the needle for 12th-graders, as well as continue to improve opportunities and achievement for all students?






July 10, 2015

‘Proficient’ in the eye of the beholder

While we often talk about the American educational system, in truth we have 50 systems, each with the latitude to define its own academic standards. A newly published analysis  by the National Center of Education Statistics shows just how widely those expectations for student learning differ among states. Moreover, the findings suggest that most states could be aiming too low.

For the last ten years, NCES has conducted periodic statistical analyses that map student proficiency on state tests to their respective performance on NAEP. This national assessment is administered in all states and it is, by large consensus, considered the gold standard both in the richness of content and the quality of the assessment itself. As such, states where their students perform at about the same level on the state test as they do on NAEP can be considered to have high performance standards.

Some partial findings:

  • Grade 4: Only two states (New York and Wisconsin) had state proficiency standards equivalent to NAEP-proficient in both reading and math; an additional three states (Massachusetts, North Carolina and Texas) were aligned with NAEP-basic in reading and NAEP-proficient in math. Four states (Alabama, Georgia, Idaho and Maryland) had proficiency levels aligned with NAEP-below basic. A whopping 22 states were in the NAEP-below basic rate in reading.
  • Grade 8: Only New York’s proficiency levels aligned with NAEP-proficient in both reading and math, while North Carolina and Texas were within NAEP-basic in reading and NAEP-proficient in math. Five states (Alabama, Connecticut, Georgia, Idaho and Ohio) were in the below basic range in both subjects. Unlike grade 4, only three states’ grade 8 performance (DC, Indiana and Mississippi) was at the NAEP-below basic level in reading. The majority of states were within the NAEP-basic range in reading and math.

Alert readers will note, of course, that some high-performing states like Connecticut and Maryland had proficiency levels that aligned with NAEP’s lowest performance designation. The analysis is, to be sure, an imperfect comparison. Even so, the relationship between state alignment to NAEP-proficient and their relative performance is fairly consistent, as you can see in the chart featured below as well as in the full report.

Despite the study’s limitations, NCES provides important context for states to help them gauge the quality of their standards. According to the Atlantic , Peggy Carr, NCES’s acting commissioner, explained to reporters that NAEP-proficient is considered to be at a level that shows students are on track to be “college-ready.” The most recent administration showed that only 35 percent of the nation’s fourth-graders performed at proficient or above on NAEP-reading; about the same proportion of eighth-graders (36 percent) were proficient in math. Clearly, we have our work cut out for us in order to meet the goal of all graduates prepared for college and careers.

The NCES study was based on 2013 data so it’s too early to see the impact of the common core standards and aligned assessments in those states that have adopted them. Several states that opted out, however, are also committed to the college and career-ready agenda. NCES’s next iteration of this series should, therefore, give us more insight into how well we are advancing.

NAEPmap

 

Filed under: Assessments,Common Core,standards — Tags: , , — Patte Barth @ 3:42 pm





May 14, 2015

Proficiency Rates Differ Between State and National Tests

Large gaps in proficiency rates still exist between state and national tests according to a new report by Achieve, Inc. It has been known for several years that more students reach the proficiency benchmark on their state assessment than on the National Assessment of Education Progress (NAEP), and that gap remains today. In fact, proficiency rates on most state assessments are 30 percentage points higher than they are on NAEP.  What this means is that if one of these states reported 80 percent of their students reached the proficiency benchmark on their state assessment, than just 50 percent likely reached it on NAEP.

In some states the gap was even larger. In Georgia, for example, the difference was 60 percentage points in 4th grade reading which was the largest difference in the country. In this case 94 percent of 4th graders were deemed proficient on the Georgia state assessment while just 34 percent reached the proficiency level on NAEP. Georgia wasn’t alone. Louisiana, Alaska, and Arkansas all had gaps of at least 50 percentage points. Similar results were found in 8th grade math as well.

However, there were states with small if any gaps. In fact, in New York more students were deemed proficient on NAEP than on the state assessment in both 4th grade reading and 8th grade math. The report also singled out a dozen or so states that had similar proficiency rates on their state assessments as on NAEP, or as the report called them the “Top Truth Tellers.”

The results aren’t entirely surprising. The Achieve report is based on results from the 2013-14 state assessments when nearly all states were still using older tests. Most states will be giving new Common Core aligned tests for the first time this year which will likely lead to lower proficiency rates as was seen in Kentucky and New York — states that have been administering Common Core aligned assessments for a couple years already. What will be interesting is how this analysis will look a year from now when state scores are based on more rigorous Common Core aligned assessments. I’m guessing the Common Core states will see their scores more aligned with NAEP while those who don’t will still have significant gaps. The question remains, will there be more pushback in states with lower proficiency rates or in those with larger gaps? I guess we will have to wait until next year to find out.—Jim Hull






June 28, 2013

Minority students make big gains on NAEP but gaps remain

Minority students have made significant gains over the past four decades in both math and reading, according to the 2012 long-term NAEP results. While most white students made significant gains as well, achievement gaps narrowed considerably since minority students made much larger gains than their white peers. However, large achievement gaps still remain.  

Reading Results

9 Year olds

  • U.S. 9 year old have made significant gains.
    • Since the first year of NAEP in 1971, student achievement in reading has increased significantly from 208 to 221 (13 points, or just over a year’s worth of learning). There was also significant growth from 2004 to 2012 (5 points), but it remained relatively flat from 2008 until the present.  
    • Gains were made by students at all performance levels.
      • Students scoring in the 10th and 25th percentiles each saw gains of 19 points, thus strengthening the lower percentile performance overall.
      • Students performing at the 50th, 75th, and 90th percentiles each saw gains from 1971 to 2012 by 15, 9, and 6 points, respectively.
      • These increases indicate an overall trend of improvement across all performance subgroups.
  • Racial achievement gaps have narrowed significantly over the past four decades.
    • The Black-White achievement gap narrowed from 44 points in 1971 to 23 points in 2012.
      • Black students increased there scores 36 points over this time period, while White students improved their scores 15 points.
    • The Hispanic-White achievement gap narrowed from 34 points in 1975 (the first year for which data was available for Hispanic students) to 21 points in 2012.  
      • Hispanic students increased their scores 25 points from 1975 to 2012, while White students nudged up 12 points in the same time period.
  • Nine year-olds were the only age group to see a significant decrease in the gender gap from 1971 to 2012.
    • In 1971, boys earned an average score of 201, while girls scored 214. By 2012, this 13-point gap shrunk to a 5-point deficit with boys scoring 218 and girls scoring 223.

13 Year Olds

  • U.S. 13 year olds have made significant long- and short-term gains.
    • Since 1971, student scores in reading has increased significantly from 255 to 263 (8 points, or nearly a year’s worth of learning). Scores also improved from 2008, the last time NAEP was administered.
    • Students made improvements in reading scores across the spectrum of performance levels, with significant gains from 1971 as well as short-term gains since 2008.
      • Lower-achieving students made the most modest gains (up 6 points from 1971), while each of the other higher-performing quintiles gained 8 or 9 points on average since 1971.
  • Racial achievement gaps have narrowed significantly between initial testing and 2012.
    • The Black-White achievement gap diminished from 39 points (1971) to 23 points (2012).
      • Black students increased their scores by 25 points (roughly 2.5 years of learning), while White students achieved a 9-point gain over this time.
    • The Hispanic-White achievement gap narrowed from 30 points in 1975 to 21 points in 2012.  
      • Hispanic students increased their scores by 17 points from 1975 to 2012, while White students achieved an 8-point gain over this time.  
  • The percentage of 13- and 17-year-olds who read for fun has diminished over time
    • The percentage of 13-year-olds reported they read for fun dropped from 35 (1984) to 27 (2012) percent, while 17-year-olds saw their percentages drop off from 31 (1984) to 19 (2012).   

17 Year olds

  • On average U.S. 17 year olds have made little progress since 1971.
    • Overall scores were not significantly different between the first NAEP reading testing in 1971 (score of 285) and 2012 (score of 287).
    • Lower performing students have made modest gains
      • Scores at the 10th percentile were 7 points higher in 2012 than in 1971.
      • Scores that 25th percentile increased by 4 points between 1971 and 2012, while scores at the 50th percentile increased by 1 point.
      • Students at the highest percentiles (75th and 90th) saw modest decreases in both long-term (since 1971) and short-term (since 2008) average scores.
  • Achievement gaps have narrowed significantly between 1971 and 2012.
    • The Black-White achievement gap narrowed by 27 points (from a 53 to a 26 point gap) between 1971 and 2012.
      • Black students increased their scores by 30 points (roughly 3 years of growth) since 1971, while White students saw a 4-point improvement.
      • Black students also showed short-term growth (from 2008) with a 3-point increase, while White students’ average reading scores remained constant.
    • The Hispanic-White achievement gap narrowed by 20 points (41 to 21 point gap) from 1975 to 2012, while Hispanic enrollment was rapidly expanding.
      • Hispanic students increased their scores by 22 points from 1975 to 2012, while White students saw only a 2-point gain in the same time period.

 

Math Results

9 Year olds

  • U.S. 9 year olds made significant gains.
    • Since the first year of NAEP in 1973, student achievement in math has increased by two and half years’ worth of learning (25 points).  However, there as been no significant improvement since 2004.
    • Similar gains were made by students at all performance levels.
      • In fact, students currently scoring at the 10th percentile score about the same as students at the 25th percentile did in 1973.
      • Furthermore, students currently scoring at the 75th percentile score about the same as students at the 90th percentile did in 1973.
  • Achievement gaps have narrowed significantly over the past four decades.
    • The Black-White achievement gap narrowed from 35 points in 1973 to 25 points in 2012.
      • Black students increased there scores by 36 points while at the same time White students improved their scores by 27 points.
      • Today’s Black students score as well as White students did in 1986.
    • The Hispanic-White achievement gap narrowed from 23 points in 1973 to 17 points in 2012 while the Hispanic enrollment increased from 5 percent in 1978 to 26 percent in 2012.
      • Hispanic students increased there scores by 32 points from 1973 to 2012.
      • Hispanic students score similarly as White students did in 1992.

13 Year Olds

  • U.S. 13 year olds have made significant long- and short-term gains.
    • Since 1973, student scores have increased by 19 points which is nearly two years’ worth of learning.  Scores also improved from 2008 the last time NAEP was administered.
    • While students at all levels made improvements, lower-achieving students made greater improvements.
      • Scores at the 10th percentile were 27 points higher in 2012 than in 1978.
      • While scores at the  90thpercentile increased 16 points between 1978 and 2012.
  • Achievement gaps have narrowed significantly between 1973 and 2012.
    • The Black-White achievement gap narrowed by 18 points (46 to 28 point gap).
      • Black students increased there scores 36 points while at the same time White students improved their scores 19 points.
      • Black students acquired about three and half more years of learning than they did in 1973.
    • The Hispanic-White achievement gap narrowed by 14 points (35 to 21 point gap), while the Hispanic enrollment increased from 6 percent in 1978 to 21 percent in 2012.
      • Hispanic students increased their scores by 32 points from 1973 to 2012.
      • Hispanic students acquired about three more years of learning than they did in 1973.
  • More 13 year olds are taking Algebra than ever before.
    • In 2012 34 percent of 13 year olds took Algebra compared to just 16 percent in 1986.
    • Nearly three-quarters of 13 year olds had taken at least Pre-Algebra in 2012, up from just 39 percent in 1986.

17 Year olds

  • On average U.S. 17 year olds have made little progress since 1973.
    • Overall scores were not significantly different between 1973 and 2012.
    • However, lower performing students have made modest gains.
      • Scores at the 10th percentile were 12 points higher in 2012 than in 1978.
      • Scores at the 25th percentile increased 11 points between 1978 and 2012 while scores at the 50th percentile increased by 6 points.
  • Achievement gaps have narrowed significantly between 1973 and 2012.
    • The Black-White achievement gap narrowed by 14 points (40 to 26 point gap) between 1973 and 2012.
      • Black students increased their scores 18 points while at the same time White students improved their scores 4 points.
      • Black students acquired about two more years of learning than they did in 1973.
    • The Hispanic-White achievement gap narrowed 14 points (33 to 19 point gap) while the Hispanic enrollment increased from 4 percent in 1978 to 22 percent in 2012.
      • Hispanic students increased their scores by 17 points from 1973 to 2012.
      • Hispanic students acquired nearly three more years of learning than they did in 1973.
  • Nearly four times as many students took Calculus or Pre-Calculus in 2012 than in 1978.
    • In 2012 23 percent of students took Calculus or Pre-Calculus compare to 6 percent in 1978. Just two decades ago just 10 percent did so.
    • In 2012 just 22 percent of students’ highest math course was geometry compared to 53 percent in 1978. In 1992 44 percent of students did so.

For more information on NAEP, check out the Center’s report The Proficiency Debate: A guide to NAEP achievement levels.






March 7, 2013

John Stossel, funky charts and Simpson’s paradox

John Stossel was on Fox and Friends this morning to promote an upcoming show about public schools. Remember, this is the guy who gave us Stupid in America — his ABC documentary from a few years back about our allegedly failing schools. During his segment, he claimed that “America has tripled spending, but test scores haven’t improved.”  The culprits? Teachers unions, school boards and other unnamed bureaucrats. Viewers were then shown a graph that indeed featured a flat line representing test scores over 40 years (improvement 1 point) with a second line escalating to $149,000 over the same period. The source was given as NCES. This got my fact-checking synapses sparking.

While I could not find the exact graph they showed on TV, Stossel did post this rather snazzy display on his blog with the same data:

Go ahead and take a moment to admire the work of the Fox News graphics department. Ok, now let’s talk data. This chart shows scores for three subjects (math, reading and science) and dollar figures (the “cost of education”) from 1970 to 2010. While not noted, I’m assuming the data source is still NCES.

This may get a little wonky, but stay with me.  NCES reports trend data over four decades for only two tests:  the National Assessment of Educational Progress (NAEP) Long-Term Trends (LTT) and the SAT. NCES also has international test scores, but that data only goes back to the 1990s so that couldn’t be what Stossel used.  The SAT does not assess science, which leaves NAEP LTT as the only possibility. It’s not a perfect match. The last NAEP LTT administration was in 2008 although Stossel’s chart shows data to 2010. But I’m going to assume that he fudged a little on the timeframe because nothing else qualifies.

NAEP LTT is given to a representative sample of students age nine, 13 and 17. I’m also going to assume that his analysis is based on 17-year-olds because the data matches his in reading and comes closest in math (more on this later).  Between 1971 and 2008, LTT reading scores for 17-year-olds have been relatively flat, posting an increase of just 1 point (not 1% as shown on Stossel’s chart, but we’ll blame the designer for that common mistake).  Here’s what it looks like:

Now let’s have some fun. Let’s look at the same test scores disaggregated by race and ethnicity:

Note that every group improved more than the overall score did: White 17-year-olds by 2 points with their Black and Hispanic classmates gaining a whopping 25 and 17 points respectively. This gives me a chance to talk about Simpson’s paradox.  The paradox occurs when “a trend that appears in different groups of data disappears when these groups are combined, and the reverse trend appears for the aggregate data.”  In this case, the overall trend for 17-year-olds is flat while each group gained, some groups by a lot. The reason is that the distribution of racial/ethnic groups has changed significantly between 1975 and 2008. Here is the distribution of the NAEP samples for the two years:

The proportion of Black and Hispanic 17-year-olds is larger while the proportion of White students in 2008 is 25 percentage points lower than it was in 1975. Even though Black and Hispanic performance also increased by a lot, they were still lower-performing than their White peers in 2008. Thus, all groups gain, but when their performance is combined the overall trend is flat.

Clearly, no one would argue that an achievement gap, though improving, is acceptable and we can move on to other things. But it’s just as absurd to look at these gains and find evidence of failing schools, as Stossel does.  And the absurdity doesn’t end there. Stossel, in turns out, is a master cherry picker of data. Let’s look at the rest of NAEP Long Term Trends:

  • Reading, 13-year-olds, 1971-2008: Overall scores +12; Black students +23; Hispanic +24.
  • Reading, 9-year-olds, 1971-2008: Overall +5; Black +21, Hispanic +10.
  • Mathematics, 17-year-olds, 1978 (first year tested)-2008: Overall +6, Black +19, Hispanic +17.
  • Mathematics, 13-year-olds, 1978-2008: Overall +17, Black +32, Hispanic +17.
  • Mathematics, 9-year-olds,  1978-2008: Overall +24, Black +32, Hispanic +30.

Notice a pattern?  If one were to apply Stossel’s grossly oversimplified analysis of education cost to scores — and I’m not saying you should — but if you did, you would have to say our public schools are producing a return on our investment.   Then again, how he got those cost figures is another topic for another day.

Filed under: Achievement Gaps,Data,Demographics,Public education — Tags: , , , — Patte Barth @ 2:46 pm





Older Posts »
RSS Feed