Yesterday, the National Center for Education Statistics (NCES) at the U.S. Department of Education released a new report, Mapping State Proficiency Standards onto NAEP Scales: 2005-2009. The report enables states to compare the rigor of their standards for proficiency in fourth and eighth grades in both math and reading to that of other states. To do so, it places each state’s assessment cut-score for proficiency — the score which students much reach to be considered proficient — onto NAEP’s scoring scale using statistical mapping techniques. This means it shows where on NAEP’s scoring scale a student would fall if that student scored right at the state’s cut-score for proficiency on the state assessment.
Example: If a fourth grader in Vermont scored at the proficient cut-score on the Vermont state assessment, that score would correspond to a score of 214 on NAEP, which falls within NAEP’s Basic Achievement Level.
What did the report find?
- The differences where states set their proficiency standards vary greatly.
- The difference in scores between the states with the five highest and lowest standards is comparable to the difference in scores between NAEP’s Basic and Proficient levels.
- The range of state standards is between 60 and 71 NAEP points, which equates to about six or seven years of learning. It is also more than twice the size of the Black/White achievement gap in 4th grade reading, which is 25 NAEP points.
- Most state’s proficiency standards are at or below NAEP’s definition of Basic performance.
- In grade 4 reading, 35 of 50 states set their standard for proficiency lower than NAEP’s cut-score for its Basic level. For grade 8 reading, 16 out of the 50 states did so.
- In grade 4 math, seven of 50 states set their score for proficiency below the cut score for NAEP’s Basic level, with 42 states setting their proficiency score within NAEP’s Basic level. One state—Massachusetts—set its proficiency score within NAEP’s Proficiency level. Similar results were found in at the 8th grade level.
- The rigor of state standards increased in states that substantively changed their assessments between 2007 and 2009.
- Across the 34 math and reading assessments that substantively changed between 2007 and 2009, in 21 cases the rigor of the standards increased.
- In just 5 cases did the rigor of the state standards decrease.
- Most state results show more positive changes in the proportion of students reaching proficiency than NAEP results.
- The change in the percent of students reaching proficiency between 2007 and 2009 was more positive in 17 of 22 state assessments than on NAEP.
Keep in mind when reading the report that NAEP does not necessarily define proficiency the same way states do. NAEP defines Proficiency as competency over challenging subject matter, not grade-level performance as states attempt to do. It is also worth mentioning that no country, not even the highest performing countries, would have 100 percent of their students reach NAEP’s Proficiency level. and that some leading assessment experts have stated that proficiency for accountability purposes probably lies somewhere between NAEP’s Basic and Proficient levels.
Even with that in mind, the results should be a warning flag to many states, especially those who set their proficiency standard below NAEP’s Basic level. But this could be a moot point in the coming years, as most states have signed on to the Common Core of Standards, where the goal is college and career readiness, not proficiency as both state assessements and NAEP are currently setup to measure. In the meantime, states should still ensure they set their proficiency standards at a level where students demostrate they have the skills necessary to get into college or get a good job after high school. – Jim Hull
For more information on how NAEP’s proficiency levels compare to states’, check out the Center for Public Education’s The proficiency debate: A guide to NAEP achievement levels.
The graph below from today’s report shows where your state’s cut-score for proficiency falls on the NAEP 8th grade math assessment score scale (Other grade and subjects can be found on pages 10, 11, and 12 of the report). Scores above 299 fall in or above NAEP’s Proficient level, while scores above 262 but below 299 fall within NAEP’s Basic level.