Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

October 17, 2011

Others agree, Fordham’s claims about high achievers not supported by data

Filed under: Achievement Gaps,research — Tags: , , , — Jim Hull @ 10:27 am

Last month I wrote about how the Fordham Institute’s claim that our nation’s high achievers are losing ground wasn’t supported by evidence.  Well, it is good to know I am not alone. First, the National Education Policy Center (NEPC) supported my critique that the data didn’t back up Fordham’s claim– but then again NEPC disagrees with just about everything Fordham says. Then the Center for American Progress (CAP)—which agrees with Fordham on several issues–released a critique of the Fordham report that raised similar concerns about the conclusions as I did. CAP’s main criticisms were:

1.     Fordham claimed that the federal No Child Left Behind law might have caused high-flying students to do worse over time. All of Fordham’s data, however, came from the post-NCLB time period. Without a pre-NCLB comparison, there is no way to make a claim that NCLB caused the decline.

2.     The report fails to acknowledge the true consequences of poverty on student achievement. The Fordham researchers note that “high achievers in high-poverty schools grew slightly less than those in low-poverty schools,” but use this finding to argue that poverty is not a strong predictor of student progress. Ample evidence proves, however, that low-income children need more resources in order to overcome the disadvantages they bring with them to school.

3.     A broader look at the data suggests that the nation’s top students have actually been gaining ground in a number of areas. For example, from 2000 to 2009, the percentage of eighth graders scoring at the highest level in math jumped 3 percentage points on the National Assessment of Educational Progress.

I’d have to agree with CAP on each of these points. It will be interesting to hear Fordham’s rebuttable when they host a conference on their report that includes one of the CAP authors on Monday October 17th. I will certainly be watching. – Jim Hull





October 13, 2011

Using growth in NCLB’s reauthorization

Filed under: Achievement Gaps,Assessments,Growth Models,Testing — Tags: , , , — Jim Hull @ 4:30 pm

On Wednesday, Senator Harkin released his bill to reauthorize the Elementary and Secondary Education Act (ESEA), better known as No Child Left Behind (NCLB). I haven’t read the bill yet, but I have read that Senator Harkin is proposing to drop the current Adequate Yearly Progress (AYP) requirement and instead evaluate schools based on “continuous improvement.” This would mean that all students are no longer expected to be proficient by 2014; instead, they are expected to make a certain amount of academic gains from year to year.

Sounds simple enough. Critics and proponents of NCLB alike have been pushing for the inclusion of a measurement of student growth since NCLB was enacted nearly a decade ago. However, back in 2002 less than a handful of states had the assessments and the data systems in place to measure how much academic gains individual student made from year to year. Now, thanks to NCLB, all states have the capacity to make such calculations. Almost all would agree that including such measures would greatly improve the fairness of any accountability system.  

Yet, incorporating student growth into a federal accountability system is not as straightforward as it seems. First of all, as my report Measuring Student Growth illustrates, there is no single method to measuring student growth. Choosing which method is best depends on the data available and how the data is going to be used. For example, a growth model, which identifies students who are not gaining as much as similar students, will look a lot different than a model that is used to identify students who gained enough in the past year to be on track to reach a certain benchmark such as being college and career ready when they graduate high school.

So before a growth model is used for accountability, policymakers need to state a clear purpose for what the growth data is to evaluate. For example, is the purpose to ensure schools are closing achievement gaps? Is the purpose to ensure all students are college or career ready by the end of high school? Or is it to identify schools where students are making fewer gains than students in schools with similar student populations?  For each of these questions, an adequate answer would require a different growth model.

Second of all, simply moving from a proficiency-based accountability system such as NCLB to a continuous improvement based system as proposed by Senator Harkin overlooks the fact that most state assessments are not designed to effectively measure student growth from grade to grade. Most states have developed their assessments to evaluate if a student is proficient or not proficient each year. As such, many state assessments are unable to reliably determine how much a student has learned from year to year, especially a student who scored at the very high or very low end of the test’s achievement scale. Yes, states can calculate a growth measure using the assessments they now have in place, but in many cases the result will not be as accurate as if the assessments were designed specifically to measure student growth.

These are just two major issues when it comes to including a growth measure for federal accountability. Yes, evaluating schools based on student growth is much fairer than how schools are currently evaluated under NCLB. But just simply including a growth measure does not automatically make it a better accountability system. Policymakers need to set a clear purpose for accountability systems and then incorporate a growth model that would best evaluates whether schools are meeting their goals. From what I know now about the Harkin bill, it is not clear whether the purpose is to ensure all students are college or career ready or if all students are making a year’s worth of gains. Without having a clear purpose for what to hold schools accountable for, adding a growth model will not be any fairer than NCLB. – Jim Hull





October 5, 2011

How does your district compare to Finland?

Filed under: International Comparisons,Public education,Report Summary,research — Tags: , , , — Jim Hull @ 8:24 pm

Last week the Web site Global Report Card (GRC) was launched by the George W. Bush Presidential Center. It enables the public to compare their school district’s academic performance in math and reading to that of students in 25 developed countries around the world, including top-achieving Finland, Canada, Japan, and Singapore.

Although the Web site is easy to use, actually making such comparisons is not. There are significant limitations in making fair comparisons of districts across states, never mind across countries. However, Jay Greene and Josh McGee, who created the GRC, have postulated that their comparisons of Boston to Finland (for example) are fair and reliable.

You can put me in the skeptical camp on this one. Not only are they comparing results across countries, they are doing it across grade levels as well. For U.S. school districts, they use scores from state assessments from all tested grades, which is grades 3 through 8 and 10th grade in most states. Other countries’ results are based on the international assessments in which they participated, which would at most include 4th, 8th and 10th grade in math and 4th and 10th grade in reading.

Keep in mind different assessments with significantly different purposes and given in different years were used in different grades and subjects. For example, 4th and 8th grade math scores are derived from TIMSS, which is designed specifically to measure how well students have learned what they were expected to be taught in school. Tenth grade reading and math scores come from PISA, which measures how well students can apply their math and reading knowledge to real life problems, no matter if they attained that knowledge in school or not. To add even more complexity to the comparisons, not all 25 countries participated in each of the assessments at each of the grade levels. Hence, districts’ results on their state assessments across multiple grade levels are compared to each country’s results across different assessments that not all comparable countries took part in.

You may remember I was skeptical in a post earlier this year of another report that compared the U.S. to other countries and that comparison was based on one grade level, in one year, on one assessment for each country. And then countries were only compared to U.S. states who had only taken one assessment, in one grade level, in one year. A far more straightforward comparison than the GRC, yet still statistically questionable.

Both report cards, however, attempt to make important comparisons that – if fair and reliable — would provide valuable information on how our students compare to their peers in other countries. Yet, we don’t know how reliable the comparisons actually are, especially at the district level, where smaller districts appear to have a distinct advantage over larger districts with similar demographics.

However, maybe the GRC with all its question marks will lead to accurate international comparisons at both the state and district levels. Because it really is an important question to answer whether our students in our best districts are as prepared as students in the highest performing countries. The answer could have a tremendous impact on the focus of our education reform efforts. – Jim Hull





September 23, 2011

Are our top students being left behind?

Filed under: Achievement Gaps,Public education,Report Summary — Tags: , , , , — Jim Hull @ 1:35 pm

It’s déjà vu all over again. Back in 2008 the Fordham Institute claimed in this report that our nation’s best students were being hurt by current education reform efforts, particularly NCLB. Fast forward to earlier this week where Fordham released another report to once again try to show that our education reforms are being targeted at our low performing students at the expense of our top students. The similarities don’t end with both studies examining the performance of high achieving students. In both reports Fordham’s conclusions don’t fit what their own data says.

In the 2008 study Fordham argued our top students were being left behind because their gains were not as large as the gains low performing students made post-NCLB. I argued then that their own data didn’t fit their claim. Once again, Fordham’s claim that our top students are being left behind doesn’t fit their own data. As a matter of fact, according to Fordham’s report the gap in math scores between low- (those scoring below 10th percentile) and high-performing (those score above the 90th percentile) did not significantly change as students moved from 3rd to 8th grade or from 6th to 10th grade. The good news is that all students made consistent gains. Unfortunately for low-performing students, their performance still lagged way behind. The story is a bit different in reading where gaps did close between the lowest and highest performing students. However, Fordham sees this gap closing as a negative even though high performing students continued to make significant gains between the 3rd and 8th grades.

Just as I argued in 2008, this is how gaps should be narrowed, where everyone improves but the lowest performers improve at a faster rate. However, Fordham didn’t agree with me then and I’ll safely assume they won’t agree with me now. We will just have to agree to disagree because I don’t believe the data shows our best students are being short changed simply because our lowest performers are making more progress than our highest performing students.

Now that doesn’t mean our schools or our education policies should focus solely on our lowest performing students. Educators and policymakers need to ensure that all students have an opportunity to reach their highest academic potential before they go onto college or the workplace. Yet, neither Fordham study provides compelling data that our schools are short changing our highest performing students.

Yes, educators and policymakers need to focus on our highest achieving students. International test scores show we have a much smaller proportion of advanced students than the leading countries such as South Korea and Finland. But the same international tests show we also have a much larger proportion of very low performers than most other industrialized nations. And students with such low achievement have little chance to go onto any sort of postsecondary education or find a good job that pays a living wage and offers benefits. So we need to at least sustain the gains our highest achievers are making since many will be our country’s future innovators, policymakers and business leaders. At the same time, we need to accelerate the gains our lowest achieving students are making so they at least have the minimal skills necessary to either go onto earn some sort of postsecondary degree/certificate or find a good job. Doing so is not a zero-sum game. If we provide our teachers with the training, resources, and support they need, they can improve the performance of all students. – Jim Hull





January 11, 2011

Maryland schools take top honors again

Filed under: Achievement Gaps,college,funding,Public education,Report Summary — Tags: , , , — Jim Hull @ 4:02 pm

This morning, Education Week (EdWeek) released its annual special report Quality Counts 2011, which included its annual State of the States report card. Maryland once again earned top honors with a B plus while the nation as a whole received a C. The report card shows that states have been very active in easing the transition for students from high school to college by defining college readiness. However, defining such goals for achievement at the end of high school has yet to significantly improve student achievement.

Here are some of the key findings from this year’s report card:

Summative Grades

How did the nation as a whole and each individual state perform across all policy and performance areas?

  • Overall, the nation received a grade of a C across all policy and performance areas, which remained the same as a year ago.
  • Maryland earned the highest grade (B plus) for the third consecutive year, followed by New York and Massachusetts who both earned a B. Nine states earned a B minus.
    • The vast majority of states earned grades between a C plus and C minus.
    • No states received an F, but Nebraska, South Dakota and the District of Columbia each earned a D plus.

Chance for Success

What are the odds that the average child who grows up in a particular state will do as well as the average child in the top-ranked state, at each stage of his or her educational life? (the early childhood years, participation and performance in formal education, and educational attainment and workforce outcomes during adulthood)

  • Massachusetts ranked first for the fourth consecutive year by being the only state to receive an A, while once again Connecticut, New Hampshire, and New Jersey each received an A minus. 
    • This means that children in Massachusetts have the best chance of achieving positive life outcomes.
  • On the other hand, children in Nevada and New Mexico have the least chance of achieving positive life outcomes by earning a D and D plus, respectively.
  • The nation as a whole earned a C plus.

K-12 Achievement

How do states compare on the academic achievement of their students in elementary through high school?

  • Overall, our nation’s schools received a D plus in the academic preparation of our school children.
    • The grade is based on the academic status and growth over time in math and reading scores, narrowing of poverty-based achievement gaps, as well as high school graduation rates and the performance on the advanced placement test.
  • Once again Massachusetts received the highest grade with a B, although Maryland and New Jersey performed nearly as well by earning a B minus.
  • New Mexico, Louisiana West Virginia, Mississippi, and the District of Columbia all received failing marks.

Transition and Alignment

How do states compare on implementing various education policies to better coordinate the connections between K-12 schooling and other segments of the education pipeline such as early-childhood education, college readiness, and links to the world of work?

  • Arkansas, Maryland, Tennessee, Texas, and West Virginia are leaders in ensuring students are ready to move up the education ladder, earning A’s for their policy work in this area.
    • Each of these states have implemented at least 12 of 14 alignment policies tracked by EdWeek.
  • In contrast, Montana and South Dakota have implemented just three such policies, earning them a D minus, while Nebraska earned an F for implementing just two policies.
  • Although the nation as a whole earned just a C plus, 33 states have defined college readiness, which is an increase of 13 states since 2009. 

School Finance

How much do states spend on their schools? Is the spending distributed equitably?

  • Although no state received a failing grade, Wyoming was the only state to receive an A minus for adequately funding its schools.
    • On the other hand, five states — Idaho, Mississippi, Nevada, Tennessee, and Utah received grades of D or D minus.
  • As a whole, the nation received a C on funding education. However, on average the nation spends more money on wealthier school districts than poorer districts nationwide.
    • Wyoming, New Jersey, Alaska, Kansas, Nebraska, Utah, and Nevada are the only states to spend as much or more on their poorer districts than their wealthier counterparts.
    • On average, across the nation over $4,000 more is spent per-pupil in each state’s wealthiest districts than in their poorest districts.
  • When adjusting for regional cost differences, the U.S. spends on average $11,223 per pupil.
    • This ranges from $17,114 (adjusted for regional cost differences) in Wyoming to $6,525 in Utah.

–Jim Hull





Older Posts »
RSS Feed