Learn About: 21st Century | Charter Schools | Homework
Home / Edifier

The EDifier

October 31, 2011

Merit pay revisited- Is Denver’s pay for performance a model plan?

Although it remains a controversial issue, merit pay has long since evolved from the days when test scores were the single factor in determining whether a teacher would get paid for performance.  Nowadays a number of school districts across the country have developed multi-pronged plans aimed at equitably rewarding teachers for their accomplishments.  Nonetheless, the question still remains: Is there actually a way to fairly reward a professional who deals with the advancement of human capital?  No plan is perfect, but one district might have come close.

In 2009, The Center took a look at merit pay and made mention of Denver’s ProComp Pay for Performance plan.  Now, a three year study, conducted by Dan Goldhaber and Joe Walch of the Center for Education Data and Research, has come out.  The study was conducted between the fall of 2006 and spring of 2010 on Denver’s ProComp plan. Denver Public Schools (DPS) requires all teachers who were hired in 2006 or later to be a part of the ProComp plan and gives veteran teachers the choice whether to opt in or not.  ProComp offers teachers four opportunities to receive bonuses, which include:

  1. Knowledge and Skills: Teachers may earn pay for completing one professional development unit per year (and can bank extra PDU’s), getting advanced  degrees and licenses, and can even receive tuition and student loan reimbursement (50 to 65 percent received this pay)
  2. Comprehensive Professional Evaluation: Based on principal evaluations, which are every 1 to 3 years (5 to 14 percent received this pay)
  3. Market Incentives: Aimed at teachers who work in hard-to-serve schools and/0r in hard-to-staff subject areas, as reviewed by school demographics and market supply (35 to 65 percent received this pay)
  4. Student Growth: Teachers set up student growth objectives, based on what they expect students to learn, which are approved by the principal (example: I expect x number of students to  exceed expectations in Reading on the Colorado Student Assessment Program (CSAP).) (70 to 80 percent received this pay)

The study suggests that the ProComp plan made teachers feel more supported and in turn, allowed them to more consistently meet their goals (Robles 2011).  In fact, between 2006 and 2010, 15 percent of the non-ProComp teachers even switched over to join the plan after seeing the positive results ProComp had on their schools and colleagues.  Not only has ProComp made the teaching profession more attractive, Goldhaber and Walch conclude that:

  • There were significant learning gains across grades and subjects;
  • The benefits of tracking data and evaluating educators spread from ProComp teachers to the entire district;
  • There was an expectation that the program would cause a negative atmosphere between team members but the opposite actually occurred and role models were bred;
  • ProComp teachers’ students had larger than expected gains on the state assessment.

Skeptics argue that these rewards focus more on classroom instruction than student test achievement and that ProComp is inconsistent with the value-added approach. Goldhaber and Walch point out that, “whether this is good or bad is clearly a normative question” but that “overall, ProComp has had a positive effect.”  They also suggest that states might want to consider investing in similar programs, especially for their Race to the Top objectives. Yesenia Robles of the Denver Post notes that ProComp has helped propel infrastructure reforms to change recruitment practices and enhance methods of data gathering.  She goes on to point out that the difference between non-ProComp and ProComp teachers’ student growth objectives are comparable to the difference between a first and second year teacher’s.   Her article, DPS Teacher-Pay System Likely Boosting Student Achievement, Study Finds, also points out that Denver Public Schools has retained 160 more teachers per year since 2006 and that 80 percent of all DPS teachers currently participate in the program.   Robles notes that, “The ProComp system is already in the process of changing with the implementation of the district’s evaluation-and-support system, known as LEAP, now being tested in 94 percent of DPS schools.”  Right now it is still too early to tell if ProComp can survive these alterations. 

ProComp is an even-handed, well-formed pay for performance plan that other districts can use as a model and will hopefully emulate.  The research shows that ProComp was not only received well by DPS teachers but most significantly, student success consistently progressed. –M. Newport

(To see whether similar pay for performance plans have been successful, check out this ECS report.)

July 22, 2011

New York City stops pay for performance

The New York Times is reporting that a “program that distributed $56 million in performance bonuses to teachers and other school staff members over the last three years will be permanently discontinued” because a study by the RAND corporation found that “the bonuses had no positive effect on either student performance or teachers’ attitudes toward their jobs.”

Before you react, consider this encouraging quote from the article: 

“The results add to a growing body of evidence nationally that so-called pay-for-performance bonuses for teachers that consist only of financial incentives have no effect on student achievement….Teachers also reported that improving as teachers and seeing their students learn were bigger motivators than a bonus, [the researchers] said.”

In an arena where attacks can often be cloaked under the phrase “It’s for the sake of the children,” it’s refreshing to see tangible evidence that children’s learning truly is a powerful motivator. And some of the other reasons given by the researchers for the program’s ineffectiveness, such as not understanding how the awards were determined, also point out that the logistical can be just as important (and far less divisive to fix) than the ideological.

In fact, several of the results of the RAND study seem to correlate with the Center’s earlier research review, “Promise or peril? Teacher pay for performance,” which pointed out that it is often how these programs are developed and structured that makes a difference.

So if your state or district is considering pay for performance, educate yourself first. There’s no use spending money on something that doesn’t work. —Rebecca St. Andrie

Filed under: Pay for Performance,teachers,Uncategorized — Tags: , , — rstandrie @ 10:01 am

April 26, 2011

Here comes another one

This time it’s Ohio: Its S.B. 5 would cut automatic pay raises for public school teachers and replace it with a statewide pay-for-performance system based on teachers’ evaluations. According to cleveland.com, fifty percent of a teacher’s evaluation would come from a yet-to-be-developed test measuring students’ academic growth over a year (commonly known as a growth model).

The test still has to be developed, there’s the possibility of a voter referendum on the bill — there’s a lot to happen in Ohio before this becomes reality. But it’s another example of how teacher evaluation systems are getting a major overhaul. Are you ready for it to happen where you live?

Is Ohio’s proposal right? Is it fair? Is it practical? Judge for yourself, but make sure to read the Center’s guides on teacher evaluation systems and pay for performance systems first.  –Rebecca St. Andrie

Filed under: Pay for Performance,Teacher evaluation,Uncategorized — rstandrie @ 11:48 am

January 7, 2011

What does baseball have to do with evaluating teachers?

“Look on the back of their baseball card.”

That’s the common reply from baseball managers and general managers around the major leagues when asked about what kind of production they expect from a player in an upcoming season.

It is a reference to the career stats of baseball players, which appear on the back of their baseball cards. Most often, at the end of the season a player’s stats will be more or less as their career stats.

This basic assumption is so accepted in Major League Baseball that teams offer tens of millions of dollars to those players with the strongest career stats, while players with the weakest stats tend to disappear after a few years.  The stakes don’t get much higher than that.

But in education, it’s still rare for similar high stakes decisions about school personnel (teachers, principals, and other administrators) to be made using data such as student test scores. Critics argue that evaluating educators primarily based on student test scores would be unfair. That even the most complex statistical models designed to isolate educators’ impact, such as value-added growth models, are not reliable enough to base high-stakes decisions such as salaries or tenure.

Now, these critics argue correctly that there is only a moderate correlation between a teacher’s past performance, as measured by value-added models, and their future effectiveness. For example, a Brookings report on value-added models stated that the correlation between test-based measures of teacher effectiveness between one school year and the next is between .30 and .40 (1.0 would mean a perfect correlation and 0.0 would be no correlation). This is a low to moderate correlation in statistical terms. Sounds like the critics have a strong point.

However, Brookings also points out when comparing that correlation to statistics used in other professions, such as Major League Baseball, the correlation doesn’t sound very weak.

Take batting average—one of the most widely used statistics to evaluate a player. The between-season correlation for batting averages for Major League Baseball players is just .36. Yet, a team will pay a player millions for hitting .300 and cut a player for hitting .250.  

Of course batting average is just one statistic teams use to evaluate a player and value-added scores are just one statistic that could be used to evaluate educators, but both can be key measures to determine how effective they will be in the future if used correctly.  

Do teams sometimes make the wrong decision by paying millions for the player who hit .300 last season while cutting the player who hit .250? Yes, it does happen. But overall, the teams are better off signing the players who had the strongest stats on back of his baseball card. Our schools could be more effective if they kept the educators with the strongest stats as well.– Jim Hull

Filed under: Growth Models,Pay for Performance,teachers — Tags: , , — Jim Hull @ 12:50 pm

December 22, 2010

Be bold, but maybe not too bold

Across the country, policies to link teachers to student performance are on a fast track. Spurred by the competition for federal Race to the Top and Teacher Incentive Grant dollars, several states changed laws and regulations that govern the use of teacher-student data in order to put themselves in a favorable position to win the grants. At the same time, numerous urban districts are at work with teachers to implement pay for performance plans, often with the aid of sizable private foundation support. 

The focus on teachers is well-placed. Of all the factors that influence student learning, research shows that teachers are the most significant. Yet, with some notable exceptions, teachers  haven’t exactly embraced the idea of being judged by the performance of their students. Many aren’t convinced that the data is valid, and the effort remains controversial.

Two recent reports shed valuable light on the teacher-student link and show how complex the task of teacher evaluation is.  The sources themselves are interesting — One report was published by the Gates Foundation and the other by Center for American Progress. Both organizations are high-profile advocates for using value-added measures of student growth to identify and reward effective teachers as the key strategy in school improvement plans. Both reports see a lot of value in value-added, especially when used for school and staff improvement.  But while neither report flat out says “don’t do it” they also advise care when using student performance for high stakes decisions about teachers.

 The Gates study reports the preliminary findings from their multi-year “Measuring Effective Teaching” project. The executive summary is here.  Among the more interesting findings: classroom observations and student perceptions have strong correlations to teacher value-added data. If this relationship holds, this could go a long way toward building up teachers’ trust in the data as districts move forward with teacher-student links.

The Center American Progress report is by Dan Goldhaber – a widely respected researcher of teacher effectiveness — who draws on his considerable background to explore the uses of value added data in high stakes decisions. Goldhaber provides a good, accessible description of the quirks in value-added models and shows how to improve the stability of the data, for example, by using multiple years of data.  He also compares value-added measures to current methods for evaluating  teachers, which he argues do little or nothing to distinguish good teaching from mediocre or just short of bad teaching, and finds value-added to be the better option.  

 My takeaway – we should keep moving ahead to develop effective and fair means to link teachers with student performance. But we might want to ease off of the accelerator a bit. 

Also check out the Center’s report, Promise or Peril: Teacher pay for performance plans. – Patte Barth

Filed under: Growth Models,Pay for Performance,teachers — Tags: , — Patte Barth @ 3:34 pm

« Newer PostsOlder Posts »
RSS Feed