Learn About: 21st Century | Charter Schools | Homework
Home / Edifier

The EDifier

September 18, 2014

PDK/Gallup poll Part 2 shows teachers matter

The folks at PDK and Gallup apparently had so much to report in this year’s annual poll of public attitudes toward public schools, they had to release it in two parts. Part 1, which we summarized here, addressed the Common Core state standards and perceptions about public schools more generally. Part 2, released this week, focuses primarily on public attitudes about the teaching profession. What they have to say should provide comfort to beleaguered teachers.

First, nearly two-thirds of the public expresses “trust and confidence in the men and women teaching children in the public schools.” The study’s authors note that this represents a decline from previous years. Nonetheless, it must be a refreshing show of support for teachers who have many reasons to feel beat up by the punditry.

The public also recognizes the key role teachers play in student learning and by large margins would welcome policies to bolster their preparation and training. A full 81% believe prospective teachers should be required to “pass board certification” similar to that for other professionals like doctors and lawyers on top of their college degree in order to be licensed to teach. Likewise, 60% thought that there should be higher entrance requirements into teacher prep programs at the front end. The majority also support the idea of requiring a longer period of supervised practice before teachers take charge of their own classrooms. A plurality of 44% thought such a bridge period should last one year with 27% saying new teachers need two (see chart).




Political affiliation had almost no effect on opinions about teacher preparation. Republicans, Democrats and Independents alike called for increasing rigor. Not so when asked about teacher evaluation, however, possibly in response to growing political controversy over new evaluation policies that are based in part on teachers’ impact on student learning. Nationally, 61% of the public opposes evaluations that include “how well a teacher’s students perform on standardized tests.” Yet only 50% of Republicans were opposed compared to 68% of Democrats. Interestingly, overall opposition to using evaluation in this way is much higher now than it was just two years ago when slightly less than half of the overall public thought it was a bad idea.

Similar party-affiliation gaps were evident when pollsters asked about the purposes of evaluation: 86% of Democrats said using evaluation to help “teachers improve” was “very important” compared to 71% of Republicans. In contrast, Republicans were much more favorable to linking evaluation to salaries or bonuses: 51% of Republicans said this was “very important” compared to only 41% of Democrats.

Other questions explored whether the public thinks their schools need to change “to meet today’s needs” and if so, how. More than half (58%) said that schools need to change compared to 47% who though so in 2006. The biggest needed change the public would like to see is a greater emphasis on career-technical education: 60% “strongly agreed” that “high school students should receive more education about possible career choices” while 32% said the same about placing “more emphasis” on college preparation for all.

This doesn’t necessarily mean that the public thinks career education is more important than college readiness. It could indicate that they think high schools are doing ok with college prep but they need to do more to get students ready for work. But either way, the message is clear that the public is looking toward high schools to make sure all graduates are able to thrive in the new workplace.

Along those lines, CPE’s Jim Hull has been analyzing work and other outcomes for the group of high school graduates who do not go to a two- or four-year college. His findings should produce some valuable insights into what career-readiness should look like. His first report will be released in the next two weeks so stay tuned. — Patte Barth

July 5, 2012

Poverty Not the Reason Teachers Labeled Ineffective

It’s no secret that most teachers are not fans of being evaluated based on student test scores.  This is true even though, according to the findings of an upcoming teacher survey from EdSector, the majority of teachers believe they should receive pay increases if they consistently receive outstanding evaluations from their principals . The survey also found that support for performance pay fell to less than a third of teachers if pay increases were based on student test scores. So teachers don’t mind being paid based on their performance but they don’t want their performance determined by test scores.

So why is this the case? Why are teachers so against being evaluated, even in part, based on how their students perform on standardized tests? Of course, there is no single answer but EdSector provides insights into the minds of many teachers. They highlight one teacher who said:

While she loved her job, if her pay depended on student test scores, she “would be crazy” to continue working in her school, which has a student body that is 90 percent low-income and speaks 18 different languages.

She says this because she, like many teachers, believes that standardized tests do not accurately measure the progress low-income and other traditionally disadvantaged students make during a given school year. Essentially, many teachers, just like the one highlighted, believe that their evaluations scores would suffer if they taught a class mostly made up of students from disadvantaged backgrounds.

I certainly don’t blame teachers for having such concerns. There have been many attempts to pay teachers based on their students’ performance that did penalize teachers for working with disadvantaged students.

However, times have changed and statisticians have developed tools to take on this problem. For example, value-added models have been designed to specifically isolate the impact a teacher has on their students no matter their students’ background or previous achievement. Value-added models are not perfect but they do a pretty good job of determining if a student would be better off with one specific teacher instead of another.

Despite the use of value-added models, teachers still believe they would essentially be penalized for having a class of disadvantaged students.  Even researchers who are critical of using value-added data to evaluate teachers believe teachers of disadvantaged students are penalized when they are evaluated based on value-added data.

However, findings from a recent report from the CALDER Center run counter to such criticism. The report found that a teacher’s evaluation score does not suffer if they teach more disadvantaged students. Specifically, CALDER found that teachers in North Carolina that transferred from a low-poverty school to a high-poverty school actually received higher value-added scores in the high-poverty school than they had in the low-poverty school.

Such finding provides strong evidence that teachers of disadvantaged students are not penalized if they are evaluated using high quality value-added data. Even so, as the Center’s report Building a Better Evaluation System argues, no teacher should be evaluated based only on student test scores – or any single measure for that matter. For a teacher evaluation system to yield both accurate results and provide information to help all teachers improve, multiple measures should be used, including student test scores. – Jim Hull

October 31, 2011

Merit pay revisited- Is Denver’s pay for performance a model plan?

Although it remains a controversial issue, merit pay has long since evolved from the days when test scores were the single factor in determining whether a teacher would get paid for performance.  Nowadays a number of school districts across the country have developed multi-pronged plans aimed at equitably rewarding teachers for their accomplishments.  Nonetheless, the question still remains: Is there actually a way to fairly reward a professional who deals with the advancement of human capital?  No plan is perfect, but one district might have come close.

In 2009, The Center took a look at merit pay and made mention of Denver’s ProComp Pay for Performance plan.  Now, a three year study, conducted by Dan Goldhaber and Joe Walch of the Center for Education Data and Research, has come out.  The study was conducted between the fall of 2006 and spring of 2010 on Denver’s ProComp plan. Denver Public Schools (DPS) requires all teachers who were hired in 2006 or later to be a part of the ProComp plan and gives veteran teachers the choice whether to opt in or not.  ProComp offers teachers four opportunities to receive bonuses, which include:

  1. Knowledge and Skills: Teachers may earn pay for completing one professional development unit per year (and can bank extra PDU’s), getting advanced  degrees and licenses, and can even receive tuition and student loan reimbursement (50 to 65 percent received this pay)
  2. Comprehensive Professional Evaluation: Based on principal evaluations, which are every 1 to 3 years (5 to 14 percent received this pay)
  3. Market Incentives: Aimed at teachers who work in hard-to-serve schools and/0r in hard-to-staff subject areas, as reviewed by school demographics and market supply (35 to 65 percent received this pay)
  4. Student Growth: Teachers set up student growth objectives, based on what they expect students to learn, which are approved by the principal (example: I expect x number of students to  exceed expectations in Reading on the Colorado Student Assessment Program (CSAP).) (70 to 80 percent received this pay)

The study suggests that the ProComp plan made teachers feel more supported and in turn, allowed them to more consistently meet their goals (Robles 2011).  In fact, between 2006 and 2010, 15 percent of the non-ProComp teachers even switched over to join the plan after seeing the positive results ProComp had on their schools and colleagues.  Not only has ProComp made the teaching profession more attractive, Goldhaber and Walch conclude that:

  • There were significant learning gains across grades and subjects;
  • The benefits of tracking data and evaluating educators spread from ProComp teachers to the entire district;
  • There was an expectation that the program would cause a negative atmosphere between team members but the opposite actually occurred and role models were bred;
  • ProComp teachers’ students had larger than expected gains on the state assessment.

Skeptics argue that these rewards focus more on classroom instruction than student test achievement and that ProComp is inconsistent with the value-added approach. Goldhaber and Walch point out that, “whether this is good or bad is clearly a normative question” but that “overall, ProComp has had a positive effect.”  They also suggest that states might want to consider investing in similar programs, especially for their Race to the Top objectives. Yesenia Robles of the Denver Post notes that ProComp has helped propel infrastructure reforms to change recruitment practices and enhance methods of data gathering.  She goes on to point out that the difference between non-ProComp and ProComp teachers’ student growth objectives are comparable to the difference between a first and second year teacher’s.   Her article, DPS Teacher-Pay System Likely Boosting Student Achievement, Study Finds, also points out that Denver Public Schools has retained 160 more teachers per year since 2006 and that 80 percent of all DPS teachers currently participate in the program.   Robles notes that, “The ProComp system is already in the process of changing with the implementation of the district’s evaluation-and-support system, known as LEAP, now being tested in 94 percent of DPS schools.”  Right now it is still too early to tell if ProComp can survive these alterations. 

ProComp is an even-handed, well-formed pay for performance plan that other districts can use as a model and will hopefully emulate.  The research shows that ProComp was not only received well by DPS teachers but most significantly, student success consistently progressed. –M. Newport

(To see whether similar pay for performance plans have been successful, check out this ECS report.)

July 22, 2011

New York City stops pay for performance

The New York Times is reporting that a “program that distributed $56 million in performance bonuses to teachers and other school staff members over the last three years will be permanently discontinued” because a study by the RAND corporation found that “the bonuses had no positive effect on either student performance or teachers’ attitudes toward their jobs.”

Before you react, consider this encouraging quote from the article: 

“The results add to a growing body of evidence nationally that so-called pay-for-performance bonuses for teachers that consist only of financial incentives have no effect on student achievement….Teachers also reported that improving as teachers and seeing their students learn were bigger motivators than a bonus, [the researchers] said.”

In an arena where attacks can often be cloaked under the phrase “It’s for the sake of the children,” it’s refreshing to see tangible evidence that children’s learning truly is a powerful motivator. And some of the other reasons given by the researchers for the program’s ineffectiveness, such as not understanding how the awards were determined, also point out that the logistical can be just as important (and far less divisive to fix) than the ideological.

In fact, several of the results of the RAND study seem to correlate with the Center’s earlier research review, “Promise or peril? Teacher pay for performance,” which pointed out that it is often how these programs are developed and structured that makes a difference.

So if your state or district is considering pay for performance, educate yourself first. There’s no use spending money on something that doesn’t work. —Rebecca St. Andrie

Filed under: Pay for Performance,teachers,Uncategorized — Tags: , , — rstandrie @ 10:01 am

January 7, 2011

What does baseball have to do with evaluating teachers?

“Look on the back of their baseball card.”

That’s the common reply from baseball managers and general managers around the major leagues when asked about what kind of production they expect from a player in an upcoming season.

It is a reference to the career stats of baseball players, which appear on the back of their baseball cards. Most often, at the end of the season a player’s stats will be more or less as their career stats.

This basic assumption is so accepted in Major League Baseball that teams offer tens of millions of dollars to those players with the strongest career stats, while players with the weakest stats tend to disappear after a few years.  The stakes don’t get much higher than that.

But in education, it’s still rare for similar high stakes decisions about school personnel (teachers, principals, and other administrators) to be made using data such as student test scores. Critics argue that evaluating educators primarily based on student test scores would be unfair. That even the most complex statistical models designed to isolate educators’ impact, such as value-added growth models, are not reliable enough to base high-stakes decisions such as salaries or tenure.

Now, these critics argue correctly that there is only a moderate correlation between a teacher’s past performance, as measured by value-added models, and their future effectiveness. For example, a Brookings report on value-added models stated that the correlation between test-based measures of teacher effectiveness between one school year and the next is between .30 and .40 (1.0 would mean a perfect correlation and 0.0 would be no correlation). This is a low to moderate correlation in statistical terms. Sounds like the critics have a strong point.

However, Brookings also points out when comparing that correlation to statistics used in other professions, such as Major League Baseball, the correlation doesn’t sound very weak.

Take batting average—one of the most widely used statistics to evaluate a player. The between-season correlation for batting averages for Major League Baseball players is just .36. Yet, a team will pay a player millions for hitting .300 and cut a player for hitting .250.  

Of course batting average is just one statistic teams use to evaluate a player and value-added scores are just one statistic that could be used to evaluate educators, but both can be key measures to determine how effective they will be in the future if used correctly.  

Do teams sometimes make the wrong decision by paying millions for the player who hit .300 last season while cutting the player who hit .250? Yes, it does happen. But overall, the teams are better off signing the players who had the strongest stats on back of his baseball card. Our schools could be more effective if they kept the educators with the strongest stats as well.– Jim Hull

Filed under: Growth Models,Pay for Performance,teachers — Tags: , , — Jim Hull @ 12:50 pm

Older Posts »
RSS Feed