Learn About: 21st Century | Charter Schools | Homework
Home / Edifier

The EDifier

August 21, 2014

Test scores aren’t perfect— and neither are classroom observations

Classroom Observations Critics of using student test scores to evaluate teachers often point to research that shows there is a limitation to using student test scores to evaluate teachers. Due to such limitations, critics argue, student test scores shouldn’t be used to evaluate teachers. Critics typically go on to argue, if teachers are to be evaluated they should be evaluated based on how well they teach by conducting classroom observations. The problem, however, is this argument is based on the assumption that classroom observations are a more accurate measure of a teacher’s true effectiveness.

To this end, researchers at the American Institutes of Research (AIR) recently attempted to determine how accurate classroom observations were at measuring a teacher’s effectiveness. What they found is contrary to popular belief. In fact, classroom observations are not better at evaluating teacher effectiveness than using student test scores—when used within quality value-added models.

Now, this doesn’t mean classroom observations shouldn’t be used to evaluate teachers. In fact, it is critical, as they not only provide information on the quality of a teacher’s instruction but they also provide valuable feedback to teachers about the strengths and weaknesses of their instructional practices which test score can’t provide. However, it is not only important to measure how well a teacher teaches, it is just as important to determine what impact they have on their students as well. Just as in baseball, a player may have the most technically sound swing but if he can’t hit the ball they’re not going to remain on the team for very long.

This is why teachers must and are being evaluated on multiple measures, not just test scores or classroom observations.  As I laid out in Building a Better Evaluation System, to accurately evaluate the effectiveness of teachers their evaluation should be based on multiple measures such as student test scores, classroom observations or other measures. No single measure, whether classroom observations or test scores, can possibly provide a full and accurate picture of the quality of the instruction a teacher provides. By incorporating multiple measures into a comprehensive teacher evaluation program, districts can not only evaluate the quality of teachers, but more importantly identify those areas where teachers can improve.

While neither test scores nor classroom observations are perfect measures of teacher quality when used in isolation, when combined they work together to provide a valuable tool in evaluating and supporting our nation’s teachers. – Jim Hull

July 1, 2014

When teacher tenure is denied

Denied Teacher tenure has been a hot topic lately, thanks to the Vergara decision a couple weeks ago in California that found the state’s teacher tenure laws were unconstitutional. As I wrote at the time, the decision is unlikely to spur a wave of similar lawsuits across the country, in large part, because many states have or are in the process of revising their tenure laws to incorporate measures of teacher effectiveness when granting tenure—-unlike in California. Many critics of teacher tenure hailed the decision believing it will now be easier to remove ineffective teachers.

However, simply removing ineffective teachers will not positively impact student achievement if those teachers aren’t replaced by more effective teachers. Recent reports from the CALDER Center at the American Institutes of Research (AIR) shed an all too important light on this question. One such report on New York City found teachers were only slightly more likely to be denied tenure after the city revamped their tenure process following the 2008-09 school year—an increase from a 2 to 3 percent denial rate. Yet, the rate at which teachers were approved for tenure declined dramatically because many more teachers had their probationary period extended instead of having their tenure denied or approved. Prior to the tenure reforms, 94 percent of eligible teachers had their tenure approved. Following the tenure reforms, that rate dipped to just 56 percent.

While New York City’s new tenure law did not force ineffective teachers to leave the classroom, many chose to leave on their own. The CALDER report found teachers who had their probationary period extended were 50 percent more likely to transfer to another school and 66 percent more likely to exit the system compared with teachers in the same school who were approved for tenure.

While critics of teacher tenure would hold this finding up as a success of tenure reform, it would in fact be a failure if ineffective teachers simply replaced the departing ones. Luckily for the students this is not what has happened in New York City. The teachers who had their probationary period extended and then left the school were on-average less effective than the teachers who replaced them in terms of both value-added scores and principal ratings. And the differences in effectiveness was fairly substantial as well.

If these trends continue students in New York City will be much less likely to be taught by an ineffective teacher as less effective teachers leave the profession while being replaced by more effective teachers. As previous research has shown, this can have a substantial impact on student achievement but such an impact will not happen over night. As effectiveness data comes in from states across the country that have implemented more accurate teacher evaluation systems, it is clear there isn’t a large proportion of consistently ineffective teachers that should be removed from the classroom. In fact, the vast majority of teachers are quite good so the focus should be on improving the performance of all teachers. Such focus will likely have a significant impact on student achievement as well. Yet, removing consistently ineffective teachers shouldn’t be ignored either. Even if just a small proportion of ineffective teachers left the profession each year and are replaced by more effective teachers, over time the impact would compound, such that students 10 years from now would be much less likely to be taught by an ineffective teacher than a student today.

Reforming teacher tenure laws and creating more accurate teacher evaluation systems have the potential to improve the overall effectiveness of teachers– if done correctly– but such impacts will not happen overnight. It will take time to make a significant impact on the overall effective of our teachers but it will likely have a tremendous impact on the achievement of those students who are just entering our public schools. – Jim Hull

June 18, 2014

The future of teacher tenure

NSBA_HomePageSlider_CPE_TeacherEvalLast week’s Vergara decision, where a California judge ruled the state’s teacher tenure law was unconstitutional sent shock waves across the education world. Debates simmered over what the decision would mean for teacher tenure in other states. The big question being asked: Is this the beginning of the end of tenure for teacher’s nationwide?

Time will only tell but it is unlikely teacher tenure will be going away on a mass scale anytime soon. That’s because California has long had some of the most lenient tenure requirements in the country. It was this low bar for obtaining tenure that the judge cited in his decision, which found the state’s teacher tenure laws deprived students of their right to an education under the state constitution and violated their civil rights. Even so, this low bar for tenure may still have passed constitutional muster if they weren’t so closely tied to key personnel decisions such as determining which teachers could be laid off and the ability to dismiss ineffective teachers, which the plaintiffs claimed disproportionately impacted disadvantaged students.

If this decision had happened five years ago I would have predicted a larger ripple effect. However, as our Trends in Teacher Evaluations report found, a number of states have already made the changes to their laws that the Vergara decision found unconstitutional. For instance, 15 states now recommend or require that the results of a teacher’s evaluation—which include measures of student achievement– be included in determining whether a teacher is granted tenure. Moreover, 32 states allow districts to use teacher evaluation results to dismiss low-performing teachers as well. Making such personnel decisions, at least in part, on the basis of teacher evaluation results was almost unheard of just five years ago but now it is almost the norm. This wasn’t the case in California where the state has lagged behind nearly every other state in implementing a comprehensive teacher evaluation system that incorporates measures of student achievement and then enables districts to use those results to make more informed personnel decisions.

With most states in the process of updating their tenure laws it is unlikely the Vergara verdict will elicit a wave of similar court cases. These new teacher evaluation systems are still in the infancy stage in most states and more time is needed to determine how accurate these systems are and how they can best be used to make personnel decisions like granting tenure. As the Trends in Teacher Evaluations report found no two states evaluate teachers the same way,  so there is no example yet on how to best evaluate teachers. So while it appears that teacher tenure took a couple steps back in California, it may actually be the case that it will push California forward to catch up to nearly every other state that now uses teacher evaluation results to make personnel decisions like granting tenure. – Jim Hull

Correction July 9, 2014: This entry previously referred to “passing constitutional mustard.” We meant “constitutional muster.” The EDifier was obviously thinking ahead to our 4th of July hot dog instead of reflecting on our youth in Henry VIII’s army.

February 27, 2014

The public doesn’t always have the right to know

Don’t get me wrong I am a huge proponent of transparency. The more information the public—especially parents—have about our public schools the better. However, there is just some information the public  shouldn’t have the right to see. Unfortunately, courts in a handful of states have declared that the public has a right to incomplete information about individual teachers. Florida is the latest example, where the courts have ruled that value-added scores for individual teachers are to be made public.

Publicly releasing value-added scores for individual teachers is wrong for a number of reasons but I will limit my critique to the usefulness of the information. As those you who follow this blog and read CPE’s research know, I am an avid proponent of using value-added scores to evaluate teachers. But I have said time and again neither value-added scores nor any other single measure should be used to evaluate teachers by themselves. To truly capture the true performance of a teacher they need to be evaluated using multiple measures. Thankfully, nearly every state has heeded that advice and now in fact evaluate their teachers using multiple measures- including value-added scores or other measures of student achievement gains. Such comprehensive approaches to evaluating teachers not only more accurately evaluates the true performance of a teacher but also provides valuable feedback to help all teachers improve- even the most effective teachers. Such improvements can have a profound effect on student achievement but only if teachers trust their evaluation is being used as a flashlight not a hammer to steal a phrase from our friends at the Data Quality Campaign. As such, the release of value-added scores for individual teachers can only undermine the efforts policymakers and educators have put in to create an evaluation system that will ultimately improve the performance of students in Florida.

I understand the argument that parents have a right to know if their child is being taught by an ineffective teacher. As a parent myself I want to know that information as well. But the answer is not providing parents incomplete information like in Florida. Providing parents only a teacher’s value-added score to determine if their child’s teacher is effective is like your doctor evaluating your health on your weight alone. Just like a doctor needs more information to evaluate your health, parents need more information on how the teacher actually performs in the classroom to determine if they are effective. Basing decisions on incomplete information will likely lead to bad decisions.

As states continue to implement these new teacher evaluation systems it will be an important debate as to what information about teachers the public is entitled and what information should only be used by administrators.  Currently some states do release aggregate teacher data at the school level but not about individual teachers, while other states like Rhode Island mandate that a student cannot be taught by an ineffective teacher for two years in row. However, most states have no policies on what information should be made public and in what form. It is a conversation that needs to be happening now in states throughout the country or more courts will likely force the release of incomplete data like in Florida that may undermine the years of work states have been putting in to create fair and accurate evaluation systems. – Jim Hull

Filed under: Teacher evaluation,teachers — Jim Hull @ 9:53 am

February 24, 2014

State Teacher Evaluation Policy – Change for the Better?

The U.S. Department of Education recently released a report titled State Implementation of Reforms Promoted Under the Recovery Act  that examines the changes in state policies between the 2009-2010 school year (the year the Recovery Act was enacted) and the 2010-2011 school year (the first school year after funds were awarded). Between 2009-2010 and 2010-2011, the number of state education agencies that reported supporting differentiated teacher compensation based on student achievement gains increased from 7 to 9. That said, the number of states that used teacher evaluation systems that measured performance with multi-level ratings, were based on multiple observations, and were based in part on student achievement gains only increased from one to two in that same time period. Overall, we can see from this report that while state education agencies were using money from the Recovery Act to build up evaluation and data systems, it was a very slow process. Although a priority, most states were unable to make dramatic changes to their teacher evaluation methods in the first year following the enactment of the American Recovery and Reinvestment Act.

However, CPE’s Trends in Teacher Evaluation report paints a slightly rosier picture when it comes to progress on teacher evaluation policy. Forty-one states now require or recommend that multiple measures be used when evaluating teacher performance, including student achievement data, classroom observations, and other factors such as student surveys and lesson plan reviews. The National Council on Teacher Quality recently released its 2013 State Teacher Policy Yearbook National Summary  which offered some interesting insight on the trends in evaluation methods over the last three school years.  One of the most striking things to come out of this report was that in 2010-2011, only 10% of states had policies in place that required teacher evaluations to include student achievement as a part of the teacher’s overall rating, but by 2012-2013, that number had increased to 80% of the states.

These changes can be attributed in large part to states seeing teacher evaluation with a new sense of urgency once their eligibility for Race to the Top money and No Child Left Behind waivers was tied to student achievement and using multiple measures in their evaluation systems. It seems promising that more states are enacting policies that use multiple measures to evaluate teachers, but we have yet to see how most of the policies will be implemented, much less how effective they will be once in place. If implemented well, they have the potential to help struggling teachers get the help they need and encourage good teachers to be great, but without proper implementation, these new policies on teacher evaluation could easily fall flat. While the efforts to enact these standards are admirable, it is too early to tell what impact these changes in evaluation will have on the teaching profession.

Filed under: CPE,Teacher evaluation,teachers — Patricia Campbell @ 4:07 pm

Older Posts »
RSS Feed