Using research to inform policy without understanding the research

Tennessee appears to be another example of how using research to inform policymaking can backfire if policymakers do not fully understand the research. This article in the New York Times examines Tennessee’s new teacher evaluation system, which is based on some good ideas but has some worrisome implementation flaws.

You can see why I’m concerned when you look at Florida and California, two other states where policymakers’ lack of understanding of research led to the implementation of flawed policies. In those cases, the states used research that showed the benefits of small class sizes. However, they failed to recognize that the benefits were primarily for students in 1st through 3 grades and particularly for poor and minority students. Yet, policymakers in both states implemented one-size-fits-all policies to lower class sizes in all grades for all schools instead of targeting the reduction to the lower grades and schools serving poor and minority students as the research suggested. This lack of understanding of the class size research led to policies that provided limited benefits to students at a significant cost to taxpayers.

Tennessee appears to be on the same track  in creating a state policy on how teachers are evaluated. In my report earlier this year, Building a Better Evaluation System, I pointed out that teachers who do not teach in subjects that are tested could be evaluated in part by their school’s overall performance. For reasons I cannot even fathom, Tennessee policymakers decided that teachers in untested subjects could choose a tested subject within their school to be evaluated even if it had nothing to do with the subject area they teach.

I’ll give you an example since you probably think I miswrote that last sentence. In Tennessee, a 1st grade teacher—a grade not tested in Tennessee—can choose to be evaluated using the school’s 5th grade writing scores.

Why can 1st grade teachers decide to be evaluated on their school’s 5th grade writing scores? I have no idea. The reason I suggested in the report that teachers in untested subjects be evaluated on school-wide results is that all teachers contribute in some way to their school’s results. Furthermore, it promotes more of a team atmosphere, since all teachers would benefit from the success of the school as a whole. But basing a teacher’s evaluation on overall school-wide scores is quite different from letting teachers choose one subject in one grade within their school. I think the Times article described this situation best when it stated:

“It’s a bit like Vegas, and if you pick the wrong academic subject, you lose and get a bad evaluation.”

Not a great way to get teachers to buy in to a system that is completely different from how they have been evaluated for decades and could cost them their jobs. That is why I stressed two other points in my report. One, any evaluation system needs to be thoroughly tested before being fully implemented. It is impossible for policymakers to predict all unintended consequences and testing would help minimize those consequences. Tennessee would have learned a lot if they had thoroughly tested the system.

Second, evaluation systems should be developed at the local level. State policymakers can play a role by requiring some general requirements or a framework so they can provide districts the data and resources needed for them to implement an effective evaluation system. But the needs of every district are different and districts should be able to design their teacher evaluation systems around those needs. For example, a district that is performing well in math but lagging in reading may want to put more weight on teaching reading when evaluating their teachers. On the other hand, if a district is hoping to lure technology firms to their district they may want to put more emphasis on math and science. These are just a few examples of why districts need flexibility in how they evaluate teachers. They illustrate how it is impossible to create a one-size-fits-all teacher evaluation system.

I commend Tennessee for putting together a teacher evaluation system, but policymakers there need to take a closer look at the research to improve their evaluation system. Teachers are already quite wary of new evaluation systems, since many previous attempts have unfairly penalized teachers who taught lower-performing students. Current research shows that this threat can be minimized in a properly designed evaluation system. Tennessee needs to re-evaluate its system before they fully implement it. If they don’t, some wrong decisions are going to be made, to the detriment of some teachers and their students. – Jim Hull

Comments (1)

  1. Pingback: The week in blogs « School Board News

Leave a comment

Your email address will not be published.