As Naomi wrote yesterday the results from the final report from the Gates Foundation’s MET study are not groundbreaking. A number of researchers–including yours truly–policymakers and advocates have been saying for years that the most accurate way to evaluate teachers is by employing multiple measures of teacher performance that also includes measures of student achievement.
Even so, a number of highly respected education policy and research experts such as Jack Jennings and Linda Darling-Hammond have argued that measures of student performance such as value-added measures are too unreliable to accurately evaluate a teacher’s true performance. However, such critiques assume that value-added measures are the sole measure of a teacher’s performance which magnifies its limitations. These and other critics of value-added typically claim that other measures of teacher performance such as teacher observations are more accurate and should be used to evaluate teachers in lieu of value-added.
However, the claim that observations are a more accurate tool in evaluating teachers turns out not to be true at all. This is where the MET study really gets interesting. MET researchers specifically examined teacher observations and found that there are more limitations to only using observations to evaluate teachers than to only use value-added measures. Specifically, they found that a teacher’s observation score differed significantly depending on who did the observing and which lessons were observed. As Jennings and Darling-Hammond point out researchers have long known that value-added scores fluctuate significantly from year to year and even from assessment to assessment as well.
What gets lost in the rhetoric is the fact both tools can be made more accurate. For example, value-added scores are more accurate when they are averaged over multiple years—a point critics often leave out. On the other hand, observations scores are more accurate when teachers are observed multiple times by multiple people. This goes to show that no measure is perfect but there are ways to make them more accurate. Most importantly, the MET study found that when these and other measures were used together they were an accurate predictor of how they would perform in the future. So those students who currently have a teacher who previously obtained high value-added and observations scores are more likely to make greater achievement gains than similar students who currently have a teacher who earned lower scores.
It is important to point out that even when using multiple measures some very good teachers will be identified as ineffective and vice versa. However, an evaluation system that is based on a combination of value-added, observations and other measures is much better than using any one of these measures alone. By using multiple measures to more accurately evaluate teachers, administrators and policymakers can make personnel decisions based on how it will likely impact student achievement. This is a great improvement over the current system that simply evaluates teachers based on their years of experience and the highest degree they have earned which the MET Study found is the least accurate way to evaluate teachers. – Jim Hull