In previous posts, I’ve taken issue with many of the criticisms leveled by Diane Ravitch and Valerie Strauss, who disputed the findings from the recent study by Harvard and Columbia economists. That study found students who had teachers with high value-added scores were more successful later in life than students who had teachers with lower value-added scores.

Since I believe many of the criticisms were based on common misconceptions about value-added measures, let’s keep going:

Criticism 6: The study predominately studied teachers and students in the 1990’s when there was no test-based accountability. Value-added measures as well as future student outcomes may differ if consequences were attached to test scores since teachers may focus more on raising scores than providing richer instruction and would likely negatively impact students’ future outcomes.

Response 6: Basing the study on teachers and students from the 1990’s was necessary in order to examine the impact of teachers on elementary students ten years after they graduated from high school. So the findings on the long-term impact are actually based on very recent data.

 It is also true that the results may differ if high stakes were placed on the test results.  Keep in mind, however, that if the data did indeed come from New York City (as Ravitch and Stauss assume), test-based accountability had been in place since the late 1980’s, and that accountability was particularly focused on low-performing schools enrolling predominately poor students.  Even if the data is not from New York City, the fact that the district was testing students and collecting so much data shows it is likely there was some sort of test-based accountability in place at the time.

Yet the study found the impact of having a high value-added teacher was the same for students of all income levels. So even the poorest students, who were much more likely to attend schools where teachers were under pressure to raise test scores, benefited as much from high value-added teachers as higher-income students, who were less likely to attend a school that faced any accountability pressure.

 Criticism 7: The study doesn’t show that value-added can accurately identify individual teachers as effective or ineffective.

Response 7: As I stated previously, no one seriously argues that that individual teachers should be evaluated using value-added measures alone. I am not aware of any proposed or current teacher evaluation systems that have value-added scores account for more than 50 percent of a teacher’s overall evaluation. As the Center for Public Education report Building a Better Evaluation System states, value-added scores can be an effective tool in accurately identifying effective and ineffective teachers, but they should be used within the context of a comprehensive evaluation system that includes observations and other qualitative measures of a teacher’s performance.

Tomorrow I’ll finish the list of misconceptions and criticisms by Strauss and Ravitch. – Jim Hull

Yesterday I responded to some common criticisms of value-added measures to evaluate teachers. As I mentioned yesterday, these criticisms have been used by critics such as Diane Ravitch and Valerie Strauss to  dispute the findings from the recent study by Harvard and Columbia economists. That study found students who had teachers with high value-added scores were more successful later in life than students who had teachers with lower value-added scores. While many, including me, believed that the study provides strong evidence that value-added measures can be an effective tool in identifying effective teachers, Ravitch and Strauss took exception to such a conclusion.

To be clear, neither Ravitch nor Strauss is questioning the report’s methodology or its findings.  As Ravitch notes “The problems of the study are not technical, but educational.” However, I believe that the criticisms they raise stem more from common misconceptions about value-added scores and how they are used. Here are some additional criticisms, with my responses.

Criticism 3: Teachers in subjects not subject to testing, such as art, music, physical education, and many times social studies do not have value-added scores.

Response 3: I took on this issue in the Center’s Building a Better Evaluation System report. It is a limitation of value-added models, but there are systems in place that still are able to evaluate teachers in non-tested subjects. One technique is to use the school’s value-added score in place of the individual teacher’s value-added score, since all teachers contribute to the success of the school. The school’s value-added score, combined with other measures like observations, would provide a more accurate measure of teacher effectiveness. Not even the most ardent support of value-added scores believes that a decision about an individual teacher should be made based on a value-added score alone.

Criticism 4: It is no wonder higher performing students are more likely to go to college.

Response 4: The key word here is ‘higher.’ If the study found that previously high-performing students were more likely to go on to college, there’d be little to talk about. But it didn’t. The study found a student who had a teacher with a high value-added score was more likely to go to college than if that same student had had an average teacher, and much more likely than if that student had had a teacher with a low value-added score.

Criticism 5: The impact of highly effective teachers on student incomes is relatively small at $250 per year.

Response 5: The number that Ravitch and Strauss cite is the impact of having one high value-added teacher for one year and having average teachers for all other years. Much like compound interest, having more than one highly effective teacher really starts to pay off, especially when you compare it to the impact of poor teachers. For instance, having four highly effective teachers instead of four teachers with low value-added scores would improve a student’s future income by $2,000 per year. That’s not chump change! And the income difference was calculated at age 28 – just before the income gap between college and non-college students really starts to widen.

Keep reading tomorrow for the final set of criticisms. – Jim Hull