Sometimes I agree with Jay Mathews and sometimes I don’t. Either way I always have the utmost respect for his opinion. The same is true when it comes to his recent blog on teacher evaluations where he argues that rating teachers based on test scores will never work. While I disagree with his conclusion, I do agree they will not work if teacher evaluations:
- Are based on only one year of data (even if value-added measures are used).
- Do not include multiple measures of teacher performance.
- Scoring is not transparent.
- Do not encourage teamwork.
As a matter of fact I make the same points in my Building a Better Evaluation System report.
Where I disagree with Jay is his implicit assumption that these are not part of most teacher evaluation systems. As matter of fact they are. Most teacher evaluation systems based on student test scores that I am aware of include each of these elements. Jay is correct to bemoan the fact that DC teacher’s are evaluated on only one year’s worth of value-added scores. I would much rather see those scores averaged over 2 or 3 years like many other evaluations system do. But a teacher’s individual value-added score is just one part of DC’s evaluation system. Teachers are still evaluated on their instructional techniques and abilities as well as their contribution to the school community, so multiple measures are used to evaluate teachers.
Furthermore, DC as well as many other districts use their evaluation systems to encourage teamwork by evaluating teachers on how well their school performed as a whole. By giving credit to all teachers when the school improves, that gives incentives for teachers to work as a team. So basing teacher evaluations on student test scores can actually encourage rather than discourage teamwork within a school.
Finally, I have to take issue with Jay’s comment “New value-added assessments in the District, New York and elsewhere carry a whiff of Stalinist economic planning: secretive measures immune to review or logic.” Of course he is referring to a lack a transparency. However, with just one quick Google search I found a DC document clearly defining how teachers are measured. In the document I found it clearly spelling out how teacher evaluation scores are calculated:
- Individual value-added scores: 50%
- Student test score improvement
- Teaching and learning frame work: 35%
- Observations and other qualitative measures of teacher abilities
- Commitment to the school community: Up to 10%
- School value-added scores: 5%
- School’s test score improvement
Furthermore, the same document provided more detail into how each of the categories is calculated. Not exactly Stalinistic. Maybe there isn’t enough detail for Jay because it doesn’t provide the code used to calculate the value-added measures. But most non-statisticians wouldn’t have a clue what the code said anyway. That’s why DC had their value-added model examined by two well respected experts who have worked extensively with value-added models. Again, I don’t think Stalin would have asked for second or third opinions.
What Jay may not realize is that the lack of transparency may have less to do with the teacher evaluation system itself and how it was developed, but about who runs DC public schools. DC, or New York City for that matter, doesn’t have an elected school board, so DC residents likely didn’t have as much of a say in the development of the evaluation system as residents in districts run by elected school boards.
So my response to Jay is that teacher evaluations based on test scores can and do work but they have to be done right, including public input through their local school boards. – Jim Hull