Learn About: 21st Century | Charter Schools | Homework
Home / Edifier

The EDifier

January 17, 2017

How schools hire teachers

Much attention has been paid recently to teacher shortages.  However, less attention has been paid to how schools hire teachers, which has a direct impact on the shortage issue (which varies by subject and region).  Ineffective teachers are more likely to leave, as are those who don’t “fit” well with the culture or demographic of the school.  This, of course, makes sense – people don’t typically want to stay in jobs they’re horrible at, especially ones that are high-stress and low-pay.  There are lots of other reasons teachers leave, but principals don’t typically have a lot of influence over employees needing more time for family, a job closer to home, or higher salaries.  Teachers’ satisfaction with their work environment and school leadership is paramount to their retention, as well, but today we’ll just focus on who gets hired.  Previous research has shown that principals and other hiring managers tend to hire teachers based on their relational skills – motivation, honesty, enthusiasm, and caring – over their track record for student achievement.

As a personal anecdote, I was hired as a teacher by three different schools in two Texas districts. I never had to provide a sample lesson plan, teach a sample lesson, provide prior test scores, or provide prior evaluations. It’s possible that they checked my references, but I’m not sure. I was certified for the grades and subject areas I was hired to teach, which are often hard to staff, and was willing to teach in high-poverty schools, so maybe the lack of data-collection was due to teacher demand being higher than the supply of qualified teachers.

Image result for teacher hiring

A recent study shows that my experience is not unique.  The researchers performed interviews with principals and school administrators in six large public districts and two smaller charter districts to determine how data was used.  About 70 percent of the principals reported using teachers’ previous observation or test score data when determining whether or not to hire teachers who might transfer into their campus from elsewhere in the district.  This seems to point toward a trend in which principals increasingly use data; previous studies showed that only 40 percent of principals used student achievement data in hiring decisions.  As more states and districts develop teacher evaluation systems, we may see this trend continue upward.  Performance-based assessments are even less commonly used.  The Center for American Progress, a left-leaning think tank, found that only 13 percent of surveyed school districts required teachers to teach a sample lesson with students.

You may be asking why 100 percent of principals aren’t using data, and I’d have to answer that there are multiple reasons.  First, in most teacher evaluation systems in which a teacher is observed by a school administrator, over 90 percent of teachers are typically found to be effective or highly effective.  The lack of distinctions and the subjectivity of these evaluations render them virtually meaningless in the hiring process.  While principals may look negatively on someone falling below this threshold, they may also balance this data with personal recommendations or personal experience with the subjectivity of the evaluation system.  They may also have limited options for qualified candidates to hire.

Even value-added models, often used as components of teacher evaluations systems, are not fully reliable.  Teacher scores may change from one year to the next, or from one test to another.  If principals are aware of the shortcomings of the testing system, they may be hesitant to rely on it for hiring. Many subjects are not tested on an annual basis, making it difficult to use related data for hiring teachers who don’t teach math and language arts.

Finally, teacher observations are often biased against teachers in high-poverty and high-minority schools, as well as against men.  Given that high-poverty, high-minority schools are also the most difficult to staff, it makes sense that principals might take prior evaluation scores for teachers who have taught at other high-needs schools with a grain of salt.

While data on teacher effectiveness may be flawed, it doesn’t mean that we should throw it out entirely.  If a teacher consistently has low scores or has bounced between schools due to ineffective ratings, principals would be wise to heed this information.  We can use data to weed out some bad apples; we can’t necessarily depend on it to be 100 percent accurate or predict success.

March 4, 2016

Teacher evaluation systems: Major changes, similar results

Since TNTP’s  groundbreaking report, The Widget Effect, was released in 2009, nearly every state has made dramatic changes to the way teachers are evaluated. In that report the authors found less than 1 percent of teachers were rated below proficient, while nearly all other teachers were rated simply as proficient or satisfactory. Just a small proportion of teachers were recognized for being exceptional. Such results didn’t pass the sniff test. As anyone who has ever stepped foot in a classroom knows, there are a number of exceptional teachers as well as a number who just aren’t making the grade. Under previous teacher evaluation systems, few of these teachers were ever identified.

Despite the dramatic changes in evaluation, such as including measures of student achievement, not much has changed, according to a recent report aptly titled Revisiting the Widget Effect. Less than 3 percent of teachers are being rated below proficient. While this is more than 3 times as many teachers than found in the original Widget Effect, it is still an extremely small number.

Of course, it could be claimed that this is because there are so few ineffective teachers and not due to a failure of how teachers are evaluated. Which is a legitimate argument. In fact, this was a question the authors wanted to examine themselves by asking principals in a sample of schools how many of their teachers they felt were below proficient prior to conducting the new evaluation systems. What they found was a huge disconnect between how many teachers principals perceived to be less than proficient (27 percent) and how many teachers actually were rated below proficient (2.7 percent).

So why are so few teachers identified below proficient? Did states expend valuable time and resources simply to rate a few more teachers as ineffective? While the findings from this report are sobering, it provides important insights for policymakers in making changes to the current systems and understanding how the system is really being used at the ground level. What was fascinating is what principals said about why they didn’t rate some teachers as below proficient.  For example, some principals reported not assigning a below proficient rating to some teachers who were on the borderline because they felt their time was better spent focusing on providing extra supports to the lowest performing teachers. Other principals stated they avoided rating teachers below proficient who showed potential to be effective teachers in the future. While technically these teachers should have received poor ratings, these are actions good supervisors should take to build an effective staff.

Yet, these are examples of why it is imperative to include objective measures of effectiveness in any teacher evaluation system. States now have more flexibility in designing their own evaluation systems with the passage of the federal Every Student Succeeds Act (ESSA). They should include objective measures such as student test scores in their revised evaluations. Doing so still provides an opportunity for evaluators, such as the principals in this report, to focus their time and resources on the teachers that need them most. At the same time it lessens the chances evaluators will simply identify nearly all teachers as proficient because it is just easier to do so just as it was prior to 2009 in most states.

What states need to do now is find a proper balance between objective measures and trusting the professional judgment of evaluators, especially principals, who know what is best for the teachers and the students in that school. As the report shows, states have not yet found that right balance. Keep in mind, however, these new teacher evaluation systems are still quite new and will take some time to make the adjustments needed to have a significant positive impact on both teachers and students. – Jim Hull

Filed under: Teacher evaluation,teachers — Jim Hull @ 2:52 pm

December 10, 2015

The future of using student achievement measures to evaluate teachers

With the president signing the new Every Student Succeeds Act (ESSA) into law today the days of the No Child Left Behind Act waivers (NCLB) as well as Race to the Top grants (RTTT) have officially come to an end. The elimination of these programs also ends the ‘incentive’ for states to evaluate teachers based, at least in part, on measures of student achievement. Keep in mind, however, over 40 states currently evaluate teachers partially on their students’ achievement.

Less than a decade ago this was far from the case. Prior to NCLB waivers and RTTT only a small number of districts included student achievement measures when they evaluated their teachers.  In fact, a number of states prohibited using these measures in evaluation as a matter of law.

More recently, NCLB waivers and RTTT grants provided ‘incentives’ to include student achievement measures as a significant portion of how teachers are evaluated. In response, the vast majority of states have made significant changes to their teacher evaluation systems. However, developing these new evaluation systems was no easy task. In fact, most states have just recently fully implemented such systems and some are still in the process of doing so.  So it is far too early to tell what impact these new evaluation systems have had on teachers and student learning.

Now that federal ‘incentives’ have been lifted, the question is will the states stay the course when it comes to evaluating teachers or will they shift gears? Of course, only time will tell but with the pushback against testing in a number of states I’m guessing at least some states will change their evaluation system, especially as it pertains to including student achievement measures.

Yet, even if states pull back on linking teachers’ ratings to student performance, these systems are likely to be significantly better than what states had in place prior to NCLB waivers and RTTT. As discussed in our Trends in Teacher Evaluation report, for decades most evaluation systems were little more than a bureaucratic exercise that failed to recognize either excellence or mediocrity in teaching. This is no longer the case. States have vastly improved their teacher evaluation systems in recent years and not just by including measures of student achievement. Nearly every state has vastly improved the way classroom observations are conducted. Now it’s the norm for teachers to be observed every year–in many cases, multiple times a year—and then provided immediate feedback to inform and improve their instruction. Moreover, nearly every state now evaluates teachers on multiple measures even when tests scores are not used. Such indicators include the quality of their lesson plans, feedback from their students, and the quality of their classroom assignments among others. These measures are then combined to provide a more accurate measure of a teacher’s true effectiveness as well as provide valuable information to help teachers improve their instruction.

States and districts have worked extremely hard over the past several years to design and implement these new teacher evaluation systems so it is unlikely they will be going to back to the old days when teachers were evaluated every couple of years and rarely provided useful information. While including objective measures of student achievement like test scores can be a valuable part of an effective teacher evaluation system, the new evaluation systems even without the student link are much more likely to accurately identify effective teachers as well as provide useful information to improve instruction. And that is good news for all teachers and students. – Jim Hull

Filed under: Teacher evaluation,teachers — Tags: — Jim Hull @ 1:42 pm

March 18, 2015

Put teacher data in the hands of those who know how to use It

While every parent wants as much information as possible to do what is best for their child, it doesn’t mean that parents have the right to their child’s teacher’s evaluation data. That information should only be used by administrators to support the continuous improvement of their teachers and make more informed decisions on which teachers are best suited to teach which students. As I argue in Trends in Teacher Evaluations, this is the best way teacher evaluation systems can improve the effectiveness of all teachers. On the other hand, providing individual teacher evaluation data to parents, as one parent in Virginia is going to court over, will likely lead to a pitchfork mentality where parents will demand their child be placed in the highest rated teacher’s class and that low performing teachers be fired without any context on what the evaluation results actually mean.

Such rush to judgments on evaluating talent happens all too often by those only looking at the short-term gains. Sports provides the most vivid examples of this phenomena. One of the best examples is when the Boston Red Sox brought up Dustin Pedroia to play second base in 2007. But Pedroia’s numbers were downright awful the first month of the season and fans wanted him replaced. However, the manager kept playing him despite the bad numbers because his experience showed him that Padroia would someday become a very good player. And the manager was right; Padroia went on to win the Rookie of the Year award in 2007 and the American League’s MVP award the next. This illustrates how data is most effective in the hands of those who not only know how to use the data correctly but will use the data for the best possible outcomes in the long-term.

This isn’t to say that parents shouldn’t have any information about the quality of those teaching their children. They certainly should. The question is what information should be provided to parents. This is a question states and districts are still struggling with. Some states provide aggregate teacher effectiveness data by school while others notify parents that their child is being taught by a teacher rated as ineffective for multiple years in a row. There is no right answer to what information parents should have but it is clear just handing parents a teacher’s evaluation data would do more harm than good.

A far more effective strategy, would be for parents, teachers and policymakers to come together to find the best solution for all involved. Together they can come to an agreement on what not only is best for individual students in the short-term but what will allow for what is best for all students in the long-term.  – Jim Hull

Filed under: Data,Teacher evaluation,teachers — Tags: , — Jim Hull @ 11:29 am

November 18, 2014

High performing students ≠ Highly effective teacher

Sheri Lederman may, in fact, be an excellent teacher. But the fact that her fourth-grade class performed well above the New York state average on both the math and English tests is not evidence of her effectiveness. That’s because overall scores such as these have long been known to be more a measure of the quality of the students a teacher is assigned than the actual effectiveness of that teacher. As such, such scores should never be used to evaluate teachers.

Why would this be the case? Let me give you an example. Say Teacher A is assigned a remedial class where just 15 percent of students pass the state test, while Teacher B is assigned to a gifted class where 70 percent of the students pass the state test. However, 70 percent of Teacher A’s students made more than a year’s worth of growth, while just 15 percent of Teacher B’s students did so as well. Which teacher do you think is more effective?

Based on these test scores alone, Teacher A would be considered the more effective teacher since the amount of growth individual students make from one year to the next is more indicative of a teacher’s true effectiveness than scores for individual students at one point in time. So, the fact that Lederman’s students scored above the state average is basically meaningless when it comes to evaluating her effectiveness.

What needs to be known is how much growth her students made over the past year and how that compares to other teachers. But simply comparing the growth Lederman’s students made to the state average isn’t a very accurate measure of a teacher’s effectiveness either since a student’s prior achievement and background can influence how much growth they make in a given year. Such differences need to be accounted for before growth measures can be used to evaluate teachers.

In fact, New York state does account for such student differences when evaluating teachers based on student test scores—including Lederman. The statistical term for including such measures is called the Value-Added Model or VAM which simply determines if a teacher’s students made more, less, or similar academic gains had those same students been taught by an average teacher.

In 2014, the statistics show that Lederman’s students, for the most part, would have made greater academic gains if they had an average teacher. This was not the case for her previous class. So one must ask why the difference? How can a teacher’s performance change so much from year to year?

The answer is VAM’s are not perfect measures of a teacher’s effectiveness, so results may vary from year to year even if the teacher’s actual performance hadn’t changed. Keep in mind, while VAM’s get criticized for their inaccuracy, other measures including classroom observations are in many cases less accurate measures of a teacher’s effectiveness. However, research shows combining student growth measures along with other measures of teacher effectiveness– like classroom observations—provides a pretty accurate measure of a teacher’s true effectiveness.

Are teacher evaluations perfect? No, which is why decisions about individual teachers should be made based on multiple evaluations, not a single one. Even then, results should be used to inform personnel decisions not mandate specific actions. Those decisions should be left in the hands of school and district leaders who know what is best for their students. For Lederman, those who know her best consider her a good teacher despite her last evaluation. The question is: would their opinion change if she continues to receive poor evaluations? Time will only tell. – Jim Hull

Older Posts »
RSS Feed