Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

July 12, 2013

Greater flexibility in teacher assignments has its benefits

Research has consistently shown that the most disadvantaged students are less likely to have an effective teacher than their more advantaged peers. This is not only true across districts but within districts as well. Some have blamed this phenomenon—at least in part—on policies that prevent school leaders from replacing ineffective teachers and have argued that if principals were given the flexibility to move an ineffective teacher to another school within the district (called forced transfer) more disadvantaged students would have access to high quality teachers.

To date, there has been little research on whether such flexibility would lead to the intended outcomes. But a recent study found in fact that flexibility can lead to greater teacher effectiveness in low-performing urban schools. The study is not definitive since it is based on only one school district (Miami-Dade). But it found that principals in low-performing schools identified low-performing teachers– as measured by their value-added scores and number of days the teacher was absent—for transfer and were able to replace those teachers with more effective teachers in terms of both test scores (value-added scores) and fewer teacher absences. While the less effective teachers were more likely to be transferred to a much higher performing school with fewer disadvantaged students, unfortunately they were no more effective in their new school. Of course this leads to bigger discussion about what districts should do with ineffective teachers which is a topic for another post.

Such findings give credence to the theory that greater district flexibility in teacher assignments will lead to a more equitable distribution of teachers so disadvantaged students can get their fair share of effective teachers. In the long-term such a policy can have a big effect on narrowing achievement gaps as research has consistently shown the tremendous impact effective teachers have on student achievement.—Jim Hull

Filed under: Achievement Gaps,research,Teacher evaluation,teachers — Tags: , , — Jim Hull @ 4:07 pm




June 11, 2013

A teacher’s misconceptions of using test scores to evaluate teachers

I certainly understand teachers’ angst when it comes to being evaluated based on their student test scores. Being evaluated is not a fun experience but when that evaluation is also based on the performance of others it makes the process that much more stressful. To make matters worse there appears to be a number of misconceptions regarding exactly how test scores are used to evaluate teachers.

A USA Today editorial by Alexandria, VA high school teacher Patrick Welch is an example of some of the misconceptions many teachers have about using tests to evaluate them. Mr. Welch’s observations are far from unique. I have heard similar misconceptions from other teachers and even researchers which makes identifying such misconceptions all the more important.

New teacher evaluation systems more accurate than previous evaluations systems

First of all, it must be recognized that there is no such thing as a perfect evaluation system in any profession. So while even the best evaluation systems will certainly identify some truly effective teachers as ineffective it doesn’t mean teachers shouldn’t be evaluated at all. However, evaluation systems should be designed to minimize the chances of such occurrences.  As explained in our Building a Better Evaluation report, there are a number of tools available to make evaluations more accurate.

Test scores can accurately measure a teacher’s performance when used with other measures

Mr. Welch cites research showing the inaccuracies of using test scores to evaluate teachers. For example, some teachers are identified as effective using one test but identified as ineffective when based on a different test. While the research he cites is sound, they do not tell the full story because they are based on using one test to measure a teacher’s effectiveness. In no state that requires teachers to be evaluated based on their impact on student achievement is a teacher ever evaluated based on one measure. In fact, every state requires at least one measure of instructional practice such as classroom observations to make up at least half a teacher’s evaluation. Even in states where half of the teacher’s evaluation is based on student achievement measures, those states require multiple measures of student achievement, not just results from the state assessment.

Teachers are not penalized for teaching lower-performing students

Keep in mind when evaluating teachers based on measures of student achievement it doesn’t mean that teachers are evaluated based on their students’ overall scores. Instead teachers are evaluated on the growth of their students’ test scores from one year to the next. So teachers are evaluated based on the growth their students made while in their teacher’s class.

This is something Mr. Welch appears to misunderstand. This is typified by his story about how he improved the test scores from his Advanced Placement class by encouraging lower-performing students to dropout. He also quotes Jesse Rothstein who said of ranking teachers based on test scores “rewards or penalizes teachers for the kids they teach, not just for how well they do it.”

Fortunately for teachers and students teaching lower-performing students will not likely hurt a teacher’s evaluation. A well designed teacher evaluation system will not penalize teachers for teaching the hardest to teach students. Even if student test scores are a significant factor in the teacher’s evaluation. That is because teachers are evaluated based on the growth their students make from year to year and not on their overall achievement.

Truly rigorous evaluations will assess student growth using a value-added model (VAM) that takes into account each students prior achievement and other background factors to determine if they made as much or more growth than if those students had an average teacher. By making such comparisons, teachers are evaluated based on the quality of their instruction not on the students assigned to their class that particular year. So while Mr. Welch would be correct to worry about being evaluated on his students’ overall test scores, this is not what is happening across the country— although a report CPE will release later this summer finds teachers in some states are more likely to be evaluated accurately than in other states.

Recent research shows teachers can be evaluated accurately

While no evaluation system is perfect and all states should continually update their systems to make them more accurate, the Measures of Teacher Effectiveness (MET) study shows that combining measures of student growth with measures of instructional practice to evaluate teachers provides a reasonably accurate measure of a teacher’s true effectiveness. And when the results are looked at over multiple years they are a more accurate measure.

So while it is understandable that Mr. Welch and many teachers across the country are apprehensive about being evaluated in part on their student test scores, such anxiety should be put at ease knowing they are being evaluated based on the growth their students make and not on the overall performance. And those teachers who are evaluated using value-added scores should have even less anxiety as they provide the most accurate measure of a teacher’s true effectiveness especially when combined with measures of instructional practice. – Jim Hull

Filed under: Teacher evaluation,teachers — Tags: , — Jim Hull @ 11:34 am




May 3, 2013

Exciting possibilities: Coursera and professional development courses

Coursera, an organization currently facilitating free online access to courses taught by college professors, has announced it will be dipping its toes into the professional development arena.  I have to admit that when I read this headline, I was thrilled.  For teachers to have free, online access to courses offered by experts on education research and teaching methods is a step in the right direction.

First, these courses could allow schools to have resources for teachers to improve their skills that are differentiated for the specific content teachers teach.  Because hiring consultants is expensive, districts often rely on generic workshops that they offer to all teachers.  I’ve sat through my fair share of these: classroom management, assessment, alignment.  However, research shows that teachers aren’t interested in generic professional development, and it doesn’t have an impact on teacher practice or student achievement.  On the other hand, professional development that is tailored to the content one teaches, specifically exploring the elements of the course students struggle with, has been shown to make a real difference in teachers’ practice and students’ learning.  With free online courses, teachers could focus on courses tailored to their content area.

Furthermore, each teacher brings his or her own unique set of strengths and weakness to the profession.  Teaching is a job that demands a lengthy list of skills which are both emotional and cognitive.  Just as some students have more natural talents in certain areas than others, the same is true with teachers.  When I co-taught a class with another teacher, I got to see this full force.  My co-teacher managed the emotional needs of a class flawlessly, while my own strengths were in lesson planning.  Working together, we got to improve our areas of weakness.  Having online courses which are free for teachers allows teachers to think about what areas they need to improve on, taking courses focused on those areas instead of sitting through PD sessions not tailored to their area of need.  Just like we urge teachers to differentiate for students, recognizing that not all students are the same, access to online PD taught by experts allows for differentiation for teachers.

Second, it could save districts lots of professional development money that they can spend more wisely.  There’s a decent amount of evidence to show that districts spend a substantial amount of money on professional development, anywhere from 2 to 7% of their total budget.  Unfortunately, most of that spending is going towards one-shot, generic workshops.  Consultants are expensive, certainly one reason that districts can only afford to have whole-school, generic sessions instead of content-specific sessions.  Nonetheless, by spending copious amounts of money on consultants and staff for workshops, districts often don’t build in professional development support as teachers aim to implement those new skills into the classroom.  The reason that’s problematic is that research studies consistently show that teachers struggle immensely with new skills during implementation of those skills in the classroom, and that without support at this stage, teachers are likely to get frustrated and simply abandon the new skill altogether.  Of course, this makes sense.  Learning how to write is easier than actually writing; learning how to ride a bike is easier than actually riding a bike. Implementation is challenging.  Therefore, schools need to develop support during the implementation stage.   When schools do this, through individual instructional coaches who observe and conference with teachers or through time for collaboration, teachers improve their teaching and students learn more.

However, having teachers meet with coaches or collaborate with colleagues takes time, and teacher time is exceptionally expensive.  School districts either have to buy this time in a teacher’s contract, pay substitutes to cover classes, or hire more staff to reduce teaching loads.  Despite that fact, research on professional development shows that opening up this time and having teachers supported during implementation of new skills is exceptionally important.  In an analysis of over 1,300 studies of professional development programs, researchers found that programs that were less than 14 hours had no impact on student achievement .   But if schools were able to cut down on some of their consultant costs by having teachers participate in free, open Coursera courses, schools might be able to buy more teacher time for deep learning experiences such as coaching or collaboration.

Of course, in all discussions about the role of online courses, it’s important to note that they can never stand alone as one’s only exposure to learning, something that’s been validated repeatedly .  However, there’s good reason to think these courses could be a nice addition to a school’s professional development tool kit.  –Allison Gulamhussein

Filed under: CPE,instruction,online learning,teachers — Tags: , — Allison @ 10:06 am




April 4, 2013

Blame the adults not the tests

I have no problem with a debate about the proper role of standardized testing in our public schools. Standardized tests are not perfect but play a vital role in improving student achievement. However, exactly what role is not clear and is certainly open for debate.

While Eugene Robinson of the Washington Post admits that standardized tests are a vital tool in improving student achievement, he essentially blames them for the cheating crisis in Atlanta and elsewhere. Arguing, the reliance on standardized test scores to make decisions about bonuses drove otherwise honorable adults in Atlanta’s classrooms to cheat.

I whole heartily disagree with Mr. Robinson’s thesis. The tests didn’t make the adults cheat, the adults made that decision. Whether it was the teachers themselves or administrators that encouraged such actions, no one is to blame for the cheating than those adults who took part in the cheating directly or ordered that wrong answers be changed. Those are the ones to blame not the tests.

It should be noted Atlanta made significant gains on NAEP– where there is no evidence of cheating –during this time as well which shows that most Atlanta teachers and administrators responded to the pressures to raise student achievement by actually teaching and not cheating. Unfortunately, their hard work is being overshadowed by a number of teachers and administrators who decided to cheat instead of facing the reality they were not meeting the needs of their students.

It is true that Atlanta may have put too much emphasis on test scores alone. As I showed in Building a Better Evaluation System and CPE encourages with data-first.org, no one indicator can accurately evaluate the true effectiveness of a teacher, administrator or school. If Atlanta wanted to reward their most effective teachers and administrators with bonuses their effectiveness should have be measured using multiple measures such as observations, student learning objectives, or portfolios along with standardized test scores. Using multiple measures not only provides a more accurate measure of  true effectiveness but it would also lessen the likelihood those being evaluated can manipulate the results by cheating.

Another reason using multiple measures would help is while teachers should be evaluated on how much they improve their students’ achievement—as measures by their students’ standardized test scores—such measures do not provide any feedback on how they can improve their students’ performance in the future. As such, a teacher who received a low rating based solely on their students’ test scores may resort to cheating to save their job or get a bonus while a teacher who received a low rating but was provided feedback on how to improve their performance is more likely to modify their instructional techniques instead.

Using multiple measures to evaluate teachers and administrators lessens the chances of cheating but that does not mean that Atlanta’s reliance on test scores caused those involved to cheat. If only those teachers and administrators who cheated had more faith in their students and their abilities to teach, more Atlanta students would be better off today. – Jim Hull

Filed under: NAEP,Pay for Performance,Public education,Teacher evaluation,Testing — Tags: , , , , — Jim Hull @ 9:36 am




School-wide data should not be used in teacher evaluations without collaboration

A baseball player doesn’t make as much if his team doesn’t do well, right? Should we pay a band’s drummer the big bucks if the band isn’t popular? Absolutely not. Even our representatives in Congress have their job stability linked to one another; if the party doesn’t do well, neither do individual members. For many people, there really is no “I” in team when it comes to pay and job security.

On first blush, tying individual performance to team performance makes perfect sense. After all, in organizations such as a baseball team, band, or political party, each member works together towards a common goal; therefore, whether or not the team achieved that goal should be considered.

Similarly, doesn’t it make sense to tie a teacher’s evaluation to the overall performance of the school? After all, teachers in one building should all be working towards a common goal—the learning of students. Why not encourage teamwork by tying teachers’ job security and pay to the performance of the school as a whole? In fact, that’s just what several states and big cities have done in their new teacher evaluation systems. Arizona, Colorado, Connecticut, Indiana, Maryland, Pennsylvania, South Carolina, Tennessee, West Virginia, Wisconsin, as well as the cities of Chicago and Pittsburgh, have made some portion of a teacher’s evaluation score tied to how well the school as a whole does on standardized tests or other indicators (such as graduation rates or promotion rates). While these states have good intentions in doing so, tying school-wide data to individual teachers’ evaluation is a bit like giving your child a cough drop for a fever. It’s the wrong medicine for the ailment.

Let me present another series of hypotheticals. Should a local eye doctor’s pay and job security be linked to the town’s ear and sinus health? Should one therapist be held accountable for the mental health of patients in his or her city who are not his or her own? Most of us would argue in these scenarios, the answer is no. What accounts for the difference?

The difference is that doctors and therapists are private practitioners. While their work contributes to a greater goal that many might work towards—the physical and mental health of a community—their job isn’t structured in a way that demands teamwork. For the most part, your eye doctor does not collaborate with, practice with, or learn with your ear doctor, foot doctor, or cardiologist. Likewise, while therapists may provide similar services with similar goals, for the most part, they do not collaborate together on a patient, practice with each other, or have mandated time together where they share practices to improve. These fields are by their nature more individualistic. We seem to demand consideration of team performance when the job itself is structured in such a way as to require collaboration. Baseball players practice together. Band members write music and perform together. Congressmen and women write bills together, discuss issues together, and compromise on laws together. Doctors and therapists in private practice do not have a job which requires collaboration, so they aren’t judged based on it.

It may surprise many Americans to find out that teaching mirrors more of the private practitioner structure than the team structure. When I first started teaching, I was certainly shocked by this as well. As a new teacher, I had envisioned meeting frequently with other teachers to share what was happening in our class or to discuss the best teaching strategies for a particular concept. The reality was quite different. Almost all of my day was taken up with teaching my classes, and in the short preparation period I had, I graded papers and planned my lessons alone. Even if I wanted to collaborate, it wouldn’t have been possible. Few teachers had off the same periods as I did, and those who did were busy planning and grading their own papers. I quickly realized that teaching was an exceptionally isolated job.

In his 1975 landmark study of the lives of American teachers, Dan Lortie deemed schools “egg crates,” where teachers worked close proximity to one another, but their work never touched, much like eggs in a crate.  Today, thirty-eight years later, not much has changed. A recent study found that teachers spend a whopping three percent of their day collaborating with colleagues. It’s not that teacher don’t want to collaborate; it’s that the structure of the school itself impedes it.  Work is done individually, and there’s no time carved into the school day for collaboration.

However, calls for evaluating teachers based on school-wide indicators of achievement seem to ignore the widely recognized truth that the teaching profession is structured as a field of private practitioners. States and districts plod on, arguing teachers lack incentives to work together, something remedied by inclusion of school-wide data in individual evaluations. This is a flawed premise supported by no research. What research does show is that teachers want time for collaboration with their colleagues and engage willingly and happily when given this time. The recent MetLife survey of American teachers found that collaboration contributes to teacher satisfaction.  These findings certainly don’t suggest that teachers are resistant to working with one another and are in need of incentives to do so. They do suggest that the school structure itself presents major impediments to a teamwork approach to learning in a school building, something that’s not remedied with a carrot and stick approach to evaluations, but which requires a structural revamp of a teacher’s day and work.

Furthermore, weighting school-wide data so heavily in teacher evaluations works to dilute the primary purpose of the Race to the Top brand of teacher evaluations—identifying the individual contribution of a teacher to student learning. In Tennessee, school-wide achievement data accounts for 35 percent of evaluations of teachers of non-tested subjects. But a report on the first year of the evaluation system suggested decreasing that percentage because teachers expressed that the data didn’t reflect their individual performance.

Before we jump to making teacher evaluations the cure all for everything wrong with public school education, we’d be wise to think deeply about what’s causing the problem we’re aiming to fix. That’s a more complicated question for some aspects of schooling. However, it’s not that complex when we ask ourselves why teachers aren’t collaborating towards greater student learning. Instead of firing teachers based on scores which aren’t their own, and for which they have no time within their school day to work towards improving, let’s think about breaking the egg crate to create a more effective school structure that’s less “I” and more “team.” Maybe then we can think about evaluating teachers based on school-wide scores. Until then, let’s stop giving a cough drop for a fever.   -Allison Gulamhussein

Filed under: CPE,Teacher evaluation,teachers — Tags: , — Allison @ 9:15 am




Older Posts »
RSS Feed