Learn About: 21st Century | Charter Schools | Homework
Home / Edifier

The EDifier

August 21, 2014

Test scores aren’t perfect— and neither are classroom observations

Classroom Observations Critics of using student test scores to evaluate teachers often point to research that shows there is a limitation to using student test scores to evaluate teachers. Due to such limitations, critics argue, student test scores shouldn’t be used to evaluate teachers. Critics typically go on to argue, if teachers are to be evaluated they should be evaluated based on how well they teach by conducting classroom observations. The problem, however, is this argument is based on the assumption that classroom observations are a more accurate measure of a teacher’s true effectiveness.

To this end, researchers at the American Institutes of Research (AIR) recently attempted to determine how accurate classroom observations were at measuring a teacher’s effectiveness. What they found is contrary to popular belief. In fact, classroom observations are not better at evaluating teacher effectiveness than using student test scores—when used within quality value-added models.

Now, this doesn’t mean classroom observations shouldn’t be used to evaluate teachers. In fact, it is critical, as they not only provide information on the quality of a teacher’s instruction but they also provide valuable feedback to teachers about the strengths and weaknesses of their instructional practices which test score can’t provide. However, it is not only important to measure how well a teacher teaches, it is just as important to determine what impact they have on their students as well. Just as in baseball, a player may have the most technically sound swing but if he can’t hit the ball they’re not going to remain on the team for very long.

This is why teachers must and are being evaluated on multiple measures, not just test scores or classroom observations.  As I laid out in Building a Better Evaluation System, to accurately evaluate the effectiveness of teachers their evaluation should be based on multiple measures such as student test scores, classroom observations or other measures. No single measure, whether classroom observations or test scores, can possibly provide a full and accurate picture of the quality of the instruction a teacher provides. By incorporating multiple measures into a comprehensive teacher evaluation program, districts can not only evaluate the quality of teachers, but more importantly identify those areas where teachers can improve.

While neither test scores nor classroom observations are perfect measures of teacher quality when used in isolation, when combined they work together to provide a valuable tool in evaluating and supporting our nation’s teachers. – Jim Hull

July 1, 2014

When teacher tenure is denied

Denied Teacher tenure has been a hot topic lately, thanks to the Vergara decision a couple weeks ago in California that found the state’s teacher tenure laws were unconstitutional. As I wrote at the time, the decision is unlikely to spur a wave of similar lawsuits across the country, in large part, because many states have or are in the process of revising their tenure laws to incorporate measures of teacher effectiveness when granting tenure—-unlike in California. Many critics of teacher tenure hailed the decision believing it will now be easier to remove ineffective teachers.

However, simply removing ineffective teachers will not positively impact student achievement if those teachers aren’t replaced by more effective teachers. Recent reports from the CALDER Center at the American Institutes of Research (AIR) shed an all too important light on this question. One such report on New York City found teachers were only slightly more likely to be denied tenure after the city revamped their tenure process following the 2008-09 school year—an increase from a 2 to 3 percent denial rate. Yet, the rate at which teachers were approved for tenure declined dramatically because many more teachers had their probationary period extended instead of having their tenure denied or approved. Prior to the tenure reforms, 94 percent of eligible teachers had their tenure approved. Following the tenure reforms, that rate dipped to just 56 percent.

While New York City’s new tenure law did not force ineffective teachers to leave the classroom, many chose to leave on their own. The CALDER report found teachers who had their probationary period extended were 50 percent more likely to transfer to another school and 66 percent more likely to exit the system compared with teachers in the same school who were approved for tenure.

While critics of teacher tenure would hold this finding up as a success of tenure reform, it would in fact be a failure if ineffective teachers simply replaced the departing ones. Luckily for the students this is not what has happened in New York City. The teachers who had their probationary period extended and then left the school were on-average less effective than the teachers who replaced them in terms of both value-added scores and principal ratings. And the differences in effectiveness was fairly substantial as well.

If these trends continue students in New York City will be much less likely to be taught by an ineffective teacher as less effective teachers leave the profession while being replaced by more effective teachers. As previous research has shown, this can have a substantial impact on student achievement but such an impact will not happen over night. As effectiveness data comes in from states across the country that have implemented more accurate teacher evaluation systems, it is clear there isn’t a large proportion of consistently ineffective teachers that should be removed from the classroom. In fact, the vast majority of teachers are quite good so the focus should be on improving the performance of all teachers. Such focus will likely have a significant impact on student achievement as well. Yet, removing consistently ineffective teachers shouldn’t be ignored either. Even if just a small proportion of ineffective teachers left the profession each year and are replaced by more effective teachers, over time the impact would compound, such that students 10 years from now would be much less likely to be taught by an ineffective teacher than a student today.

Reforming teacher tenure laws and creating more accurate teacher evaluation systems have the potential to improve the overall effectiveness of teachers– if done correctly– but such impacts will not happen overnight. It will take time to make a significant impact on the overall effective of our teachers but it will likely have a tremendous impact on the achievement of those students who are just entering our public schools. – Jim Hull

June 18, 2014

The future of teacher tenure

NSBA_HomePageSlider_CPE_TeacherEvalLast week’s Vergara decision, where a California judge ruled the state’s teacher tenure law was unconstitutional sent shock waves across the education world. Debates simmered over what the decision would mean for teacher tenure in other states. The big question being asked: Is this the beginning of the end of tenure for teacher’s nationwide?

Time will only tell but it is unlikely teacher tenure will be going away on a mass scale anytime soon. That’s because California has long had some of the most lenient tenure requirements in the country. It was this low bar for obtaining tenure that the judge cited in his decision, which found the state’s teacher tenure laws deprived students of their right to an education under the state constitution and violated their civil rights. Even so, this low bar for tenure may still have passed constitutional muster if they weren’t so closely tied to key personnel decisions such as determining which teachers could be laid off and the ability to dismiss ineffective teachers, which the plaintiffs claimed disproportionately impacted disadvantaged students.

If this decision had happened five years ago I would have predicted a larger ripple effect. However, as our Trends in Teacher Evaluations report found, a number of states have already made the changes to their laws that the Vergara decision found unconstitutional. For instance, 15 states now recommend or require that the results of a teacher’s evaluation—which include measures of student achievement– be included in determining whether a teacher is granted tenure. Moreover, 32 states allow districts to use teacher evaluation results to dismiss low-performing teachers as well. Making such personnel decisions, at least in part, on the basis of teacher evaluation results was almost unheard of just five years ago but now it is almost the norm. This wasn’t the case in California where the state has lagged behind nearly every other state in implementing a comprehensive teacher evaluation system that incorporates measures of student achievement and then enables districts to use those results to make more informed personnel decisions.

With most states in the process of updating their tenure laws it is unlikely the Vergara verdict will elicit a wave of similar court cases. These new teacher evaluation systems are still in the infancy stage in most states and more time is needed to determine how accurate these systems are and how they can best be used to make personnel decisions like granting tenure. As the Trends in Teacher Evaluations report found no two states evaluate teachers the same way,  so there is no example yet on how to best evaluate teachers. So while it appears that teacher tenure took a couple steps back in California, it may actually be the case that it will push California forward to catch up to nearly every other state that now uses teacher evaluation results to make personnel decisions like granting tenure. – Jim Hull

Correction July 9, 2014: This entry previously referred to “passing constitutional mustard.” We meant “constitutional muster.” The EDifier was obviously thinking ahead to our 4th of July hot dog instead of reflecting on our youth in Henry VIII’s army.

June 11, 2014

Common Core no longer OK in Sooner state

Oklahoma just became the latest state to jettison the Common Core standards that they adopted in 2010. The Sooner State joins Indiana and South Carolina which have also experienced grassroots opposition to the college- and career-ready standards, leading them to opt out of a nationwide effort they had not too long ago voluntarily opted into.

Interestingly— and unlike her Indiana and South Carolina colleagues— Oklahoma Gov. Mary Fallin was a public supporter of the standards who spoke in their favor as recently as January of this year at a meeting of the National Governors Association. But the bill, which passed with overwhelming support from both chambers of the Oklahoma legislature, had the backing of a vocal group of parents and small-government conservatives who saw the effort as a sign of federal over-reach. On June 5, Gov. Fallin signed HB3399 into law.

The Oklahoma opt-out differs from Indiana and South Carolina in another way. The latter two states both called for the development of new college- and career-ready standards to eventually replace the Common Core, which will continue to be used in the interim. In contrast, Oklahoma’s law calls for the immediate repeal of the Common Core. The state will revert back to the PASS standards they were using prior to 2010 until the replacements are finalized sometime in 2016.

A report by the Oklahoma Business Education Coalition and the Fordham Foundation calculated the total costs of writing new standards, assessments, and training to run upwards to $125 million. On top of that are the interim costs related to reverting to old standards while schools await the new. The authors wrote:

A harder cost to quantify is the impact repeal will have within the classroom…. [F]or nearly four years, teachers and students have been preparing for the Common Core Standards. A sudden departure from that course will create greater uncertainty in curriculum planning, and inevitably introduce several shifts as schools readjust to PASS standards and then again to new standards in two years. While some teachers might remember PASS standards from prior years, many newer teachers will have little to no exposure to these standards.

Clearly from our point of view, providing a sound public education to every child is the most important responsibility state and local governments have to fulfill. Each state therefore needs to consider standards from its own context and come to its own decisions. It’s not only appropriate for states to re-examine the standards they hold their schools to, it’s something they should do periodically.

So it’s entirely reasonable for states to have second thoughts about the Common Core, especially if they now believe they rushed into the relationship. But they should also be extra careful in deciding what to do about it. For one, there are some very practical costs involved, as the Oklahoma report points out. Over 40 states have spent the last three to four years retooling their school programs to align with the Common Core. That’s a large investment that should not be easily disregarded.

There are educational considerations, too. Putting aside for a moment how states and their public may view the federal role in Common Core, they should examine the standards on their own merits. There is a lot in the substance to be commended: the emphasis on using evidence, reading and writing in the subject areas, and the articulation of mathematical reasoning are just a few.

The first matter for public discussion then is, do these standards represent what we want for our students? I’m not at all sure this conversation happened in a lot of states, even though it should have. But it’s not too late to have it now. If the answer is “yes,” the Common Core can at least be on the table when the state develops its own standards even if the state wants to bail on the national effort. But if the answer is “no,” the state faces the challenge to define standards that will prepare all students for college and careers — standards that will likely need to be higher than what the state had before.  — Patte Barth

May 6, 2014

More teachers think ‘just the right amount’ of time spent testing

image for blog
A recently released study from the Northwest Evaluation Association finds that both teachers and administrators view the amount of time spent on testing more favorably now than they did two years ago.  The Make Assessments Matter report found that compared to 2011, increasing numbers of teachers and administrators believe that “just the right amount” of time is spent on assessments. While the majority of teachers still think that too much time is spent on testing, there was a notable increase in the number of teachers who think the amount of time spent on assessments is appropriate.

In 2011, 28 percent of teachers thought students spent “just the right amount” of time preparing for and taking tests; by 2013 this number had increased to 38 percent. When it comes to how much of their own time they have to invest in assessments, 36 percent of teachers in 2011 believed they spend “just the right amount” of time preparing for and administering assessments to their students. By 2013, 42 percent of teachers surveyed believed the amount of time they spent preparing for and administering assessments was appropriate.

District administrators’ favorable views on the appropriate amount of time spent testing increased even more than those of teachers over the same period.  While in 2011 only 29 percent of administrators believed students were spending “just the right amount” of time on testing, in 2013 48 percent believed that the amount of time spent on testing was suitable. The increase for the amount of time teachers spend on testing is slightly smaller for administrators, with the 31 percent viewing it as “just right” in 2011 increasing to 42 percent in 2013.

The study also reports on student experiences with assessments. Somewhat surprisingly, more than 90 percent of students agreed that assessments are either “very important” or “somewhat important” for a variety of purposes, including helping their teachers chart their progress, understanding what they’re learning, helping them get into college, and knowing whether they will be promoted to the next grade.

This seems to be a good start, but there is still work to be done, as 53 percent of teachers and 40 percent of administrators still think that students spend too much time preparing for and taking assessments. However, considering the political polarization and public scrutiny that has been following the early implementation and field testing of the Common Core State Standards, it appears that many teachers and administrators are actually happier with the current level of testing than they were two years ago. -Patricia Campbell

Filed under: Assessments,teachers,Testing — Patricia Campbell @ 3:58 pm

Older Posts »
RSS Feed