Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

October 28, 2014

Building a better reporting system

As readers know, CPE is all about the importance of using data and research to craft effective school policy and practice. We also encourage everyone who has an interest in public schools to look at data when gauging their quality. Unfortunately, getting that data isn’t always as straightforward as it could be. Even when found, it’s often presented in long tables, complicated graphs and confusing formats that obscure rather than shed light on school performance.

The Data Quality Campaign sought to address the all-too-common lack of quality in the way states report school data to the public. DQC recently convened a task force of national education experts and advocates — an effort we were proud to be part of — to identify best practices in state reporting systems. The results of our meetings are contained in the publication, Empowering Parents and Communities through Quality Public Reporting, released today.

The recommendations are intended for state policymakers to inform their design of state data systems. These systems should feature the following characteristics:

  • First, the data is trustworthy. There’s obviously little value in data that is wrong or out-of-date so every effort must be made to ensure accuracy. In addition, an essential part of gaining the public’s trust in data systems is protecting student privacy. Indeed, parental concerns about who has access to their child’s data and how it is used have grown a lot over the last year. Both states and districts have a role in putting fail-safe limits on access to individual students’ data. (To learn more about how, NSBA’s general counsel’s office produced this excellent guide on assuring student data privacy and CPE/DQC developed this data privacy fact sheet for school boards.)
  • Good systems are also focused on meeting people’s information needs. Many state systems were developed with a view toward compliance with federal and state regulations. If this information was also useful to educators, administrators, policymakers, parents and press, it was merely by happenstance, not design. A good data system, however, is designed with the consumer in mind, going beyond compliance to provide a real service to education stakeholders.
  • State reporting is timely and ongoing. The data collection and vetting process can often delay public reporting for as long as two years. This doesn’t help teachers or parents who need to respond to students’ needs in real time, not long after the fact. The same is true for administrators and school boards who need current data to inform their decisions.
  • Finally, in a good system, data is easy to find, access and understand. Some state report cards are buried in the department of education website. Many others are hard to navigate or present the data in ways that are difficult to interpret. The DQC report features state-of-the-art data systems from Illinois, Ohio and others that demonstrate the possibilities in presenting data that is easy for non-statisticians to locate, understand and, ultimately, use effectively.

While the Empowering Parents report is intended for state policymakers, the accompanying fact sheets are written specifically for parents, administrators and school boards and they discuss how these different stakeholders can use data and be strong advocates for better data systems. – Patte Barth

Filed under: CPE,Data,Parents,School boards,teachers — Tags: , , , — Patte Barth @ 3:43 pm




September 18, 2014

PDK/Gallup poll Part 2 shows teachers matter

The folks at PDK and Gallup apparently had so much to report in this year’s annual poll of public attitudes toward public schools, they had to release it in two parts. Part 1, which we summarized here, addressed the Common Core state standards and perceptions about public schools more generally. Part 2, released this week, focuses primarily on public attitudes about the teaching profession. What they have to say should provide comfort to beleaguered teachers.

First, nearly two-thirds of the public expresses “trust and confidence in the men and women teaching children in the public schools.” The study’s authors note that this represents a decline from previous years. Nonetheless, it must be a refreshing show of support for teachers who have many reasons to feel beat up by the punditry.

The public also recognizes the key role teachers play in student learning and by large margins would welcome policies to bolster their preparation and training. A full 81% believe prospective teachers should be required to “pass board certification” similar to that for other professionals like doctors and lawyers on top of their college degree in order to be licensed to teach. Likewise, 60% thought that there should be higher entrance requirements into teacher prep programs at the front end. The majority also support the idea of requiring a longer period of supervised practice before teachers take charge of their own classrooms. A plurality of 44% thought such a bridge period should last one year with 27% saying new teachers need two (see chart).

 

PDK2

 

Political affiliation had almost no effect on opinions about teacher preparation. Republicans, Democrats and Independents alike called for increasing rigor. Not so when asked about teacher evaluation, however, possibly in response to growing political controversy over new evaluation policies that are based in part on teachers’ impact on student learning. Nationally, 61% of the public opposes evaluations that include “how well a teacher’s students perform on standardized tests.” Yet only 50% of Republicans were opposed compared to 68% of Democrats. Interestingly, overall opposition to using evaluation in this way is much higher now than it was just two years ago when slightly less than half of the overall public thought it was a bad idea.

Similar party-affiliation gaps were evident when pollsters asked about the purposes of evaluation: 86% of Democrats said using evaluation to help “teachers improve” was “very important” compared to 71% of Republicans. In contrast, Republicans were much more favorable to linking evaluation to salaries or bonuses: 51% of Republicans said this was “very important” compared to only 41% of Democrats.

Other questions explored whether the public thinks their schools need to change “to meet today’s needs” and if so, how. More than half (58%) said that schools need to change compared to 47% who though so in 2006. The biggest needed change the public would like to see is a greater emphasis on career-technical education: 60% “strongly agreed” that “high school students should receive more education about possible career choices” while 32% said the same about placing “more emphasis” on college preparation for all.

This doesn’t necessarily mean that the public thinks career education is more important than college readiness. It could indicate that they think high schools are doing ok with college prep but they need to do more to get students ready for work. But either way, the message is clear that the public is looking toward high schools to make sure all graduates are able to thrive in the new workplace.

Along those lines, CPE’s Jim Hull has been analyzing work and other outcomes for the group of high school graduates who do not go to a two- or four-year college. His findings should produce some valuable insights into what career-readiness should look like. His first report will be released in the next two weeks so stay tuned. – Patte Barth

Filed under: High school,Public education,Teacher evaluation,teachers — Tags: , , , , — Patte Barth @ 9:02 am




August 28, 2014

Success of new teacher evaluation systems in districts’ hands

In just the past few years just about every state has revamped how their teachers are evaluated. In 2010 the vast majority of teachers were evaluated by being observed for 45 minutes every couple years. Now, most teachers are evaluated annually based on multiple measures of their effectiveness. Although these new comprehensive evaluation systems have the opportunity to significantly impact the overall quality of our nation’s teachers, a new report from Bellwether Education Partners shows that they are still in need of improvement.

While these new evaluation systems are superior to previous evaluation systems, the report points out there is still room for improvement and provide five major findings with lessons for policymakers:

  1. Districts are starting to differentiate between poor, fair, and great educator performance, rather than treating all teachers as interchangeable widgets.
  2. Schools are using higher-quality classroom observation rubrics to provide teachers with better, timelier feedback.
  3. Despite state policy changes, many districts still don’t factor student growth into teacher evaluation ratings.
  4. Districts have wide discretion even under “statewide” evaluation systems—meaning that evaluation systems within the same state may look very different from one another.
  5. Districts continue to ignore performance when making decisions about teacher hiring, compensation, tenure, and dismissal.

 

As you can see “district” is explicitly mentioned in four of the five lessons and the fifth lesson about classroom observations typically falls under the domain of districts as well. This pretty much means the success or failure of these new evaluation systems depends in large part on our nation’s school boards as they are the policymakers at the district level.  As the report points out, even in states that mandate the use of a statewide evaluation system school districts have significant discretion over how their teachers are evaluated.

As I argue in our report Trends in Teacher Evaluations it is imperative that districts have flexibility in how they evaluate their teachers. And it is good to see that this is the case, as the Bellwether report stated, “Evaluation reform has not meant the end of local discretion.”

While flexibility is necessary, districts also need support too. Few districts have the resources and expertise to implement an accurate and effective teacher evaluation system on their own. Districts need support from their states to help them align these new evaluation systems to the unique needs of their district.

While responsibility for designing teacher evaluation systems was originally placed on states, this new report clearly shows that these new teacher evaluation systems will only be successful if school boards are provided the resources not only to implement these evaluation systems but also to provide professional development opportunities that are aligned with the results of each teacher’s evaluation. Without proper support for districts, teacher evaluations are unlikely to have much of an impact on the quality of our nation’s teachers. – Jim Hull

Filed under: Teacher evaluation,teachers — Tags: , , — Jim Hull @ 10:37 am




July 12, 2013

Greater flexibility in teacher assignments has its benefits

Research has consistently shown that the most disadvantaged students are less likely to have an effective teacher than their more advantaged peers. This is not only true across districts but within districts as well. Some have blamed this phenomenon—at least in part—on policies that prevent school leaders from replacing ineffective teachers and have argued that if principals were given the flexibility to move an ineffective teacher to another school within the district (called forced transfer) more disadvantaged students would have access to high quality teachers.

To date, there has been little research on whether such flexibility would lead to the intended outcomes. But a recent study found in fact that flexibility can lead to greater teacher effectiveness in low-performing urban schools. The study is not definitive since it is based on only one school district (Miami-Dade). But it found that principals in low-performing schools identified low-performing teachers– as measured by their value-added scores and number of days the teacher was absent—for transfer and were able to replace those teachers with more effective teachers in terms of both test scores (value-added scores) and fewer teacher absences. While the less effective teachers were more likely to be transferred to a much higher performing school with fewer disadvantaged students, unfortunately they were no more effective in their new school. Of course this leads to bigger discussion about what districts should do with ineffective teachers which is a topic for another post.

Such findings give credence to the theory that greater district flexibility in teacher assignments will lead to a more equitable distribution of teachers so disadvantaged students can get their fair share of effective teachers. In the long-term such a policy can have a big effect on narrowing achievement gaps as research has consistently shown the tremendous impact effective teachers have on student achievement.—Jim Hull

Filed under: Achievement Gaps,research,Teacher evaluation,teachers — Tags: , , — Jim Hull @ 4:07 pm




June 11, 2013

A teacher’s misconceptions of using test scores to evaluate teachers

I certainly understand teachers’ angst when it comes to being evaluated based on their student test scores. Being evaluated is not a fun experience but when that evaluation is also based on the performance of others it makes the process that much more stressful. To make matters worse there appears to be a number of misconceptions regarding exactly how test scores are used to evaluate teachers.

A USA Today editorial by Alexandria, VA high school teacher Patrick Welch is an example of some of the misconceptions many teachers have about using tests to evaluate them. Mr. Welch’s observations are far from unique. I have heard similar misconceptions from other teachers and even researchers which makes identifying such misconceptions all the more important.

New teacher evaluation systems more accurate than previous evaluations systems

First of all, it must be recognized that there is no such thing as a perfect evaluation system in any profession. So while even the best evaluation systems will certainly identify some truly effective teachers as ineffective it doesn’t mean teachers shouldn’t be evaluated at all. However, evaluation systems should be designed to minimize the chances of such occurrences.  As explained in our Building a Better Evaluation report, there are a number of tools available to make evaluations more accurate.

Test scores can accurately measure a teacher’s performance when used with other measures

Mr. Welch cites research showing the inaccuracies of using test scores to evaluate teachers. For example, some teachers are identified as effective using one test but identified as ineffective when based on a different test. While the research he cites is sound, they do not tell the full story because they are based on using one test to measure a teacher’s effectiveness. In no state that requires teachers to be evaluated based on their impact on student achievement is a teacher ever evaluated based on one measure. In fact, every state requires at least one measure of instructional practice such as classroom observations to make up at least half a teacher’s evaluation. Even in states where half of the teacher’s evaluation is based on student achievement measures, those states require multiple measures of student achievement, not just results from the state assessment.

Teachers are not penalized for teaching lower-performing students

Keep in mind when evaluating teachers based on measures of student achievement it doesn’t mean that teachers are evaluated based on their students’ overall scores. Instead teachers are evaluated on the growth of their students’ test scores from one year to the next. So teachers are evaluated based on the growth their students made while in their teacher’s class.

This is something Mr. Welch appears to misunderstand. This is typified by his story about how he improved the test scores from his Advanced Placement class by encouraging lower-performing students to dropout. He also quotes Jesse Rothstein who said of ranking teachers based on test scores “rewards or penalizes teachers for the kids they teach, not just for how well they do it.”

Fortunately for teachers and students teaching lower-performing students will not likely hurt a teacher’s evaluation. A well designed teacher evaluation system will not penalize teachers for teaching the hardest to teach students. Even if student test scores are a significant factor in the teacher’s evaluation. That is because teachers are evaluated based on the growth their students make from year to year and not on their overall achievement.

Truly rigorous evaluations will assess student growth using a value-added model (VAM) that takes into account each students prior achievement and other background factors to determine if they made as much or more growth than if those students had an average teacher. By making such comparisons, teachers are evaluated based on the quality of their instruction not on the students assigned to their class that particular year. So while Mr. Welch would be correct to worry about being evaluated on his students’ overall test scores, this is not what is happening across the country— although a report CPE will release later this summer finds teachers in some states are more likely to be evaluated accurately than in other states.

Recent research shows teachers can be evaluated accurately

While no evaluation system is perfect and all states should continually update their systems to make them more accurate, the Measures of Teacher Effectiveness (MET) study shows that combining measures of student growth with measures of instructional practice to evaluate teachers provides a reasonably accurate measure of a teacher’s true effectiveness. And when the results are looked at over multiple years they are a more accurate measure.

So while it is understandable that Mr. Welch and many teachers across the country are apprehensive about being evaluated in part on their student test scores, such anxiety should be put at ease knowing they are being evaluated based on the growth their students make and not on the overall performance. And those teachers who are evaluated using value-added scores should have even less anxiety as they provide the most accurate measure of a teacher’s true effectiveness especially when combined with measures of instructional practice. – Jim Hull

Filed under: Teacher evaluation,teachers — Tags: , — Jim Hull @ 11:34 am




Older Posts »
RSS Feed