Home / Edifier

The EDifier

January 25, 2012

More misconceptions about value-added measures

Filed under: Growth Models,Teacher evaluation,teachers,Uncategorized — rstandrie @ 4:17 pm

Yesterday I responded to some common criticisms of value-added measures to evaluate teachers. As I mentioned yesterday, these criticisms have been used by critics such as Diane Ravitch and Valerie Strauss to dispute the findings from the recent study by Harvard and Columbia economists. That study found students who had teachers with high value-added scores were more successful later in life than students who had teachers with lower value-added scores. While many, including me, believed that the study provides strong evidence that value-added measures can be an effective tool in identifying effective teachers, Ravitch and Strauss took exception to such a conclusion.

To be clear, neither Ravitch nor Strauss is questioning the report’s methodology or its findings. As Ravitch notes “The problems of the study are not technical, but educational.” However, I believe that the criticisms they raise stem more from common misconceptions about value-added scores and how they are used. Here are some additional criticisms, with my responses.

Criticism 3: Teachers in subjects not subject to testing, such as art, music, physical education, and many times social studies do not have value-added scores.

Response 3: I took on this issue in the Center’s Building a Better Evaluation System report. It is a limitation of value-added models, but there are systems in place that still are able to evaluate teachers in non-tested subjects. One technique is to use the school’s value-added score in place of the individual teacher’s value-added score, since all teachers contribute to the success of the school. The school’s value-added score, combined with other measures like observations, would provide a more accurate measure of teacher effectiveness. Not even the most ardent support of value-added scores believes that a decision about an individual teacher should be made based on a value-added score alone.

Criticism 4: It is no wonder higher performing students are more likely to go to college.

Response 4: The key word here is ‘higher.’ If the study found that previously high-performing students were more likely to go on to college, there’d be little to talk about. But it didn’t. The study found a student who had a teacher with a high value-added score was more likely to go to college than if that same student had had an average teacher, and much more likely than if that student had had a teacher with a low value-added score.

Criticism 5: The impact of highly effective teachers on student incomes is relatively small at $250 per year.

Response 5: The number that Ravitch and Strauss cite is the impact of having one high value-added teacher for one year and having average teachers for all other years. Much like compound interest, having more than one highly effective teacher really starts to pay off, especially when you compare it to the impact of poor teachers. For instance, having four highly effective teachers instead of four teachers with low value-added scores would improve a student’s future income by $2,000 per year. That’s not chump change! And the income difference was calculated at age 28 – just before the income gap between college and non-college students really starts to widen.

Keep reading tomorrow for the final set of criticisms. – Jim Hull

Permalink | Trackbacks | Comments (1) |

January 19, 2012

Evaluating teachers: look at the data, please

Filed under: Teacher evaluation,teachers,Uncategorized — rstandrie @ 3:55 pm

A new report by the Measures of Effective Teaching project and supported by the Bill & Melinda Gates Foundation looks into the accuracy of classroom observations, as compared to and combined with other things such as value-added measures. The results are too long to discuss in a blog post, but one point caught my eye.

Much of the report compared the “predictive power” of different evaluations. That is, it looked at how likely any given type of evaluation (classroom observations, rigorous student surveys, value-added measures, or a combination), was to predict student gains on state tests in math and reading.

The graph that caught my eye compared the predictive value of a combination of these evaluations vs. the predictive value of graduate degrees or years of experience, the two measures used most often in determining raises and layoffs. For graduate degrees and experience, the predictive value was slight; but for the combined evaluation measures, the predictive value was noticeably more significant. Students who had the top 25% of teachers (as identified by the combined evaluation) gained roughly between one and five more months’ worth of learning than those who had teachers in the bottom 25%.

The point is this: a combined evaluation, using multiple measures of a teacher’s effectiveness including value-added measures, was more effective in identifying the teachers who boosted students learning than any of the traditional measures such as experience or degrees. And the report showed elsewhere that a combined evaluation was more effective than any single evaluation measurement.

My takeaway? Please, let’s use data in evaluating teachers. It doesn’t just have to be numbers — for instance, those rigorous student surveys did a good job, too. But as we argued in Building a Better Evaluation System, value-added models have a place in evaluating teachers as part of a system of multiple measures.

It’s time to build a better evaluation system. It won’t be perfect, but the evidence is mounting that it will be better than what we’ve got. –Rebecca St. Andrie

Permalink | Trackbacks | Comments (3) |

December 30, 2011

Taking the test

Filed under: Testing,Uncategorized — Patte Barth @ 4:24 pm

A lot of us practically breathe test data in our various duties related to public education. But my guess is not a lot of us have taken the trouble to understand what the data means about the knowledge and skills students are actually being asked to demonstrate. Most of us have not taken the test.

Probably for this reason, a Washington Post blog post from earlier this week has gone viral. The article describes the experience of a Florida school board member who decided to find out for himself what the state test was demanding. I love it. When talking to audiences about standards and assessments, I always like to show them real test items because they very tangibly illustrate what is expected of students. And I always ask audiences to consider if this is something they would expect their own child or child they know to be able to handle at 4^th grade, 10^th grade or whatever. It is simply a really effective way to bring the conversation down to earth.

So I applaud this board member for taking the initiative to discover what it means to pass the FCAT from the student’s perspective. Unfortunately, it didn’t go so well for this highly educated professional. He reports:

The math section had 60 questions. I knew the answers to none of them, but managed to guess ten out of the 60 correctly. On the reading test, I got 62%. In our system, that’s a “D”, and would get me a mandatory assignment to a double block of reading instruction.

He concludes that

[a] test that can determine a student’s future life chances should surely relate in some practical way to the requirements of life. I can’t see how that could possibly be true of the test I took.

He was particularly critical of the mathematics that he says most students won’t need.

I understand his reaction, but don’t agree. For one thing, he admits he has not been in school for a long time. There is a test-taking mindset that we lose in the real world but is familiar to students who would not be thrown by items that may seem impractical but in fact get at some important skills. Moreover, the changing world demands knowledge and skills way beyond what was required even two decades ago, especially in math. The Center’s report, Defining a 21^st century education, makes this case very well.

Anyway, I looked at the FCAT myself and that’s how I see it. But how about all of you? You can take a complete Florida grade 10 released test here. Download “test with answers” to see how Florida students performed on each item. The most recent released test is from 2006 and Florida has developed a version 2.0. To get a sense of the new tests, go here and here.

Most states release test items to the public. Check out your department of education to see tests students take in your state. And for a national comparison, you can search the NAEP released test items here.

Let us know what you think and happy test-taking! – Patte Barth

Permalink | Trackbacks | Comments (0) |

December 8, 2011

What common core standards means for preschools

Filed under: instruction,Pre-k,preschool,standards,Uncategorized — Mandy @ 3:06 pm

Juliet D. Wolf, a Montgomery County kindergarten teacher, points out that “thinking P-12 is not the status quo.” However, recent studies, such as the Center for Public Education’s Starting Out Right: Pre-K and Kindergarten, have brought attention to the significance early childhood preparation can have on college and career-readiness.

Jaclyn Zubrzycki’s recent Ed Week article, Common Core Poses Challenges for Preschools, addresses the tightrope that early childhood educators are currently tiptoeing on as they determine “how to balance the common standards’ emphasis on increasing and measuring academic rigor with research findings on young children’s developmental needs.” Upon first reading this article, I thought to myself that standards should be solely for academics, but this begs the question: Can successful academic standards for early childhood grades (preschool through 3^rd grade) ever really be separated from child development?

Every state has a set of preschool skill guidelines but few states actually put “issues of school readiness and the content of those preschool skill guidelines in the spotlight.” However, this school year (2011-2012) marked the first year the common-core standards were used for kindergarten in fourteen states, which in turn, marked a need for more calibrated preschool standards. Some preschool programs are being proactive, such as Head Start, which “recently aligned its Child Development and Early Learning Framework with the common core.” Aligning these standards seemed logical since Head Start is a program that, like the common-core standards, crosses many state lines. However, according to the article, the National Governor’s Association and the Council of Chief State School Officers has “no plans to create a nationwide common-core standards” set for preschool, after all “early-childhood education is not funded or regulated in the same way as K-12 (and) providers range from tiny, private, home-based daycare centers to fully aligned, state-funded programs.” The lack of alignment concerns some early childhood educators who recognize that “time for play, the arts, and developing communication and executive-function skills are more important for young children” than rigorous standards.

So to answer the question; Can successful academic standards for early childhood grades (preschool through 3^rd grade) ever really be separated from child development? I would say the annoyingly indefinite, “it depends.” First off, I have not read all of the kindergarten common-core standards to know if they are reasonable and second, new reforms always require some tweaking and patience. I would, however, say with some conviction that standards should be solely for academics whereas stages of childhood development should be “how” the standards are taught, which should be up to the teacher’s discretion. Just because a standard may list a set of academic goals does not mean an early-childhood educator has to teach them in a bland fashion. Rather, most standards can be taught effectively in young grades by hands-on instruction, art, and engaging a child’s creative side… even the rigorous ones. –M. Newport

Permalink | Trackbacks | Comments (0) |

« Newer Posts — Older Posts »