The EDifierJuly 27, 2010 The ability to relate various school factors to student outcomes is a powerful tool for educational improvement. This data allows educators to examine their own practices for their effect on student learning and helps them to identify what is working and what they need to improve.
Policymakers also see a lot of potential in this data as a key element in school reform policies, notably those intended to assure that all students have access to effective teachers. The federal Race to the Top competition awards its sizable grants in part on states’ capacity to link teachers to student performance, and judging by the amount of state action in this area, the federal strategy seems to be working.
In my work, I love having data to help me gauge how I’m doing, whether it’s in the form of survey feedback or analyzing web traffic. I think educators would likewise welcome having really good information about the impact of their instruction. But like all tools, you need the right one to fit the purpose.
As we have written in these pages before, the statistical methods that produce this data are not designed to do all things (see in particular Measuring student growth). For example, data that is used for hiring or compensation decisions must meet a more rigorous standard than data that teachers use to reflect on their own practices. In addition, it can be difficult to get data for teachers who teach subjects that aren’t tested.
The Data Quality Campaign recently published a very useful policy brief that lays out major implications that policymakers should consider in putting together a valid and reliable teacher-student link (Effectively Linking Teachers and Students). Foremost among these are purely logistical: how do you define the teacher of record? what about team teaching? teachers of untested subjects? virtual classes? are student rosters up to date and accurate? DQC emphasizes that while IT and data staff will necessarily be at the table to solve these problems, the issue is fundamentally a policy matter. They recommend that any system be designed in collaboration with all the stakeholders, especially the teachers it will affect. We agree. –Patte Barth
July 21, 2010 NBC has had a great idea and decided to put education in the spotlight with its Education Summit this September. However, I have to say NBC needs a little education itself on how to read data.
First off, NBC provides the statistic that U.S. students ranked 21st in the world in math and 25th in science on the international assessment PISA. First, PISA results are not designed to rank countries, as if testing was like a horse race (read more about in the Center’s Guide to international assessments).
Second, PISA is just one international assessment. In TIMSS, which assesses the math and science performance of 4th and 8th graders, U.S. students compare favorably to their peers by scoring significantly (see statistically significant) above the international average in math in both grades. At the 8th grade level, U.S. students scored significantly better than 37 out of the 47 participating countries. Only 5 countries (all Asian countries) scored significantly above U.S. 8th graders. We may not be number one, but we’re doing a lot better internationally than you would think from NBC’s press release.
When using international assessments NBC just didn’t give a full picture of how U.S. students really perform. When using national assessments, NBC just got it plain wrong. NBC claims that NAEP scores show that 68% of eighth graders can’t “read at grade level.” This is not true. Sixty-eight percent of eighth graders scored below Proficient in 2009.
If NBC had read our report The proficiency debate, it would know that NAEP’s “Proficient” level is not synonymous with “on grade level.” Proficient is a higher standard than being on grade level. So saying that two-thirds of eighth graders can’t read at grade level is plain wrong.
It’s not that all our schools are doing so well that we don’t need to do anything. NAEP’s “Below Basic” level shows that far too many students perform poorly in our schools. But exaggerating the problems and painting all schools with the same brush will do nothing to help the students that need the most help. Across the country, there are countless traditional public schools that are just as or even more successful than highly touted successful charter schools such as KIPP and Harlem’s Children Zone. Although you can read about the traditional public schools’ success stories here, they rarely garner the national media attention successful charter schools receive. If we focus too much on national averages, as NBC has done, we will lose sight of these successful public schools – and their lessons on how to improve other public schools.
So, when evaluating our public schools we have to be careful to get the picture right. We have numerous great public schools that parents, communities, and local businesses know are meeting the needs of their students. We also have too many schools that are struggling to meet tremendous challenges. Instead of painting all schools with the same negative brush, NBC should ask these questions: what are the problems, where are they, and what can we do about them? Basing our decisions about education on a couple of misinterpreted pieces of data will only impair our effective schools while denying ineffective schools the support they need. – Jim Hull
June 9, 2010 What are “virtual charter schools,” and what does it mean for education that they seem to be growing? Right now, the answers include way too many unknowns.
In our recent report on charter schools, authors Eileen O’Brien and Chuck Dervarics noted, “Education management organizations (EMOs) are creating a growing number of virtual or cyber charter schools, which deliver curriculum and provide instruction via the Internet and electronic communication. For-profit EMOs have increased the number of virtual charter schools they operate from thirteen in the 2003–2004 school year to fifty by 2008–2009…There is little research on virtual charters, and a recent review of research (Cavanaugh 2009) indicated “mixed outcomes” when comparing virtual charter school achievement with classroom-based charter programs.” Read the full report to see how little is formally known about virtual charter schools.
Now a recent article in The New York Times questions the funding received by one of these online virtual charter schools. As presented by the article, it’s unclear how the state and federal money received from these schools is used. While some goes toward curriculum, computers, and support, the lack of other services (transportation, for one) brings up questions of where the “extra” money goes.
But there are differing reports about the amount of money received, and no clear explanation in the article on how these students are, or are not, part of the public school system. Charter schools, remember, are public schools. But the students described in the article, while they seem to be tested under the public school accountability system, are more like homeschool students than public school students.
The lack of clarity raises a lot of questions. Is the federal and state money received being used for its intended purpose? Should we evaluate these students’ performance as part of public schools in general, charter schools, or home schools? They seem to be a hybrid of both. And who is ultimately responsible for the students’ learning? Such confusion can’t be good for anybody, no matter what kind of education the students are receiving.
To me, it just seems like one of those situations where each of the different groups involved thinks the other group is responsible for or taking care of a specific task. I’ve been in enough of those situations myself to know that they usually end badly. With all these virtual unknowns floating around, I think somebody had better find out something soon. –Rebecca St. Andrie
January 12, 2010 A group supporting charter schools in DC, FOCUS (Friends of Charters in Urban Schools) has put up a new web site, http://focusdc.org/schoolquality.swf. The site offers several different ways to look at the school achievement scores and growth over time of all DC schools.
My focus isn’t on charter vs. non-charter performance, but rather on the tool they created to look at the data. It’s clear and easy to use. But most importantly, the scatterplot graph they use presents some interesting questions. This type of graph takes the usual percentages you hear and forget (85% proficiency, 5% gain) and transforms them into an active picture of who’s moving up, who’s moving down, and who’s staying stable (or stagnant).
Take a look. What do you think? What questions do graphs like these raise for you? –Rebecca St. Andrie
November 11, 2009 On Monday I attended the U.S. Chamber of Commerce’s National Education and Workforce Summit, where they released the second installment of their Leaders and Laggards report. The Chamber released the first installment two years ago which graded states on their education achievement. This installment, however, graded each state on their educational innovation in eight categories using 47 indicators.
The categories include:
- School management
- Finance
- Staffing: Hiring and evaluation
- Staffing: Removing ineffective teachers
- Data
- Technology
- Pipeline to postsecondary
- State reform environment
For each of these categories states were given a grade (A – F), mainly on the basis of whether a state had implemented a policy or not. In most of the cases, it didn’t matter if the policy was effective, it just mattered if there was one in place.
For example, part of a state’s Finance grade was based on whether states allow teachers to be paid based on their performance. However, the Center’s Teacher pay for performance report shows that such pay for performance plans have shown only modest gains in student achievement and that there is little evidence yet on what type of rewards are most effective.
Furthermore, the Center’s Wanted: Good teachers report also points out that the impact of alternative certification programs are quite mixed. Yet, part of each state’s Staffing: Hiring and Evaluation grade is based on the percent of teachers who entered the profession through an alternative certification program regardless of its quality. So the grades in these areas do not necessarily represent how effective state policies are at improving student learning.
Yes, our schools do need to be innovative. But just implementing policies with no track record of improving student learning isn’t exactly innovative. As a matter of fact, it is something that has been done for far too long in education.
What would be truly innovative is if states were graded on how successful they were at scaling up successful programs and eliminating those programs that aren’t effective. Another truly innovative approach to grading states and our schools would be to grade them on how much they improved their student’s learning. The Chamber didn’t give grades for either of these measures in either installment of their report. –Jim Hull
Older Posts »
 RSS Feed |
|
Print