Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

March 14, 2013

Study examines links between the rigor of Algebra I and Geometry course content and test scores

The National Center for Education Statistics recently released a study examining the relationship between the rigor in Algebra I and Geometry courses high school students take and student test performance in those areas on the 12th grade National Assessment of Educational Progress (NAEP).  This study was spurred by positive findings from the 2005 NAEP  High School Transcript study, which found that in 2005 high school students earned more math credits, took higher level math courses, and obtained higher math course grades than in 1990. 

While it is certainly good news that more students are taking more math courses at higher levels and earning higher grades, it’s not clear whether students are taking courses that are truly rigorous or whether or not this uptick in math course enrollment is resulting in more student math achievement.  This study aims to answer that question.  The study was unable to actually observe classroom instruction in order to measure rigor; however, the researchers gained access to math textbooks used in 550 public schools, analyzing the rigor of the problems in the textbooks to determine how demanding classes are.  Previous studies have shown that math textbooks are closely related to math classroom instruction and serve as a good proxy for actual course rigor.  After coding the textbooks to determine whether or not they represented basic, intermediate, or advanced levels of rigor, the study matched those rigor levels to math NAEP scores to see if there’s a relationship. 

The overwhelming finding is there is a clear relationship between classroom rigor and NAEP scores.  Students in rigorous Algebra I courses and Geometry courses scored higher on NAEP than students in basic or intermediate courses.  On the other hand, the study also finds that the labeling of a course (i.e.-whether a course is regular or honors) often has little relationship to the true rigor offered in a course.

Here’s a detailed breakdown of the findings:

  • Graduates in rigorous Algebra I courses and Geometry courses score higher on NAEP.
    • Algebra I rigor level with corresponding NAEP scores (10 points is roughly equivalent to a year’s worth of learning):
      • Beginner: 137 points
      • Intermediate: 143
      • Rigorous: 146
    • Geometry rigor level with corresponding NAEP scores:
      • Beginner: 148
      • Intermediate: 152
      • Rigorous: 159
  • School course titles often do not truly represent the level of rigor a course offers.
    • 73% of students who took an honors Algebra I course actually received an intermediate Algebra I course.
    • In fact, a higher percentage of students in a regular class received a rigorous course than those in courses labeled “honors”
      • Regular title, but curriculum was rigorous: 34%
      • Honors title, but curriculum was rigorous: 18%
    • In Geometry classes, only 33% of courses title honors were actually rigorous, while 62% were intermediate, and the rest were basic.
  • Generally, about two-thirds of an Algebra I or Geometry course covers core content; the rest is a review of lower level material or a preview of higher level material.
  • Most students, regardless of race or course title, took an intermediate level Algebra I course.
    • 54% of high school students took an intermediate Algebra I course, while 14% had a beginner course, and 32% had a rigorous course.
  • Most students, regardless of course title, took an intermediate level Geometry course.
    • Classes titled “Informal”: 54% had an intermediate course (30% basic, 14% rigorous)
    • Classes titled “Regular: 68% had an intermediate course (11% basic, 19% rigorous)
    • Classes titled “Honors”: 62% had an intermediate course (4% basic, 33% rigorous)
  • While racial differences weren’t present in differences in rigor level for all other courses, racial differences were present for Honors Geometry rigor levels.
    • 37% of white students had a rigorous Honors Geometry course, while 21% of Black graduates and 17% of Hispanic graduates had a rigorous Honors Geometry course.
  • While a higher level of rigor in a Algebra I or Geometry course resulted in higher NAEP scores, white graduates still scored higher than Black or Hispanic graduates on the Algebra I and Geometry portion of NAEP, regardless of the rigor level of their math test:
    • White students rigor level of Algebra I and Geometry course and corresponding NAEP scores:
      • Algebra I
        • Basic: 142
        • Intermediate: 148
        • Rigorous: 151
      • Geometry
        • Basic: 155
        • Intermediate: 159
        • Rigorous: 165
    • Black students rigor level of Geometry course and corresponding NAEP scores:  
      • Algebra I
        • Basic: 128
        • Intermediate: 129
        • Rigorous: 134
      • Geometry
        • Basic: 120
        • Intermediate: 129
        • Rigorous: 133
    • Hispanic students rigor level of Geometry course and corresponding NAEP scores:
      • Algebra I
        • Basic: 127
        • Intermediate: 132
        • Rigorous: 132
      • Geometry
        • Basic: 140
        • Intermediate: 138
        • Rigorous: 138

 

Filed under: Achievement Gaps,Course taking,instruction,NAEP,Report Summary,research — Jim Hull @ 3:06 pm




Misconceptions about new teacher evaluation systems

I was checking out Diane Ravitch’s blog and came across this posting about the dangers of the new teacher evaluation systems currently being implemented across the country. The post really caught my attention because it was written by JoAnn Bartoletti, executive director of the National Association of Secondary School Principals (NASSP).  In many of these new teacher evaluation systems, more responsibilities are being placed on principals that already have too much on their plates as it is. So, I was really interested in what Ms. Bartoletti had to say.

While she made some very important points about the unworkable time commitments some of these new evaluation systems require of principals, most of the criticisms lacked a thorough understanding of what research says about effective teacher evaluation systems.

Here are a couple of misconceptions she and others have about new evaluation systems:

The new Common Core State Standards (CCSS) and teacher evaluation systems could discredit and dismantle public education because student test scores are likely to decline under the CCSS assessments.

While test scores are likely to decline under the CCSS since students in most states will be held to a higher standard than they have been under their current state standards, this will not lead to more teachers being identified as ineffective. For one, half or more of teacher evaluations are based on qualitative measures such as principal observations which would be unaffected by lower test scores. Second, the most common student achievement measures used in teacher evaluations such as value-added are normative measures that basically compare each teacher’s student’s achievement to the student achievement of an average teacher so even if every student’s score dropped teachers would not receive a lower evaluation score.

Having 30 percent or more of a teacher’s evaluation based on student achievement is not supported by research

Not true, the Measures of Effective Teachers (METS) study found that student achievement should account for between 33 and 50 percent of a teacher’s evaluation. Such a proportion provides the most accurate assessment of a teacher’s true effectiveness.

Teacher evaluations shouldn’t be used for both formative and summative purposes

I’d have to disagree as teacher evaluation systems should be designed for dual purposes. Unlike student assessments, where it isn’t wise to use an assessment for dual purposes, evaluation systems for any industry are specifically designed to be both formative and summative. There is no research to say that they should be used for one purpose or the other.

  • The problem is actually that some states may use the same student achievement measure to evaluate both teachers and schools. This is a bad idea. The student achievement measure in teacher evaluations should be designed specifically to evaluate teachers. Research on this point is solid.
  • If principals are reluctant to give a teacher an honest critical review in fear it would leave a black mark on the teacher’s record then outside observers should rate teachers, as is the case in a number of states.

Principals are required to spend too much time evaluating teachers

While I disagree with the criticism that training principals how to evaluate teachers is a waste of time, I do agree that principals shouldn’t have to spend as much time as they are now required in many states to evaluate teachers. However, states have been adjusting how much time principals have to spend evaluating teachers in response to this concern.

Much time is wasted since these new systems are not identifying more ineffective teachers

It is true that just about 3 percent of teachers are being identified as ineffective under new evaluation systems. Although small, it is much more than the mere 1 percent of teachers that previous evaluation systems had identified as ineffective. So I wouldn’t say they are identifying the same.

Remember too, these new evaluation systems are not simply designed to identify ineffective teachers. They are also designed to improve the performance of all teachers. Previously, ineffective teachers remained in the classroom with little support to improve. Furthermore, what the article neglected to mention is that these new evaluations are able to identify highly effective teachers as well and not just lump the best teachers with the average teachers.

Teacher evaluation systems have a negative impact on a school culture

There is no evidence that teacher evaluation systems have a negative impact on school culture. Yes, there is anxiety during evaluation time (as in any organization) but that is short-lived. A good principal will ensure that doesn’t happen as research shows good principals provide constant feedback to teachers so that their final evaluation is not a surprise. Furthermore, evaluations could and should be designed to encourage teamwork which would positively impact a school’s culture.

Truly effective teachers are being fired for poor performance

Unfortunately there has and likely will be more examples of where this is the case. There will be truly effective teachers identified as ineffective and may be fired but no evaluation system is perfect. On the other hand, previous teacher evaluation systems had consistently rated ineffective teachers as effective and left alone to teach for years without intervention which has been a disservice to millions of students.  Keep in mind, however, that evaluation systems are being tweaked to minimize the chances a truly effective teacher is rated as ineffective.

Evaluating teachers serving traditional low-performing populations is unfair

This would be true if the new evaluation systems used student overall test scores, which virtually none (if any) do. Teachers are typically evaluated on how much growth their students make during the year which is a much fairer way to evaluate teachers. However, the fairest way is to also account for the type of students they teach by using a value-added model. Research has shown that teaching traditional low-performing students does not negatively impact a teacher’s value-added score. – Jim Hull

For more on how to accurately evaluate teachers check out CPE’s Building a Better Evaluation System.

Filed under: Assessments,CPE,Growth Models,Principals,research,Teacher evaluation,teachers — Tags: , , , — Jim Hull @ 1:42 pm




February 27, 2013

Great gains in high school graduation rates

 

Great news!! The U.S. is on-track to have a 90 percent on-time graduation rate by 2020 according to a new report from America’s Promise Alliance.

Such news should be plastered all over the newspapers and lead the nightly news. Unfortunately, that hasn’t been the case. But it is news worth celebrating nonetheless.

Specifically, the report found:

 

  • As of 2010, the national on-time graduation rate is 78.5 percent up from 71.7 percent in 2001, a 6.5 percentage point increase.
  • The graduation rate has increased by 5 percentage points between 2006 and 2010.
  • Due to this increase, nearly 200,000 more students graduated in 2010 than would have if the rate remained the same as in 2006.
  • If this annual improvement is maintained over the next decade the U.S. will have a 90 percent on-time graduation rate by 2020.
  • Hispanic and African American graduation rates made significant rains during this time period.
  • Two states, Wisconsin and Vermont, already have achievement a 90 percent on-time graduation rate.
  • Over one million (1.1 million) fewer students attended so-called ‘Dropout Factories’ in 2011 compared to 2002. Such schools graduate less than half of their incoming 9thgraders within four years.
  • The percent of African American students attending dropout factories declined from nearly 50 percent in 2002 to just 25 percent in 2011.
  • Hispanic students were also less likely to attend a dropout factory, where the percent of Hispanic students attending such schools declined from nearly 39 percent to 17 percent over the same time period.

Of course, there is a lot more work to be done but our schools should be given credit where credit is due. For decades our on-time graduation rate stubbornly hovered around the 70 percent mark with no signs of budging. And now it is closing in on 80 percent and heading towards 90 percent by 2020.

Our teachers and administrators should take pride in making such progress and policymakers should support them with the resources they need by investing in early dropout warning systems and high quality dropout preventions as well as ensure all students have access to effective teachers and a rigorous curriculum. These are tools our research shows have helped move our on-time graduation rate to nearly 80 percent and will be vital in graduating 90 percent of students by 2020.

Filed under: Graduation rates,High school,Public education,Report Summary,research — Tags: — Jim Hull @ 7:35 am




February 14, 2013

What is the economic benefit of vouchers?

A recent study of the Washington, DC voucher program purports to show that the voucher program yields a whopping 162 percent rate of return because more students graduate due to the voucher program than would have without the program in place. Not too shabby. However, a closer look at how they came to this number raises some serious questions about such large rates of returns.

When it comes to calculating the economic benefits of the voucher program the study does use a similar methodology as other studies that have calculated rate of returns on other education policies such as high quality Pre-k and dropout prevention programs, among others.

In this, as well as, previous studies the economic benefit was based on the increased wages and lower unemployment rates of those who graduate high school compared to dropouts, so more high school graduates means more tax revenues.

Moreover, high school graduates tend to require fewer government services so they will use less taxpayer money as well.  While there is certainly room to disagree with the accuracy of these measures, they do provide some estimate of the economic benefit of such education programs. So the question is not how much students benefit from the program.

The big question is actually how many students benefit from the voucher program. The economic benefits may be large but if very few students graduate high school due to the voucher program then it doesn’t matter how great the economic benefits are. So the larger question really is whether the voucher program improves graduation rates as much as the report states it does?

To determine how many more students graduated high school due to the voucher program, the report used the official evaluation of the voucher program which found graduation rates were 12 percentage points higher for those students who used the voucher and those students who entered the lottery but did not receive a voucher. According to the study, if this rate held true more than 400 students per year would graduate high school due to the voucher program.

However, I’ve looked over the report and while its methodology is sound, there are number of issues that raise the question whether the increase in graduation rates can really be attributable to the voucher program?

A major issue to keep in mind is that the voucher evaluation was about determining the impact of having a voucher system not about whether students were better off using the voucher. In fact, the majority of students who were offered vouchers never used them. They either attended a charter school, remained in their traditional public school, or attended a non-participating private school.

Moreover, half of the students who entered the voucher lottery but did not win a voucher (control group) went on to attend to a charter school or a private school. So the 12 percentage point increase in the probability that a student will graduate is based on whether a student was ‘offered’ a voucher, not whether the student actually ‘used’ the voucher.

What the evaluation actually found was simply offering a student a voucher— even if they remained at their local public school or charter school— improves a student’s chances of graduating high school. I am a strong believer in the ‘invisible hand’ and the power of competition. However, this just doesn’t make sense to me. If the evaluation was simply measuring the impact of vouchers to create competition then that impact should be felt by all students even those who didn’t enter the voucher lottery.

This makes the claim that vouchers increase graduation rates 12 percentage points quite suspect, especially since the same evaluation found that the voucher program had no impact on student achievement. One reason this may be the case is that test scores are objective measures provided by the schools. However, the graduation rates of those students who took part in the voucher program are based on parent responses to follow-up surveys.

It could be the parents of students who were offered a voucher, claimed their child graduated but did not. Also, we don’t know if students who graduated from private schools were required to meet as rigorous requirements as students in public schools.  So, it really is questionable whether the DC Voucher program has produced more high school graduates prepared for the real world.

What this new study shows is that any program that increases high school graduates will yield significant economic benefits in the form of additional tax revenue and less demand for government services.  But whether the DC voucher program does actually increase the chances a student will graduate high school is still not known, as the original evaluation failed to isolate the impact of using a voucher. Without an accurate measure of the number of students who graduated high school due to the voucher program it is impossible to say that the DC voucher program provides a 162 percent rate of return.

Filed under: Charter Schools,Graduation rates,Public education,research,vouchers — Tags: , , — Jim Hull @ 7:45 am




February 7, 2013

Are charter schools better than traditional public schools?

Been there done that. As CPE’s report Charter Schools: Finding out the facts found a couple years ago, ‘on-average’ charter schools perform slightly better than traditional public schools. However, the report noted that the quality of charter schools varies dramatically from very effective to very ineffective. In fact, 83 percent of charter schools were no more effective than the neighborhood traditional public school their students would have attended. Yes, a couple of years have gone by since we released that report but no new research has come out showing anything different.

In fact, several studies examining the effectiveness of charter schools have been released over the past couple of years that have solidified those findings.  A number of these studies were conducted by CREDO the Stanford University research organization that conducted the most rigorous and comprehensive study of charter schools across 16 states. Since that time they have conducted several other similar studies based on individual states like New Jersey, Indiana, Michigan, and Pennsylvania which provided similar findings to their 16 state study  where they found that ‘on-average’ charter schools are just as, or slightly more, effective than traditional public schools but that the charter school average is pulled up by a small number of very effective charter schools typically located in large urban districts.

To counter these findings some charter school proponents have stated that charter schools don’t stack up as well in these studies due to the large number of new charter schools. The point being that just like new teachers and principals it takes a couple of years for a school to be as effective as they are capable of being. Certainly a valid point. However, a new CREDO report found that this is not likely the case as nearly all charter schools tend to be as effective in their first year as they will be five years later. In particular, CREDO found that 80 percent of the lowest performing charter schools remain low performing five years later. So growing pains are not the reason charter schools don’t stack up better compared to traditional public schools in research.

After years of debating whether charter schools are more effective it is time to move on.  Charters schools are here to stay but are certainly no silver bullet. Now researchers should focus on what makes those high quality charter schools so effective. Not only to improve charter schools but to take those lessons and apply them to traditional public schools where applicable. The most recent CREDO report does just that in fact. The latest report is a must-read for any school board or other organization that has the power to authorize charter schools as it provides insights into what makes for an effective charter school. Authorizers, policymakers, and educators need more research like this so they can make more informed decisions about what is best for our all students not just those who attend the highly effective charter schools.

Filed under: Charter Schools,research — Tags: , , , , — Jim Hull @ 12:27 pm




Older Posts »
RSS Feed