Learn About: 21st Century | Charter Schools | Homework
Home / Edifier


The EDifier

June 13, 2017

New research: Community schools are an evidence-based strategy for school improvement

Last week, in a ceremony at the National Press Club in Washington D.C., six schools and community-based initiatives across the country were recognized for their excellence in utilizing the community schools model. The Coalition for Community Schools highlighted the considerable achievements of schools from New York City, Nashville, Chicago and Oakland.

CommunitySchoolsShotThe National Education Policy Center (NEPC) and the Learning Policy Institute (LPI) also presented new research at the event that supported the use of the community schools model as an evidence-based strategy for school improvement under the Every Student Succeeds Act (ESSA). ESSA requires that all interventions meet the “evidence-based” requirement, and this new research suggests that community schools more than meet that standard.

The community schools model may be a particularly effective strategy for improving schools in areas that struggle with high rates of poverty, because it creates a support system for students and families that addresses needs outside of the academic curriculum. Community schools create a system of partnerships and collaborations that address the needs of each child not only as a learner but also as a community member.

Because the needs and assets of each community are unique, there is no one formula for creating a community school. Each community school takes a unique approach to the model depending upon the circumstances of its students and families. However, all form partnerships and collaborations to create a set of integrated services that meet the needs of the whole child. Most are open before and after school—some even on weekends and during the summer—to provide students with wraparound support. Community schools provide services such as physical and mental health screenings, parent and community resources, and expanded learning opportunities like sports and arts programs.

Despite the variety of approaches, NEPC and LPI were able to identify common aspects of the community schools model that lead to success, including a wraparound student support system and a high degree of community collaboration and engagement. The newly released research also found that for every dollar invested in a community school, there will be a $10 to $15 return on investment within the community. In the awardee schools, chronic absenteeism and discipline referrals have decreased, test scores have increased, and fast academic growth has resulted in rising state ratings. Across the board, students and families report closer school and community ties. Using a wraparound support system, community schools may be a tool to close achievement gaps, prepare students for college and future careers, and promote positive outcomes throughout the broader community.






February 7, 2017

School Improvement Grants: Why didn’t $7 billion change results for students?

Mathematica recently released a study of the federal program of Student Improvement Grants (SIG). Their findings? Schools receiving the extra funds showed no significant improvement over similar schools that did not participate. With a price tag of $7 billion (yes, with a “b”), this strikes many as a waste of taxpayer dollars. Interestingly, the study also found no evidence that the SIG schools actually had significantly higher per-pupil expenditures than similar schools that didn’t receive the grants, which may have contributed to the mediocre results.

SIG awarded up to $2 million annually to 1,400 schools, which was administered by states. The program began in the 2010-11 school year and continues through the end of the 2016-17 year. Starting in 2017-2018, the new Every Student Succeeds Act (ESSA) will allow states to use up to seven percent of their Title I allotments to improve the bottom five percent of schools. States may choose to dole out funds via formula or competitive grants, but districts are the ones responsible for using evidence-based practices to improve schools.

Under the old SIG rules, the federal government required schools to choose from one of these four turnaround models:

SIG 1

The new report analyzed transformation, turnaround, and restart models, and found no statistically significant effects for any of them. The authors did find positive, but not statistically significant, effects on math and reading scores for schools receiving the grant, but lower high school graduation rates. Critics of the new report have noted that the mathematical model chosen was not sensitive enough to detect small effects. The authors did find mixed effects each year, which many studies would have the power to find as significant, but due to the design, these remain insignificant. To give perspective of the magnitude of these effects, the effect of decreasing elementary class sizes by seven students is about 0.2 standard deviations; the effect of urban charter schools compared to their neighborhood schools after one year is 0.01 in math and -0.01 in reading (0.15 and 0.10 after four years). According to the Mathematica study, the results of SIG in 2012-2013 were 0.01 standard deviations in math and 0.08 standard deviations in reading, with a drop of in the graduation rate (note that SIG had a positive impact on the graduation rate in 2011-2012, which suggests that these results are not statistically significant, or could be zero). Not enough to conclude a positive effect, for sure, but not nothing, either.

 

SIG3

I’ll offer a couple of my own thoughts (based on research, of course) on why SIG didn’t have the success that was hoped for:

1. The authors found no evidence that the grant funds actually increased per-pupil spending. In government-speak, the funds may have supplanted other funding streams instead of supplementing them, even though the law states that federal funds are supposed to supplement other funds spent. They found that SIG schools spent about $245 more per student than similar non-SIG schools in 2011-2012, and only $100 more in 2012-2013 (again the results are not statistically significant, meaning that we can’t confidently say that the difference isn’t zero). Recent studies have shown that spending makes a difference in education, so this may help explain why we didn’t see a difference here.

2. Students in many priority schools (the bottom five percent of schools), which are the ones that qualified for SIG grants, may have had the option to transfer to higher-performing schools. While the report doesn’t address this, it seems that students with more involved parents and better academic achievement may have been more likely to utilize this offer, thus lowering the average scores of the schools they left behind. Students perform better when surrounded with higher-performing peers, which means that the lack of overall effect could have been influenced by the loss of higher achieving students.

3. Schools receiving SIG grants were high-poverty and high-minority. The average rate of students eligible for free-and-reduced price (FRL) lunches in the study group was 83 percent, with non-white students making up 91 percent of the school populations (as compared with the overall school population being about 50 percent FRL-eligible and 50 percent non-white). While the resources allocated through SIG to these schools should have made spending more equitable, schools may have still struggled with recruiting and retaining experienced, qualified teachers, which is often a challenge for high-poverty, high-minority schools. Research is clear that integrated schools have better outcomes for students than segregated schools. Yet, the reform strategies used under SIG (replacing school staff and/or converting to a charter school) did little to improve school integration.

Hopefully, states and districts will learn from these lessons and use school reforms that fundamentally change the practices of the school, not just a few personnel: increased funding, school integration, changes in instructional practices, meaningful teacher/principal mentoring and development, and/or wrap-around services for students in poverty or who have experienced trauma.






February 6, 2017

School district partnerships with afterschool can help meet ESSA goals

Today’s post is from guest-blogger Jillian Luchner, who is a Policy Associate with the Afterschool Alliance.  The Afterschool Alliance is a nonprofit public awareness and advocacy organization working to ensure that all children and youth have access to affordable, high-quality afterschool programs.

 

The passage of the Every Student Succeeds Act (ESSA), the long-awaited successor to No Child Left Behind, creates a unique framework for school boards, teachers, administrators and communities to work together to make sure all children have access to high-quality, well-rounded education.

At the NSBA’s January 19 forum, “Public Education Agenda for America’s Success,” panelists discussed how the new law, new administration, and new Congress would affect education across the nation. Despite some level of uncertainty, panelists spoke to how school boards and local – even family level -decision making could be expected to play a larger role than in the recent past. When asked specifically about what school boards might do, much of the panelists’ conversation focused on the regular school day, but panelist Gerard Robinson of the American Enterprise Institute noted that afterschool programs are a time-tested, research-based part of the solution that should not be overlooked.

Afterschool and summer programs across the nation have a strong history of supporting school systems’ efforts to provide students with a well-rounded education that puts them on the path to wellness and success. These out-of-school-time programs provide students with educational opportunities, enrichment activities, access to physical activity and nutritious meals and snacks, as well as opportunities to build leadership and social connections. Notably, afterschool programs do all that during what’s sometimes called “prime time for juvenile crime” – the afternoon hours when children are most likely to be either perpetrators or victims of crime  and when working parents worry most about their children’s safety.

Research shows that students who regularly attend quality afterschool programs improve their academics, have better school attendance and are more likely to graduate. Moreover,  , the Afterschool Alliance’s recurring, nationally representative parent survey, consistently finds that parents strongly support afterschool programs. In the 2014 survey, the most recent, 89 percent of parents with a child in a program reported being satisfied with the program. In addition, 84 percent of all parents supported public funding for afterschool, while more than 7 in 10 said they think afterschool programs reduce the chance that their child will participate in risky behavior. Additionally, 80 percent of parents report that their children’s programs offer students opportunities for physical activity, and three in four parents are happy with the healthy snacks their student’s program provides. Despite high demand, for every child enrolled in an afterschool program, the parents of two more children say they would sign their children up, but cannot either because a program isn’t available or because it isn’t affordable.

District school boards often play an important role in leveraging resources to expand access to afterschool and summer opportunities. Afterschool and summer programs are frequently operated as a partnership among community nonprofits and school districts, with funding from federal, state and local sources as well as businesses, foundations, parent fees and other contributions. The average 21st Century Community Learning Center (a federally funded program that supports competitive grants in every state) has nine partner organizations with which it coordinates, which contribute in financial and in-kind support.

Across the nation, school system partnerships with afterschool programs have expanded opportunities for students while helping districts meet their goals for student success and family involvement.

  • The Pittsburgh, Pennsylvania Public School System has an out-of-school-time office that works with 80 different district partners to oversee summer programming for thousands of district students. Using research-based systems of support, the office coordinates closely with out-of-school time providers in the district and supports data, quality and systems-building to meet the city’s goals of graduation and college- and career-ready students.
  • In the early 1990s, the Corbin Independent School District (Kentucky) created the Redhound Enrichment afterschool program after conducting a community needs assessment in the district. Originally focused on providing a much-needed safe place for children in the afternoon hours, the program subsequently expanded its offerings to include more academic components, with the support of a 21st Century Community Learning Center grant. Twenty-five years later, the program is still in operation, employing a project-based learning model to provide a mix of academic supports, hands-on activities, physical activity and opportunities to primary and secondary students.
  • In Redwood City, CA the district builds partnerships with non-profit and private providers including youth centers, parks and recreation, Boys and Girls Clubs and the YMCA to offer afterschool opportunities. The programs give children opportunities for academic support, developing new skills and relationship building as well as other opportunities. Parent surveys show 97% satisfaction with the programs.

Such efforts are the tip of the iceberg, as afterschool programs across the nation work with school districts to promote student success.

With the Every Student Succeeds Act going into full implementation this year, now is a perfect time for districts to coordinate more closely with afterschool and summer learning programs. Final state plans are due to the federal Department of Education in either April or September and many state drafts (see our map) are out now. As part of that process, school districts will engage parents and other community stakeholders to consider how to meet state goals for improving graduation rates, academic achievement and student engagement and reducing chronic absenteeism.

Afterschool and summer programs are well-poised to help meet all these goals, and the 50 statewide afterschool networks stand ready to help connect school districts with afterschool and summer learning programs in their communities.

 

Jillian joined the Afterschool Alliance team as a Policy Associate in 2015. Her work involves tracking trends in afterschool policy and programs at the state and federal level and communicating successful and innovative approaches toward supporting youth during out-of-school time. Jillian worked for years as a teacher and afterschool educator in the Washington D.C. region. She also served as an AmeriCorps VISTA and community development director in California’s Central Valley. She holds undergraduate degrees in Economics and Geology and a Master’s in Public Policy specializing in education from the University of Maryland at College Park.

Filed under: After School,ESSA,Guest Blog,Parents,Public education — Tags: — Chandi Wagner @ 12:53 pm





November 17, 2016

What does “evidence-based” mean?

The Every Student Succeeds Act requires schools to use “evidence-based interventions” to improve schools.  The law also includes definitions of what evidence means, and recent guidance from the Department of Education has provided additional clarification on what passes as “evidence-based.”  Mathematica has also put out a brief guide on different types of data that have similar categories as the Department of Education, but also provide explanations for data we may see in the media or from academic researchers that do not qualify as hard data but can still help us understand policies and programs.

ESSA Evidence

What follows is a brief summary of what qualifies as “evidence-based” starting with the strongest first:

Experimental Studies:  These are purposefully created experiments, similar to medical trials, that randomly assign students to treatment or control groups, and then determine the difference in achievement after the treatment period.  Researchers also check to make sure that the two groups are similar in demographics.  This is considered to be causal evidence because there is little reason to believe the two similar groups would have had different outcomes except for the effect of the treatment.  Studies must involve at least 350 students, or 14 classrooms (assuming 25 students per class) and include multiple sites.

Quasi-experimental Studies:  These still have some form of comparison group, which may be between students, schools, or districts that have similar demographic characteristics.  However, even groups that seem similar on paper may still have systematic differences, which makes evidence from quasi-experimental studies slightly less reliable than randomized studies.  Evidence from these studies are often (but not always) considered to be causal, though experiment design and fidelity can greatly affect how reliable these conclusions are across other student groups.  Studies must involve at least 350 students, or 14 classrooms (assuming 25 students per class) and include multiple sites.

Correlational Studies: Studies that result in correlational effects can’t necessarily prove that a specific intervention caused students in a particular program to have a positive/negative effect.  For example, if Middle School X requires all teachers to participate in Professional Learning Communities (PLCs), and they end up with greater student improvement than Middle School Y, we can say that their improved performance was correlated with PLC participation.  However, there could have also been other changes at the school that truly caused the improvement, such as greater parental participation, so we cannot say that the improvement was caused by PLCs, but that further study should be done to see if there is a causal relationship.  Researchers still have to control for demographic factors; in this example, Middle School X and Middle School Y would have to be similar in both their teacher and student groups.

With all studies, we also have to consider who was involved and how the program was implemented.  A good example of this is the class-size experiment performed in Tennessee in the 1980s.  While their randomized control trial found positive effects of reducing class size by an average of seven students per class, when California reduced class sizes in the 1990s they didn’t see as strong of effects.  Part of this was implementation – reducing class sizes means hiring more teachers, and many inexperienced, uncertified teachers had to be placed in classrooms to fill the gap, which could have reduced the positive effect of smaller classes.  Also, students in California may be different than students in Tennessee; while this seems less likely for something like class size, it could be true for more specific programs or interventions.

An additional consideration when looking at evidence is not only statistical significance (whether or not we can be certain that the effect of a program wasn’t actually zero, using probability), but the effect size.  If an intervention has an effect size of 0.01 standard deviations* (or other units), it may only translate to the average student score changing a fraction of a percentage point.  We also have to consider if that effect is really meaningful, and if it’s worth our time, money, and effort to implement, or if we should look for a different intervention with greater effects.  Some researchers would say that an effect size of 0.2 standard deviations is the gold standard for really making meaningful changes for students.  However, I would also argue that it depends on the cost, both of time and money, of the program.  If making a small schedule tweak could garner 0.05 standard deviations of positive effect, and cost virtually nothing, then we should do it.  In conjunction with other effective programs, we can truly move the needle for student achievement.

School administrators should also consider the variation in test scores.  While most experimental studies report on the mean effect size, it is also important to consider how high- and low-performing students fared in the study.

Evidence is important and should guide policy decisions.  However, we have to keep in mind its limitations and be cautious consumers of data to make sure that we’re truly understanding how the study was done to see if its results are valid and can translate to other contexts.

 

*Standard deviations are standardized units used to help us compare programs, considering that most states and school districts use different tests.  The assumption is that most student achievement scores follow a bell curve, with the average score being at the top of the curve.  In a standard bell curve, a change of one standard deviation for a student at the 50th percentile would bump him/her up to 85th percentile, or down to the 15th percentile, depending on the direction of the change.  A report of the effect size of a program typically indicates how much the mean of the students who participated in the program changed from the previous mean or changed from the group of students who didn’t receive the program.

Filed under: CPE,Data,ESSA — Tags: , — Chandi Wagner @ 3:39 pm





November 2, 2016

Thoughts on nuance and variance

As we approach the 2016 general election, I’ve heard public officials, family, and friends make very clear statements regarding which side of the aisle they support.  Yet, I find it hard to believe that the average American falls in line 100% with either political party, or supports every word and tenet of a particular public policy.  We are nuanced people.  Very few issues are as black-and-white as we’d like them to be.  Here’s a guide for things to consider when considering your stance on a particular issue, candidate, or political party, put in the context of educational issues.

  1. Most issues have an “it depends” clause.

With the onslaught of information available today, it makes sense that we want answers that are black-and-white.  The reality, though, is that there’s gray area for most policies and practices.  We also have to balance our ideological values with evidence.  Charter school proponents may believe in free-market values and choice to improve public schools through vouchers and charter schools, but I haven’t seen widespread evidence that choice in and of itself actually improves academic achievement or long-term outcomes in significant ways.  Yes, there are individual students who have benefited, but there are also individual students who have lost out.  Charter school opponents claim that taking away publicly-elected oversight through school boards is detrimental to the public’s ability to provide free and quality education to all.  Yet, the reality is that some public schools have dismal records, and charter or private schools have sometimes had success with the same students.  We have to acknowledge that we all want good things for our kids, and then use the evidence to figure out what that looks like without demonizing the other side.

  1. Most policies rely heavily on the quality of their implementation to be successful.

Common Core seems to be a prime example of this.  Two-thirds of Americans are in support of some sort of common standards across the country.  Yet, barely half of Americans are in support of Common Core.  Support for both questions have dwindled significantly from about 90% of public support in 2012.  Even presidential candidate Hillary Clinton has called the roll-out of Common Core “disastrous,” despite supporting them overall.

CommonCore

Source: http://educationnext.org/ten-year-trends-in-public-opinion-from-ednext-poll-2016-survey/

They were implemented quickly in many states, often without the curriculum materials or professional development to help teachers succeed in teaching the new standards.  While support for Common Core seems to be leveling off with teachers, who are most familiar with them, several states have repealed or are considering repealing the Common Core.  The new state standards that have been written in South Carolina and Indiana are extremely similar to the Common Core, which means that it may not be the concept or content that people disagree with so much as how they were implemented and the ensuing political backlash.

 

  1. Statistics usually tell us about an average (the typical student) but variance is also important.

Charter schools are a prime example of this.  On average, they have similar student achievement outcomes as traditional public schools.  But, there are schools that outperform their counterparts and schools that woefully underperform.  We have to think about those schools, too.

This is also clear in school segregation.  The average black student in the U.S. attends a school that is 49% black, 28% white, 17% Latino, 4% Asian, and 3% “Other,” but that doesn’t mean that every black student has this experience.  At the edges of the spectrum, however, 13% of U.S. public schools are over 90% black and Latino, while 33% of schools are less than 10% black and Latino.  To understand the reality, we need to look at the variety of students’ experiences (known in statistic-speak as “variance”) not just the average.

  1. There’s always room for improvement. “Fixing” a policy may mean making adjustments, not abandoning it altogether.

Student assessments under No Child Left Behind (2001) resulted in the narrowing of curriculum.  But, we also learned more about disadvantaged student groups and have continued closing the achievement gap for students of color.  Should we throw out testing altogether? Some would say yes, but most Americans say no.  Graduation rates, college enrollment, and achievement scores have all increased since NCLB passed in 2001.  What we can do is improve on student assessments.  Adjusting consequences for students, teachers, and schools could result in less narrowing of curriculum and subjects taught.  Involving more well-rounded tests that encourage creative and critical thinking would help teachers emphasize these skills in class.  Continued improvement in data use can help teachers and school administrators adjust their practices and policies to see continued student growth.  States have the power to make some of these changes under the new Every Student Succeeds Act without dismantling gains made under No Child Left Behind.






Older Posts »
RSS Feed