On June 11, Secretary of Education Duncan attended the National PTA Convention where he stated “Only by moving beyond basic skills and bubble tests can children develop the critical thinking skills that will give them the ability to compete successfully in the global economy.” 

There is growing consensus that multiple-choice based assessment should be augmented by written-response and performance items that test the application of skills in practical contexts. The elephant in the room is cost.  Across studies, a 2 to 1 cost ratio of “high quality” assessments (HQA) to multiple-choice assessments is fairly consistent.

The Stanford Center for Opportunity Policy in Education (SCOPE), led by Linda Darling-Hammond recently released a collection of reports intended to summarize current research on HQA’s, and the most important of them might be Topol, Olson and Roeber’s “The Cost of New Higher Quality Assessments:  A Comprehensive Analysis of the Potential Costs for Future State Assessments.”

Using a sophisticated cost model, Topol and colleagues estimate that traditional multiple-choice driven assessments cost approximately $20/student to administer, compared to about $56 for HQA’s.  Through cost savings strategies related to economies of scope, technology and teacher involvement, the authors suggests that HQA costs could be reduced to as little as $10/per student.

Here’s how they break it down:

 $55.67 (per pupil)  average starting cost of HQA (scored by vendor)

-$24.26                        savings from having teachers score non-multiple choice items as part of professional development

-$16.84                        savings from economies of scale over a 30-state assessment consortium

-$3.49                          savings from having students take tests online rather then with pencil-and-paper

-$0.93                          savings from using technology instead of humans to grade short written responses

-$0.71                          savings from having scorers grade questions on their work-based or personal computers, instead of at overhead-intense grading centers

$9.44                           TOTAL COST

Reducing the per-pupil costs of high quality assessments to half that of traditional assessment is obviously a bold, dramatic assertion.  There are a number of assumptions Topol and colleagues make that can be questioned:

That a 30-state assessment consortium is feasible.  The current common standards consortium would suggest it is, but it has yet to prove its worth in implementation, and abstract goal-setting is a lot easier to commit to then assessment implementation.

That the true long-term cost of implementing high quality assessment doesn’t actually include all alignment costs associated with a comprehensive cycle of meaningful assessment, training costs associated with integrating test results into instruction, altering teacher evaluation systems to account for new instructional values, modifying systemwide information systems, etc.  That’s the thing about HQA.  You’re not just changing the test—if you’re doing it right, you’re changing a district’s entire culture.  This cultural change would most likely need to include subjects other than math and English, the only subjects included in the report’s projections.

That the costs of purchasing, installing and administering computers in schools wouldn’t nullify the savings, even over an extended period of time.  The authors note that PC purchase costs were not considered in their figures and that the current student-to-PC ratio is estimated at around 4 or 5 to 1.

That teachers wouldn’t put up an intractable fight to avoid assuming what they might perceive to be unfair required duties.  As Darling-Hammond and many international studies have noted, scoring of contructed-response questions is a rich professional development activity, but asking teachers do it “on their own time”, outside of more easily fulfilled workshop requirements is in a different class of time-demands.

That such heavy reliance on technology wouldn’t be a mistake.  Remember a couple of election cycles ago when there was a big movement to replace manual voting systems with electronic touch screens?  It didn’t take because electronic systems failed in grand scales in high stakes situations.  How long would it take to get even more complex assessment applications working on a large scale in education?

Despite these considerations, there is a modified approach to the proposed framework that is quite promising.  Topol considers the impact of actually paying teachers a $126/day stipend.  Eliminating all of the technology-based improvements and implementing this stipend yields an overall cost of $25.71 versus the traditional multiple-choice test rate of $20/model. 

Now we’re talkin’.  If we could get HQA costs down to just 25% percent more than traditional assessment, we have a serious discussion on our hands as to why we aren’t, as an education community, prioritizing the development and implementation of such a critical step in the development of college- and career-ready skills. –David English