The Failed Case for Super-NAEPery

At The74 (the nation's most uneven education coverage), Goldy Brown (Whitworth U and AEI/CERN) and Christos Makridis (Labor Economics and ASU) have a bold idea that involves putting fresh paint on a bad old idea--the national Big Standardized Test.

Their set-up is the usual noise about how the National Assessment of Educational Progress (NAEP) peaked around 2013, which is true if you also believe that the rise that carries I-80 across the Bonneville Salt Flats is also a peak. They are more accurate when they say that "student outcomes" (aka "Big Standardized Test scores") have "largely stagnated" over recent decades.

Yep, it's a roller coaster

Let me digress for just a moment to note the oddness of that idea of stagnation--as if test scores should keep rising like stock prices and property values. Each cohort of students should be smarter and better than the one before, a thing that would happen... why? What's the theory here? Each year's children will be genetically better than those that came before? That every teacher will significantly up her game with every passing year (because the students rotate out at a much higher rate than the teachers)? Schools get better at gaming the tests? If the expectation is that each successive group of students will score higher than the group before, what is supposed to cause that to happen? And how does it square with the people who think that education should be going "back" to something like "basics"? I mean, doesn't the vision of non-stagnating test scores include students who are all smarter and more knowledgeable than their parents?

Okay, digression over. The authors also point out that Dear Leader and his crew have "downsized" the staff that oversees the NAEP (while simultaneously insisting that NAEPing will continue normally)-- but they argue that the kneecapping will "create an opportunity to rethink the role this tool can play."

In particular, the Trump Administration could explore using the NAEP to promote greater transparency among schools, parents, and local communities, as well to enhance academic rigor and ensure genuine accountability in a comparable way across schools and states. That would mean replacing a disparate collection of state tests will a single national assessment administered to every fourth and eighth grade student every year.

Yikes. I checked quickly to see if Brown and Makridis are over 15 years of age, because if so, they should remember pretty clearly that the feds have tried this exact thing before. Every state was supposed to measure their Common Core achievements by taking the same BS Test, except then that turned out to be two BS Tests (PAARC and SBA) but then those turned out to be expensive and not-very-good tests and states started dumping them, while folks from all ends of the spectrum noted that this sure looked like an illegal attempt to control curriculum from the federal level.

With national standards and national testing, supporters argued, we would be able to compare students from Utah and Ohio, as if that was something anyone actually wanted to do. As if in Utah parents were saying, "Nice report card, Pat, but what I really want to know is how your test scores compare to the test scores of some kid in Teaneck, New Jersey."

No, these guys have to remember those days, because they are well versed in all the same bad arguments made at the time.

Parents, educators, and state leaders agree that more information — not more bureaucracy — is needed to make informed decisions for their children and communities, as well as to foster greater competition. Making the NAEP a truly national assessment would provide this information in a consistent, credible, and actionable manner.

Right. Test scores would be great for unleashing free market forces in a free market, education-as-a-commodity choice system. Also, competition doesn't unleash anything useful in education. Also also, choice fans have mostly stopped using this talking point because it turns out charter and voucher schools don't actually do any better on BS Tests. Get up to date, guys-- today it's all "choice is a virtue in and of itself" and "parents should get to choose a school that matches their values."

The writers call for the NAEP to be cranked out every year instead of every other, and for every student instead of the current sampling. No sweat, they say, because every state already has stuff in place for their own state test.

But an annual universal NAEP would be great because it's a "consistent and academically rigorous measure of student performance." There's a huge amount of room to debate that, but it only sort of matters because the writers have fallen into the huge fallacy of NAEP and PISA and all the rest of these data-generating numbers. "If we had some good solid data," says the fallacy, "then we could really Get Shit Done." We would Really Know how students are doing, we would Really Know about how bad the state tests are, and we would Really Know where the issues in the system are.

It's an appealing notion, and it has never, ever worked. For one thing, nobody can even agree on what critical terms like "proficient" mean when it comes to NAEP. But more importantly, the solid data of NAEP never solves anything. Everyone grabs a slice, applies it to the policies they were busy pushing anyway, and NAEP solves nothing, illuminates nothing, settles nothing.

The writers also want to use the test illegally in a method now familiar to both political parties. Tie Title I funding to compliance with NAEP testing mandates and presto-- "States would have a stronger incentive to align their instructional practices with higher expectations." In other words, test + money = federal control of local curriculum. Not okay.

They would also like the test to provide feedback to parents about their individual students. This also repeats a critical error of every BS Test to come down the pike. Tests are designed for a particular purpose and one should not attempt to apply them to a host of other purposes-- doing so gets you junk. Also, I still don't believe that conversation in Utah is happening. But this notion--

A national benchmark can support local autonomy while enabling cross-district comparisons that inform parents, educators, and policymakers alike.

Producing a test that generates data useful to all three groups is less likely than capturing a yeti riding a unicorn that is pooping rainbows.

The writers also argue that states could save money if the feds forced them to replace their current batteries of BS Tests with NAEP instead in just 4th and 8th grade. I suppose that depends on the test manufacturer who secures this national testing monopoly.

Their last argument is that universal NAEPery would "offer a balanced form of federal oversight." That means "less intrusive than programmatic mandates" which are not so much intrusive as they are illegal. At any rate, national standardized tests intended to drive programmatic choices are still pretty damned intrusive.

Now for the wrap up. Starting with this understatement:

Federal initiatives to improve student outcomes have historically produced mixed results.

Yes, and theater trips to see "Our American Cousin" have historically produced mixed results for Presidents. Of the whole list of "mixed" results, they include just the Obama era attempt to use test scores to drive teacher improvement (well, not "improvement" exactly, but teaching to the test in order to raise scores).

They say one right thing, which is "that policy tools must be both well-designed and responsive to local implementation contexts." But they follow that with "designating NAEP as the national assessment meets both criteria." And no, no it wouldn't, and we know it wouldn't because the last time we tried this national BS Test thing, it went very poorly. This is such a classic reformster construct-- "Historically this thing has failed, so we think the solution is to do it some more, harder."

In an era of educational fragmentation, the NAEP stands out as a uniquely credible and underutilized tool. Repurposing it as the primary national assessment — administered annually to all 4th and 8th graders in states receiving Title I dollars — would promote transparency, reduce redundant testing, and align incentives around higher academic standards. This reform would offer a shared benchmark to evaluate progress across states and districts. At a time when parents, educators, and policymakers are calling for both accountability and flexibility, a restructured NAEP provides a rare opportunity to deliver both.

Is that what parents, e3ducators, and policymakers are calling for, really? Doesn't matter, because NAEP provides nothing special for accountability (certainly not before we have a long, long conversation about accountability to whom and for what) and it certainly doesn't provide flexibility, not even under their repeat of the old argument that states could decide how to meet the national test standards, which is like telling someone "You can get to Cleveland any way you want as long as you arrive at E.9th and Superior within the next six hours seated in a blue Volkswagon, listening to Bob Marley, and eating a taco. Totally up to you what meat is in the taco, though. See? Flexible."

You know what's really flexible? An end to federal mandates for a nationalized Big Standardized Test.