The Opportunity Myth Myth

From October, 2018

When it comes to slick, pretty education "research," the folks at TNTP know their stuff. Reformsters have been milking the slick-but-hollow "Widget Effect" for years, and now TNTP has whipped up another sure-to-be-referenced-way-too-much "report" entitled "The Opportunity Myth."

If ed reform eats its own tail, what does it poop out?

Who are these folks? TNTP used to stand for The New Teacher Project; She Who Will Not Be Named created it as a spin-off of TFA, designed to put older career-changers into the classroom. At some point it changed into an advocacy group pushing a redesign of teaching (current slogan: reimagine teaching). TNTP is led by Daniel Weisberg, who started out as a lawyer and then served as a labor specialist under Joel Klein in NYC. The board is packed with entrepreneurs, PR specialists, and reform CEOs. You can hunt through the whole list of TNTP leaders and find that this organization devoted to teaching has no teachers in leadership positions (just a few TFA temps and other alternative paths to one or two resume-building years in the classroom).

So this report comes straight from the heart of reformdom.

The report is a slick piece of graphic-soaked digitized niftiness (Look! This part scrolls sideways! And here are more graphics!!) but it features the TNTP sleight of hand. The subheading is the first piece of misdirection-- "What students can show us about how school is letting them down-- and how to fix it." But despite a lot of dressing about Listening To The Students, at its heart, this report is making a familiar point.

The Remediation Myth

After opening with a story about an individual student, TNTP lays out a familiar picture of the problem.

While more students than ever before are enrolling in college, far fewer are succeeding once they get there. Nationwide, 40 percent of college students (including 66 percent of Black college students and 53 percent of Latinx college students) take at least one remedial course, where they spend time and money learning skills they were told they'd already mastered in high school. A recent study found that college remediation costs students and their families $1.5 billion annually...

There are several corners cut here on a subject that deserves attention. Were those students told they had mastered the skills? Because high school teachers have been part of the following conversation for the past several years:

College: This student is not ready to be here!
High school: Did you look at his transcript? Did you look at his grades? Did you look at which courses he took? Because we told you pretty clearly that he was not ready for college, and you took him anyway.
College: This student is not ready to be here!

By the way-- "learning skills"? Common Core and other reform movements may keep insisting it's all about skills, but higher learning also requires some content knowledge, too.

The rise of college remediation courses is a subject that deserves examination, because something's going on and we really need to know what. Possible explanations include:

1) Colleges desperate to fill seats accept underqualified students.

2) The college eligibility test, the one that determines who needs remediation, is not a good test.

3) Students need more remediation these days because more of the year is spent on test prep and testing instead of actual education.

4) Colleges are pushing maybe-not-necessary remediation because it makes them a whopping $1.5 billion each year.

I taught 11th graders at both ends of the academic spectrum. Here are two things I can tell you I've seen multiple times:

Student visits from college and says, "Yeah, they tried to push me into a remedial course, but I just didn't do it, and I've been fine."

Student in vocational prep class insisting they want to go to college; they just don't want to take the "hard" college prep course right now.

As I said, there's a huge conversation to be had here, because those remediation numbers are trying to tell us something important. But this report is not interested in that conversation-- it brings up the subject only so it can make the same old reform point---

Public schools are failing.

The We Did Research Myth

There are somewhere under 15,000 public, private, whatever school districts in the country. TNTP "partnered" with five of them. That is not an impressive sample size-- and they weren't even randomly selected. This is like deciding that, since you have a problem with geese running into jets at the airport, you will go study some penguins in the zoo.

They observed 1,000 lessons-- but that's 200 per system. They followed 4,000 students, but that's just 800 per school. Within each system, according to their technical appendix, they divided the district into elementary, middle and high grades. Then they split those subgroups between 1/2 above-average students and 1/2 below-average students. So the study literally does not include any average students. Ten teachers volunteered from each school, and each teacher picked two classes to have included in the study.

For the three urban districts, percentage of students involved in the study was 5% or lower. In the single rural district, 64% of the district's students were in a study classroom.

But wait-- there's less!! In each of the two classrooms, the participating teachers selected six students-- two way below level, two just below level, and two above level. So once we dig down, we find a tiny sampling of students that is skewed toward underachievers. Well-- relative underachievers, because these six were within one classroom. So presumably we've got the student in an AP classroom who is in over her head, and we've got the student who is taking the low-level class that is way below his ability.

From this tiny not-even-sort-of-random sampling, the authors are able to make all sorts of sweeping statements about how different sorts of students are being educated in this country.

The Grade Level Myth

The big sexy headline-generating takeaway from the report is that all these students are being failed by a system that doesn't require them to do work at their level. The lessons aren't challenging enough.

Students spend most of their time in school without access to four key resources: grade-appropriate assignments, strong instruction, deep engagement, and teachers who hold high expectations.

Because numbers are sexy, the report also tells us exactly how much time is wasted--  "Students spent more than 500 hours per school year on assignments that weren't appropriate for their grade."

The entire report rests on this assertion about the grade level of the instruction in these courses. But what does "grade level instruction" even mean? Once again we delve into the technical appendix (which, I should note, involves following a series of links to a pdf file written in about .2 font). Once again, the rigor is underwhelming.

Assignments were collected from the six special students (photographed by the teacher) and the assignment quality was rated by "raters," all of whom had "at least two hours of training." TNTP gave them a rubric to judge three domains--

1) Content. Does the assignment align with the expectations defined by grade-level standards.
2) Practice. Does the assignment provide meaningful practice opportunities for this content area and grade level?
3) Relevance: Does the assignment give students an authentic opportunity to connect academic standards to real-world issues and/or contexts?

By "standards" we mean the Common Core (NGSS for science). But if we're parsing Core standards by grade level, I foresee some problems. Here are two different years for the same standard (RL.1):

Cite textual evidence to support analysis of what the text says explicitly as well as inferences drawn from the text.

Quote accurately from a text when explaining what the text says explicitly and when drawing inferences from the text.

That's the fifth grade and sixth grade standard. Can you tell which is which? I know-- it's an unfair question because you haven't had your two hours of training. TNTP does site some subject-specific criteria for the rubric-- but the link is dead and the address leads to an error page.

The raters are also supposed to determine if the student completed the assignment successfully-- in other words, grade the assignment-- which is pretty impressive work for someone with two hours of training. They were also supposed to see if the student met the standards' requirement. It is not clear if these raters were hired through Craigslist or the TNTP intern program or some other agency.

The Strong Instruction Myth

This is good old fashioned tautological ouroborean research. First, you decide how to identify "strong instruction" based on what you consider identifying characteristics. Then you check the "strong instruction" to see if it shows signs of the characteristics you believe are important. Voila. It mostly does. In this case, "strong instruction" includes lessons that reflect "the demands of the standards."

And they watched two whole lessons by each volunteacher.

Moving on. Nothing to see here.

The Chicken Littling Myth

The biggest bit of misdirection in this report is the great amount of weight thrown behind the narrative that these students have dreams and ambitions, but they are being lied to, lied to by school systems that tell them that they are on a path to college and career readiness. There are plenty of charts and numbers to quantify the size and shape of student aspirations, and to suggest that teachers and school systems somehow don't know this about their students, as if the entire system is that guy sitting at the desk behind a newspaper that he never lowers to actually look at his students.

This is the oldest routine in the reformster playbook-- lean heavily on explaining just how bad the problem is. Lay out the problem in gut-wrenching detail. Make sure to define the problem in terms that fit your proposed solution. But while you have research and data and details about the problem, the part where you insist on your solution remains unsupported by anything except your assertions.

Reformster: Look at these test results. Look at these x-rays. You definitely have a brain tumor, and if  it's not fixed, you'll soon lose feeling in your limbs and your legs will stop working properly.
Patient: Oh my God! Save me!
Reformster: Certainly. I'm just going to use this chain saw to cut off your legs.
Patient: Wait! What? How will that help with my brain tumor?
Reformster: Look at these x-rays! Look at how big it is! Right there in your brain! This is terrible!
Patient: But how will hacking off my legs-
Reformster: X-rays! Brain! Terrrrrrrible!

Oh, and the writers also want you to know that socio-economics are no explanation or excuse for students' low performance.

The Recommendations Myth

The recommendations portion of this report starts with a reminder that we should be doing this For The Kids, and that the classroom experience of students should be the center of policy decisions. Then it calls for two commitments: First, give every student grade appropriate assignments, strong instruction, deep engagement, and teachers with high expectations regardless of any part of their identity. Second, every student and family should be "an authentic partner" and should have a serious role indecision-making-- a real choice, one might say.

This leads to five recommendations:

1) Listen to parents and students.

2) Grade-appropriate assignments for everybody. Regardless of whatever.

3) Give all students, especially the behind ones that is [insert rhetoric that gets you around actually saying "rigorous" here].

4) High expectations. Stop being racist.

5) Conduct an equity audit to make sure everyone is getting the good stuff.

The Deja Vu Myth

So, to sum up....

Schools are lying to families by telling them that their children are college and career ready when they actually aren't. Teachers should have higher expectations, which can be put into action by aligning class work with the very best college and career ready standards-- and poverty and racism, while bad, are no excuse for low performance on the school's part. Do it For The Kids- don't put adult concerns ahead of children's needs. The school system should be responsive first and foremost to parents, who should have choices available that suit their goals.

Does this all sound familiar? It should-- this entire report is the report that Reformsters (particularly Arne Duncan) wish they had come up with a decade ago when trying to pitch Common Core and charters.

We could spend even more time picking apart the research techniques here, though Matt Barnum at Chalkbeat hit the basic issues pretty well, but looking at the details is really beside the point. The details are window dressing. This is the same pitch we've been hearing for over a decade, adroitly stripped of the test-centric angle and the demand to evaluate teachers-- arguably the two most hackle-raising obstacles to reformster goals.

The authors have tried to ease up on teachers, even going so far as to write at one point "for those of us working in the school system..." which-- well, no. The folks at TNTP are not down there in the trenches with actual trained educators. But the repeated calls to listen to the students and teach them the right stuff make it clear that the premise here is that teachers currently do neither. It's an odd charge, given that teachers have been laboring under reform programs for years, but then the other curious feature of the report is that it does not acknowledge how NCLB, RttT, Common Core, and test-centered accountability have changed the landscape.

This report could have been written fifteen or twenty years ago. Again, I'll bet some Reformsters wish it had been.  But there is nothing new here.

Here is probably the report's biggest flaw. If you really thought the problem was that students aren't ready for college and career, as evidenced primarily by the rate of college remediation, why wouldn't you gather up students who needed remediation and those who didn't, and compare and contrast them. Why wouldn't you study them to see what one group got that the other lacked? You wouldn't do that because you have a point you want to make, and your research "design" is based not on a quest for answers, but on building a scaffold for the answers you already want to sell. This report was funded by the Joyce Foundation, the Chan Zuckerberg Initiative, the Nellie Mae Education Foundation, the Walton Family Foundation, the Overdeck Family Foundation, and the Barr Foundation.

The Unfortunate Truth

As Barnum notes:

The research of TNTP, previously known as the The New Teacher Project, has a track record of shaping policy, particularly with an influential 2009 report known as The Widget Effect, which focused on perceived flaws of teacher evaluation systems.

The report is old wine in a slick new skin. But it looks really, really cool, and it layers on the For The Children with a shovel and a backhoe, which is always compelling because we really should do it for the children-- but not if "it" means launch a highly profitable education-flavored industry based on pumping amateur hour standards out into a top-down imposed framework that stomps on actual education professionals.

I don't know that anyone will be "influenced" by the report, because it is old news. But people who are already invested in this reform agenda will have a slick new publication that they can wave around to bolster their position. The Widget Effect mattered because instead of just saying, "I think this should be policy because I like the idea," reformsters could call policy-makers and wave around an official-looking report-thing. The same will happen here. TNTP has provided the striking headline-- "Students are wasting 500 hours a year on work that's below their level"-- and a whole bunch of policy makers will never look any deeper than that.

So that's a bummer.

When someone waves this in your face, you can tell them that it's a report based on a tiny number of students in a tiny number of schools about which we know no pertinent details. You can tell them that the report may back up the notion that students have aspirations and sometimes have trouble achieving them, but that TNTP made up its own notion of what "grade level work" means, and someone else's notion would yield different results. You could point out that this is one more call for schools to buy material aligned to the Common Core standards, and if the Core was going to fix all this, wouldn't it have done so by now? You could ask them to tell you what, exactly, is a new idea included in this report?

Here's hoping that more people listen to you than are going to listen to this report.