Two Sigma Cyber-Tutoring For The Poors

Much of the US education policy has been driven by a simple enough issue-- a lot of people who would like to spend as little as possible educating other people's children. Especially when those other people are poor and/or of color.

The last few decades of the school choice movement has been driven in large part by Milton Friedman's dream of a country where the government is not involved in schools at all and an education is a consumer good that parents are fully responsible for purchasing on their own. That would, as with any other market sector, result in tiers of service. The well-to-do would get nice schools, and the less well-to-do would get the Dollar Generals of education, and people located in certain communities would get the equivalent of food deserts for education-- little chunks of the market that no vendor wants to serve.

But the dream has some obstacles to overcome. One of the largest is that we like the idea of America as a nation that educates everyone, that schools are our great equalizing engine and we've worked hard to pursue that idea. It's hard to reconcile ourselves to saying, "From now on, you only get the education that your parents can afford, and if that's not much, well, now you know what your station is in life." To give up on equity in education for all is to give up on the whole "all humans are created equal" thing.

Yes, lots of folks have always believed that some people are better than others, and that the betters should rule over the lessers (and we're living with the effects of that right now)-- but it's still hard to say it out loud and admit that we aren't quite who we like to think we are.

So the attempt to install a tiered education system stalls on the messaging problem. How can we short-change the not-so-wealthy families of this country while somehow making it look like we aren't, that our cool new system is still making a quality education available to all?

One proposed solution is microschools. Microschools are the answer to the complaint, "What good does a voucher do me when there isn't a private school that will accept my child within fifty miles of me?" You can start a microschool with a computer, an internet connection, any adult to be a "coach," and a license for some set of software. Gather a few neighborhood kids around the computer desk and voila! You have your own private school! (The overlap between microschool fans and those still angry about COVID distance learning is a monument of cognitive dissonance).

The other idea used to paper over the inherent inequities of a market-based commodified education system is tutoring, Betsy DeVos liked to harken back to the days when Alexander the Great skipped public education and was tutored instead by Aristotle. Let's do that!

Specifically, we find folks touting Two Sigma tutoring, a magical kind of tutoring that creates magical education achievement. There are tutoring companies waving the Two Sigma Tutoring flag all over the place, including Sal Khan presenting a TedX Talk on how his AI-flavored Khanmigo tutoring service would provide the Two Sigma Solution.

When you hear about Two Sigma tutoring, you're hearing about a 1984 essay by Benjamin Bloom that has become a classic. In it Bloom argues that super-duper tutoring can raise student performance by two whole standard deviations. That would mean, for instance, that students scoring in the 50th percentile would be moved up to the 98th percentile (God only knows what would happen if all students were given the 2 Sigma treatment).

If that sounds like it might be bunk--well, yes. Education Next has a new piece by Paul T. von Hippel that is the most thorough look at Bloom's work that you could ask for. Bookmark that puppy for the next time some tech company shows up to sell your district AI-driven Two Sigma tutoring.

I'm not going to cover the whole article, but here are just a few highlights to keep in mind.

A chart often shown to illustrate Bloom's "findings" (including by Sal Khan) is not an illustration of actual data, but Bloom's hard-drawn illustration of "this is what it would look like."

Bloom's essay leans on the work of two grad students working with a tiny sample size. As von Tippel notes, these grad students, having supposedly discovered the secret of super-tutoring, did not go on to make it big in the tutoring world.

There was a lot more than simple tutoring involved. Extra tutor training, tests, feedback, and, most crucially, a focus on topics about which the tutees initially knew nothing; when student knowledge starts at zero, you have a lot of room to improve dramatically.

The two-sigma effects obtained in the 1980s by Anania and Burke were real and remarkable, but they were obtained on a narrow, specialized test, and they weren’t obtained by tutoring alone. Instead, Anania and Burke mixed a potent cocktail of interventions that included tutoring; training and coaching in effective instructional practices; extra time; and frequent testing, feedback, and retesting.

And for the purposes of all the AI-powered tutoring being hyped, Bloom's results relied entirely on tutoring by actual human beings. Though von Tippel doesn't get into this, I will-- any value of one-on-one tutoring includes a closer connection between tutor and student, increasing the tutor's ability to get a sense of what is going on in the student's head, which in turn makes it easier to address precisely what the student isn't getting. AI can't do that.

What von Tippel does point out is that chatbots aren't necessarily very good at this. He found that a chatbots "quickly get lost when trying to teach common math concepts like the Pythagorean theorem." And he rightly questions how well students will engage with a chatbot tutor. Ultimately, he's pretty gentle with the two sigma promise of AI, calling it "rash," when perhaps "highly improbable" or even "bunk" might be accurate.

But what can AI tutoring do? It can allow supporters of commodified education to point and say, "See? Top-quality education available at low, low prices, so we are absolutely fulfilling our promise to get every child a decent education." The supporters will probably not go on to say, "And I don't have to pay for it, which is awesome."

Every bit of the school choice "revolution" is about creating a multi-tiered system of education, pretty much like what we have for higher education (complete with the chance to take on crippling debt in hopes of getting ahead in life).

AI just facilitates that, providing one more way to paper over the idea of abandoning the lessers. I will believe otherwise the day I see wealthy parents pulling kids out of elite academies and plunking them down with an AI tutor instead. "Why would I send you to Philips Exeter when you can get an equally awesome education here at home on your Macbook and AI-ristotle?"

This is the choice argument again and again-- not that choice won't usher in an age of upper and lower strata in education, but that the lower tiers will actually not be so bad. Separate, but equal, one might say, even if such claims seem rash. Or even bunk.