Reverse Centaurs, AI, and the Classroom

Cory Doctorow gave us “enshittification” to explain much of what has gone wrong, and he is already moving on to explain much of what we suspect is wrong with the push for AI. There’s a book coming, but he has already laid out the basic themes in a presentation that he shared with his on-line audience. It doesn’t address teaching and education directly, but the implications are unmistakable.

We start with the automation theory term “centaur.” A centaur is a human being assisted by a machine. Doctorow cites as an example driving a car, or using autocomplete. “You’re a human head carried around on a tireless robot body.”

A “reverse centaur” is a machine head on a human body, “a person who is serving as a squishy meat appendage for an uncaring machine.” Here’s his example, in all its painful clarity:

Like an Amazon delivery driver, who sits in a cabin surrounded by AI cameras, that monitor the driver’s eyes and take points off if the driver looks in a proscribed direction, and monitors the driver’s mouth because singing isn’t allowed on the job, and rats the driver out to the boss if they don’t make quota.

The driver is in that van because the van can’t drive itself and can’t get a parcel from the curb to your porch. The driver is a peripheral for a van, and the van drives the driver, at superhuman speed, demanding superhuman endurance. But the driver is human, so the van doesn’t just use the driver. The van uses the driver up.

Doctorow explains that tech companies are highly motivated to appear to be growth industries, and then explains how they’re selling AI as a growth story, and not a pretty one. AI is going to disrupt labor.

The promise of AI – the promise AI companies make to investors – is that there will be AIs that can do your job, and when your boss fires you and replaces you with AI, he will keep half of your salary for himself, and give the other half to the AI company.

The thing is-- AI can’t do your job. So the radiology department can’t fire all the radiologists and replace them with AI to read scans-- they have to hire someone to sit and check the AI’s work, to be the “human in the loop” whose job is to catch the rare-but-disastrous case where the AI screws up.

That last radiologist is a reverse-centaur, and Doctorow cites Dan Davis’ coinage for the specific type-- the Last Radiologist is an “accountability sink.” Says Doctorow, “The radiologist’s job isn’t really to oversee the AI’s work, it’s to take the blame for the AI’s mistakes.”

In education, there is potential for AI to create centaurs and reverse centaurs, and I think the distinction is useful for parsing just how horrible a particular AI application can be.

The most extreme version of a reverse centaur is any of the bullshit AI-driven charter or mini-schools, like the absurd Alpha school chain that promises two hours on a screen will give your child all the education they need. Just let the AI teach your child! All of these models offer a “school” that doesn’t need teachers at all--just a “guide” or a “coach” there to be make sure nothing goes wrong, like an AI that offers instruction on white racial superiority or students who zone out entirely. The guide is a reverse centaur, an accountability sink whose function is to be responsible for everything the AI screws up, while allowing the investors in these businesses (and they are always businesses, usually run by business people and not educators) to save all sorts of costs on high-priced teachers by hiring a few low-cost guides.

For teachers, AI promises to make you a high-powered centaur. Let the AI write your lessons, correct your papers, design your teaching materials. Except that AI can’t do any of those things very reliably, so the teacher ends up checking all of the AI’s work to make sure it’s accurate. Or at least they should, providing the human in the loop. So the teacher ends up as either a reverse centaur or, I suppose, a really incompetent reverse centaur who just passes along whatever mistakes the AI makes.

Almost nobody is sales-arguing that AI can make teaching better, that an AI can reach students better than another human; virtually all arguments are centered on speed and efficiency and time-saving, and while that is appealing to teachers, who never have enough time for the work, the speed and efficiency argument is appealing to management because to them speed and efficiency mean fewer meat widgets to hire, and in a field where the main expense is personnel, that’s appealing.

Public schools don’t have investors to make money from cutting teachers (though private and charter schools sure do), but for AI businesses (as with all other ed tech businesses before them) cannot help but salivate at just how huge the education market could be, a $6 million mountain just waiting to be chewed up. So education gets an endless barrage of encouragements to join the AI revolution. Don’t miss out! It’s inevitable! It’s shiny! To teachers, the promise that it will convert them into powerful cybernetic centaurs. To managers, the promise that it will convert teachers into more compliant and manageable reverse centaurs, controlled by a panel on the screen in your office.

And both snookered, because an AI can’t do a teacher’s job. “Don’t worry,” the boosters say. “There will always be a human in the loop.” Of course there will be--because AI can’t do a teacher’s job. The important question is whether the AI will serve the teachers or be served by them. As a teacher in the classroom being pushed to incorporate AI (”C’mon! It’s so shiny!!”), you should be asking whether the tech will be empowering you and giving you new teacher arms of steel, or will it be converting you to some fleshy support for a piece of tech.

Right now, far more pressure is being put on the Be A Fleshy Appendage side of the discussion. Here’s hoping teachers find the strength to stand up to that pressure.

Oh, and a side point that I learned in Doctorow’s article that’s worth remembering the next time a company wants to offer AI-generated materials-- the courts have repeatedly ruled that AI-generated materials cannot be copyrighted (because they aren’t human-made).