AI, Facing the Dark, and Human Sparknotes

The New York Times unleashed a feature section about AI, and it is just a big fat festival of awful.

There's a conversation between Kevin Roose and Casey Newton, hosts of the podcast Hard Fork, named, perhaps, after the object I want to drive into my own brain while reading this conversation.

These days I read this kind of stuff for the same reason that I leave many far right voices unblocked on my social media-- because if you're going to face reality, you have to face the dark parts where people believe awful stuff. It's ugly, but it won't go away just because you ignore it.

So here's Roose saying that AI has replaced Google to answer questions like "What setting do I put this toaster oven on to make a turkey melt?" Or his friend who now gets through the morning commute by putting ChatGPT on voice mode and asking it to teach them about modern art or whatever. And "another person I know just started using ChatGPT as her therapist after her regular human therapist doubled her rates."

The piece is loaded with quotable foolishness, like this:

But I confess that I am not as worried about hallucinations as a lot of people — and, in fact, I think they are basically a skill issue that can be overcome by spending more time with the models. Especially if you use A.I. for work, I think part of your job is developing an intuition about where these tools are useful and not treating them as infallible. If you’re the first lawyer who cites a nonexistent case because of ChatGPT, that’s on ChatGPT. If you’re the 100th, that’s on you.

Intuition? I suppose if you lack actual knowledge, then intuition will have to do. But this will be a recurring theme-- AI's lack of expertise in a field can be compensated for by a human with expertise in that field. How does that shake out down the road when people don't have expertise because they have leaned on AI their whole lives? Hush, you crazy Luddite.

Newton says he uses LLM for fact checking spelling, grammatical, and factual errors, and of course the first two aren't really AI jobs, but these days we just slap an AI label on everything a computer can do. Factual errors? Yikes. Roose says he likes AI for tasks where there's no right or wrong error. They both like it for brainstorming. Also for searching documents, because AI is easier than Control F? Mistakes? Well, you know, humans aren't perfect, either.

Roose notes that skeptics say that the bots are just predicting the next word in a sentence, that they aren't capable of creative thinking or reasoning, just a fancy autocomplete, and that all that will just turn this into a flash in the pan, and Roose has neatly welded together two separate arguments because A) bots aren't actually thinking, just running word token prediction models and B) AI will wash out soon-- those are not related. In fact, I think I'm not unusual in thinking that A is true, and B is to be hoped for, but unlikely. Anyway, Roose asks Newton to respond, and the response is basically, "Well, a lot of people are making a lot of money."

Roose and Newton are not complete dopey fanboys, and at one point Roose says something I sort of agree with:

I think there are real harms these systems are capable of and much bigger harms they will be capable of in the future. But I think addressing those harms requires having a clear view of the technology and what it can and can’t do. Sometimes when I hear people arguing about how A.I. systems are stupid and useless, it’s almost as if you had an antinuclear movement that didn’t admit fission was real — like, looking at a mushroom cloud over Los Alamos, and saying, “They’re just raising money, this is all hype.” Instead of, “Oh, my God, this thing could blow up the world.”

"Clear view of the technology" and "hype" are doing a lot of work here, and Roose and Casey fall into the mistake of straw manning AI skeptics by conflating skeptics and deniers (a mistake Newton has made before and to which Ben Riley responded well).

The other widely quoted chunk of the discussion is this one from Roose:

The mental model I sometimes have of these chatbots is as a very smart assistant who has a dozen Ph.D.s but is also high on ketamine like 30 percent of the time. But also, the bar of 100 percent reliability is not the right one to aim for here: The base rate that we should be comparing with is not complete factuality but the comparable smart human given the same task.

But the bots don't have Ph.D.s, and I don't want to work with someone juiced up on ketamine, and if bots aren't any better than humans, why am I using them?

The article is entitled "Everyone Is Using AI for Everything," which at least captures the concerning state of affairs.

Take the re-emergence of disgraced author and professional asshat James Frey (the guy who was shamed by Oprah for his fake memoir) who just put an AI-created book on the Book of the Month list. If that seems like a problem, Frey explained why he was happy to let AI do most of his work back in 2023.

I have asked the AI to mimic my writing style so you, the reader, will not be able to tell what was written by me and what was generated by the AI. I am also not going to tell you or make any indication of what was written by me and what was generated by the AI. It was I, the writer, who decided what words were put on to the pages of this book, so despite the contributions of the AI, I still consider every word of this book to be mine. And I don’t care if you don’t.

And there's the other article in the NYT section, a piece about using NotebookLM, a bot designed to help writers. "AI Is Poised To Rewrite Hostory," says editorial director Steve Wasik. He talks about how author Steven Johnson used the bot (which he had helped build) to sift through the research and generate story ideas. Muses Wasik:

Like most people who work with words for a living, I’ve watched the rise of large-language models with a combination of fascination and horror, and it makes my skin crawl to imagine one of them writing on my behalf. But there is, I confess, something seductive about the idea of letting A.I. read for me — considering how cruelly the internet-era explosion of digitized text now mocks nonfiction writers with access to more voluminous sources on any given subject than we can possibly process. This is true not just of present-day subjects but past ones as well: Any history buff knows that a few hours of searching online, amid the tens of millions of books digitized by Google, the endless trove of academic papers available on JSTOR, the newspaper databases that let you keyword-search hundreds of publications on any given day in history, can cough up months’ or even years’ worth of reading material. It’s impossible to read it all, but once you know it exists, it feels irresponsible not to read it.

What if you could entrust most of that reading to someone else … or something else?

On one level, I get it. I do a ton of reading. Did a ton of reading when I was teaching so that I could better represent the material. I do a ton of reading for the writing I do, and yes-- sometimes you tug on a string and a mountain falls in your lap and you despair of reading enough of it to get a picture of what's going on.

But, you know, working out is sweaty and painful. What if I could entrust most of that exercising to someone or something else? Keeping in touch with the any farflung members of my family is really hard and time consuming. What if I could entrust most that work to someone or something else? Preparing and eating food is time consuming and not always fun. What if I could entrustmost of that work to someone or something else?

Humaning is hard. Maybe I could just get some tech to human for me.

Any day now

I know. It's not a simple issue. I wear glasses and, in fact, have plastic lenses inserted in my human eyeballs. I drive a car. I enjoy a variety of technological aids that help me do my humaning both personally and professionally. But there's a line somewhere, and some of these folks have uncritically sailed past it, cheerfully pursuing a future in which they can hand off so many tasks to the AI that they can... what? Settle down to a happy life as a compact, warm ball of flash in a comfortable plasticene nest, lacking both cares and autonomy?

At what point do folks say, "No, you can't have that. That business belongs to me, a human."

But back to the specifics at hand.

I don't know how one separates the various parts of writing into categories like Okay If AI Cuts This Corner For Me and This Part Really Matters So That I Should Do It Myself (or, like Frey, simply decide that none of it is important except the part where you get to sign checks). Brainstorming, topic generation, research-- these are often targeted for techification, but why? I am often asked how I am able to write so much and so quickly, and part of my answer has always been "low standards," but it is also that I read so much that I have a ton of stuff constantly being churned over in my brain and my writing is just the result of a compulsion to process all that stuff into a written form.

That points to a major issue that Roose and Newton and Wasik all miss. Using the bot as a research assistant or first reader or brainstormer can only hope to be useful to a human who is already an expert. Steven Johnson can only use what his AI research bot hands him because he is expert enough to understand it. The notion that a human can use intuition to check the AI's work is a dodge-- what the human needs is actual expertise.

That may be fine for the moment, but what happens when first hand experience and expertise are replaced by "I read an AI summary of some of that stuff"?

At least one of Wasik's subjects wrestles with the hypocrisy problem of an educator who tells students to avoid the plagiarism machine and then employs the same bots to help with scholarship. But I wish more were wrestling with the basic questions of what parts of writing and reading shouldn't be handed over to someone or something else.

In some ways, this is an old argument. I talked to my students about Cliff notes and, later, Sparknotes, and I always made two points. First, what you imagine as an objective judgment is not, and by using their work instead of your own brain, you are substituting their judgment for your own. Not only substituting the final project, but skipping your own mental muscle-building exercise. Second, you are cheating yourself of the experience of reading the work. It's like kissing your partner at the end of an excellent date-- if it's worth doing, it's worth doing yourself.

No doubt there are some experiences that aren't necessarily worth having (e.g. spending ten years scanning data about certain kinds of tumors). But I'd appreciate a little more thoughtfulness before we sign everyone up to use sparknotes for humaning.