Scary AI Teacher Coaching Tool

This seems like several kinds of bad ideas, and some schools are absolutely going to go for it.

What if teachers could have a sort of feedback and self-evaluation tool working with them every day, powered, of course, by AI? Well, dream no more.

Teachers get little feedback on classroom performance, nor is it possible to collect a ton of data while simultaneously doing the job. Kathleen Moore at the Times Union reports cheerfully:

Enter AI. The AI tool uses cameras and audio recordings to report on whether the teacher looked at or walked through each section of the classroom, how often they used group work, and many other techniques. Even the words the teacher and students use are tracked.

The AI works with the Mathematical Quality of Instruction, which comes to us from the Center for Education Policy Research. It is, they tell us, "a Common Core-aligned observational rubric that provides a framework for analyzing mathematics instruction in several domains." Ruh-roh.

It considers five domains-- common core-aligned student practices, working with students and mathematics, richness of mathematics, errors and imprecision, and classroom work is connected to mathematics. And I'm not going to dig any deeper because 1) I've got my doubts about how much of that an AI can actually measure and 2) common core.

The fresh-faced assistant professor promoting this AI eval is Jonathan Foster, who started out teaching math (well, he really started out in South Carolina's Teacher Cadet program, a pretty nifty program aimed at getting high school students started on the path to teaching) at the Montessori Academy of Spartanburg. He was hired by SUNY at Albany in 2023.

GT reports that Peter Youngs and Scott Acton of University of Virginia (education and computer engineering, respectively) are leading the project.

The money? Why, from the Bill and Melinda Gates Foundation-- a cool $1.4 million.

They are field testing the tool on some early career teachers, who "have been receptive" but also complain that the AI "isn't good at noticing everything." No kidding. AI cannot understand or interpret in any conventional sense of the words. It can only scan for particular words or positions in the room. Which means that this system will inevitably train teachers to incorporate an assortment of odd behaviors and vocabulary for no reason other than it will game the AI.

While all of that provides reason enough to give this coaching tool a big fat side eye, here's a sentence that just hints at bigger issues:

But the AI can give them daily feedback, without it going on performance reports.

Yet.

I'm trying to imagine a universe in which administrators and policymakers say something along the lines of, "Well, the computer is just sitting there chock full of performance data on the teachers, but we should definitely not use that at all."

No, this tool is just a half-step away from being your computerized teacher evaluator, counting every day of your work as part of your professional measure. Some administrators would love it because it would save them time. Policymakers would love it because it generates numbers, so you know it's reliable hard data and all scientific and stuff. And if that's not scary enough, let's imagine what happens when someone hacks into the system.

"Oh, but it's just math teachers," I hear you say. I invite you to travel to that imaginary universe where there is definitely nobody saying, "Yeah, with just a few tweaks we could totally use this to evaluate reading or history or home ec or phys ed teachers."

Foster gamely tries to head off the idea that this AI could be used to substitute for teachers. "I see the act of teaching as a human endeavor," he tells Moore, and I agree, but how human is a human who is taking career advice from an AI coach?

It could be worse. I expect that right now, someone is out there programming an AI to work with the Charlotte Danielson framework. In the meantime, it's one more sign that teachers should prepare to meet their new robot overlords.