Grading AI Tutors
Here’s how gen-AI can really help students.
I gave a talk at the XQ Institute’s Exchange this week inspired by a conversation I was having with Emma Brunskill, a Stanford Computer Science Professor (who also happens to be my wife). She was looking at a data set of students using AI tutors. Data can tell a vibrant story. In this data set, it’s late afternoon – and a student is struggling with an algebra problem. He’s tried once to solve the problem, and got it wrong. He tried it five times. Twenty times. Sixty times.
Look, I'm a technologist. I love technology, and am fascinated by the opportunities that it can (and will) create. Like many of you, I've been incredibly excited watching tools like Khanmigo emerge. The demos are amazing, right? An AI tutor that can work with every student individually, understand their unique challenges, adapt to their learning style. It feels like we're finally on the verge of giving every child access to the kind of personalized support that was once available to only a few. I want this to exist, and it will. We’re really just at the beginning.
But here's what's fascinating: When researchers studied what actually happens when students use these AI tools today — granted, in a college setting — they found something unfortunate. Students weren't learning more. Often, they were learning less. They weren't engaging more. They were engaging less.
Are we focused on the wrong things? As a builder, I want to make sure we’re building the most impactful educational tools — ones that teachers can scale. Otherwise, what is AI good for?
Helping teachers helps students
Here's where things get really interesting: While we've been focused on creating AI teachers, maybe, just maybe, we've been missing something obvious. Maybe we need to rethink the frame. AI is actually incredible at one specific thing: handling repeated tasks at massive scale. And you know who deals with more repeated tasks than almost anyone? Teachers. Endless grading. Lesson planning. Documentation. Progress reports. The very things that steal their superpowers.
Because great teachers are superheroes. They have this almost magical ability to spot the exact moment a student's eyes light up with understanding. They can tell when a quiet student isn't just shy, but also confused. They can sense when a high-achieving student is about to hit a wall. But instead of using these superpowers, teachers are buried in paperwork.
What if we could give teachers back their superpowers? In one study, researchers did exactly that. They gave teachers AI tools — not to replace them, but to handle the routine tasks that were bogging them down. Suddenly, these teachers had their powers unleashed. They had time. Real time. Time to notice when Johnny in the back row started staring out the window. Time to realize that Maya wasn't just getting answers wrong, she was making the same mistake over and over, pointing to a specific misconception that needed addressing.
Don't get me wrong: I deeply believe we'll figure out how to make these generative AI tools work for students someday. I want every child to have access to a personalized tutor, to have the world's knowledge at their fingertips. But we can't wait for someday.
We can scale care
We've been so caught up in the promise of AI teaching that we forgot to ask how it can help teachers teach. Because here's what we know works, and what's always worked: Students thrive when they have teachers who can see them, understand them, connect with them. When they have someone who notices not just what they got wrong, but why they got it wrong.
Think about that kid struggling with the algebra assignment. What if, instead of facing their sixtieth attempt alone, they had a teacher who had the time to notice their struggle in class? A teacher who wasn't buried in paperwork, but could actually sit down and understand where they're getting stuck?
We’ve talked about this repeatedly on this newsletter and on the podcast – most opportunities with AI may be with focusing on giving people their superpowers back, instead of finding ways to replace these people with machines.
In this case, for now, AI won't solve education's challenges by teaching our students. It'll help solve them by giving our teachers back their time to teach. By handling the routine so we can focus on the remarkable. By creating space for what matters most: those moments of connection that change lives.
That's the future we can build. Today. Because somewhere, there's a student at a kitchen table, trying for the sixty-first time. And they shouldn't have to do it alone.
Worth the Read
Mozilla removed the Do Not Track feature that’s been on Firefox since 2009. Before you get mad: It’s basically useless today anyway. (Apple stopped using it in 2019.) Instead, Firefox users can select the “Tell websites not to sell or share my data” feature, which is honored by more web sites. (NB: I’m on the board of Mozilla.org.)
The cofounder of the new private social network Mozi wrote a nice reminder of what social networks were supposed to be — before they became social media.