Last week, as soon as my team and I posted the last episode of Season Two of the podcast, my family and I got on a plane to a place with lots of nature and very little connectivity. But in the world of AI, two weeks is an eternity: So much can happen. So I wanted to weigh in on the latest conversation.
There’s an essay going around Silicon Valley right now titled “Situational Awareness: The Decade Ahead.” It was published by a former OpenAI researcher who claims he’s one of a few hundred people who have “situational awareness” that computers will be smarter than humans by 2030. How afraid should we be?
The paper starts with the premise that there has been exponential growth in AI over the past few years, and contends that it won’t be stopping for the next few years. It’s as simple, he says, as “counting the OOMs,” or orders of magnitude. By 2027, the paper posits, large language models will be as smart as an AI researcher/engineer. Continuing on the curve, that exponential growth, he says, is enough to get to AGI (artificial general intelligence) – colloquially meaning that AI will match or surpass human capabilities across a wide range of cognitive tasks in the next few years.
First off, humans are really bad at thinking about exponential growth. Take, for example, folding pieces of paper: If you were to fold a piece of paper, end to end, on top of itself so that the thickness is doubling each time. How many folds to reach the moon? Only 42 folds to reach the moon — way less than most people would think. It’s because the thickness of the folded paper is growing exponentially, and we’re really bad at thinking that way. Do you want one more? How many more folds to come all the way back from the moon? Just one more. That’s exponential growth.
We have probably been at exponential growth in AI for a few years now. Things are progressing at a rate that is so much faster than any of us (who think linearly) can possibly imagine. And “Situational Awareness” claims that this crazy growth is going to continue toward AGI, or artificial general intelligence.
If we are racing that quickly towards AGI, maybe the question to ask is: What is it? When I was in university (what now feels like eons ago), we would talk about the Turing Test as the benchmark. That test would go the following way: Take two computer terminals, one of which is connected to another terminal that a human is sitting at, and the other is connected to software. You would chat using the two terminals, and if you couldn’t tell which one was the human responding and which was the software, then the software would have passed the Turing Test.
ChatGPT can probably pass the Turing Test now, but would we say it was AGI? There is a paper making its way to the 2024 International Conference on Machine Learning that takes the first stab at saying what constitutes AGI, proposing “‘Levels of AGI’ based on depth (performance) and breadth (generality) of capabilities.” It argues that there are different benchmarks ranging from “Emerging,” meaning “equal to or somewhat better than an unskilled human,” to “Superhuman,” meaning “outperforms 100% of humans,” with stops at “Competent,” “Expert” and “Virtuoso” in between. Is this the right way to think about it?
There is probably more than one dimension to look at AGI. ChatGPT can almost replace most executive assistants – is that AGI? EAs are superhuman in a very particular skill set. How do we rate AGI when it comes to artistic and creative endeavors? As we’ve spoken about before, these generative systems are really good at taking unusual combinations of different styles or things they’ve seen before – is that AGI, or is that averaging?
What about the physical world? Most examples we’ve spoken about live completely in cyberspace – do they need to have some form of robotic manipulation to make changes or sense the real world? What about when they make mistakes? Do they need to make mistakes like humans do — and what is the potential impact of those mistakes? (In the last newsletter, I looked at fatal accidents caused by self-driving cars beyond the data.)
Why is this important? From the marketing side, it's going to be easy for people to try to fool people into thinking that we have intelligence. Take Tesla and their hype of the “Full Self Driving” car, which it’s clearly not: A person needs to monitor it at all times. But also, from a legislative and regulatory side, does AGI imply some form of “trust”? That the decisions it's making, or the things it is outputting, are correct?
We need a common language to define what it is. Then we can have conversations around regulation, etc. But it starts with definitions: Human-made ones.
Worth the Read
On Friday, the Supreme Court’s decision to rein in the power of the executive branch will impact the ability to keep big tech in check, Axios reports. That means setting guardrails around AI will be even trickier.
In lighter news, a post in a group chat called out the podcast Science of Birds, which dedicated a 2021 episode to artificial intelligence in bird research. For example, machine learning is being used to identify individual birds, not just species.