Excerpt from Chapter Five

The Elements of Understanding

If you did a survey of the world’s philosophers, asking them to name the most famous argument deployed by skeptics of artificial intelligence, the consensus would be clear. “Chinese Room” would win in a landslide.

The Chinese Room thought experiment was put forth in 1980 by John Searle, a professor at the University of California, in a paper called “Minds, Brains, and Programs” published in the journal Behavioral and Brain Sciences. Searle argued that, no matter how successful an AI is in imitating human behavior—even if it answers questions the way a person would, thus passing the Turing Test—it can’t be said to have “understanding.”

The Stanford Encyclopedia of Philosophy says that “the Chinese Room argument has probably been the most widely discussed philosophical argument in cognitive science to appear since the Turing Test. By 1991 computer scientist Pat Hayes had defined Cognitive Science as the ongoing research project of refuting Searle’s argument.”

Three decades after Hayes identified that project, it got a big boost. The large language models that started gaining prominence in the 2020s posed a new kind of challenge to Searle’s argument.

Here is a fragment from Searle’s paper that will give you some idea why. He writes that the kind of information processing that AIs do has “only a syntax but no semantics.” That was arguably true of the kinds of AIs he was talking about. But, as we’ve seen, large language models do have a kind of “semantics.” They depict the meaning of words by locating them in high-dimensional semantic space.

Still, it would be a mistake to dismiss Searle’s argument on the basis of that quote alone. Though he wrote his paper before the coming of large language models, it turns out that his Chinese Room argument raises a valid and vital question about those models—namely, whether they really do represent meaning in a complete way.

What’s more, it turns out that the answer to that question is no. The semantic mapping that LLMs do is impressive and powerful, but if your goal is to build AIs that “understand” things the way humans do, you need to add at least one major tool to their machinery for handling meaning. It’s a credit to Searle that he identified this tool long before LLMs existed.

However, this credit comes with a big asterisk. This “missing” major tool already exists. It just exists in the form of a different kind of AI—an AI that might not be considered a large language model in the narrow sense of the term but that, in any event, can be integrated into such models, greatly expanding their power. In fact, that integration started happening in 2023, two years before Searle died at age ninety-three. LLMs equipped with this missing tool do have the things that, according to Searle’s famous 1980 paper, were prerequisites for true understanding.

That doesn’t mean these AIs are capable of true understanding. There may be other prerequisites for understanding that AIs still lack. It all depends on how you define “understanding”—you’re free to add as many prerequisites as you like.

But I think it’s going to get harder and harder to define understanding in a way that keeps AIs from qualifying for it. Because this “missing tool” is actually more like a “missing link”—a link that, in principle, can connect large language models to everything they need in order to become the functional equivalent of the entire human mind. Though Searle in 1980 aimed to undermine great expectations about the coming power of AI, his argument, viewed forty-six years later, validates them…

Excerpted from The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning by Robert Wright. Copyright © 2026 by Robert Wright. Published by Simon & Schuster.