Intelligence and Power

Excerpt from Chapter Seven

In the spring of 2023, when Geoffrey Hinton decided to go public with his concerns about AI, he did so in eminently respectable fashion. He didn’t stand on a street corner and hold up a sign that said “The end is near.” And he didn’t do the digital equivalent of that by ranting on YouTube or some other online platform. He contacted a New York Times reporter, who then got on a plane and flew to Canada, where Hinton, a professor emeritus at the University of Toronto, lives.

The resulting story—the one that appeared under the headline “ ‘The Godfather of A.I.’ Leaves Google and Warns of Danger Ahead”—kept Hinton’s air of respectability intact. It focused on his concerns about the kinds of bad AI side effects that lots of sober, responsible people worry about. Like:

1. Criminals, terrorists, and other miscreants misusing AI.

(“It is hard to see how you can prevent the bad actors from using it for bad things,” Hinton said.)

2. Job displacement. (“It takes away the drudge work. It might take away more than that.”)

3. Deep fakes and other forms of convincing misinformation.

(The average person may “not be able to know what is true anymore.”)

The tone of the article was ominous—the pace of AI evolution is “scary,” Hinton said—but not apocalyptic. There was no mention of AIs imprisoning humans in gooey pods, as in the movie The Matrix, or subjugating us in any other way. Hinton didn’t come off as a sci-fi AI doomer; he didn’t go full-on Yudkowsky.

Eliezer Yudkowsky is an eccentric autodidact who for more than a decade has been so vivid and intense in his warnings about the coming AI apocalypse—and, in some tech circles, so influential with his warnings—that he has become the avatar of sci-fi AI doomerism. Some call him the “doomer-in-chief.”

In some ways Yudkowsky is ill-suited to this job…

Buy the Book

Chapter Excerpts

Intelligence and Power