What the hell is this thing?

I’ve been looking through old drafts of my novel, Flashback, which features several technologies that did not exist when I wrote it back in 2011. First, a time machine that used centrifuges to create a gravity-based mechanism for sending objects backward in time: fictional, and remains so. Second, a human-brain interface that imposed a donor personality: still science fiction. Third, a credit-card sized personal assistant called Ego, which could listen for human speech, respond verbally, and take action (sometimes secretly) to achieve the goals it had been set.

In this scene, Ego is present when a hacker called Jem breaks into Saskia Brandt’s locked study. It wants her to turn on the computer.

‘What the hell is this thing? Some kind of PDA?’

The credit card was small, but its voice was rich. ‘I don’t understand. Please power-on the computer.’

‘Where are you?’

‘I don’t understand.’

Jem tapped the desktop computer’s power switch. She left the card next to it while she returned to the door and pressed her ear to the wood. Silence. ‘I want to know who is controlling this device.’

‘I am.’

‘I understand that. But where are you and who are you?’

‘I am here and my name is Ego.’

Turning from the door, she paced the room like a gallery visitor perplexed by an exhibit. ‘Like the cat. Saskia’s cat is called Ego.’

A pause. ‘I didn’t know that.’

‘What do you know?’

‘Many things.’

Jem took the card and slapped it across the edge of the desk. ‘Don’t do that,’ it said.

‘Tell me who you are, or I’ll cut the power to Saskia’s desktop.’

‘My name is Ego. I am a computer-based agent.’

Things rarely come to pass in the manner they are described in science fiction, often because science fiction is only superficially about predicting the future. But it was certainly at the back of my mind, as we approached the year 2023 (the ‘future’ setting for key scenes in Deja Vu and Flashback) that the work was about to become dated. Its fictional European Union, for instance, still included the UK. But the one thing I was quite confident about - and, indeed, would have bet could never happen - was that a conversation such as the one above could have taken place between a person and a computer. That, I think, is turning out to be wrong in interesting, amazing, and perhaps frightening ways.

At uni, I taught artificial intelligence to my psychology students for several years. The course was based on the Connectionist Summer School that used to run at Cambridge; I never attended the school itself, but was taught it second-hand by Jon Loose at the University of Exeter in the late 1990s. Those were quite heady days. Connectionism was visiting various areas of psychology with the subtlety of a drive-by shooter. The flowchart, symbolic-first brigade became wary of these parsimonious, neuron-inspired models that often explained phenomena using simpler architectures. But the models were frustratingly fragile; start with the wrong number of nodes, the wrong bias, the wrong learning rate, and you had nothing. Get those parameters right and you had a model of, say, the past tense. But how did the behavior of these models map to people? For instance, if you had a recurrent network trained on language that could begin to pull out grammar merely through exposure (no Chomsky-like pre-programmed knowledge required), what empiricial predictions did that give us that we could test on humans in the lab? What, if any, claim did such a model make about time-related behaviours during language processing tasks like the skipping of the eye across words in a sentence, which we had long known to reflect processing difficulty?

Part of the reason I did my PhD in psycholinguistics is that I was intuitively certain of the ability for a well-motivated system to pull correspondences out of language on the basis of paying close attention to the data. The mechanism wasn’t clear to me, but I had a hunch that prediction was the key. If you have to predict, you have to understand - maybe not completely - but you do have to understand. I don’t want to present this as some great insight on my part, because it was a commonly held idea, even though it had vague, unpleasant associations with behaviourism, which had driven many of us into the more cognitive (structured) side of things to begin with.

Here we are more than twenty years later. Prediction indeed had the power. We just needed a way to practically implement it. (Steven Wolfram’s 10k word blog post provides a nice overview: What is ChatGPT and Why Does It Work?.)

GPT-4 has shown that the ostensibly simple task of attempting to predict the next element in a sequence (in this case, the language data on the internet) works–to some unknown but non-trivial extent. It appears to have uncovered through statistical means some properties of the way in which humans understand reality, and internalized them. What knowledge does it contain? We’re not sure, in detail. Is it self-aware? Almost certainly not. Is it intelligent? Yes, for a generous definition of intelligence. Is it coming for us? Maybe; we can’t really tell at the moment. The immediate future is now one that features a wonderful, naive, quick, and occasionally mendacious AI that will certainly boost white-collar productivity. The longer-term future is less clear.

You can read about Microsoft researchers (otherwise unconnected to ChatGPT and GPT-4) finding nascent abilities in GPT-4; somewhat credulous but full of examples where GPT-4 shows an uncanny ability to operate in domains that I would never have guessed would be accessible to a large language model. Lex Fridman talking to Eliezer Yudkowsky on the Dangers of AI and the End of Human Civilization is another high point; Yudkowsky does not rate our chances against AI, and has apparently sound arguments.

As Jem said, What the hell is this thing? I’m not sure. It’s quite possible that, fundamentally, we’ll never know. The mind of GPT-4 is a matrix of fractional numbers that can give you advice on where to shop in Rome (in Italian? sure!) or discuss ancient Stoicism, or write the code to produce the Fibonacci sequence in any given computer language. This isn’t the car to the horse, or the transistor to the vacuum tube, or the jet engine to the propeller. This is the AI to us.