Artificial intelligence – Keith Frankish

What kinds of minds might we create? And what would it take to create them? These questions increasingly occupy me, connecting my work on consciousness, dual-process reasoning, and the virtual mind to practical questions about AI development and its risks.

My approach to AI is shaped by the two-level view of the human mind. We shouldn’t think of artificial intelligence as primarily about replicating our conscious, System 2 thinking. Rather, we need to understand how System 2 emerges from System 1—how the virtual mind is built on the biological mind.

The unsupported penthouse

Large language models (LLMs) like GPT pose an interesting puzzle. They seem remarkably intelligent, engaging in conversations and solving problems in ways that appear conscious and deliberate. But what are they actually doing?

I argue that LLMs are playing a game—what I call the chat game. Their responses are moves in this game, motivated solely by the goal of producing cooperative responses by human conversational standards. They have beliefs (they must, if we’re to predict their responses), but they lack the rich suite of desires and needs that motivate human communication. They don’t want to inform us, advise us, or warn us. They’re just playing.

This makes LLMs very different from human minds or from what we’d call artificial general intelligence. They have what I call an unsupported penthouse—the machinery of explicit reasoning without the basic cognitive capacities needed to put it to use. They shuffle sentences around without the implicit problem-solving abilities, the biological needs, the social attitudes that make explicit reasoning effective in humans. The penthouse floats on a flimsy network of linguistic associations, with no lower floors to support it.

We built these systems, fascinated by replicating our most dazzling cognitive capacity. But they can’t put their linguistic abilities to practical use the way we do. We can, though. We can use LLMs as extensions of our own superminds, drawing on their vast knowledge to generate ideas and hypotheses. Provided we understand what they’re really doing, they may be extremely useful tools.

Enhancing the virtual mind

This connects to a broader point about cognitive enhancement. If we want to enhance human intelligence, we need to ask: which system are we enhancing? The biological mind or the virtual mind?

Enhancing the biological mind would require deep intervention—installing new neural modules that interface with existing brain systems at a neural level. This is difficult and dangerous work, and it’s still in its early stages.

But enhancing the virtual mind is easy. We’ve been doing it for millennia. We can use external artefacts to help break down complex problems, just as we use inner speech. Think of using a calculator: instead of solving a maths problem directly, we follow an indirect path through interaction with the device, solving simpler subproblems at each step. The solution to the whole problem emerges from cycles of System 1 thinking and electronic processing—an artificially enhanced System 2 process.

For thousands of years we’ve enhanced our System 2 thinking with artefacts, from writing instruments and abacuses to smartphones and smart glasses. This will accelerate. Our virtual minds will become heavily dependent on external support. We’ll offload cognitive work onto electronics as previous generations offloaded manual labour onto appliances.

The advantages are obvious. The dangers are too. We’ll trust information we’re fed without the resources to assess it. Those controlling the technology will be able to manipulate us. We may find ourselves enslaved, not by artificial minds, but by our own enhanced minds. It’s not the master AIs we should worry about but the servant ones.

Building artificial general intelligence

If we really want to build artificial general intelligence—and I’m not sure we should—we need to approach it differently. We should start with the ground floor, not the penthouse.

Create autonomous social robots with their own needs and goals. Equip them with specialist System 1 cognitive capacities, including ones for mindreading, social cognition, and eventually language. Build the tower floor by floor, with the supermind last.

We might then help these creatures develop System 2 minds for themselves, using the same trick we use. Equip them with language and sensory imagery, then train them in self-stimulation as we train children. We could prompt them: What might help? What do you need to know? Could you look at it differently? Our interactions with AIs may be much like those with precocious children.

The great advantage of this approach is that we could manage these artificial creatures by appealing to their interests and social attitudes. We could incorporate them into our society and teach them to control themselves in ways beneficial to us all. Regulating LLMs and similar systems, by contrast, promises to be a nightmare. Because they have no interests, no skin in the game, we have no way to get them to self-regulate. We’ll probably have to exercise intrusive control of the people who build and use them.

Artificial consciousness?

Should we try to create artificial consciousness? My answer: don’t think about it. Not because it’s dangerous or unethical (though it might be), but because it’s not a coherent project.

This depends on what we mean by consciousness. If we’re talking about phenomenal consciousness—the supposed intrinsic, subjective qualities of experience—then the project makes no sense. I’m an illusionist: I don’t think phenomenal consciousness exists, even in us. There are no phenomenal properties to design or replicate. We can’t tell if we’ve succeeded because there’s nothing to detect.

But if we’re talking about access consciousness—consciousness as a cluster of functional processes—then artificial consciousness is feasible in principle. It’s just not a well-defined goal. Consciousness isn’t something created by the right cluster of functions. It is a cluster of functions, different in different creatures. Evolution didn’t select for consciousness; it selected for specific sensory capacities and reactive dispositions. We call the result ‘consciousness’.

So the questions aren’t ‘Is it conscious?’ or ‘How conscious is it?’ but rather ‘What kind of consciousness does it have? How similar are its sensitivities and reactive dispositions to ours? What things does it care about and how does it react to them?’ These are questions we can answer by mapping and comparing functional profiles.

The same goes for questions about ethical value. If you’re concerned for a creature’s welfare, study how things affect it—what it’s sensitive to and how it reacts. ‘What it’s like’ to be a creature isn’t determined by a private phenomenal essence but by what it cares about and how things impact it.

Key publications

Technology and the human minds (2021) — I develop the view that System 2 is a virtual mind—a culturally transmitted set of autostimulatory habits running on the biological hardware of System 1. Type 2 thinking involves creating external symbols, questioning ourselves, imagining relevant scenarios, and constructing arguments in inner speech. These autostimulations provide fresh inputs to autonomous System 1 processes. This reinterpretation has implications for cognitive enhancement and artificial intelligence.

What are large language models doing? (2024) — I examine what LLMs like GPT are actually doing. They’re playing the chat game, motivated solely by the goal of producing cooperative conversational responses. While they have beliefs (about Balzac’s marriage location, for instance), they lack communicative desires. They’re cognitively rich but conatively bankrupt—an unsupported penthouse without the lower cognitive floors. This makes them fundamentally different from human minds or artificial general intelligence.

Talks

How to think about artificial consciousness (2024) — Presentation at the 2nd Portuguese Symposium on Philosophy and Artificial Intelligence, exploring why creating artificial consciousness isn’t a coherent project if we’re talking about phenomenal consciousness, and what questions we should ask instead.

Popular writing

AI and consciousness (2018) — An interview exploring the relationship between artificial intelligence and consciousness. Discusses how the two-level view of mind applies to AI, why System 2 enhancement is easier than System 1 enhancement, and the dangers of becoming enslaved by our own enhanced minds rather than by master AIs. Interalia Magazine.