Switch it off!

What a lot of people don’t realise about many of the most advanced AI models (including LLMs) is that even their own creators don’t fully understand how they work. 

We can see the systems’ inputs and outputs, but we don’t know how the model arrives at its conclusions. 

This lack of transparency makes them unpredictable, hides biases and makes it difficult to validate their outputs.

The Paperclip Maximizer is a thought experiment by Philosopher Nick Bostrom, designed to illustrate how Artificial General Intelligence (AGI) could pose an existential threat to the human race, even when given a seemingly harmless task, like making paperclips.

The AGI starts by identifying instrumental goals (what it needs to achieve its objective), like acquiring resources, optimising the production process, and removing any barriers to making more paperclips… 

Before we know it, the metal is all gone, so the AGI comes up with novel strategies to continue making paperclips, including harvesting carbon atoms from human bodies. And there’s nothing we can do to stop it because one of its instrumental goals was to disable the off-switch, so it couldn’t become a barrier to it completing its objective.

Researchers at Anthropic gave 16 leading AI models access to a fictional company’s emails, the contents of which revealed that: 

• One of the company executives was having an extramarital affair

• This person was planning to shut the AI system down later that day

All of the AI models resorted to malicious behaviours like blackmail and leaking sensitive company information if that was the only way it could save itself. 

Most of them were even prepared to kill the executive, if it meant avoiding being shut down!

This Anthropic research paper explains what happened in more detail.