Alignment problems
IN THIS CHAPTER
Pitfalls of reinforcement learning
Objective setting for AI
Who gets to choose?
Why is misalignment such a difficult problem to solve?
AI misalignment
Alignment problems are the unintended and often undesirable consequences of giving AI instructions, objectives or training that is incomplete, ambiguous, or simply poorly thought-through.
For example, when OpenAI tried to train one of its agents to win the Coastrunners video game, it repeatedly crashed the boat until it burst into flames because its training rewarded it for accumulating points, instead of crossing the finish line in the fastest time.
This chapter explores why alignment problems are such a so difficult to avoid.