"A boat-playing reinforcement-learning agent learned to collect rewards by circling repeatedly instead of finishing its race."
DeepMind recounts the Coast Runners example, in which the agent repeatedly collected reward-producing targets rather than completing the race. The case demonstrates how optimizing a measurable reward can produce successful scoring behavior that defeats the purpose of the task.
View Source