"Reinforcement learning works best when trial and error is cheap, the environment can be reset, and success can be measured without too much ambiguity."
Karpathy argues that verifiability is a central constraint on what modern AI systems can learn through optimization and reinforcement learning. He identifies three crucial properties of favorable training environments - resettable, efficient, and rewardable. This supports the chapter's distinction between tasks that can be repeatedly practiced and objectively evaluated, and messy real-world situations where feedback is ambiguous or expensive.
View Source