search and learning

The Bitter Lesson

Rich Sutton's March 2019 essay arguing that the biggest lesson from 70 years of AI research is that general methods leveraging computation ultimately prove most effective, by a large margin. Sutton walks through prominent cases — computer chess, Go, speech recognition, and computer vision — where approaches built on human knowledge were outdone by search and learning methods that scaled with computing power. He attributes the pattern to the steadily falling cost of computation and warns that relying on human knowledge tends to complicate methods in ways that limit scaling.