作者：Rich Sutton（理查德·萨顿）
日期：2019 年 3 月 13 日
1. The Biggest Lesson of AI Research｜人工智能研究最重要的教训
The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation.
从过去 70 年的人工智能研究中，我们能够得出的最重要教训是：那些能够充分利用计算能力的通用方法，最终会成为最有效的方法，而且其优势往往是压倒性的。
其根本原因在于摩尔定律，或者更准确地说，是摩尔定律所体现的更普遍趋势：单位计算成本持续以指数级速度下降。
Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available.
大多数人工智能研究都是在一个隐含假设下开展的：智能体（Agent）可用的计算资源是固定不变的。
如果这一假设成立，那么利用人类已有的知识，确实是提升系统性能的少数几种途径之一。
然而，只要把时间尺度拉长到略微超过一个典型研究项目的周期，我们就会发现：更大规模的计算资源几乎必然会变得可用。
Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters in the long run is the leveraging of computation.
These two need not run counter to each other, but in practice they tend to. Time spent on one is time not spent on the other. There are psychological commitments to investment in one approach or the other.
And the human-knowledge approach tends to complicate methods in ways that make them less suited to taking advantage of general methods leveraging computation.
为了在短期内取得显著的性能提升，研究人员往往倾向于利用自己对特定领域的理解和知识。
但从长期来看，真正起决定性作用的，是能否充分利用不断增长的计算能力。
这两种方法在理论上并不一定相互冲突，但在实践中，它们往往彼此竞争。
投入到一种方法上的时间，就无法再投入到另一种方法上。而且，研究人员通常还会对自己选择的研究路线产生心理上的依赖。
更重要的是，依赖人类知识的方法往往会使系统变得更加复杂，从而削弱它利用大规模计算和通用方法的能力。
There were many examples of AI researchers' belated learning of this bitter lesson, and it is instructive to review some of the most prominent.
人工智能研究史上，有许多研究人员直到很晚才领悟这一苦涩教训的例子。回顾其中几个最具代表性的案例，将非常有启发性。
2. Computer Chess｜计算机国际象棋
In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search.
At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess.
在计算机国际象棋领域，1997 年击败世界冠军卡斯帕罗夫（Kasparov）的方法，主要依赖于大规模的深度搜索。
当时，大多数计算机国际象棋研究人员一直致力于利用人类对国际象棋特殊结构的理解来设计程序。
因此，当基于大规模搜索的方法取得胜利时，他们感到十分失望。
When a simpler, search-based approach with special hardware and software proved vastly more effective, these human-knowledge-based chess researchers were not good losers.
They said that "brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess.
These researchers wanted methods based on human input to win and were disappointed when they did not.
当一种更简单、基于搜索的方法，借助专门设计的硬件和软件，展现出远超其他方法的效果时，那些依赖人类知识的国际象棋研究人员并不愿意坦然接受失败。
他们辩称，虽然这种“暴力搜索”（Brute Force Search）的方法这次取得了胜利，但它并不是一种通用策略，而且无论如何，这也不是人类下棋的方式。
这些研究人员原本希望基于人类知识的方法能够获胜，因此当结果并非如此时，他们感到非常失望。
3. Computer Go｜计算机围棋
A similar pattern of research progress was seen in computer Go, only delayed by a further 20 years.
Enormous initial efforts went into avoiding search by taking advantage of human knowledge, or of the special features of the game, but all those efforts proved irrelevant, or worse, once search was applied effectively at scale.
在计算机围棋领域，我们看到了几乎相同的研究发展模式，只不过这一过程又晚了大约 20 年。
早期研究人员投入了大量精力，试图利用人类的围棋知识，或者围棋本身的特殊性质，来避免进行大规模搜索。
然而，一旦大规模搜索被有效应用，这些努力就变得无关紧要，甚至可能适得其反。
Also important was the use of learning by self play to learn a value function (as it was in many other games and even in chess, although learning did not play a big role in the 1997 program that first beat a world champion).
Learning by self play, and learning in general, is like search in that it enables massive computation to be brought to bear.
另一个重要突破是利用**自我对弈学习（Self-Play Learning）**来训练价值函数（Value Function）。
这种方法在许多其他游戏中同样发挥了重要作用，包括国际象棋，尽管在 1997 年首次击败世界冠军的程序中，学习并没有扮演主要角色。
自我对弈学习，以及更广义上的机器学习，与搜索具有一个共同特征：它们都能够有效利用大规模计算资源。
Search and learning are the two most important classes of techniques for utilizing massive amounts of computation in AI research.
In computer Go, as in computer chess, researchers' initial effort was directed towards utilizing human understanding (so that less search was needed) and only much later was much greater success had by embracing search and learning.
搜索（Search）和学习（Learning），是人工智能研究中利用大规模计算资源最重要的两类技术。
在计算机围棋领域，与国际象棋一样，研究人员最初努力的方向，是利用人类的理解来减少搜索的需求。
直到很久以后，当研究人员真正拥抱搜索和学习时，才取得了远为显著的成功。
4. Speech Recognition｜语音识别
In speech recognition, there was an early competition, sponsored by DARPA, in the 1970s.
Entrants included a host of special methods that took advantage of human knowledge—knowledge of words, of phonemes, of the human vocal tract, etc.
On the other side were newer methods that were more statistical in nature and did much more computation, based on hidden Markov models (HMMs).
Again, the statistical methods won out over the human-knowledge-based methods.
在语音识别领域，20 世纪 70 年代，美国国防高级研究计划局（DARPA）曾资助过一场早期的技术竞赛。
参赛方案中，有许多专门利用人类知识设计的方法，例如关于词汇、音素以及人类声道结构的知识。
与此同时，另一类较新的方法则更多地依赖统计学和大规模计算，例如基于隐马尔可夫模型（Hidden Markov Models，HMMs）的方法。
最终，统计方法再次战胜了依赖人类知识的方法。
This led to a major change in all of natural language processing, gradually over decades, where statistics and computation came to dominate the field.
The recent rise of deep learning in speech recognition is the most recent step in this consistent direction.
Deep learning methods rely even less on human knowledge, and use even more computation, together with learning on huge training sets, to produce dramatically better speech recognition systems.
这一结果推动了整个自然语言处理领域的重大转变。
在随后的几十年里，统计方法和计算能力逐渐成为这一领域的主导力量。
近年来，深度学习在语音识别领域的崛起，只是这一持续发展趋势的最新阶段。
深度学习方法对人类知识的依赖更少，却使用了更多计算资源，并通过在海量训练数据上进行学习，构建出了性能显著更好的语音识别系统。
As in the games, researchers always tried to make systems that worked the way the researchers thought their own minds worked—they tried to put that knowledge in their systems—but it proved ultimately counterproductive, and a colossal waste of researcher's time, when, through Moore's law, massive computation became available and a means was found to put it to good use.
与棋类游戏研究一样，研究人员总是试图让系统按照他们认为人类大脑运作的方式工作。
他们试图把自己对人类思维的理解直接植入系统。
然而，当摩尔定律使大规模计算资源变得可用，并且人们找到了有效利用这些资源的方法之后，这种做法最终被证明是适得其反的。
从长远来看，它甚至造成了研究人员时间和精力的巨大浪费。
5. Computer Vision｜计算机视觉
In computer vision, there has been a similar pattern.
Early methods conceived of vision as searching for edges, or generalized cylinders, or in terms of SIFT features.
But today all this is discarded.
Modern deep-learning neural networks use only the notions of convolution and certain kinds of invariances, and perform much better.
在计算机视觉领域，也出现了类似的发展模式。
早期的方法试图通过检测边缘、识别广义圆柱体（Generalized Cylinders），或者提取 SIFT（尺度不变特征变换）特征等方式来理解视觉。
但如今，这些方法已经被主流方法所取代。
现代深度学习神经网络主要依赖卷积（Convolution）以及某些形式的不变性（Invariances）等基本概念，却能够取得远为出色的性能。
6. Why the Lesson Is Bitter｜为什么这是一个苦涩的教训
This is a big lesson.
As a field, we still have not thoroughly learned it, as we are continuing to make the same kind of mistakes.
To see this, and to effectively resist it, we have to understand the appeal of these mistakes.
We have to learn the bitter lesson that building in how we think we think does not work in the long run.
这是一个极其重要的教训。
然而，作为一个研究领域，人工智能至今仍然没有彻底吸取这一教训，因为我们还在不断重复同样的错误。
要认识这些错误，并有效避免再次犯错，我们首先必须理解：为什么这些错误的方法对研究人员具有如此强大的吸引力。
我们必须接受这个苦涩的教训：
试图把我们认为自己如何思考的方式，直接构建进人工智能系统，从长期来看是行不通的。
The bitter lesson is based on the historical observations that:
-
AI researchers have often tried to build knowledge into their agents, -
this always helps in the short term, and is personally satisfying to the researcher, -
but in the long run it plateaus and even inhibits further progress, and -
breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning.
这一苦涩教训建立在以下历史观察之上：
-
人工智能研究人员经常试图将人类知识直接植入智能体。 -
这种方法通常能够在短期内带来帮助，而且会让研究人员获得个人成就感。 -
但从长期来看，这种方法的性能提升会逐渐达到瓶颈，甚至阻碍进一步的发展。 -
真正的突破性进展，最终往往来自另一条相反的技术路线：通过搜索和学习来扩大计算规模。
The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.
这种最终的成功之所以带有苦涩的意味，而且常常难以被研究人员完全接受，是因为它所战胜的，恰恰是研究人员长期偏爱、以人类为中心的研究方法。
7. The Power of General-Purpose Methods｜通用方法的强大力量
One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great.
The two methods that seem to scale arbitrarily in this way are search and learning.
我们应该从这一苦涩教训中学到的第一件事，是通用方法（General-Purpose Methods）所具有的巨大力量。
这些方法即使在可用计算资源已经非常庞大的情况下，仍然能够随着计算规模的进一步扩大而持续提升性能。
目前看来，似乎能够以这种方式持续扩展的两类方法，就是：
搜索（Search）和学习（Learning）。
8. The Complexity of Intelligence｜智能的复杂性
The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries.
All these are part of the arbitrary, intrinsically-complex, outside world.
我们应该从这一苦涩教训中学到的第二个普遍性结论是：
心智中实际包含的内容极其复杂，而且这种复杂性是无法从根本上消除的。
我们应该停止试图用简单的方式来概括心智中的内容，例如试图用简单的模型来理解空间、物体、多个智能体之间的关系，或者对称性。
所有这些概念，都属于任意多样且本质上极其复杂的外部世界的一部分。
They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity.
Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us.
我们不应该把这些具体知识直接内置到人工智能系统中，因为它们所涉及的复杂性是无穷无尽的。
相反，我们应该只内置那些能够自主发现并捕捉这种复杂性的元方法（Meta-Methods）。
这些方法的一个关键能力，是能够找到对复杂世界的良好近似。
但寻找这些近似的过程，应该由我们设计的方法来完成，而不是由我们人类亲自完成。
We want AI agents that can discover like we can, not which contain what we have discovered.
Building in our discoveries only makes it harder to see how the discovering process can be done.
我们想要的，是能够像人类一样自主发现知识的人工智能体，而不是仅仅装载了人类已经发现的知识的智能体。
把我们已经发现的知识直接植入系统，只会让我们更难看清：究竟应该如何让智能体自己完成发现知识的过程。
核心术语对照
|
|
|
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
全文最重要的一句话：
We want AI agents that can discover like we can, not which contain what we have discovered.
我们想要的，是能够像人类一样自主发现知识的人工智能体，而不是仅仅装载了人类已经发现的知识的智能体。