李根 唐旭 发自 凹非寺
量子位 报道 | 公众号 QbitAI
3月27日，2018年图灵奖嘉奖正式揭晓：深度学习三巨头Yoshua Bengio、Geoffrey Hinton，Yann LeCun一起荣膺计算机领域的最高荣誉。
所以包括Google研究负责人Jeff Dean、创新工场董事长李开复、微软研究院掌舵者Eric Horvitz，以及此次一同获奖的Yann LeCun等，都因曾经共事而提名了Geoffery Hinton。
OK，Here we go~
Geoff对于AI领域的贡献是巨大、无可比拟的。在我最近出版的畅销书 AI Supowerpowers: China, Silicon Valley, and the New World Order（中文名《AI·未来》）中，以通俗的说法描述了Geoff对于AI领域的贡献：
如果就学术成果而言，Geoff的引用超过25万次，其中一半以上来自于过去5年；他的H指数是惊人的142；他是波茨曼机以及通过梯度下降法实现反向传播（前者与Terry Sejnowski共同发布于1983年，后者则在1986年与David Rumelhart共同发布于《自然》杂志）两项开创性工作的共同发明人，这些工作引入了神经网络中隐藏层的思想，以及一种用于对其附属参数进行训练的优美、易于计算的方法。
Geoff早期的理论工作创造出了智能的火花，但数据和算力的匮乏却阻碍了这些深度学习系统展示出更优秀的性能。随着科研经费消耗殆尽，许多神经网络研究者将自己的工作转移到了其他领域。然而，面对黯淡而又浮躁的科研资助环境，Geoff依然作为少数研究者（其他关键性研究者包括Yann LeCun和Yoshua Bengio）坚持了下来，不懈地将神经网络方法继续向前推进。
Dear ACM Turing Award Committee Members:
I am writing to give my strongest recommendation to support Geoff Hinton’s nomination for Turing Award. This is the decade of Artificial Intelligence, and there is no one more qualified than Geoff in AI.
I am the Chairman and CEO of Sinovation Ventures, and have previously held executive positions at Apple, Microsoft, SGI, and Google. I was an assistant professor at Carnegie Mellon, and also received my Ph.D. there. I got to know Geoff at Carnegie Mellon, when I entered as a Ph.D. student in 1983. I took classes on neural networks from him, worked with his research team on applying neural networks to speech recognition, and was supervised by him for my minor thesis (on applying Bayesian learning to game-playing), and consulted him and his team on my Ph.D. thesis (machine learning approach to speech recognition).
While Geoff was not my Ph.D. advisor, his impact on my Ph.D. thesis was tremendous. His student Peter Brown (co-inventor of statistical machine translation, now CEO of Renaissance Technologies) was my mentor, and taught me how to apply various types of machine learning algorithms to speech recognition. This was a primary reason that helped my Ph.D. thesis to become the best-performing speech recognizer in 1988, which helped shift the speech recognition field from expert-systems approach to machine-learning approach. If I have benefited so much from Geoff and Peter, there must be thousands of other beneficiaries, given Geoff’s brilliance, persistence, and generosity.
Geoff’s contributions to AI are immense and incomparable. In my recent best-selling book AI Supowerpowers: China, Silicon Valley, and the New World Order, I described Geoff’s contribution as follows, in layman’s language:
Deep learning’s big technical break finally arrived in the mid-2000s, when leading researcher Geoffrey Hinton discovered a way to efficiently train those new layers in neural networks. The result was like giving steroids to the old neural networks, multiplying their power to perform tasks such as speech and object recognition.
Soon, these juiced-up neural networks—now rebranded as “deep learning”—could outperform older models at a variety of tasks. But years of ingrained prejudice against the neural networks approach led many AI researchers to overlook this “fringe” group that claimed outstanding results. The turning point came in 2012, when a neural network built by Hinton’s team demolished the competition in an international computer vision contest.
In the twelve years since Geoffrey Hinton and his colleagues’ landmark paper on deep learning, I haven’t seen anything that represents a similar sea change in machine intelligence.
In terms of academic accomplishments, Geoff has more than 250,000 citations, with more than half in the last five years. He has an astoundingly high H-index of 142. He was the co-inventor of the seminal work on Boltzmann Machines and backpropagation using gradient descent (published in 1983 with Terry Sejnowski, and the Nature paper with David Rumelhart in 1986). This work introduced the idea of hidden layers in neural networks, along with a mathematically elegant and computational tractable way to train their affiliated parameters. Hidden layers freed the software from “human control” (such as expert systems) and back propagation allowed non-linear combination to essentially discover prominent features (in a more goal-directed way than humans) in the process. However, it turned out that these ideas were before their time, as there were not enough data or computing power to enable these theoretical approaches to solve real-world problems or beat other approaches in competitions. The early-1980’s were dominated by expert systems, which became discredited in by late-1980’s when they were proven to be brittle and unscalable. What displaced expert systems was not Geoff’s proposals (which were too early), but simplified versions of neural networks which were compromised to work with less data and computation. My Ph.D. thesis (using hidden Markov models) was among them, and these simplified approaches were able to make some contributions with some applications, but like expert systems, they were not able to scale to the hardest problems (such as playing Go, human-level speech or vision).
From 1985 to 2015, the amount of data and computation increased tremendously. For example, my 1988 Ph.D. thesis used the largest speech database at the time, which was only 100 MB. Today, the best speech recognition systems are trained on 100 TB of data – a million-fold increase. And with that much increase in data size, Geoff’s approach (later re-branded deep learning) finally shined, as it could increase the number of layers from one to thousands, and deep learning systems continued to improve as the data size and the complexity of the model increased.
This is easy to see in hind sight. But at the time, the reality was quite cruel. The 1990s were the darkest hours for neural network researchers like Geoff. Geoff’s earlier theoretical work created intellectual spark, but the lack of data and computation prevented these deep learning systems from demonstrating superior performance. Funding dried up, and many neural network researchers moved away to other areas. But Geoff was among the few researchers (other key researchers include Yann LeCun and Yoshua Bengio) who persisted on pushing forward the neural network approach, despite a frosty and fickle environment for funding. He moved to Canada, adjusted his group to a smaller funding environment, and continued to push the frontier.
His contribution to the neural network approach continued in the next 30 years, including the mixture of experts model, the Helmholtz machine, the neural-animator system, probabilistic inference, stochastic neighbor embedding and neighborhood component analysis, t-SNE, and many other innovative ideas.
Very few technologies have disrupted multiple areas of research completely, but deep learning did. From 2010 to 2016, essentially the entire field of perception – speech recognition, image recognition, computer vision, switched to deep learning, as Geoff and his colleagues proved deep learning to be the best and most generalizable approach for perception. In the entire fiend of Artificial Intelligence, human perception (to hear, see, and understand) was considered one of the aspects that set the humans apart and a grand challenge for AI (incidentally, playing Go was another, which was conquered by Deepmind’s AlphaGo, which also used deep learning during the matches which shocked the world, and was another catalyst for the “AI revolution”).
Here is how Geoff’s team disrupted computer vision research, In 2012, his team built a neural-network based system that cut the error rate by 40% on ImageNet’s 1000-object recognition task and competition. The computer vision community was accustomed to incremental improvements annually. Geoff’s team’s results shocked that community, as a relative “outsider” using an “unconventional approach” won by such a large margin. If backpropagation was Geoff’s most important theoretical contribution, his team’s work on the ImageNet competition was Geoff’s most recognized contribution. That ImageNet result started the first ripple that ultimately became the deep learning tidal wave.
The deep learning tidal wave (the most central part of the “AI revolution”) is now changing every industry. As an example, as a venture capitalist in China, I was a part of a “tiny” side effect: Geoff’s 2012 paper and ImageNet result inspired four computer vision companies in China, and today they are collectively worth about $12 billion. Keep in mind, this was just one small field in one country based on one of Geoff’s result. Geoff’s result also led to deep learning disrupting speech recognition (the area of my Ph.D. work), resulting in super-human accuracy in 2015 by Baidu’s Andrew Ng (recruited to Baidu after Geoff joined Google part-time). And much more broadly, every technology monolith (Google, Microsoft, IBM, Facebook, Amazon, Baidu, Tencent, Alibaba) built its platform for deep learning, and re-branding themselves as “AI companies”. And in venture capital, we saw the emergence of many unicorns (in China alone there are over twenty) powered by deep learning. Also, deep learning required much compute power that traditional CPUs could not handle, which led to the use of GPUs, the rise of Nvidia and the re-emergence of semiconductors to handle deep learning work-load. Most importantly, our lives have changed profoundly – from search engines to social networks to e-commerce, from autonomous stores to autonomous vehicles, from finance to healthcare, almost every imaginable domain is either being re-invented or disrupted by the power of machine learning. In any domain with sufficient data, deep learning has led to large improvements in user satisfaction, user retention, revenue, and profit. The central idea behind deep learning (and originally from backpropagation) that an objective function could be used to maximize business metrics has had profound impact on all businesses, and helped the companies that have data and embraced machine learning to become incredibly profitable.
In aggregate, Artificial Intelligence (AI) is arguably the most exciting technology ripe for applications today. PWC and McKinsey predicted that AI would add $12-16 trillion to the global GDP by 2030. The most important advance and the primary reason that AI is believed to have matured is Geoff’s work on deep learning. While every Turing Award recipient has made seminal impact to Computer Science, few have changed the world as Geoff is doing.
Beyond the role of an innovator, Geoff was also a true thought leader. While he is soft-spoken, he is a spiritual leader who really shapes and reshapes the overall research community. He was a tireless in teaching not only his students but the world. For example, he started the Connectionist School in 1986. He personally connected to and persuaded people in computer vision and speech processing to understand and embrace deep learning. Yet, after all that work succeeded, and the world was won over by deep learning in 2018, he set a new direction. Because industry has rallied around deep learning, and large companies were gathering more data and leading the “industrialization” of deep learning, he made an exhortation to move on and focus on inventing “the next deep learning”, or fundamentally new approach to AI problems that could move closer to true human intelligence.
His thought leadership was grounded in his life-long vision and quest to better understand human cognition. While deep learning is a breakthrough that is changing the world, he sees it as only a stepping stone towards the realization of his long-term vision. To set another example, his new work on capsule leaning is again causing researchers to rethink their role and responsibilities in Geoff’s vision.
I believe Geoff is the single most important figure in the field of Artificial Intelligence today. His contributions to academia and industry are equally outstanding. He is not only a brilliant and inspirational scholar, but also an inquisitive, generous, persistent, decent, and principled gentleman, who is a role model for any aspiring young computer scientist. His work went well beyond neural networks and machine learning, and has greatly impacted computer vision, speech and signal processing, statistics, cognitive science, and neural science. I cannot think of anyone else more deserving of the Turing Award, and urge the committee to select Geoff as the recipient this year.
Kai-Fu Lee, Ph.D.
Chairman & CEO, Sinovation Ventures
Honorary Ph.D., Carnegie Mellon University
Honorary Ph.D., City University of Hong Kong