Machine learning (ML) is the science of credit assignment. It seeks to find patterns in observations that explain and predict the consequences of events and actions. This then helps to improve future performance. Minsky's so-called "fundamental credit assignment problem" (1963) surfaces in all sciences including physics (why is the world the way it is?) and history (which persons/ideas/actions have shaped society and civilisation?). Here I focus on the history of ML itself. Modern artificial intelligence (AI) is dominated by artificial neural networks (NNs) and deep learning, both of which are conceptually closer to the old field of cybernetics than what was traditionally called AI (e.g., expert systems and logic programming). A modern history of AI & ML must emphasize breakthroughs outside the scope of shallow AI text books. In particular, it must cover the mathematical foundations of today's NNs such as the chain rule (1676), the first NNs (circa 1800), the first practical AI (1914), the theory of AI and its limitations (1931-34), and the first working deep learning algorithms (1965-). From the perspective of 2025, I provide a timeline of the most significant events in the history of NNs, ML, deep learning, AI, computer science, and mathematics in general, crediting the individuals who laid the field's foundations. The text contains numerous hyperlinks to relevant overview sites. With a ten-year delay, it supplements my 2015 award-winning deep learning survey which provides hundreds of additional references. Finally, I will put things in a broader historical context, spanning from the Big Bang to when the universe will be many times older than it is now.
翻译:机器学习(ML)是一门研究信用分配的科学。它致力于从观测数据中发现能够解释和预测事件及行为结果的规律,从而提升未来的表现。明斯基提出的所谓“基本信用分配问题”(1963)普遍存在于包括物理学(世界为何呈现当前形态?)和历史学(哪些人物/思想/行动塑造了社会与文明?)在内的所有科学领域。本文聚焦于机器学习自身的发展历程。现代人工智能(AI)主要由人工神经网络(NNs)和深度学习主导,这两者在概念上更接近早期的控制论领域,而非传统意义上的人工智能(如专家系统和逻辑编程)。一部现代AI与ML的发展史必须突破浅层AI教科书的局限,着重阐述关键性突破。尤其需要涵盖当今神经网络的理论基础,例如链式法则(1676)、早期神经网络雏形(约1800年)、首个实用AI系统(1914年)、AI理论及其局限性研究(1931-34年),以及首批可运行的深度学习算法(1965年至今)。站在2025年的视角,本文梳理了神经网络、机器学习、深度学习、人工智能、计算机科学及广义数学史上最具里程碑意义的事件时序,并致敬了奠定该领域基石的先驱者。文中包含大量相关综述站点的超链接。作为延迟十年的补充,本文对笔者2015年获奖的深度学习综述进行了增补,新增数百篇参考文献。最后,我将把这一切置于更宏大的历史背景中——从宇宙大爆炸开始,直至宇宙年龄远超当下的未来纪元。