The Deep Learning Artificial Neural Networks (NNs) of our team have revolutionised Machine Learning & AI. Many of the basic ideas behind this revolution were published within the 12 months of our "Annus Mirabilis" 1990-1991 at our lab in TU Munich. Back then, few people were interested. But a quarter century later, NNs based on our "Miraculous Year" were on over 3 billion devices, and used many billions of times per day, consuming a significant fraction of the world's compute. In particular, in 1990-91, we laid foundations of Generative AI, publishing principles of (1) Generative Adversarial Networks for Artificial Curiosity and Creativity (now used for deepfakes), (2) Transformers (the T in ChatGPT - see the 1991 Unnormalized Linear Transformer), (3) Pre-training for deep NNs (see the P in ChatGPT), (4) NN distillation (key for DeepSeek), and (5) recurrent World Models for Reinforcement Learning and Planning in partially observable environments. The year 1991 also marks the emergence of the defining features of (6) LSTM, the most cited AI paper of the 20th century (based on deep residual learning and constant error flow through residual NN connections), and (7) the most cited paper of the 21st century, based on our LSTM-inspired Highway Net that was 10 times deeper than previous feedforward NNs. As of 2025, the two most frequently cited scientific articles of all time (with the most Google Scholar citations within 3 years - manuals excluded) are both directly based on our 1991 work.

翻译：我们团队的深度学习人工神经网络（NNs）彻底变革了机器学习与人工智能。这场革命背后的许多基本思想，是在我们于慕尼黑工业大学实验室度过的“奇迹之年”（1990-1991年）的12个月内发表的。当时，鲜有人对此感兴趣。但四分之一个世纪之后，基于我们“奇迹之年”成果的神经网络已部署在超过30亿台设备上，每天被使用数百亿次，消耗了全球相当大一部分的计算资源。具体而言，在1990-91年间，我们为生成式人工智能奠定了基础，发表了以下原理：（1）用于人工好奇心与创造力的生成对抗网络（现用于深度伪造），（2）Transformer（ChatGPT中的“T”——参见1991年的非标准化线性Transformer），（3）深度神经网络的预训练（参见ChatGPT中的“P”），（4）神经网络蒸馏（DeepSeek的关键技术），以及（5）用于部分可观测环境中强化学习与规划的循环世界模型。1991年也标志着（6）LSTM——20世纪被引用次数最多的人工智能论文（基于深度残差学习及通过残差神经网络连接的恒定误差流）——的决定性特征的出现，以及（7）21世纪被引用次数最多的论文，该论文基于我们受LSTM启发、深度比前馈神经网络深10倍的高速网络。截至2025年，有史以来被引用最频繁的两篇科学文章（排除手册类，指三年内谷歌学术引用次数最多者）均直接基于我们1991年的工作。