Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic learning metamaterials offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here we introduce a nonlinear learning metamaterial -- an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR and nonlinear regression, without a computer. We find our nonlinear learning metamaterial reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.
翻译:标准深度学习算法需要对大型非线性网络进行微分,这一过程耗时且能耗极高。电子学习超材料为模拟机器学习提供了潜在的高速、高效且容错的硬件方案,但现有实现均为线性结构,严重限制了其能力。这些系统与人工神经网络及大脑存在显著差异,因此引入非线性元件的可行性与实用性尚未得到探索。本文提出了一种非线性学习超材料——基于晶体管的自调节非线性电阻元件构成的模拟电子网络。我们证明,该系统无需计算机即可完成线性系统无法实现的学习任务,包括异或逻辑与非线性的回归分析。研究发现,非线性学习超材料按顺序(均值、斜率、曲率)降低训练误差模式,这与人工神经网络中的谱偏差相似。该电路具有抗损伤能力,可在数秒内重新训练,并在微秒级时间内完成已学习任务,同时每个晶体管仅耗散皮焦级能量。这表明其在边缘系统(如传感器、机器人控制器及医疗设备)中实现快速、低功耗计算具有巨大潜力,且具备规模化制造能力,可用于执行和研究涌现学习。