Humans sometimes show sudden improvements in task performance that have been linked to moments of insight. Such insight-related performance improvements appear special because they are preceded by an extended period of impasse, are unusually abrupt, and occur only in some, but not all, learners. Here, we ask whether insight-like behaviour also occurs in artificial neural networks trained with gradient descent algorithms. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that provided a hidden opportunity which allowed to solve the task more efficiently. We show that humans tend to discover this regularity through insight, rather than gradually. Notably, neural networks with regularised gate modulation closely mimicked behavioural characteristics of human insights, exhibiting delay of insight, suddenness and selective occurrence. Analyses of network learning dynamics revealed that insight-like behaviour crucially depended on noise added to gradient updates, and was preceded by ``silent knowledge'' that is initially suppressed by regularised (attentional) gating. This suggests that insights can arise naturally from gradual learning, where they reflect the combined influences of noise, attentional gating and regularisation.

翻译：人类有时会在任务表现中出现突然的改进，这些改进与洞察时刻相关。此类与洞察相关的表现提升之所以显得特殊，是因为其发生前通常经历一段长时间的僵局，改进过程异常突然，且仅发生在部分（而非所有）学习者身上。本文探究了使用梯度下降算法训练的人工神经网络中是否也会出现类似洞察的行为。我们在一项包含隐藏机会的感知决策任务中，比较了人类与正则化神经网络的学习动态——该隐藏机会允许更高效地解决任务。结果表明，人类倾向于通过洞察而非渐进过程发现这一规律。值得注意的是，采用正则化门控调制的神经网络在行为特征上高度模仿人类洞察，表现出延迟的洞察、突现性及选择性出现。对网络学习动态的分析显示，类似洞察的行为关键依赖于梯度更新中引入的噪声，且在洞察发生前存在被正则化（注意力）门控初始抑制的"隐性知识"。这表明洞察可能自然源于渐进学习过程，其本质是噪声、注意力门控与正则化共同作用的体现。