Humans sometimes show sudden improvements in task performance that have been linked to moments of insight. Such insight-related performance improvements appear special because they are preceded by an extended period of impasse, are unusually abrupt, and occur only in some, but not all, learners. Here, we ask whether insight-like behaviour also occurs in artificial neural networks trained with gradient descent algorithms. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that provided a hidden opportunity which allowed to solve the task more efficiently. We show that humans tend to discover this regularity through insight, rather than gradually. Notably, neural networks with regularised gate modulation closely mimicked behavioural characteristics of human insights, exhibiting delay of insight, suddenness and selective occurrence. Analyses of network learning dynamics revealed that insight-like behaviour crucially depended on noise added to gradient updates, and was preceded by ``silent knowledge'' that is initially suppressed by regularised (attentional) gating. This suggests that insights can arise naturally from gradual learning, where they reflect the combined influences of noise, attentional gating and regularisation.

翻译：人类有时会在任务表现中出现突如其来的提升，这些提升与洞察时刻相关联。这种与洞察相关的表现改善之所以显得特殊，是因为它们发生前通常经历一段较长的停滞期，提升过程异常突然，且仅出现在部分而非所有学习者中。本文探究了使用梯度下降算法训练的人工神经网络是否也会出现类似洞察的行为。我们在一项包含隐藏机会（该机会可让任务解决更高效）的感知决策任务中，比较了人类与正则化神经网络的学习动态。结果显示，人类倾向于通过洞察而非渐进方式发现这一规律。值得注意的是，采用正则化门控机制的神经网络高度模仿了人类洞察的行为特征，表现出洞察延迟、突发性和选择性出现。对网络学习动态的分析表明，类似洞察的行为关键依赖于梯度更新中添加的噪声，并且其前兆是起初被正则化（注意力）门控抑制的"隐性知识"。这表明洞察可能自然产生于渐进学习过程，反映噪声、注意力门控和正则化的综合影响。