We consider dense, associative neural-networks trained with no supervision and we investigate their computational capabilities analytically, via a statistical-mechanics approach, and numerically, via Monte Carlo simulations. In particular, we obtain a phase diagram summarizing their performance as a function of the control parameters such as the quality and quantity of the training dataset and the network storage, valid in the limit of large network size and structureless datasets. Moreover, we establish a bridge between macroscopic observables standardly used in statistical mechanics and loss functions typically used in the machine learning. As technical remarks, from the analytic side, we implement large deviations and stability analysis within Guerra's interpolation to tackle the not-Gaussian distributions involved in the post-synaptic potentials while, from the computational counterpart, we insert Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of the synaptic tensors, overall obtaining a novel and broad approach to investigate neural networks in general.
翻译:我们考虑无监督训练的密集联想神经网络,并通过统计力学方法解析地以及通过蒙特卡罗模拟数值地研究其计算能力。特别地,我们得到了一个相图,总结了其性能作为控制参数(如训练数据集的质量和数量以及网络存储容量)的函数,该相图在大网络规模和结构无数据集的极限下有效。此外,我们在统计力学中常用的宏观可观测量与机器学习中常用的损失函数之间建立了桥梁。作为技术性备注,在解析方面,我们在Guerra插值中实现了大偏差和稳定性分析,以处理突触后电位中涉及的非高斯分布;而在计算方面,我们在蒙特卡罗方案中引入了Plefka近似,以加速突触张量的评估,从而总体上获得了一种新颖且广泛的方法来研究神经网络。