In this paper we investigate the equilibrium properties of bidirectional associative memories (BAMs). Introduced by Kosko in 1988 as a generalization of the Hopfield model to a bipartite structure, the simplest architecture is defined by two layers of neurons, with synaptic connections only between units of different layers: even without internal connections within each layer, information storage and retrieval are still possible through the reverberation of neural activities passing from one layer to another. We characterize the computational capabilities of a stochastic extension of this model in the thermodynamic limit, by applying rigorous techniques from statistical physics. A detailed picture of the phase diagram at the replica symmetric level is provided, both at finite temperature and in the noiseless regimes. Also for the latter, the critical load is further investigated up to one step of replica symmetry breaking. An analytical and numerical inspection of the transition curves (namely critical lines splitting the various modes of operation of the machine) is carried out as the control parameters - noise, load and asymmetry between the two layer sizes - are tuned. In particular, with a finite asymmetry between the two layers, it is shown how the BAM can store information more efficiently than the Hopfield model by requiring less parameters to encode a fixed number of patterns. Comparisons are made with numerical simulations of neural dynamics. Finally, a low-load analysis is carried out to explain the retrieval mechanism in the BAM by analogy with two interacting Hopfield models. A potential equivalence with two coupled Restricted Boltmzann Machines is also discussed.
翻译:本文研究了双向联想记忆(BAM)的平衡性质。BAM由Kosko于1988年作为Hopfield模型向二分结构的推广而提出,其最简单的架构由两个神经元层构成,仅在不同层的单元之间存在突触连接:即便各层内部没有连接,通过神经活动在层间反复传递,仍能实现信息的存储与检索。我们运用统计物理的严格方法,在热力学极限下刻画了该模型随机扩展版本的计算能力。在复本对称水平上,详细给出了有限温度和无噪声条件下的相图。针对无噪声情形,进一步探究了单步复本对称破缺下的临界负载。通过解析与数值分析,在控制参数(噪声、负载及两层大小间的非对称性)变化时,研究了过渡曲线(即分割机器不同工作模式的临界线)。特别地,当两层间存在有限非对称性时,证明了BAM相比Hopfield模型能以更少的参数编码固定数量的模式,从而更高效地存储信息。并与神经动力学的数值模拟进行了比较。最后,通过类比两个相互作用的Hopfield模型,进行了低负载分析以解释BAM的检索机制,同时讨论了其与两个耦合受限Boltzmann机之间潜在的等价性。