Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using $N$ neurons? The answer depends on the choice of weights and update rule. Inspired by setwise connectivity in biology, we extend Hopfield networks by adding setwise connections and embedding these connections in a simplicial complex. Simplicial complexes are higher dimensional analogues of graphs which naturally represent collections of pairwise and setwise relationships. We show that our simplicial Hopfield networks increase memory storage capacity. Surprisingly, even when connections are limited to a small random subset of equivalent size to an all-pairwise network, our networks still outperform their pairwise counterparts. Such scenarios include non-trivial simplicial topology. We also test analogous modern continuous Hopfield networks, offering a potentially promising avenue for improving the attention mechanism in Transformer models.

翻译：Hopfield网络是一种人工神经网络，通过选择合适的递归连接权重和更新规则，使网络的能量景观围绕记忆形成吸引子，从而将记忆模式存储在网络神经元的状态中。利用$N$个神经元，我们能在这样的网络中存储多少个稳定且具有足够吸引力的记忆模式？答案取决于权重和更新规则的选择。受生物学中集合连接方式的启发，我们通过添加集合连接并将这些连接嵌入单纯形复形来扩展Hopfield网络。单纯形复形是图的高维推广，能够自然地表示成对和集合关系集合。我们证明，单纯形Hopfield网络提高了记忆存储容量。令人惊讶的是，即使将连接限制在与全成对网络规模相当的随机子集中，我们的网络仍优于对应的成对网络。此类场景包含非平凡的单纯形拓扑结构。我们还测试了类似的现代连续Hopfield网络，这为改进Transformer模型中的注意力机制提供了潜在有前景的方向。