Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns [34], this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning.

翻译：从记忆中回忆起多年未见的朋友的面孔是一项困难的任务。然而，若你们偶然相遇，却很容易认出彼此。生物记忆配备了一种令人印象深刻的压缩算法，能够存储关键信息，随后推断细节以匹配感知。Willshaw记忆是一种用于皮层计算的简单抽象模型，实现了生物记忆的机制。利用我们近期提出的视觉模式稀疏编码方案[34]，该模型能够以容错方式存储并检索大量真实数据。本文通过使用多模态框架，扩展了基础联想记忆模型的能力。在此框架中，记忆同时存储每个模式的多种模态（如视觉或文本模态）。训练后，当仅感知到部分模态时，记忆可用于推断缺失的模态。采用简单的编码器-记忆-解码器架构，并结合我们新提出的Willshaw模型迭代检索算法，我们在MNIST数据集上进行了实验。通过将图像和标签均作为模态存储，单一记忆不仅可以用于检索和补全模式，还能进行分类和生成新数据。本文进一步讨论了该模型如何应用于其他学习任务，从而作为生物启发式学习框架发挥作用。