Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns, this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning.

翻译：时隔多年，从记忆中勾勒出朋友的面容是一项困难的任务。然而，若你们偶然相逢，却能轻易认出彼此。生物记忆配备了一种令人惊叹的压缩算法，能够存储核心信息，进而推断细节以匹配感知。威尔肖记忆是一种用于皮层计算的简单抽象模型，它实现了生物记忆的若干机制。通过我们近期提出的视觉模式稀疏编码方案，该模型能够以容错方式存储并检索大量真实世界数据。本文通过采用多模态框架，拓展了基本关联记忆模型的能力。在此设定中，记忆体同步存储每个模式的多种模态（例如视觉或文本形式）。训练完成后，记忆体可在仅感知部分模态时推断缺失的模态。借助简单的编码器-记忆-解码器架构，以及新提出的威尔肖模型迭代检索算法，我们在MNIST数据集上开展了实验。通过将图像与标签作为模态共同存储，单一记忆体不仅能用于模式检索与补全，还可实现分类与生成新样本。我们进一步讨论了该模型如何应用于其他学习任务，从而作为一种受生物学启发的学习框架。