We investigate the task of retrieving information from compositional distributed representations formed by Hyperdimensional Computing/Vector Symbolic Architectures and present novel techniques which achieve new information rate bounds. First, we provide an overview of the decoding techniques that can be used to approach the retrieval task. The techniques are categorized into four groups. We then evaluate the considered techniques in several settings that involve, e.g., inclusion of external noise and storage elements with reduced precision. In particular, we find that the decoding techniques from the sparse coding and compressed sensing literature (rarely used for Hyperdimensional Computing/Vector Symbolic Architectures) are also well-suited for decoding information from the compositional distributed representations. Combining these decoding techniques with interference cancellation ideas from communications improves previously reported bounds (Hersche et al., 2021) of the information rate of the distributed representations from 1.20 to 1.40 bits per dimension for smaller codebooks and from 0.60 to 1.26 bits per dimension for larger codebooks.

翻译：我们研究了从超维计算/向量符号架构形成的组合分布式表示中检索信息的任务，并提出了达到新信息率界限的创新技术。首先，我们概述了可用于处理检索任务的解码技术，这些技术被分为四类。随后，我们在若干设置中评估了所考虑的技术，这些设置涉及外部噪声的引入和存储单元精度降低等情况。特别地，我们发现来自稀疏编码和压缩感知文献（在超维计算/向量符号架构中很少使用）的解码技术也同样适用于从组合分布式表示中解码信息。将这些解码技术应用于通信中的干扰消除思想，进一步改进了先前报告（Hersche 等人，2021）的分布式表示信息率界限：对于较小码本，从每维度1.20比特提升至1.40比特；对于较大码本，从每维度0.60比特提升至1.26比特。