We propose Pointer-Augmented Neural Memory (PANM) to help neural networks understand and apply symbol processing to new, longer sequences of data. PANM integrates an external neural memory that uses novel physical addresses and pointer manipulation techniques to mimic human and computer symbol processing abilities. PANM facilitates pointer assignment, dereference, and arithmetic by explicitly using physical pointers to access memory content. Remarkably, it can learn to perform these operations through end-to-end training on sequence data, powering various sequential models. Our experiments demonstrate PANM's exceptional length extrapolating capabilities and improved performance in tasks that require symbol processing, such as algorithmic reasoning and Dyck language recognition. PANM helps Transformer achieve up to 100% generalization accuracy in compositional learning tasks and significantly better results in mathematical reasoning, question answering and machine translation tasks.

翻译：我们提出指针增强神经记忆（PANM），以帮助神经网络理解符号处理并将其应用于新的更长数据序列。PANM集成了一种外部神经记忆，通过使用新颖的物理地址和指针操作技术，模拟人类和计算机的符号处理能力。PANM通过显式使用物理指针访问记忆内容，支持指针分配、解引用和算术运算。值得注意的是，它可以在序列数据的端到端训练中学习执行这些操作，从而增强各种序列模型的能力。我们的实验展示了PANM在需要符号处理的任务（如算法推理和Dyck语言识别）中卓越的长度外推能力和改进性能。PANM帮助Transformer在组合学习任务中实现高达100%的泛化准确率，并在数学推理、问答和机器翻译任务中取得显著更优的结果。