The effectiveness of neural processes (NPs) in modelling posterior prediction maps -- the mapping from data to posterior predictive distributions -- has significantly improved since their inception. This improvement can be attributed to two principal factors: (1) advancements in the architecture of permutation invariant set functions, which are intrinsic to all NPs; and (2) leveraging symmetries present in the true posterior predictive map, which are problem dependent. Transformers are a notable development in permutation invariant set functions, and their utility within NPs has been demonstrated through the family of models we refer to as TNPs. Despite significant interest in TNPs, little attention has been given to incorporating symmetries. Notably, the posterior prediction maps for data that are stationary -- a common assumption in spatio-temporal modelling -- exhibit translation equivariance. In this paper, we introduce of a new family of translation equivariant TNPs that incorporate translation equivariance. Through an extensive range of experiments on synthetic and real-world spatio-temporal data, we demonstrate the effectiveness of TE-TNPs relative to their non-translation-equivariant counterparts and other NP baselines.

翻译：自神经过程（NPs）问世以来，其在建模后验预测映射（即从数据到后验预测分布的映射）方面的有效性已显著提升。这一改进主要归因于两个关键因素：（1）所有NPs固有的置换不变集合函数架构的进步；（2）利用真实后验预测映射中存在的对称性，这些对称性取决于具体问题。Transformer是置换不变集合函数领域的一项重要进展，其在NPs中的实用性已通过我们称为TNPs的模型系列得到验证。尽管TNPs引起了广泛关注，但对其对称性整合的研究却鲜有涉及。值得注意的是，对于平稳数据（时空建模中的常见假设），其后验预测映射表现出平移等变性。本文提出了一种新的平移等变TNP系列模型，该模型整合了平移等变性。通过对合成与真实世界时空数据进行广泛实验，我们证明了TE-TNPs相较于非平移等变对应模型及其他NP基线的优越性能。