Various neural network architectures rely on pooling operators to aggregate information coming from different sources. It is often implicitly assumed in such contexts that vectors encode epistemic states, i.e. that vectors capture the evidence that has been obtained about some properties of interest, and that pooling these vectors yields a vector that combines this evidence. We study, for a number of standard pooling operators, under what conditions they are compatible with this idea, which we call the epistemic pooling principle. While we find that all the considered pooling operators can satisfy the epistemic pooling principle, this only holds when embeddings are sufficiently high-dimensional and, for most pooling operators, when the embeddings satisfy particular constraints (e.g. having non-negative coordinates). We furthermore show that these constraints have important implications on how the embeddings can be used in practice. In particular, we find that when the epistemic pooling principle is satisfied, in most cases it is impossible to verify the satisfaction of propositional formulas using linear scoring functions, with two exceptions: (i) max-pooling with embeddings that are upper-bounded and (ii) Hadamard pooling with non-negative embeddings. This finding helps to clarify, among others, why Graph Neural Networks sometimes under-perform in reasoning tasks. Finally, we also study an extension of the epistemic pooling principle to weighted epistemic states, which are important in the context of non-monotonic reasoning, where max-pooling emerges as the most suitable operator.

翻译：各类神经网络架构依赖池化算子来聚合来自不同来源的信息。在此类背景下，通常隐含假设向量编码了认知状态，即向量捕捉了关于某些感兴趣属性所获得的证据，并且对这些向量进行池化可得到整合了这些证据的向量。我们针对若干标准池化算子，研究了它们在何种条件下与这一理念（我们称之为认知池化原则）兼容。我们发现，尽管所有被考虑的池化算子均能满足认知池化原则，但这仅在嵌入具有足够高维度时才成立，且对大多数池化算子而言，还需嵌入满足特定约束（例如坐标非负）。此外，我们进一步证明这些约束对嵌入在实际中的应用具有重要影响。具体而言，当认知池化原则成立时，通常在大多数情况下无法使用线性评分函数来验证命题公式的满足性，但存在两个例外：（i）使用上界约束嵌入的最大池化；（ii）使用非负嵌入的哈达玛池化。这一发现有助于澄清图神经网络在推理任务中有时表现欠佳的原因。最后，我们还研究了认知池化原则在加权认知状态下的扩展——这一扩展在非单调推理的背景下尤为重要，而最大池化在此场景中成为最合适的算子。