Conventional machine learning algorithms have traditionally been designed under the assumption that input data follows a vector-based format, with an emphasis on vector-centric paradigms. However, as the demand for tasks involving set-based inputs has grown, there has been a paradigm shift in the research community towards addressing these challenges. In recent years, the emergence of neural network architectures such as Deep Sets and Transformers has presented a significant advancement in the treatment of set-based data. These architectures are specifically engineered to naturally accommodate sets as input, enabling more effective representation and processing of set structures. Consequently, there has been a surge of research endeavors dedicated to exploring and harnessing the capabilities of these architectures for various tasks involving the approximation of set functions. This comprehensive survey aims to provide an overview of the diverse problem settings and ongoing research efforts pertaining to neural networks that approximate set functions. By delving into the intricacies of these approaches and elucidating the associated challenges, the survey aims to equip readers with a comprehensive understanding of the field. Through this comprehensive perspective, we hope that researchers can gain valuable insights into the potential applications, inherent limitations, and future directions of set-based neural networks. Indeed, from this survey we gain two insights: i) Deep Sets and its variants can be generalized by differences in the aggregation function, and ii) the behavior of Deep Sets is sensitive to the choice of the aggregation function. From these observations, we show that Deep Sets, one of the well-known permutation-invariant neural networks, can be generalized in the sense of a quasi-arithmetic mean.

翻译：传统机器学习算法传统上是在假设输入数据遵循向量格式的基础上设计的，强调以向量为中心的范式。然而，随着涉及集合输入的任务需求增长，研究界已出现向解决这些挑战的范式转变。近年来，Deep Sets和Transformer等神经网络架构的出现标志着在处理集合数据方面取得了重大进展。这些架构专门设计用于自然适应集合输入，从而能够更有效地表示和处理集合结构。因此，大量研究工作致力于探索和利用这些架构的能力，以处理涉及集合函数近似的各种任务。本综合综述旨在概述与近似集合函数的神经网络相关的多样化问题设置和持续研究努力。通过深入探讨这些方法的复杂性并阐明相关挑战，本综述旨在为读者提供对该领域的全面理解。通过这一全面视角，我们希望研究者能够获得关于基于集合的神经网络的潜在应用、固有局限和未来方向的宝贵见解。事实上，从本综述中我们得出两个见解：i) Deep Sets及其变体可通过聚合函数的差异进行泛化；ii) Deep Sets的行为对聚合函数的选择敏感。基于这些观察，我们表明Deep Sets（一种著名的置换不变神经网络）可以在拟算术均值意义上进行泛化。