Conventional machine learning algorithms have traditionally been designed under the assumption that input data follows a vector-based format, with an emphasis on vector-centric paradigms. However, as the demand for tasks involving set-based inputs has grown, there has been a paradigm shift in the research community towards addressing these challenges. In recent years, the emergence of neural network architectures such as Deep Sets and Transformers has presented a significant advancement in the treatment of set-based data. These architectures are specifically engineered to naturally accommodate sets as input, enabling more effective representation and processing of set structures. Consequently, there has been a surge of research endeavors dedicated to exploring and harnessing the capabilities of these architectures for various tasks involving the approximation of set functions. This comprehensive survey aims to provide an overview of the diverse problem settings and ongoing research efforts pertaining to neural networks that approximate set functions. By delving into the intricacies of these approaches and elucidating the associated challenges, the survey aims to equip readers with a comprehensive understanding of the field. Through this comprehensive perspective, we hope that researchers can gain valuable insights into the potential applications, inherent limitations, and future directions of set-based neural networks. Indeed, from this survey we gain two insights: i) Deep Sets and its variants can be generalized by differences in the aggregation function, and ii) the behavior of Deep Sets is sensitive to the choice of the aggregation function. From these observations, we show that Deep Sets, one of the well-known permutation-invariant neural networks, can be generalized in the sense of a quasi-arithmetic mean.

翻译：传统机器学习算法传统上是在假设输入数据遵循向量格式的基础上设计的，强调以向量为中心的范式。然而，随着涉及集合输入的任务需求增加，研究社区出现了一种范式转变，以应对这些挑战。近年来，Deep Sets和Transformer等神经网络架构的出现，在处理集合数据方面取得了重大进展。这些架构被专门设计为自然接受集合作为输入，从而能够更有效地表示和处理集合结构。因此，越来越多的研究致力于探索和利用这些架构来解决各种涉及集合函数逼近的任务。本综合综述旨在概述与逼近集合函数的神经网络相关的多样化问题设置和当前研究努力。通过深入探讨这些方法的复杂性并阐明相关挑战，本综述旨在为读者提供对该领域的全面理解。通过这一全面的视角，我们希望研究者能够深入了解基于集合的神经网络的潜在应用、固有局限和未来方向。事实上，从本综述中我们得到两个见解：i) Deep Sets及其变体可以通过聚合函数的差异进行泛化，ii) Deep Sets的行为对聚合函数的选择敏感。基于这些观察，我们证明Deep Sets（一种著名的置换不变神经网络）可以在拟算术均值意义上进行泛化。