Neural networks have proven to be a highly effective tool for solving complex problems in many areas of life. Recently, their importance and practical usability have further been reinforced with the advent of deep learning. One of the important conditions for the success of neural networks is the choice of an appropriate activation function introducing non-linearity into the model. Many types of these functions have been proposed in the literature in the past, but there is no single comprehensive source containing their exhaustive overview. The absence of this overview, even in our experience, leads to redundancy and the unintentional rediscovery of already existing activation functions. To bridge this gap, our paper presents an extensive survey involving 400 activation functions, which is several times larger in scale than previous surveys. Our comprehensive compilation also references these surveys; however, its main goal is to provide the most comprehensive overview and systematization of previously published activation functions with links to their original sources. The secondary aim is to update the current understanding of this family of functions.

翻译：神经网络已被证明是解决众多领域复杂问题的强大工具。近年来，随着深度学习的兴起，其重要性与实际可用性进一步得到强化。神经网络成功的关键条件之一在于选择恰当的激活函数，以向模型引入非线性特性。文献中虽已提出多种此类函数，但至今缺乏一部完整涵盖其详尽概览的综合性资源。即便在我们看来，这种概述的缺失也导致了对已有激活函数的重复发现与无意识冗余。为填补这一空白，本文开展了大规模调研，涵盖400种激活函数，规模远超以往综述的数倍。本综合汇编亦引用了这些综述，但主要目标在于提供最全面的先前已发表激活函数概览与系统化梳理，并附原始文献链接。次要目标则是更新学界对该函数族的现有认知。