Neural networks are the state-of-the-art approach for many tasks and the activation function is one of the main building blocks that allow such performance. Recently, a novel transformative adaptive activation function (TAAF) allowing for any vertical and horizontal translation and scaling was proposed. This work sets the TAAF into the context of other activation functions. It shows that the TAAFs generalize over 50 existing activation functions and utilize similar concepts as over 70 other activation functions, underscoring the versatility of TAAFs. This comprehensive exploration positions TAAFs as a promising and adaptable addition to neural networks.

翻译：神经网络是许多任务中最先进的方法，而激活函数是实现这种性能的主要构建模块之一。最近，一种新型可变换自适应激活函数（TAAF）被提出，允许进行任意垂直和水平平移与缩放。本研究将TAAF置于其他激活函数的背景下进行分析，表明TAAF泛化了50多种现有激活函数，并利用了类似的概念覆盖70多种其他激活函数，这突显了TAAF的多功能性。这项全面的探索将TAAF定位为神经网络中有前景且适应性强的补充。