We present a unified representation of the most popular neural network activation functions. Adopting Mittag-Leffler functions of fractional calculus, we propose a flexible and compact functional form that is able to interpolate between various activation functions and mitigate common problems in training neural networks such as vanishing and exploding gradients. The presented gated representation extends the scope of fixed-shape activation functions to their adaptive counterparts whose shape can be learnt from the training data. The derivatives of the proposed functional form can also be expressed in terms of Mittag-Leffler functions making it a suitable candidate for gradient-based backpropagation algorithms. By training LeNet-5 neural network on MNIST and CIFAR-10 datasets, we demonstrate that adopting a unified gated representation of activation functions offers a promising and affordable alternative to individual built-in implementations of activation functions in conventional machine learning frameworks.

翻译：采用微微微微微微微微微微微微微微微微的Mittag-Leffler功能,我们建议一种灵活而紧凑的功能形式,能够在各种激活功能之间进行插插,并减轻培训神经网络中常见的问题,如渐变和爆炸梯度等。我们介绍的门面代表将固定形状激活功能的范围扩大到从培训数据中可以学习成型的适应性对应方。拟议的功能形式的衍生物也可以用Mittag-Leffler功能来表示,使其成为基于梯度的反再适应算法的合适候选方。我们通过对MNIST和CIFAR-10数据集培训LeNet-5神经网络,我们证明采用统一的封闭式激活功能代表提供了一种有希望和负担得起的替代在常规机器学习框架内内在实施激活功能的备选办法。