We propose a new high-performance activation function, Moderate Adaptive Linear Units (MoLU), for the deep neural network. The MoLU is a simple, beautiful and powerful activation function that can be a good main activation function among hundreds of activation functions. Because the MoLU is made up of the elementary functions, not only it is a infinite diffeomorphism (i.e. smooth and infinitely differentiable over whole domains), but also it decreases training time.

翻译：我们提出了一种新的高性能激活函数——中度自适应线性单元（MoLU），适用于深度神经网络。MoLU是一种简洁、优美且强大的激活函数，能够在数百种激活函数中成为优秀的主流激活函数。由于MoLU由初等函数构成，它不仅是无限微分同胚（即在整个定义域上光滑且无限可微），还能减少训练时间。