Activation Functions introduce non-linearity in the deep neural networks. This nonlinearity helps the neural networks learn faster and efficiently from the dataset. In deep learning, many activation functions are developed and used based on the type of problem statement. ReLU's variants, SWISH, and MISH are goto activation functions. MISH function is considered having similar or even better performance than SWISH, and much better than ReLU. In this paper, we propose an activation function named APTx which behaves similar to MISH, but requires lesser mathematical operations to compute. The lesser computational requirements of APTx does speed up the model training, and thus also reduces the hardware requirement for the deep learning model. Source code: https://github.com/mr-ravin/aptx_activation

翻译：激活函数为深度神经网络引入了非线性特性。这种非线性有助于神经网络从数据集中更快速、高效地学习。在深度学习中，根据问题陈述的类型，人们开发并使用了多种激活函数。ReLU的变体、SWISH和MISH是常用的激活函数。MISH函数被认为具有与SWISH相似甚至更优的性能，且远优于ReLU。本文提出了一种名为APTx的激活函数，其行为与MISH相似，但计算所需的数学运算更少。APTx较低的计算需求确实加速了模型训练，从而也降低了对深度学习模型的硬件要求。源代码：https://github.com/mr-ravin/aptx_activation