In this paper, we exploit the unique properties of a deterministic projected belief network (D-PBN) to take full advantage of trainable compound activation functions (TCAs). A D-PBN is a type of auto-encoder that operates by "backing up" through a feed-forward neural network. TCAs are activation functions with complex monotonic-increasing shapes that change the distribution of the data so that the linear transformation that follows is more effective. Because a D-PBN operates by "backing up", the TCAs are inverted in the reconstruction process, restoring the original distribution of the data, thus taking advantage of a given TCA in both analysis and reconstruction. In this paper, we show that a D-PBN auto-encoder with TCAs can significantly out-perform standard auto-encoders including variational auto-encoders.

翻译：本文利用确定性投影信念网络（D-PBN）的独特性质，充分发挥可训练复合激活函数（TCAs）的优势。D-PBN是一种通过“回溯”前馈神经网络运行的自编码器。TCAs是具有复杂单调递增形状的激活函数，能改变数据分布，从而提升后续线性变换的有效性。由于D-PBN采用“回溯”机制，TCAs在重建过程中被反转，恢复数据的原始分布，从而在分析与重建两个阶段均能有效利用特定TCA。本文证明，引入TCAs的D-PBN自编码器在性能上显著优于标准自编码器（包括变分自编码器）。