Inspired by the complexity and diversity of biological neurons, a quadratic neuron is proposed to replace the inner product in the current neuron with a simplified quadratic function. Employing such a novel type of neurons offers a new perspective on developing deep learning. When analyzing quadratic neurons, we find that there exists a function such that a heterogeneous network can approximate it well with a polynomial number of neurons but a purely conventional or quadratic network needs an exponential number of neurons to achieve the same level of error. Encouraged by this inspiring theoretical result on heterogeneous networks, we directly integrate conventional and quadratic neurons in an autoencoder to make a new type of heterogeneous autoencoders. To our best knowledge, it is the first heterogeneous autoencoder that is made of different types of neurons. Next, we apply the proposed heterogeneous autoencoder to unsupervised anomaly detection for tabular data and bearing fault signals. The anomaly detection faces difficulties such as data unknownness, anomaly feature heterogeneity, and feature unnoticeability, which is suitable for the proposed heterogeneous autoencoder. Its high feature representation ability can characterize a variety of anomaly data (heterogeneity), discriminate the anomaly from the normal (unnoticeability), and accurately learn the distribution of normal samples (unknownness). Experiments show that heterogeneous autoencoders perform competitively compared to other state-of-the-art models.

翻译：受生物神经元复杂性与多样性的启发，提出一种二次神经元，以简化二次函数替代当前神经元中的内积运算。采用这种新型神经元为发展深度学习提供了新视角。分析二次神经元时发现，存在某种函数使得异构网络可用多项式数量的神经元良好逼近，而纯传统或纯二次网络需指数级数量的神经元才能达到同等误差水平。受这一关于异构网络的启发性理论结果激励，我们直接在自编码器中集成传统神经元与二次神经元，构建新型异构自编码器。据我们所知，这是首个由不同类型神经元构成的异构自编码器。进而，将所提异构自编码器应用于表格数据与轴承故障信号的无监督异常检测。该异常检测面临数据未知性、异常特征异质性及特征不易察觉性等困难，所提异构自编码器恰好适用——其高特征表示能力可表征多种异常数据（异质性）、区分异常与正常数据（不易察觉性），并准确学习正常样本分布（未知性）。实验表明，异构自编码器性能与当前最优模型相比具有竞争力。