Sparse auto-encoders are useful for extracting low-dimensional representations from high-dimensional data. However, their performance degrades sharply when the input noise at test time differs from the noise employed during training. This limitation hinders the applicability of auto-encoders in real-world scenarios where the level of noise in the input is unpredictable. In this paper, we formalize single hidden layer sparse auto-encoders as a transform learning problem. Leveraging the transform modeling interpretation, we propose an optimization problem that leads to a predictive model invariant to the noise level at test time. In other words, the same pre-trained model is able to generalize to different noise levels. The proposed optimization algorithm, derived from the square root lasso, is translated into a new, computationally efficient auto-encoding architecture. After proving that our new method is invariant to the noise level, we evaluate our approach by training networks using the proposed architecture for denoising tasks. Our experimental results demonstrate that the trained models yield a significant improvement in stability against varying types of noise compared to commonly used architectures.

翻译：稀疏自编码器在从高维数据中提取低维表示方面具有重要作用。然而，当测试阶段的输入噪声与训练阶段采用的噪声不同时，其性能会急剧下降。这一局限性阻碍了自编码器在输入噪声水平不可预测的现实场景中的应用。本文通过将单隐藏层稀疏自编码器形式化为变换学习问题，基于变换建模的阐释提出了一种优化问题，该问题导出的预测模型对测试阶段的噪声水平具有不变性。换言之，同一预训练模型能够泛化至不同的噪声水平。所提出的优化算法源自平方根套索方法，被转化为一种新型计算高效的自编码架构。在证明新方法对噪声水平具有不变性后，我们通过使用该架构训练网络进行去噪任务来评估所提方法。实验结果表明，与常用架构相比，经训练的模型在面对不同类型噪声时展现出显著提升的稳定性。