Linear autoencoder models learn an item-to-item weight matrix via convex optimization with L2 regularization and zero-diagonal constraints. Despite their simplicity, they have shown remarkable performance compared to sophisticated non-linear models. This paper aims to theoretically understand the properties of two terms in linear autoencoders. Through the lens of singular value decomposition (SVD) and principal component analysis (PCA), it is revealed that L2 regularization enhances the impact of high-ranked PCs. Meanwhile, zero-diagonal constraints reduce the impact of low-ranked PCs, leading to performance degradation for unpopular items. Inspired by this analysis, we propose simple-yet-effective linear autoencoder models using diagonal inequality constraints, called Relaxed Linear AutoEncoder (RLAE) and Relaxed Denoising Linear AutoEncoder (RDLAE). We prove that they generalize linear autoencoders by adjusting the degree of diagonal constraints. Experimental results demonstrate that our models are comparable or superior to state-of-the-art linear and non-linear models on six benchmark datasets; they significantly improve the accuracy of long-tail items. These results also support our theoretical insights on regularization and diagonal constraints in linear autoencoders.

翻译：线性自编码器模型通过带L2正则化和零对角线约束的凸优化学习物品间权重矩阵。尽管结构简单，但与复杂的非线性模型相比，它们展现出卓越性能。本文旨在从理论上理解线性自编码器中两项性质的机理。通过奇异值分解（SVD）和主成分分析（PCA）的视角，我们发现L2正则化增强了高排序主成分（PCs）的影响，而零对角线约束则削弱低排序PCs的作用，导致对非流行物品的性能下降。基于这一分析，我们提出使用对角线不等式约束的简单高效线性自编码器模型，即松弛线性自编码器（RLAE）和松弛去噪线性自编码器（RDLAE）。我们证明这些模型通过调整对角线约束程度，泛化了线性自编码器。实验结果表明，在六个基准数据集上，我们的模型与最先进的线性和非线性模型性能相当或更优，显著提升了长尾物品的准确率。这些结果也支持了我们对线性自编码器中正则化与对角线约束的理论见解。