Personalization of machine learning (ML) predictions for individual users/domains/enterprises is critical for practical recommendation systems. Standard personalization approaches involve learning a user/domain specific embedding that is fed into a fixed global model which can be limiting. On the other hand, personalizing/fine-tuning model itself for each user/domain -- a.k.a meta-learning -- has high storage/infrastructure cost. Moreover, rigorous theoretical studies of scalable personalization approaches have been very limited. To address the above issues, we propose a novel meta-learning style approach that models network weights as a sum of low-rank and sparse components. This captures common information from multiple individuals/users together in the low-rank part while sparse part captures user-specific idiosyncrasies. We then study the framework in the linear setting, where the problem reduces to that of estimating the sum of a rank-$r$ and a $k$-column sparse matrix using a small number of linear measurements. We propose a computationally efficient alternating minimization method with iterative hard thresholding -- AMHT-LRS -- to learn the low-rank and sparse part. Theoretically, for the realizable Gaussian data setting, we show that AMHT-LRS solves the problem efficiently with nearly optimal sample complexity. Finally, a significant challenge in personalization is ensuring privacy of each user's sensitive data. We alleviate this problem by proposing a differentially private variant of our method that also is equipped with strong generalization guarantees.

翻译：机器学习（ML）预测对个体用户/领域/企业的个性化对于实用推荐系统至关重要。标准的个性化方法涉及学习用户/领域专属的嵌入向量，并将其输入固定的全局模型，这种方法可能具有局限性。另一方面，为每个用户/领域个性化/微调模型本身——即元学习——会带来高昂的存储与基础设施成本。此外，针对可扩展个性化方法的严格理论研究迄今非常有限。为解决上述问题，我们提出了一种新颖的元学习式方法，将网络权重建模为低秩部分与稀疏部分之和。其中低秩部分捕捉来自多个个体/用户的共同信息，而稀疏部分则捕捉用户特有的异质性。随后我们在线性设定下研究该框架，此时问题简化为利用少量线性测量值估计秩为$r$的矩阵与$k$列稀疏矩阵之和。我们提出了一种结合迭代硬阈值的高效交替最小化方法——AMHT-LRS——用于学习低秩部分与稀疏部分。理论上，在可实现的高斯数据设定下，我们证明了AMHT-LRS能以近乎最优的样本复杂度高效求解该问题。最后，个性化面临的一个重大挑战是确保每个用户敏感数据的隐私性。我们通过提出所提方法的差分隐私变体缓解了该问题，该变体同时具备强大的泛化保障。