Personalization of machine learning (ML) predictions for individual users/domains/enterprises is critical for practical recommendation style systems. Standard personalization approaches involve learning a user/domain specific embedding that is fed into a fixed global model which can be limiting. On the other hand, personalizing/fine-tuning model itself for each user/domain -- a.k.a meta-learning -- has high storage/infrastructure cost. We propose a novel meta-learning style approach that models network weights as a sum of low-rank and sparse matrices. This captures common information from multiple individuals/users together in the low-rank part while sparse part captures user-specific idiosyncrasies. Furthermore, the framework is up to two orders of magnitude more scalable (in terms of storage/infrastructure cost) than user-specific finetuning of model. We then study the framework in the linear setting, where the problem reduces to that of estimating the sum of a rank-$r$ and a $k$-column sparse matrix using a small number of linear measurements. We propose an alternating minimization method with iterative hard thresholding -- AMHT-LRS -- to learn the low-rank and sparse part. For the realizable, Gaussian data setting, we show that AMHT-LRS solves the problem efficiently with nearly optimal samples. A significant challenge in personalization is ensuring privacy of each user's sensitive data. We alleviate this problem by proposing a differentially private variant of our method that also is equipped with strong generalization guarantees. Finally, on multiple standard recommendation datasets, we demonstrate that our approach allows personalized models to obtain superior performance in sparse data regime.

翻译：机器学习预测针对个体用户/领域/企业的个性化，对于实际推荐系统至关重要。标准个性化方法通常学习特定用户/领域的嵌入向量，并将其输入固定的全局模型，这种做法存在局限性。另一方面，为每个用户/领域单独个性化/微调模型（即元学习）则面临较高的存储/基础设施成本。我们提出一种新颖的元学习方法，将网络权重建模为低秩矩阵与稀疏矩阵之和。其中低秩部分捕捉多个个体/用户的共同信息，而稀疏部分则捕捉用户特有的特质。此外，该框架在存储/基础设施成本方面比用户特定模型微调可扩展性高达两个数量级。我们在线性设定下研究该框架，此时问题简化为通过少量线性测量估计秩-$r$矩阵与$k$列稀疏矩阵之和。我们提出一种结合迭代硬阈值法的交替最小化方法——AMHT-LRS——来学习低秩与稀疏部分。在可实现的高斯数据设定下，我们证明AMHT-LRS能以近乎最优的样本量高效求解该问题。个性化面临的一个重大挑战是确保每个用户敏感数据的隐私性。我们通过提出一种差分隐私变体来缓解此问题，该变体同时还具备强泛化保证。最后，在多个标准推荐数据集上，我们证明该方法能使个性化模型在稀疏数据场景中获得优越性能。