Mixed linear regression is a well-studied problem in parametric statistics and machine learning. Given a set of samples, tuples of covariates and labels, the task of mixed linear regression is to find a small list of linear relationships that best fit the samples. Usually it is assumed that the label is generated stochastically by randomly selecting one of two or more linear functions, applying this chosen function to the covariates, and potentially introducing noise to the result. In that situation, the objective is to estimate the ground-truth linear functions up to some parameter error. The popular expectation maximization (EM) and alternating minimization (AM) algorithms have been previously analyzed for this. In this paper, we consider the more general problem of agnostic learning of mixed linear regression from samples, without such generative models. In particular, we show that the AM and EM algorithms, under standard conditions of separability and good initialization, lead to agnostic learning in mixed linear regression by converging to the population loss minimizers, for suitably defined loss functions. In some sense, this shows the strength of AM and EM algorithms that converges to ``optimal solutions'' even in the absence of realizable generative models.

翻译：混合线性回归是参数统计学与机器学习中一个被深入研究的课题。给定一组样本（协变量与标签的元组），混合线性回归的任务是找到能最佳拟合这些样本的少量线性关系集合。通常假设标签的生成过程是：随机从两个或多个线性函数中选择一个，将选定的函数应用于协变量，并可能对结果引入噪声。在此情境下，目标是将真实线性函数估计至一定的参数误差范围内。此前已有研究对常用的期望最大化（EM）算法和交替最小化（AM）算法在此问题上的表现进行了分析。本文考虑更一般性的问题：在无此类生成模型的情况下，从样本中不可知学习混合线性回归。具体而言，我们证明，在标准的可分离性与良好初始化条件下，针对适当定义的损失函数，AM与EM算法能通过收敛到总体损失最小化器，实现混合线性回归的不可知学习。在某种意义上，这展现了AM与EM算法的强大之处——即使在没有可实现生成模型的情况下，它们仍能收敛到“最优解”。