We present a methodology for using unlabeled data to design semi supervised learning (SSL) methods that improve the prediction performance of supervised learning for regression tasks. The main idea is to design different mechanisms for integrating the unlabeled data, and include in each of them a mixing parameter $\alpha$, controlling the weight given to the unlabeled data. Focusing on Generalized Linear Models (GLM) and linear interpolators classes of models, we analyze the characteristics of different mixing mechanisms, and prove that in all cases, it is invariably beneficial to integrate the unlabeled data with some nonzero mixing ratio $\alpha>0$, in terms of predictive performance. Moreover, we provide a rigorous framework to estimate the best mixing ratio $\alpha^*$ where mixed SSL delivers the best predictive performance, while using the labeled and unlabeled data on hand. The effectiveness of our methodology in delivering substantial improvement compared to the standard supervised models, in a variety of settings, is demonstrated empirically through extensive simulation, in a manner that supports the theoretical analysis. We also demonstrate the applicability of our methodology (with some intuitive modifications) to improve more complex models, such as deep neural networks, in real-world regression tasks.

翻译：我们提出了一种利用无标签数据设计半监督学习(SSL)方法的技术框架，旨在提升回归任务中监督学习的预测性能。核心思路是构建融合无标签数据的不同机制，并在每种机制中引入混合参数$\alpha$以控制无标签数据的权重。聚焦于广义线性模型(GLM)与线性插值器模型类，我们分析了不同混合机制的特性，并证明在所有情形下，采用非零混合比$\alpha>0$整合无标签数据均能稳定提升预测性能。此外，我们建立了严谨的框架以估计最优混合比$\alpha^*$——在此比例下混合SSL能基于当前有标签与无标签数据实现最佳预测性能。通过大规模仿真实验，我们以支撑理论分析的方式实证展示了该方法在多种场景下相较于标准监督模型的显著性能提升。最后，我们证明该方法（经直观改进后）可应用于改善深度神经网络等复杂模型，并在真实回归任务中验证其实效性。