We present a methodology for using unlabeled data to design semi supervised learning (SSL) methods that improve the prediction performance of supervised learning for regression tasks. The main idea is to design different mechanisms for integrating the unlabeled data, and include in each of them a mixing parameter $\alpha$, controlling the weight given to the unlabeled data. Focusing on Generalized-Linear-Models (GLM), we analyze the characteristics of different mixing mechanisms, and prove that in all cases, it is inevitably beneficial to integrate the unlabeled data with some non-zero mixing ratio $\alpha>0$, in terms of predictive performance. Moreover, we provide a rigorous framework for estimating the best mixing ratio $\alpha^*$ where mixed-SSL delivers the best predictive performance, while using the labeled and the unlabeled data on hand. The effectiveness of our methodology in delivering substantial improvement compared to the standard supervised models, under a variety of settings, is demonstrated empirically through extensive simulation, in a manner that supports the theoretical analysis. We also demonstrate the applicability of our methodology (with some intuitive modifications) in improving more complex models such as deep neural networks, in a real-world regression tasks.

翻译：我们提出了一种利用无标签数据设计半监督学习（SSL）方法的方法论，该方法能够提升回归任务中有监督学习的预测性能。核心思路是设计不同的机制来整合无标签数据，并在每种机制中引入混合参数$\alpha$，用于控制无标签数据的权重。聚焦于广义线性模型（GLM），我们分析了不同混合机制的统计特性，并证明在所有情况下，以非零混合比率$\alpha>0$整合无标签数据在预测性能方面都是必然有益的。此外，我们提出了一个严谨的框架用于估计最优混合比率$\alpha^*$，在该比率下混合SSL能够使用手头的有标签和无标签数据实现最佳预测性能。通过大规模仿真实验，我们在多种设置下实证展示了我们的方法相较于标准有监督模型能够带来显著性能提升，这些实验结果支持了理论分析。同时，我们还展示了该方法（经过若干直观修改）在真实回归任务中提升深度神经网络等更复杂模型的可应用性。