Semi-supervised learning (SSL) is an important theme in machine learning, in which we have a few labeled samples and many unlabeled samples. In this paper, for SSL in a regression problem, we consider a method of incorporating information on unlabeled samples into kernel functions. As a typical implementation, we employ Gaussian kernels whose centers are labeled and unlabeled input samples. Since the number of coefficients is larger than the number of labeled samples in this setting, this is an over-parameterized regression roblem. A ridge regression is a typical estimation method under this setting. In this paper, alternatively, we consider to apply the minimum norm least squares (MNLS), which is known as a helpful tool for understanding deep learning behavior while it may not be application oriented. Then, in applying the MNLS for SSL, we established several methods based on feature extraction/dimension reduction in the SVD (singular value decomposition) representation of a Gram type matrix appeared in the over-parameterized regression problem. The methods are thresholding according to singular value magnitude with cross validation, hard-thresholding with cross validation, universal thresholding and bridge thresholding methods. The first one is equivalent to a method using a well-known low rank approximation of a Gram type matrix. We refer to these methods as SVD regression methods. In the experiments for real data, depending on datasets, clear superiority of the proposed SVD regression methods over ridge regression methods was observed. And, depending on datasets, incorporation of information on unlabeled input samples into kernels was found to be clearly effective.

翻译：半监督学习（SSL）是机器学习中的一个重要主题，其中我们拥有少量标注样本和大量未标注样本。本文针对回归问题中的SSL，研究了一种将未标注样本信息融入核函数的方法。作为一种典型实现，我们采用了以标注和未标注输入样本为中心的高斯核。由于在此设定下系数数量多于标注样本数量，这构成了一个过参数化回归问题。岭回归是该设定下的典型估计方法。本文则考虑应用最小范数最小二乘法（MNLS），该方法虽非面向应用设计，但被公认为理解深度学习行为的有力工具。在将MNLS应用于SSL时，我们基于过参数化回归问题中出现的Gram型矩阵的奇异值分解（SVD）表示，建立了若干基于特征提取/降维的方法。这些方法包括：基于交叉验证的奇异值幅度阈值法、基于交叉验证的硬阈值法、通用阈值法以及桥式阈值法。第一种方法等价于使用Gram型矩阵的经典低秩近似方法。我们将这些方法统称为SVD回归方法。在真实数据实验中，根据不同数据集，观察到所提出的SVD回归方法相较于岭回归方法具有明显优势。同时，在不同数据集上，将未标注输入样本信息融入核函数被证明具有显著效果。