Semi-supervised learning by self-training heavily relies on pseudo-label selection (PLS). The selection often depends on the initial model fit on labeled data. Early overfitting might thus be propagated to the final model by selecting instances with overconfident but erroneous predictions, often referred to as confirmation bias. This paper introduces BPLS, a Bayesian framework for PLS that aims to mitigate this issue. At its core lies a criterion for selecting instances to label: an analytical approximation of the posterior predictive of pseudo-samples. We derive this selection criterion by proving Bayes optimality of the posterior predictive of pseudo-samples. We further overcome computational hurdles by approximating the criterion analytically. Its relation to the marginal likelihood allows us to come up with an approximation based on Laplace's method and the Gaussian integral. We empirically assess BPLS for parametric generalized linear and non-parametric generalized additive models on simulated and real-world data. When faced with high-dimensional data prone to overfitting, BPLS outperforms traditional PLS methods.

翻译：通过自训练进行半监督学习严重依赖于伪标签选择（PLS）。该选择通常取决于基于标注数据的初始模型拟合结果。早期过拟合可能通过选择具有过度自信但错误预测的实例（通常被称为确认偏差）而传播到最终模型。本文提出BPLS，一个旨在缓解该问题的贝叶斯PLS框架。其核心是一个选择实例进行标注的准则：伪样本后验预测的解析近似。我们通过证明伪样本后验预测的贝叶斯最优性推导出该选择准则，并进一步通过解析近似该准则来克服计算障碍。该准则与边际似然的关系使我们能够基于拉普拉斯方法和高斯积分提出近似方案。我们在模拟数据和真实数据上对参数化广义线性模型与非参数化广义可加模型进行了BPLS的实证评估。当面对易于过拟合的高维数据时，BPLS优于传统PLS方法。