Semi-supervised learning by self-training heavily relies on pseudo-label selection (PLS). The selection often depends on the initial model fit on labeled data. Early overfitting might thus be propagated to the final model by selecting instances with overconfident but erroneous predictions, often referred to as confirmation bias. This paper introduces BPLS, a Bayesian framework for PLS that aims to mitigate this issue. At its core lies a criterion for selecting instances to label: an analytical approximation of the posterior predictive of pseudo-samples. We derive this selection criterion by proving Bayes optimality of the posterior predictive of pseudo-samples. We further overcome computational hurdles by approximating the criterion analytically. Its relation to the marginal likelihood allows us to come up with an approximation based on Laplace's method and the Gaussian integral. We empirically assess BPLS for parametric generalized linear and non-parametric generalized additive models on simulated and real-world data. When faced with high-dimensional data prone to overfitting, BPLS outperforms traditional PLS methods.

翻译：半监督自训练学习高度依赖伪标签选择（PLS）。该选择常取决于初始模型在标注数据上的拟合效果。早期过拟合可能通过选择具有过度自信但错误预测的样本（通常称为确认偏差）而传播至最终模型。本文提出BPLS——一种旨在缓解该问题的贝叶斯PLS框架。其核心是一个用于选择待标注样本的准则：伪样本后验预测分布的解析近似。我们通过证明伪样本后验预测分布的贝叶斯最优性推导出该选择准则，并进一步通过解析近似该准则克服计算障碍。该准则与边缘似然的关系使我们能够基于拉普拉斯方法和高斯积分提出近似方案。我们通过在模拟数据和真实数据上对参数化广义线性模型与非参数化广义可加模型进行实证评估BPLS。当面临易过拟合的高维数据时，BPLS优于传统PLS方法。