PAC-Bayes learning is a comprehensive setting for (i) studying the generalisation ability of learning algorithms and (ii) deriving new learning algorithms by optimising a generalisation bound. However, optimising generalisation bounds might not always be viable for tractable or computational reasons, or both. For example, iteratively querying the empirical risk might prove computationally expensive. In response, we introduce a novel principled strategy for building an iterative learning algorithm via the optimisation of a sequence of surrogate training objectives, inherited from PAC-Bayes generalisation bounds. The key argument is to replace the empirical risk (seen as a function of hypotheses) in the generalisation bound by its projection onto a constructible low dimensional functional space: these projections can be queried much more efficiently than the initial risk. On top of providing that generic recipe for learning via surrogate PAC-Bayes bounds, we (i) contribute theoretical results establishing that iteratively optimising our surrogates implies the optimisation of the original generalisation bounds, (ii) instantiate this strategy to the framework of meta-learning, introducing a meta-objective offering a closed form expression for meta-gradient, (iii) illustrate our approach with numerical experiments inspired by an industrial biochemical problem.

翻译：PAC-Bayes学习是一个综合性框架，可用于（i）研究学习算法的泛化能力，以及（ii）通过优化泛化界推导新的学习算法。然而，出于可处理性或计算效率的考虑，优化泛化界并非总是可行的。例如，迭代查询经验风险的计算成本可能过高。为此，我们提出了一种基于原则的新策略，通过优化一系列源自PAC-Bayes泛化界的代理训练目标来构建迭代学习算法。其核心思路是将泛化界中的经验风险（视为假设的函数）替换为其在可构建的低维函数空间上的投影：这些投影的查询效率远高于原始风险。除了提供这种通过代理PAC-Bayes界进行学习的通用方法外，我们（i）建立了理论结果，证明迭代优化代理目标意味着对原始泛化界的优化；（ii）将该策略实例化到元学习框架中，提出了一个具有闭式元梯度表达式的元目标；（iii）通过受工业生化问题启发的数值实验展示了本方法的有效性。