Meta-learning can successfully acquire useful inductive biases from data. Yet, its generalization properties to unseen learning tasks are poorly understood. Particularly if the number of meta-training tasks is small, this raises concerns about overfitting. We provide a theoretical analysis using the PAC-Bayesian framework and derive novel generalization bounds for meta-learning. Using these bounds, we develop a class of PAC-optimal meta-learning algorithms with performance guarantees and a principled meta-level regularization. Unlike previous PAC-Bayesian meta-learners, our method results in a standard stochastic optimization problem which can be solved efficiently and scales well. When instantiating our PAC-optimal hyper-posterior (PACOH) with Gaussian processes and Bayesian Neural Networks as base learners, the resulting methods yield state-of-the-art performance, both in terms of predictive accuracy and the quality of uncertainty estimates. Thanks to their principled treatment of uncertainty, our meta-learners can also be successfully employed for sequential decision problems.

翻译：元化学习可以成功地从数据中获取有用的感化偏差。 然而,它的一般特性与隐性学习任务不易理解。 特别是如果元训练任务的数量很少, 则会引起对超称的担忧。 我们利用PAC- Bayesian框架提供理论分析,并为元化学习得出新的概括性界限。 利用这些界限, 我们开发了一类PAC- 最佳元学习算法, 具有性能保障和有原则的元水平规范。 与以往的PAC- Bayesian 元学习人不同, 我们的方法导致标准随机优化问题, 并且能够高效地解决, 比例相当。 当我们以高山进程和巴伊斯神经网络为基础学习者的PAC- 最优化性极性( PACOH) (PACOH) (PAC- Oper- Oper- Offical) (PAC- Offical) 时, 由此产生的方法在预测性精确性和不确定性估计质量方面产生最先进性能。 由于对不确定性进行有原则的处理, 我们的元学习者也可以成功地被运用于相近的处理。