The estimation of the generalization error of classifiers often relies on a validation set. Such a set is hardly available in few-shot learning scenarios, a highly disregarded shortcoming in the field. In these scenarios, it is common to rely on features extracted from pre-trained neural networks combined with distance-based classifiers such as nearest class mean. In this work, we introduce a Gaussian model of the feature distribution. By estimating the parameters of this model, we are able to predict the generalization error on new classification tasks with few samples. We observe that accurate distance estimates between class-conditional densities are the key to accurate estimates of the generalization performance. Therefore, we propose an unbiased estimator for these distances and integrate it in our numerical analysis. We empirically show that our approach outperforms alternatives such as the leave-one-out cross-validation strategy.

翻译：许多分类器的泛化误差估计依赖于一个验证集，而在少样本学习场景中，验证集很难找到。这是该领域极具争议的缺陷之一。在这些场景中，常常依赖预训练神经网络提取的特征，结合最近类均值这样的基于距离的分类器。在本文中，我们介绍了一个特征分布的高斯模型。通过估计这个模型的参数，我们能够预测新分类任务的泛化误差，而样本数很少。我们发现，准确的类条件密度距离估计是精确估算泛化性能的关键。因此，我们提出了一种无偏估计这些距离的方法，并将其集成到我们的数值分析中。我们经验证明，我们的方法优于leave-one-out交叉验证策略等替代方法。