Meta learning have achieved promising performance in low-resource text classification which aims to identify target classes with knowledge transferred from source classes with sets of small tasks named episodes. However, due to the limited training data in the meta-learning scenario and the inherent properties of parameterized neural networks, poor generalization performance has become a pressing problem that needs to be addressed. To deal with this issue, we propose a meta-learning based method called Retrieval-Augmented Meta Learning(RAML). It not only uses parameterization for inference but also retrieves non-parametric knowledge from an external corpus to make inferences, which greatly alleviates the problem of poor generalization performance caused by the lack of diverse training data in meta-learning. This method differs from previous models that solely rely on parameters, as it explicitly emphasizes the importance of non-parametric knowledge, aiming to strike a balance between parameterized neural networks and non-parametric knowledge. The model is required to determine which knowledge to access and utilize during inference. Additionally, our multi-view passages fusion network module can effectively and efficiently integrate the retrieved information into low-resource classification task. The extensive experiments demonstrate that RAML significantly outperforms current SOTA low-resource text classification models.
翻译:元学习在低资源文本分类中已取得令人瞩目的性能,该类任务旨在通过从源类别迁移知识来识别目标类别,并利用名为episodes的小任务集进行训练。然而,由于元学习场景中有限的训练数据以及参数化神经网络的固有不适应性,泛化性能差已成为亟待解决的问题。为此,我们提出一种基于元学习的方法——检索增强元学习(RAML)。该方法不仅使用参数化进行推理,还从外部语料库中检索非参数化知识以辅助推理,从而显著缓解了因元学习中训练数据多样性不足而导致泛化性能差的问题。该方法不同于以往仅依赖参数的模型,它明确强调了非参数化知识的重要性,旨在平衡参数化神经网络与非参数化知识。模型需在推理过程中决定访问并利用哪些知识。此外,我们的多视角段落融合网络模块能高效且有效地将检索到的信息整合到低资源分类任务中。大量实验表明,RAML显著优于当前最先进的低资源文本分类模型。