Question answering methods are well-known for leveraging data bias, such as the language prior in visual question answering and the position bias in machine reading comprehension (extractive question answering). Current debiasing methods often come at the cost of significant in-distribution performance to achieve favorable out-of-distribution generalizability, while non-debiasing methods sacrifice a considerable amount of out-of-distribution performance in order to obtain high in-distribution performance. Therefore, it is challenging for them to deal with the complicated changing real-world situations. In this paper, we propose a simple yet effective novel loss function with adaptive loose optimization, which seeks to make the best of both worlds for question answering. Our main technical contribution is to reduce the loss adaptively according to the ratio between the previous and current optimization state on mini-batch training data. This loose optimization can be used to prevent non-debiasing methods from overlearning data bias while enabling debiasing methods to maintain slight bias learning. Experiments on the visual question answering datasets, including VQA v2, VQA-CP v1, VQA-CP v2, GQA-OOD, and the extractive question answering dataset SQuAD demonstrate that our approach enables QA methods to obtain state-of-the-art in- and out-of-distribution performance in most cases. The source code has been released publicly in \url{https://github.com/reml-group/ALO}.

翻译：问答方法以利用数据偏置而著称，例如视觉问答中的语言先验和机器阅读理解（抽取式问答）中的位置偏置。当前的去偏置方法通常以显著降低分布内性能为代价，以获得良好的分布外泛化能力；而非去偏置方法为了取得高分布内性能，则牺牲了大量分布外性能。因此，它们难以应对复杂多变的实际场景。本文提出一种简单而有效的新型损失函数，采用自适应宽松优化策略，旨在使问答方法兼顾两者的优势。我们的主要技术贡献在于：根据小批量训练数据上前后优化状态的比值，自适应地降低损失值。这种宽松优化既防止非去偏置方法过度学习数据偏置，又使去偏置方法得以保留轻微偏置学习能力。在视觉问答数据集（包括VQA v2、VQA-CP v1、VQA-CP v2、GQA-OOD）及抽取式问答数据集SQuAD上的实验表明，我们的方法在多数情况下使问答方法获得了分布内与分布外性能的最优水平。源代码已公开发布于 \url{https://github.com/reml-group/ALO}。