Robustness in Natural Language Processing continues to be a pertinent issue, where state of the art models under-perform under naturally shifted distributions. In the context of Question Answering, work on domain adaptation methods continues to be a growing body of research. However, very little attention has been given to the notion of domain generalization under natural distribution shifts, where the target domain is unknown. With drastic improvements in the quality and access to generative models, we answer the question: How do generated datasets influence the performance of QA models under natural distribution shifts? We perform experiments on 4 different datasets under varying amounts of distribution shift, and analyze how "in-the-wild" generation can help achieve domain generalization. We take a two-step generation approach, generating both contexts and QA pairs to augment existing datasets. Through our experiments, we demonstrate how augmenting reading comprehension datasets with generated data leads to better robustness towards natural distribution shifts.

翻译：自然语言处理中的鲁棒性仍然是一个重要问题，当前最先进的模型在自然分布偏移下表现欠佳。在问答任务中，关于领域适应方法的研究持续增长，但针对未知目标领域下自然分布偏移的领域泛化概念关注甚少。随着生成模型质量与获取途径的显著提升，我们回答了以下问题：生成数据集如何影响问答模型在自然分布偏移下的性能？我们在4个不同数据集上开展了不同偏移程度的实验，分析了"野外"生成数据如何助力实现领域泛化。我们采用两步生成方法，同时生成上下文和问答对以扩充现有数据集。通过实验，我们证明了使用生成数据增强阅读理解数据集能够提升对自然分布偏移的鲁棒性。