Many datasets are underspecified: there exist multiple equally viable solutions to a given task. Underspecification can be problematic for methods that learn a single hypothesis because different functions that achieve low training loss can focus on different predictive features and thus produce widely varying predictions on out-of-distribution data. We propose DivDis, a simple two-stage framework that first learns a diverse collection of hypotheses for a task by leveraging unlabeled data from the test distribution. We then disambiguate by selecting one of the discovered hypotheses using minimal additional supervision, in the form of additional labels or inspection of function visualization. We demonstrate the ability of DivDis to find hypotheses that use robust features in image classification and natural language processing problems with underspecification.

翻译：许多数据集是欠定的：对于给定任务存在多个同等有效的解决方案。欠定性对于学习单一假设的方法可能构成问题，因为不同函数在实现低训练损失的同时，可能关注不同的预测特征，从而在分布外数据上产生差异极大的预测结果。我们提出DivDis，一个简单的两阶段框架：首先利用测试分布的未标注数据，为任务学习一组多样化的假设；随后通过最小额外监督（如额外标注或函数可视化检查）来消歧，从已发现的假设中选择一个。我们证明了DivDis能够在存在欠定性的图像分类和自然语言处理问题中，找到利用鲁棒特征的假设。