Test-time domain adaptation (TTDA) is an excellent method which helps generalize models across domains, tasks, and distributions without the use of labeled datasets. Thus, TTDA is very useful in natural language processing (NLP) in the dialectal setting, since oftentimes, models are trained on Standard American English (SAE), evaluated on Indian English (IndE), Singaporean English (SingE), or Nigerian English (NgE), of which distribution differs significantly from the former. This is especially useful since dialectal datasets are scarce. In this paper, we explore one of the most famous TTDA techniques, SHOT, in dialectal NLP. We finetune and evaluate SHOT on different combinations of dialectal GLUE. Our findings show that SHOT is a viable technique when labeled datasets are unavailable. We also theoretically propose the concept of dialectal gap and show that it has a positive correlation with the effectiveness of SHOT. We also find that in many cases, finetuning on SAE yields higher performance than finetuning on dialectal data.

翻译：测试时域适应（TTDA）是一种优秀的方法，它有助于模型在无需使用标注数据集的情况下，跨领域、跨任务、跨分布进行泛化。因此，TTDA在方言场景下的自然语言处理（NLP）中非常有用，因为模型通常基于标准美国英语（SAE）进行训练，而在印度英语（IndE）、新加坡英语（SingE）或尼日利亚英语（NgE）上进行评估，这些方言的分布与SAE存在显著差异。由于方言数据集稀缺，这一方法尤其具有实用价值。本文探讨了方言NLP中最著名的TTDA技术之一——SHOT。我们在不同方言组合的GLUE数据集上对SHOT进行了微调和评估。研究结果表明，当标注数据集不可用时，SHOT是一种可行的技术。我们还从理论上提出了方言间隙的概念，并证明其与SHOT的有效性呈正相关。此外，我们发现许多情况下，在SAE上进行微调比在方言数据上进行微调能获得更高的性能。