One advantage of neural ranking models is that they are meant to generalise well in situations of synonymity i.e. where two words have similar or identical meanings. In this paper, we investigate and quantify how well various ranking models perform in a clear-cut case of synonymity: when words are simply expressed in different surface forms due to regional differences in spelling conventions (e.g., color vs colour). We first explore the prevalence of American and British English spelling conventions in datasets used for the pre-training, training and evaluation of neural retrieval methods, and find that American spelling conventions are far more prevalent. Despite these biases in the training data, we find that retrieval models often generalise well in this case of synonymity. We explore the effect of document spelling normalisation in retrieval and observe that all models are affected by normalising the document's spelling. While they all experience a drop in performance when normalised to a different spelling convention than that of the query, we observe varied behaviour when the document is normalised to share the query spelling convention: lexical models show improvements, dense retrievers remain unaffected, and re-rankers exhibit contradictory behaviour.

翻译：神经排序模型的优势之一在于其能够在近义词情境（即两个词语具有相似或相同含义）下具有良好的泛化能力。本文针对拼写规范的区域差异所导致的词语表层形式不同（例如color与colour）这一明确的近义词案例，研究并量化了多种排序模型的性能表现。我们首先探讨了美式与英式英语拼写规范在神经检索方法的预训练、训练及评估所用数据集中的分布情况，发现美式拼写规范占据主导地位。尽管训练数据存在此类偏差，但检索模型在该近义词案例中通常表现出良好的泛化能力。我们进一步研究了文档拼写归一化对检索的影响，观察到所有模型均受到文档拼写归一化的作用。当文档拼写被归一化为与查询不同的规范时，所有模型性能均有所下降；而当文档拼写被归一化为与查询相同的规范时，不同模型表现出差异化行为：词汇模型性能提升，密集型检索器不受影响，而重排序模型则呈现矛盾性表现。