Transformer-based pre-trained language models such as BERT have achieved remarkable results in Semantic Sentence Matching. However, existing models still suffer from insufficient ability to capture subtle differences. Minor noise like word addition, deletion, and modification of sentences may cause flipped predictions. To alleviate this problem, we propose a novel Dual Attention Enhanced BERT (DABERT) to enhance the ability of BERT to capture fine-grained differences in sentence pairs. DABERT comprises (1) Dual Attention module, which measures soft word matches by introducing a new dual channel alignment mechanism to model affinity and difference attention. (2) Adaptive Fusion module, this module uses attention to learn the aggregation of difference and affinity features, and generates a vector describing the matching details of sentence pairs. We conduct extensive experiments on well-studied semantic matching and robustness test datasets, and the experimental results show the effectiveness of our proposed method.

翻译：基于Transformer的预训练语言模型（如BERT）在语义句子匹配任务中取得了显著成果。然而，现有模型在捕捉细微差异方面的能力仍显不足。句子中诸如词语添加、删除或修改等微小噪声可能导致预测结果反转。为缓解此问题，我们提出了一种新颖的双注意力增强BERT（DABERT），以提升BERT捕捉句子对中细粒度差异的能力。DABERT包含：（1）双注意力模块，通过引入新型双通道对齐机制来建模亲和度与差异度注意力，从而衡量软词语匹配；（2）自适应融合模块，该模块利用注意力机制学习差异度与亲和度特征的聚合，并生成描述句子对匹配细节的向量。我们在广泛研究的语义匹配与鲁棒性测试数据集上进行了大量实验，实验结果证明了所提方法的有效性。