Transformer-based pre-trained models like BERT have achieved great progress on Semantic Sentence Matching. Meanwhile, dependency prior knowledge has also shown general benefits in multiple NLP tasks. However, how to efficiently integrate dependency prior structure into pre-trained models to better model complex semantic matching relations is still unsettled. In this paper, we propose the \textbf{D}ependency-Enhanced \textbf{A}daptive \textbf{F}usion \textbf{A}ttention (\textbf{DAFA}), which explicitly introduces dependency structure into pre-trained models and adaptively fuses it with semantic information. Specifically, \textbf{\emph{(i)}} DAFA first proposes a structure-sensitive paradigm to construct a dependency matrix for calibrating attention weights. It adopts an adaptive fusion module to integrate the obtained dependency information and the original semantic signals. Moreover, DAFA reconstructs the attention calculation flow and provides better interpretability. By applying it on BERT, our method achieves state-of-the-art or competitive performance on 10 public datasets, demonstrating the benefits of adaptively fusing dependency structure in semantic matching task.

翻译：基于Transformer的预训练模型（如BERT）在语义句子匹配任务上取得了显著进展。同时，依存先验知识在多个NLP任务中也展现出了普遍优势。然而，如何高效地将依存先验结构融入预训练模型以更好地建模复杂语义匹配关系仍是一个未解决的问题。本文提出**依存增强自适应融合注意力机制（DAFA）**，该机制将依存结构显式引入预训练模型，并将其与语义信息自适应融合。具体而言：**（i）** DAFA首先提出一种结构敏感范式来构建用于校准注意力权重的依存矩阵；**（ii）** 它采用自适应融合模块整合所获得的依存信息与原始语义信号。此外，DAFA重构了注意力计算流程，并提供了更好的可解释性。通过将其应用于BERT，我们的方法在10个公开数据集上达到了最先进或具有竞争力的性能，验证了在语义匹配任务中自适应融合依存结构的优势。