Recently, Chinese Spell Checking(CSC), a task to detect erroneous characters in a sentence and correct them, has attracted extensive interest because of its wide applications in various NLP tasks. Most of the existing methods have utilized BERT to extract semantic information for CSC task. However, these methods directly take sentences with only a few errors as inputs, where the correct characters may leak answers to the model and dampen its ability to capture distant context; while the erroneous characters may disturb the semantic encoding process and result in poor representations. Based on such observations, this paper proposes an n-gram masking layer that masks current and/or surrounding tokens to avoid label leakage and error disturbance. Moreover, considering that the mask strategy may ignore multi-modal information indicated by errors, a novel dot-product gating mechanism is proposed to integrate the phonological and morphological information with semantic representation. Extensive experiments on SIGHAN datasets have demonstrated that the pluggable n-gram masking mechanism can improve the performance of prevalent CSC models and the proposed methods in this paper outperform multiple powerful state-of-the-art models.

翻译：近期，中文拼写检查（CSC）作为一项检测并纠正句子中错别字的任务，因在各类自然语言处理任务中的广泛应用而备受关注。现有方法大多利用BERT提取语义信息以完成CSC任务，但这些方法直接将仅含少量错误的句子作为输入，其中正确的汉字可能向模型泄露答案，削弱其捕捉远距离语境的能力；而错误的汉字则会干扰语义编码过程，导致表征质量下降。基于上述观察，本文提出一种N元掩码层，通过掩码当前和/或周围标记来避免标签泄露与错误干扰。此外，考虑到掩码策略可能忽略错误所提示的多模态信息，本文进一步设计了一种新型点积门控机制，将语音与形态信息融入语义表征。在SIGHAN数据集上的大量实验证明，可插拔的N元掩码机制能显著提升主流CSC模型的性能，且本文提出的方法在多个指标上超越多种强大的最先进模型。