Most previous studies integrate cognitive language processing signals (e.g., eye-tracking or EEG data) into neural models of natural language processing (NLP) just by directly concatenating word embeddings with cognitive features, ignoring the gap between the two modalities (i.e., textual vs. cognitive) and noise in cognitive features. In this paper, we propose a CogAlign approach to these issues, which learns to align textual neural representations to cognitive features. In CogAlign, we use a shared encoder equipped with a modality discriminator to alternatively encode textual and cognitive inputs to capture their differences and commonalities. Additionally, a text-aware attention mechanism is proposed to detect task-related information and to avoid using noise in cognitive features. Experimental results on three NLP tasks, namely named entity recognition, sentiment analysis and relation extraction, show that CogAlign achieves significant improvements with multiple cognitive features over state-of-the-art models on public datasets. Moreover, our model is able to transfer cognitive information to other datasets that do not have any cognitive processing signals.

翻译：现有的大多数研究通过直接将词嵌入与认知特征拼接的方式，将认知语言处理信号（如眼动追踪或脑电图数据）整合至自然语言处理的神经模型中，却忽略了文本与认知两种模态间的差异以及认知特征中的噪声。本文针对上述问题提出CogAlign方法，通过学习将文本神经表示对齐至认知特征。在CogAlign中，我们采用配备模态判别器的共享编码器，交替编码文本与认知输入，以捕捉两者的差异性与共性。此外，我们提出一种文本感知注意力机制，用于检测任务相关信息并避免使用认知特征中的噪声。在命名实体识别、情感分析和关系抽取三项自然语言处理任务上的实验结果表明，与现有公开数据集上的最优模型相比，CogAlign在结合多种认知特征后取得了显著提升。同时，我们的模型能够将认知信息迁移至其他不具备任何认知处理信号的数据集中。