Natural language understanding (NLU) is an essential branch of natural language processing, which relies on representations generated by pre-trained language models (PLMs). However, PLMs primarily focus on acquiring lexico-semantic information, while they may be unable to adequately handle the meaning of constructions. To address this issue, we introduce construction grammar (CxG), which highlights the pairings of form and meaning, to enrich language representation. We adopt usage-based construction grammar as the basis of our work, which is highly compatible with statistical models such as PLMs. Then a HyCxG framework is proposed to enhance language representation through a three-stage solution. First, all constructions are extracted from sentences via a slot-constraints approach. As constructions can overlap with each other, bringing redundancy and imbalance, we formulate the conditional max coverage problem for selecting the discriminative constructions. Finally, we propose a relational hypergraph attention network to acquire representation from constructional information by capturing high-order word interactions among constructions. Extensive experiments demonstrate the superiority of the proposed model on a variety of NLU tasks.

翻译：自然语言理解（NLU）是自然语言处理的重要分支，其核心依赖于预训练语言模型（PLMs）生成的表示。然而，PLMs主要侧重于获取词汇语义信息，可能无法充分处理构式的语义。为解决这一问题，我们引入构式语法（CxG）——一种强调形式与意义配对的理论——以丰富语言表示。采用基于用法的构式语法作为研究基础，该理论与PLMs等统计模型高度兼容。进而提出HyCxG框架，通过三阶段方法增强语言表示：首先，采用槽约束方法从句子中提取所有构式；由于构式可能相互重叠导致冗余与不平衡，我们构建条件最大覆盖问题以选择具有判别性的构式；最后，提出关系超图注意力网络，通过捕捉构式间的高阶词交互来获取构式信息表示。大量实验表明，所提模型在多种NLU任务中具有优越性。