Symbolic sentence meaning representations, such as AMR (Abstract Meaning Representation) provide expressive and structured semantic graphs that act as intermediates that simplify downstream NLP tasks. However, the instruction-following capability of large language models (LLMs) offers a shortcut to effectively solve NLP tasks, questioning the utility of semantic graphs. Meanwhile, recent work has also shown the difficulty of using meaning representations merely as a helpful auxiliary for LLMs. We revisit the position of semantic graphs in syntactic simplification, the task of simplifying sentence structures while preserving their meaning, which requires semantic understanding, and evaluate it on a new complex and natural dataset. The AMR-based method that we propose, AMRS$^3$, demonstrates that state-of-the-art meaning representations can lead to easy-to-implement simplification methods with competitive performance and unique advantages in cost, interpretability, and generalization. With AMRS$^3$ as an anchor, we discover that syntactic simplification is a task where semantic graphs are helpful in LLM prompting. We propose AMRCoC prompting that guides LLMs to emulate graph algorithms for explicit symbolic reasoning on AMR graphs, and show its potential for improving LLM on semantic-centered tasks like syntactic simplification.
翻译:符号化的句子意义表示,如抽象意义表示(AMR),提供了富有表现力的结构化语义图,作为简化下游自然语言处理任务的中间表示。然而,大语言模型(LLM)的指令遵循能力为有效解决自然语言处理任务提供了捷径,这引发了关于语义图实用性的疑问。与此同时,近期研究也表明,仅将意义表示作为大语言模型的辅助工具存在困难。我们重新审视了语义图在句法简化(即在保持句子意义的同时简化其结构的任务,该任务需要语义理解)中的地位,并在一个新的复杂自然数据集上进行了评估。我们提出的基于AMR的方法——AMRS$^3$——表明,最先进的意义表示能够催生易于实现的简化方法,这些方法在性能上具有竞争力,并在成本、可解释性和泛化性方面具有独特优势。以AMRS$^3$为锚点,我们发现句法简化是一项语义图在大语言模型提示中有帮助的任务。我们提出了AMRCoC提示方法,该方法引导大语言模型模拟图算法,在AMR图上进行显式的符号推理,并展示了其在提升大语言模型处理以语义为中心的任务(如句法简化)方面的潜力。