We examine the extent to which, in principle, linguistic graph representations can complement and improve neural language modeling. With an ensemble setup consisting of a pretrained Transformer and ground-truth graphs from one of 7 different formalisms, we find that, overall, semantic constituency structures are most useful to language modeling performance -- outpacing syntactic constituency structures as well as syntactic and semantic dependency structures. Further, effects vary greatly depending on part-of-speech class. In sum, our findings point to promising tendencies in neuro-symbolic language modeling and invite future research quantifying the design choices made by different formalisms.

翻译：我们审视了语言图表在原则上在多大程度上可以补充和改进神经语言模型。我们发现,语言群落结构在原则上在多大程度上可以补充和改进神经语言模型。通过一个由来自7种不同形式主义之一的预先训练的变异器和地面真象图组成的组合结构,我们发现,总体而言,语义选区结构对语言模拟性能最为有用 -- -- 超越合成群群结构以及合成和语义依赖性结构。此外,影响因语系不同而有很大差异。简言之,我们的调查结果表明,神经-顺理派语言模型的有希望的趋势,并请今后开展研究,量化不同形式主义所作的设计选择。