We argue that the relative success of large language models (LLMs) is not a reflection on the symbolic vs. subsymbolic debate but a reflection on employing an appropriate strategy of bottom-up reverse engineering of language at scale. However, due to the subsymbolic nature of these models whatever knowledge these systems acquire about language will always be buried in millions of microfeatures (weights) none of which is meaningful on its own. Moreover, and due to their stochastic nature, these models will often fail in capturing various inferential aspects that are prevalent in natural language. What we suggest here is employing the successful bottom-up strategy in a symbolic setting, producing symbolic, language agnostic and ontologically grounded large language models.

翻译：我们认为，大型语言模型（LLMs）的相对成功并非反映了符号化与次符号化之争的胜负，而是体现了在大规模语言处理中采用恰当的自底向上逆向工程策略的成效。然而，鉴于这些模型的次符号化本质，它们所获得的语言知识始终深植于数百万个微观特征（权重）之中，且每个特征本身均不具有独立意义。此外，由于其随机性本质，这些模型在捕捉自然语言中普遍存在的各种推理特征时常常失效。本文建议在符号化框架中采用这种成功的自底向上策略，构建符号化、语言无关且具有本体论基础的大型语言模型。