We devise an algorithm to generate sets of propositions that objectively instantiate graphs that support coherence-driven inference. We then benchmark the ability of large language models (LLMs) to reconstruct coherence graphs from (a straightforward transformation of) propositions expressed in natural language, with promising results from a single prompt to models optimized for reasoning. Combining coherence-driven inference with consistency evaluations by neural models may advance the state of the art in machine cognition.
翻译:我们设计了一种算法,用于生成客观实例化支持一致性驱动推理的图结构的命题集合。随后,我们评估了大语言模型(LLMs)从自然语言表达的命题(经过简单转换)重建一致性图的能力,结果表明,仅通过单一提示,针对推理优化的模型即可取得有前景的效果。将一致性驱动推理与神经模型的一致性评估相结合,有望推动机器认知领域的技术前沿。