LLMs have marked a revolutonary shift, yet they falter when faced with compositional reasoning tasks. Our research embarks on a quest to uncover the root causes of compositional reasoning failures of LLMs, uncovering that most of them stem from the improperly generated or leveraged implicit reasoning results. Inspired by our empirical findings, we resort to Logit Lens and an intervention experiment to dissect the inner hidden states of LLMs. This deep dive reveals that implicit reasoning results indeed surface within middle layers and play a causative role in shaping the final explicit reasoning results. Our exploration further locates multi-head self-attention (MHSA) modules within these layers, which emerge as the linchpins in accurate generation and leveraing of implicit reasoning results. Grounded on the above findings, we develop CREME, a lightweight method to patch errors in compositional reasoning via editing the located MHSA modules. Our empirical evidence stands testament to CREME's effectiveness, paving the way for autonomously and continuously enhancing compositional reasoning capabilities in language models.

翻译：大语言模型带来了革命性的转变，但在面对组合推理任务时仍显不足。本研究旨在揭示大语言模型组合推理失败的根源，发现大多数问题源于隐式推理结果生成或利用不当。受实验发现的启发，我们借助Logit Lens和干预实验剖析大语言模型的内部隐藏状态。深入分析表明，隐式推理结果确实出现在中间层，并对最终显式推理结果的形成起到因果作用。进一步探索定位了这些层中的多头自注意力模块，这些模块在隐式推理结果的准确生成与利用中扮演关键角色。基于上述发现，我们开发了轻量级方法CREME，通过编辑定位到的MHSA模块来修补组合推理中的错误。实验证据证实了CREME的有效性，为自主且持续增强语言模型中的组合推理能力铺平了道路。