Large Language Models (LLMs) can justify or critique their predictions through discussions with other models or humans, thereby enriching their intrinsic understanding of instances. While proactive discussions in the inference phase have been shown to boost performance, such interactions have not been extensively explored during the training phase. We hypothesize that incorporating interactive discussions into the training process can enhance the models' understanding and improve their reasoning and verbal expression abilities during inference. This work introduces the SAIE framework, which facilitates supportive and adversarial discussions between learner and partner models. The learner model receives responses from the partner, and its parameters are then updated based on this discussion. This dynamic adjustment process continues throughout the training phase, responding to the evolving outputs of the learner model. Our empirical evaluation across various tasks, including math problems, commonsense reasoning, and multi-domain knowledge, demonstrates that models fine-tuned with the SAIE framework outperform those trained with conventional fine-tuning approaches. Furthermore, our method enhances the models' reasoning capabilities, improving both individual and multi-agent inference performance.

翻译：大型语言模型（LLM）可通过与其他模型或人类进行讨论来证明或批判自身预测，从而丰富对实例的内在理解。尽管推理阶段中的主动讨论已被证明能提升性能，但在训练阶段，此类交互尚未得到广泛探索。我们假设，将交互式讨论融入训练过程能增强模型的理解能力，并提升其在推理阶段的推理与语言表达能力。本文提出SAIE框架，该框架在学习模型与伙伴模型之间促进支持性与对抗性讨论。学习模型接收伙伴模型的回应，并据此更新自身参数。这一动态调整过程贯穿整个训练阶段，随着学习模型输出结果的变化而持续响应。我们在包括数学问题、常识推理及多领域知识在内的多项任务上的实证评估表明，使用SAIE框架微调的模型性能优于传统微调方法。此外，我们的方法增强了模型的推理能力，并提升了个体及多智能体推理的表现。