Logical reasoning is a crucial task for Large Language Models (LLMs), enabling them to tackle complex problems. Among reasoning tasks, multi-step reasoning poses a particular challenge. Grounded in the theory of formal logic, we have developed an automated method, Multi-step Deduction (MuseD), for deductive reasoning data. MuseD has allowed us to create training and testing datasets for multi-step reasoning. Our generation method enables control over the complexity of the generated instructions, facilitating training and evaluation of models across different difficulty levels. Through RLHF training, our training data has demonstrated significant improvements in logical capabilities for both in-domain of out-of-domain reasoning tasks. Additionally, we have conducted tests to assess the multi-step reasoning abilities of various models.

翻译：逻辑推理是大型语言模型（LLM）处理复杂问题的关键能力。在各类推理任务中，多步推理尤其具有挑战性。基于形式逻辑理论，我们开发了一种用于演绎推理数据的自动化方法——多步演绎（MuseD）。MuseD使我们能够构建用于多步推理的训练与测试数据集。我们的生成方法能够控制生成指令的复杂度，从而便于在不同难度级别上对模型进行训练和评估。通过RLHF训练，我们的训练数据在领域内和领域外推理任务的逻辑能力方面均展现出显著提升。此外，我们还测试了多种模型的多步推理能力。