Reasoning is a central capability of human intelligence. In recent years, with the advent of large-scale datasets, pretrained large language models have emerged with new capabilities, including reasoning. However, these models still struggle with long-term, complex reasoning tasks, such as playing chess. Based on the observation that expert chess players employ a dual approach combining long-term strategic play with short-term tactical play along with language explanation, we propose improving the reasoning capability of large language models in chess by integrating annotated strategy and tactic. Specifically, we collect a dataset named MATE, which consists of 1 million chess positions with candidate moves annotated by chess experts for strategy and tactics. We finetune the LLaMA-3-8B model and compare it against state-of-the-art commercial language models in the task of selecting better chess moves. Our experiments show that our models perform better than GPT, Claude, and Gemini models. We find that language explanations can enhance the reasoning capability of large language models.

翻译：推理是人类智能的核心能力。近年来，随着大规模数据集的出现，预训练大型语言模型展现出包括推理在内的新能力。然而，这些模型在处理长期、复杂的推理任务（如下国际象棋）时仍面临困难。基于观察到专业棋手采用结合长期战略走法与短期战术走法并辅以语言解释的双重方法，我们提出通过整合标注的战略与战术来提升大型语言模型在国际象棋中的推理能力。具体而言，我们收集了一个名为MATE的数据集，其中包含100万个国际象棋局面，每个局面由国际象棋专家标注了候选走法的战略与战术信息。我们对LLaMA-3-8B模型进行微调，并在选择更优国际象棋走法的任务中将其与最先进的商业语言模型进行比较。实验结果表明，我们的模型表现优于GPT、Claude和Gemini模型。我们发现语言解释能够增强大型语言模型的推理能力。