Riddle-solving requires advanced reasoning skills, pushing LLMs to engage in abstract thinking and creative problem-solving, often revealing limitations in their cognitive abilities. In this paper, we examine the riddle-solving capabilities of LLMs using a multiple-choice format, exploring how different prompting techniques impact performance on riddles that demand diverse reasoning skills. To enhance results, we introduce RISCORE (RIddle Solving with COntext REcontruciton) a novel fully automated prompting method that generates and utilizes contextually reconstructed sentence-based puzzles in conjunction with the original examples to create few-shot exemplars. Our experiments demonstrate that RISCORE significantly improves the performance of language models in both vertical and lateral thinking tasks, surpassing traditional exemplar selection strategies across a variety of few-shot settings.

翻译：谜题求解需要高级推理能力，促使大型语言模型进行抽象思维和创造性问题解决，这常常暴露出其认知能力的局限性。本文通过多项选择形式考察大型语言模型的谜题求解能力，探究不同提示技术如何影响需要多样化推理技能的谜题表现。为提升效果，我们提出RISCORE（基于上下文重构的谜题求解方法）——一种全新的全自动提示方法，该方法生成并利用上下文重构的基于句子的谜题，结合原始示例创建少样本范例。实验表明，RISCORE能显著提升语言模型在纵向与横向思维任务中的表现，在各种少样本设置下均超越传统的范例选择策略。