Rule-based reasoning, a fundamental type of legal reasoning, enables us to draw conclusions by accurately applying a rule to a set of facts. We explore causal language models as rule-based reasoners, specifically with respect to compositional rules - rules consisting of multiple elements which form a complex logical expression. Reasoning about compositional rules is challenging because it requires multiple reasoning steps, and attending to the logical relationships between elements. We introduce a new prompting method, Chain of Logic, which elicits rule-based reasoning through decomposition (solving elements as independent threads of logic), and recomposition (recombining these sub-answers to resolve the underlying logical expression). This method was inspired by the IRAC (Issue, Rule, Application, Conclusion) framework, a sequential reasoning approach used by lawyers. We evaluate chain of logic across eight rule-based reasoning tasks involving three distinct compositional rules from the LegalBench benchmark and demonstrate it consistently outperforms other prompting methods, including chain of thought and self-ask, using open-source and commercial language models.

翻译：基于规则的推理作为法律推理的基本形式，使我们能够通过准确地将规则应用于一组事实来得出结论。我们探索将因果语言模型作为基于规则的推理器，特别是在组合规则——由多个要素构成复杂逻辑表达式的规则——方面的应用。对组合规则的推理具有挑战性，因为它需要多个推理步骤，并需关注要素之间的逻辑关系。我们提出了一种新的提示方法——逻辑链（Chain of Logic），该方法通过分解（将要素作为独立逻辑线索求解）和重组（将这些子答案重新组合以解析底层逻辑表达式）来激发基于规则的推理。该方法受律师使用的顺序推理框架IRAC（Issue, Rule, Application, Conclusion，即议题、规则、应用、结论）启发。我们在LegalBench基准测试中涉及三种不同组合规则的八项基于规则的推理任务上评估了逻辑链方法，结果表明，使用开源和商业语言模型时，该方法始终优于包括思维链（chain of thought）和自问自答（self-ask）在内的其他提示方法。