Table-based reasoning has shown remarkable progress in combining deep models with discrete reasoning, which requires reasoning over both free-form natural language (NL) questions and structured tabular data. However, previous table-based reasoning solutions usually suffer from significant performance degradation on huge evidence (tables). In addition, most existing methods struggle to reason over complex questions since the required information is scattered in different places. To alleviate the above challenges, we exploit large language models (LLMs) as decomposers for effective table-based reasoning, which (i) decompose huge evidence (a huge table) into sub-evidence (a small table) to mitigate the interference of useless information for table reasoning; and (ii) decompose complex questions into simpler sub-questions for text reasoning. Specifically, we first use the LLMs to break down the evidence (tables) involved in the current question, retaining the relevant evidence and excluding the remaining irrelevant evidence from the huge table. In addition, we propose a "parsing-execution-filling" strategy to alleviate the hallucination dilemma of the chain of thought by decoupling logic and numerical computation in each step. Extensive experiments show that our method can effectively leverage decomposed evidence and questions and outperforms the strong baselines on TabFact, WikiTableQuestion, and FetaQA datasets. Notably, our model outperforms human performance for the first time on the TabFact dataset.
翻译:表格推理在深度融合模型与离散推理方面取得了显著进展,这要求同时处理自由形式的自然语言问题和结构化表格数据。然而,以往的表格推理方案通常在处理大规模证据(表格)时面临显著性能下降。此外,由于所需信息分散在不同位置,现有方法大多难以推理复杂问题。为缓解上述挑战,我们利用大型语言模型作为分解器实现高效表格推理:(i)将大规模证据(大表格)分解为子证据(小表格),以减少表格推理中无关信息的干扰;(ii)将复杂问题分解为更简单的子问题以进行文本推理。具体而言,我们首先使用大型语言模型拆解当前问题涉及的证据(表格),保留相关证据并从大表格中排除其余无关证据。此外,我们提出一种“解析-执行-填充”策略,通过解耦每一步的逻辑与数值计算,缓解思维链的幻觉困境。大量实验表明,我们的方法能有效利用分解后的证据和问题,在TabFact、WikiTableQuestion和FetaQA数据集上优于强基线模型。值得注意的是,我们的模型在TabFact数据集上首次超越了人类表现。