Chain-of-thought and more broadly test-time compute are known to augment the expressive capabilities of language models and have led to major innovations in reasoning. Motivated by this success, this paper explores latent chain-of-thought as well as the impact of depth and looping for time-series and tabular data. We propose a recurrent scheme in which a structured-data transformer, after an initial forward pass, compresses its query-position hidden states into feedback tokens that are appended to the input and processed again, allowing multiple rounds of latent computation before prediction. We compare CoT models against a same-depth no-CoT baseline, a deeper baseline matched to the CoT model in effective depth, and a looped transformer with weight-tied recurrence but no additional chain-of-thought tokens. Across 36 datasets in time-series forecasting and tabular prediction, latent chain-of-thought improves over the baseline on 7/9 time-series datasets (+12.63\% average gain) and 23/27 tabular datasets (+3.25\% average gain), with CoT models performing best on average in both settings. We also show that the benefit of CoT extends to pretrained foundation models: applying latent CoT to nanoTabPFN, a small open-source tabular foundation model, improves its performance above the much larger TabPFN-v2 on TabArena. Together, these results demonstrate that chain-of-thought is a useful axis for scaling test-time compute for structured data.

翻译：思维链以及更广义的测试时计算已知能够增强语言模型的表达能力，并在推理领域引发了重大创新。受此成功启发，本文探索了潜在思维链以及深度和循环对时间序列与表格数据的影响。我们提出了一种循环方案：结构化数据Transformer在完成一次前向传播后，将其查询位置隐藏状态压缩为反馈令牌，这些令牌被附加到输入中并再次处理，从而在预测前允许多轮潜在计算。我们将CoT模型与同深度无CoT基线、与CoT模型有效深度匹配的更深基线以及具有权重绑定循环但无额外思维链令牌的循环Transformer进行了比较。在时间序列预测和表格预测的36个数据集上，潜在思维链在7/9个时间序列数据集（平均提升+12.63%）和23/27个表格数据集（平均提升+3.25%）上优于基线，CoT模型在两种场景下平均性能最优。我们还表明，CoT的优势可扩展到预训练基础模型：将潜在CoT应用于nanoTabPFN（一个轻量级开源表格基础模型）后，其性能超越了规模更大的TabPFN-v2在TabArena上的表现。这些结果共同表明，思维链是扩展结构化数据测试时计算的一个有效维度。