Self-consistency improves LLM reasoning by sampling multiple outputs and selecting the most consistent answer, but existing formulations largely rely on exact matching and therefore remain limited to tasks with categorical outputs. In this work, we study self-consistency in open-ended generation tasks such as code synthesis and text summarization. We hypothesize that consistency can be understood as a geometric property of the generation space, where semantically compatible generations concentrate in similar regions of representation space. To study this hypothesis, we introduce Embedding-Based Agreement (EBA), a simple training-free operationalization that estimates agreement by clustering sampled generations in embedding space. Through experiments on mathematical reasoning, code generation, and summarization, we show that agreement in representation space provides a robust and scalable signal of self-consistency for open-ended tasks. In particular, EBA consistently outperforms random selection and exhibits more stable scaling behavior than recent selection approaches based on LLM evaluation or uncertainty estimation. We further show that these agreement signals remain stable across model families and embedding spaces, even with native hidden representations. Finally, our analysis shows that the geometric location occupied by sampled generations is strongly correlated with generation quality: generations concentrated near central regions of representation space tend to correspond to more reliable outputs, whereas peripheral generations are substantially less accurate. Overall, our findings support viewing self-consistency as a property of the geometric organization of sampled generations rather than exact symbolic overlap.

翻译：自我一致性通过采样多个输出并选择最一致的答案来提升大型语言模型的推理能力，但现有方法主要依赖于精确匹配，因此仅限于分类输出任务。本研究探讨开放生成任务（如代码合成和文本摘要）中的自我一致性。我们假设一致性可理解为生成空间的几何属性，其中语义兼容的生成结果在表示空间中聚集于相似区域。为验证这一假设，我们提出基于嵌入的一致性（EBA）方法，这是一种无需训练的简单操作化方法，通过在嵌入空间中对采样生成结果进行聚类来估计一致性。通过在数学推理、代码生成和摘要任务上的实验，我们证明表示空间中的一致性为开放式任务提供了鲁棒且可扩展的自我一致性信号。特别是，EBA始终优于随机选择，并且相较于近期基于LLM评估或不确定性估计的选取方法，展现出更稳定的扩展行为。我们进一步证明，这些一致性信号在不同模型家族和嵌入空间（包括原生隐藏表示）中保持稳定。最后，我们的分析表明，采样生成结果占据的几何位置与生成质量强相关：聚集在表示空间中心区域的生成结果往往对应更可靠的输出，而边缘区域的生成结果准确性显著降低。总体而言，我们的发现支持将自我一致性视为采样生成结果几何组织属性而非符号精确匹配的视角。