We investigate the patterns of incorrect answers produced by large language models (LLMs) during evaluation. These errors exhibit highly non-intuitive behaviors unique to each model. By analyzing these patterns, we measure the similarities between LLMs and construct a taxonomy that categorizes them based on their error correlations. Our findings reveal that the incorrect responses are not randomly distributed but systematically correlated across models, providing new insights into the underlying structures and relationships among LLMs.

翻译：本研究探究了大型语言模型（LLMs）在评估过程中产生错误答案的模式。这些错误展现出高度非直观且模型特有的行为。通过分析这些模式，我们测量了不同LLMs之间的相似性，并构建了一个基于错误相关性的分类体系。研究结果表明，错误答案并非随机分布，而是在模型间呈现系统性关联，这为理解LLMs的底层结构与相互关系提供了新的见解。