As large language models increase in capability, researchers have started to conduct surveys of all kinds on these models in order to investigate the population represented by their responses. In this work, we critically examine language models' survey responses on the basis of the well-established American Community Survey by the U.S. Census Bureau and investigate whether they elicit a faithful representations of any human population. Using a de-facto standard multiple-choice prompting technique and evaluating 39 different language models using systematic experiments, we establish two dominant patterns: First, models' responses are governed by ordering and labeling biases, leading to variations across models that do not persist after adjusting for systematic biases. Second, models' responses do not contain the entropy variations and statistical signals typically found in human populations. As a result, a binary classifier can almost perfectly differentiate model-generated data from the responses of the U.S. census. At the same time, models' relative alignment with different demographic subgroups can be predicted from the subgroups' entropy, irrespective of the model's training data or training strategy. Taken together, our findings suggest caution in treating models' survey responses as equivalent to those of human populations.

翻译：随着大型语言模型能力的提升，研究者已开始对这些模型进行各类调查，以探究其回应所代表的人群。本研究基于美国人口普查局成熟的美国社区调查，批判性地审视语言模型的调查回应，并考察它们是否真实地反映了任何人类群体。通过采用事实上的标准多选题提示技术，并对39种不同语言模型进行系统性实验，我们确立了两种主导模式：首先，模型的回应受排序和标注偏差影响，导致模型间的差异在调整系统性偏差后不再持续。其次，模型的回应缺乏人类群体中通常存在的熵变异与统计信号。因此，一个二元分类器几乎可以完美区分模型生成的数据与美国人口普查的回应。同时，模型与不同人口统计子群体的相对对齐程度可依据子群体的熵进行预测，而与模型的训练数据或训练策略无关。综合来看，我们的发现提示，在将模型的调查回应视为人类群体回应时需保持谨慎。