Large language models (LLMs) have shown remarkable promise in simulating human language and behavior. This study investigates how integrating persona variables-demographic, social, and behavioral factors-impacts LLMs' ability to simulate diverse perspectives. We find that persona variables account for <10% variance in annotations in existing subjective NLP datasets. Nonetheless, incorporating persona variables via prompting in LLMs provides modest but statistically significant improvements. Persona prompting is most effective in samples where many annotators disagree, but their disagreements are relatively minor. Notably, we find a linear relationship in our setting: the stronger the correlation between persona variables and human annotations, the more accurate the LLM predictions are using persona prompting. In a zero-shot setting, a powerful 70b model with persona prompting captures 81% of the annotation variance achievable by linear regression trained on ground truth annotations. However, for most subjective NLP datasets, where persona variables have limited explanatory power, the benefits of persona prompting are limited.

翻译：大型语言模型（LLM）在模拟人类语言和行为方面展现出显著潜力。本研究探讨了整合人格变量——包括人口统计、社会和行为因素——如何影响LLM模拟多元视角的能力。我们发现，在现有主观自然语言处理数据集中，人格变量仅能解释标注方差不足10%。尽管如此，通过提示机制在LLM中融入人格变量仍能带来虽有限但统计显著的改进。人格提示在众多标注者存在分歧但分歧程度相对较小的样本中最为有效。值得注意的是，我们在实验设置中发现了线性关系：人格变量与人工标注之间的相关性越强，采用人格提示的LLM预测准确性越高。在零样本设置下，配备人格提示的70b参数强大模型能够捕捉线性回归在真实标注上训练所能达到的标注方差的81%。然而，对于多数人格变量解释力有限的主观自然语言处理数据集而言，人格提示的收益仍较为有限。