Personalization of LLMs by sociodemographic subgroup often improves user experience, but can also introduce or amplify biases and unfair outcomes across groups. Prior work has employed so-called personas, sociodemographic user attributes conveyed to a model, to study bias in LLMs by relying on a single cue to prompt a persona, such as user names or explicit attribute mentions. This disregards LLM sensitivity to prompt variations (robustness) and the rarity of some cues in real interactions (external validity). We compare six commonly used persona cues across seven open and proprietary LLMs on four writing and advice tasks. While cues are overall highly correlated, they produce substantial variance in responses across personas. We therefore caution against claims from a single persona cue and recommend future personalization research to evaluate multiple externally valid cues.

翻译：通过社会人口学子群体对大语言模型进行个性化通常能改善用户体验，但也可能在不同群体间引入或放大偏见与不公结果。先前研究采用所谓角色（即传达给模型的社会人口学用户属性），通过依赖单一线索（如用户姓名或显式属性提及）来提示角色，从而考察大语言模型中的偏见。这种方法忽视了大语言模型对提示变动的敏感性（鲁棒性）以及某些线索在真实交互中的稀缺性（外部效度）。我们在四项写作与建议任务中，对七个开源与专有大语言模型比较了六种常用角色线索。虽然线索整体高度相关，但它们在不同角色间产生了显著的响应差异。因此，我们警示不应仅凭单一角色线索得出结论，并建议未来个性化研究评估多种具有外部效度的线索。