Two studies tested the hypothesis that a Large Language Model (LLM) can be used to model psychological change following exposure to influential input. The first study tested a generic mode of influence - the Illusory Truth Effect (ITE) - where earlier exposure to a statement (through, for example, rating its interest) boosts a later truthfulness test rating. Data was collected from 1000 human participants using an online experiment, and 1000 simulated participants using engineered prompts and LLM completion. 64 ratings per participant were collected, using all exposure-test combinations of the attributes: truth, interest, sentiment and importance. The results for human participants reconfirmed the ITE, and demonstrated an absence of effect for attributes other than truth, and when the same attribute is used for exposure and test. The same pattern of effects was found for LLM-simulated participants. The second study concerns a specific mode of influence - populist framing of news to increase its persuasion and political mobilization. Data from LLM-simulated participants was collected and compared to previously published data from a 15-country experiment on 7286 human participants. Several effects previously demonstrated from the human study were replicated by the simulated study, including effects that surprised the authors of the human study by contradicting their theoretical expectations (anti-immigrant framing of news decreases its persuasion and mobilization); but some significant relationships found in human data (modulation of the effectiveness of populist framing according to relative deprivation of the participant) were not present in the LLM data. Together the two studies support the view that LLMs have potential to act as models of the effect of influence.
翻译:两项研究验证了以下假设:大型语言模型(LLM)可用于模拟接触有影响力输入后的心理变化。第一项研究检验了一种通用影响模式——虚幻真相效应(ITE),即早期接触某陈述(例如通过评估其趣味性)会提升后续对其真实性的评分。通过在线实验收集了1000名人类参与者的数据,并利用工程化提示和LLM补全功能收集了1000名模拟参与者的数据。每个参与者获取64个评分,涵盖真实性、趣味性、情感性和重要性属性的所有暴露-测试组合。人类参与者的结果再次确认了ITE,并表明除真实性外的其他属性无此效应,且当暴露与测试使用同一属性时亦无效应。LLM模拟参与者呈现出相同的效应模式。第二项研究关注一种特定影响模式——以民粹主义框架呈现新闻以增强其说服力与政治动员效果。收集了LLM模拟参与者的数据,并与先前发表的15国7286名人类参与者实验数据进行比较。模拟研究复现了人类研究中先前证实的多项效应,包括令人类研究者意外且违背其理论预期的效应(反移民新闻框架降低了新闻说服力与动员效果);但人类数据中发现的部分显著关系(如参与者相对剥夺感对民粹框架效果的调节作用)未在LLM数据中呈现。两项研究共同表明,LLM具有作为影响效应模型的潜力。