Modelling users' online decision-making and opinion change is a complex issue that needs to consider users' personal determinants, the nature of the topic and the information retrieval activities. Furthermore, generative-AIbased products like ChatGPT gradually become an essential element for the retrieval of online information. However, the interaction between domainspecific knowledge and AI-generated content during online decision-making is unclear. We conducted a lab-based explanatory sequential study with university students to overcome this research gap. In the experiment, we surveyed participants about a set of general domain topics that are easy to grasp and another set of domain-specific topics that require adequate levels of chemical science knowledge to fully comprehend. We provided participants with decision-supporting information that was either produced using generative AI or collected from selected expert human-written sources to explore the role of AI-generated content compared to ordinary information during decision-making. Our result revealed that participants are less likely to change opinions on domain-specific topics. Since participants without professional knowledge had difficulty performing in-depth and independent reasoning based on the information, they favoured relying on conclusions presented in the provided materials and tended to stick to their initial opinion. Besides, information that is labelled as AI-generated is equivalently helpful as information labelled as dedicatedly human-written for participants in this experiment, indicating the vast potential as well as concerns for AI replacing human experts to help users tackle professional topics or issues.
翻译:建模用户的在线决策与观点转变是一个复杂问题,需要综合考虑用户的个人决定因素、话题性质以及信息检索行为。此外,以生成式人工智能为基础的产品(如ChatGPT)正逐渐成为在线信息检索的关键要素。然而,在线决策过程中领域专业知识与AI生成内容之间的交互机制尚不明确。为弥补这一研究空白,我们针对大学生开展了一项基于实验室的解释性序列研究。实验中,我们向参与者调查了两类话题:一组为易于理解的通用领域话题,另一组则需要具备足够的化学科学知识才能完全理解的特定领域话题。我们为参与者提供了两种决策支持信息:一种由生成式AI生成,另一种则选自专家撰写的人工来源,以探究AI生成内容相较于普通信息在决策过程中的作用。研究结果显示,参与者对特定领域话题改变观点的可能性较低。由于缺乏专业知识的参与者难以基于所获信息进行深入独立的推理,他们更倾向于依赖材料中呈现的结论,并坚持初始观点。此外,实验中标注为AI生成的信息与标注为人工专门撰写的信息对参与者具有同等的辅助作用,这表明AI在替代人类专家协助用户处理专业议题方面既展现出巨大潜力,也引发了相应担忧。