The growing use of Generative AI (GenAI) conversational search tools has raised concerns about their effects on people's metacognitive engagement, critical thinking, and learning. As people increasingly rely on GenAI to perform tasks such as analyzing and applying information, they may become less actively engaged in thinking and learning. This study examines whether metacognitive prompts - designed to encourage people to pause, reflect, assess their understanding, and consider multiple perspectives - can support critical thinking during GenAI-based search. We conducted a user study (N=40) with university students to investigate the impact of metacognitive prompts on their thought processes and search behaviors while searching with a GenAI tool. We found that these prompts led to more active engagement, prompting students to explore a broader range of topics and engage in deeper inquiry through follow-up queries. Students reported that the prompts were especially helpful for considering overlooked perspectives, promoting evaluation of AI responses, and identifying key takeaways. Additionally, the effectiveness of these prompts was influenced by students' metacognitive flexibility. Our findings highlight the potential of metacognitive prompts to foster critical thinking and provide insights for designing and implementing metacognitive support in human-AI interactions.

翻译：生成式人工智能对话式搜索工具的日益普及引发了人们对它们如何影响用户元认知参与、批判性思维和学习的担忧。随着人们越来越依赖生成式人工智能来执行分析和应用信息等任务，他们可能在思维和学习方面的主动参与度降低。本研究探讨了设计用于鼓励人们暂停、反思、评估自身理解并考虑多元视角的元认知提示，能否在基于生成式人工智能的搜索中支持批判性思维。我们开展了一项包含40名大学生的用户研究，以考察元认知提示对他们在使用生成式人工智能工具搜索时的思维过程和搜索行为的影响。研究发现，这些提示促进了更主动的参与，促使学生探索更广泛的主题，并通过后续提问进行更深入的探究。学生报告称，这些提示特别有助于考虑被忽视的观点、促进对人工智能回答的评估以及识别关键要点。此外，这些提示的有效性受学生元认知灵活性的影响。我们的研究结果凸显了元认知提示在培养批判性思维方面的潜力，并为在人机交互中设计和实施元认知支持提供了洞见。