Recent research suggests that the use of Generative AI tools may result in diminished critical thinking during knowledge work. We study the effect on knowledge work of provocations: brief textual prompts that offer critiques for and propose alternatives to AI suggestions. We conduct a between-subjects study (n=24) in which participants completed AI-assisted shortlisting tasks with and without provocations. We find that provocations can induce critical and metacognitive thinking. We derive five dimensions that impact the user experience of provocations: task urgency, task importance, user expertise, provocation actionability, and user responsibility. We connect our findings to related work on design frictions, microboundaries, and distributed cognition. We draw design implications for critical thinking interventions in AI-assisted knowledge work.

翻译：近期研究表明，使用生成式AI工具可能导致知识工作中的批判性思维减弱。本研究探讨了挑衅性提示对知识工作的影响：这些简短的文本提示会对AI建议提出批评并提出替代方案。我们开展了一项被试间研究（n=24），参与者分别在有无挑衅性提示的情况下完成AI辅助的初选任务。研究发现，挑衅性提示能够引发批判性思维和元认知思考。我们提炼出影响挑衅性提示用户体验的五个维度：任务紧急性、任务重要性、用户专业知识、提示可操作性以及用户责任感。我们将研究结果与关于设计摩擦、微边界和分布式认知的相关工作联系起来。最后，为AI辅助知识工作中的批判性思维干预措施提出了设计启示。