How much does a user's skill with AI shape what AI actually delivers for them? This question is critical for users, AI product builders, and society at large, but it remains underexplored. Using a richly annotated sample of 27K transcripts from WildChat-4.8M, we show that fluent users take on more complex tasks than novices and adopt a fundamentally different interactional mode: they iterate collaboratively with the AI, refining goals and critically assessing outputs, whereas novices take a passive stance. These differences lead to a paradox of AI fluency: fluent users experience more failures than novices -- but their failures tend to be visible (a direct consequence of their engagement), they are more likely to lead to partial recovery, and they occur alongside greater success on complex tasks. Novices, by contrast, more often experience invisible failures: conversations that appear to end successfully but in fact miss the mark. Taken together, these results reframe what success with AI depends on. Individuals should adopt a stance of active engagement rather than passive acceptance. AI product builders should recognize that they are designing not just model behavior but user behavior; encouraging deep engagement, rather than friction-free experiences, will lead to more success overall. Our code and data are available at https://github.com/bigspinai/bigspin-fluency-outcomes
翻译:用户对AI的熟练程度如何实际影响AI为其带来的结果?这一问题对用户、AI产品构建者乃至整个社会都至关重要,但至今仍未得到充分探索。基于WildChat-4.8M数据集中精心标注的27K条对话记录样本,我们展示了熟练用户比新手承担更复杂的任务,并采用根本不同的交互模式:他们与AI协同迭代,不断优化目标并批判性评估输出结果,而新手则持被动姿态。这些差异导致了一个关于AI流畅度的悖论:熟练用户经历比新手更多的失败——但他们的失败往往是可见的(这是其积极参与的直接后果),更容易实现部分恢复,并且在复杂任务上取得更大成功。相比之下,新手更常经历隐性失败:看似成功结束的对话实际上并未达到预期目标。综合来看,这些结果重新定义了与AI合作成功的决定性因素。个体应采取主动参与而非被动接受的态度。AI产品构建者应当认识到,他们设计的不仅是模型行为,更是用户行为;鼓励深度参与而非追求零摩擦的体验将带来更高的整体成功率。我们的代码和数据可通过https://github.com/bigspinai/bigspin-fluency-outcomes获取。