Can targeted user training unlock the productive potential of generative artificial intelligence in professional settings? We study this question using a randomized experiment in which 164 law students completed an issue-spotting examination under one of three conditions: no GenAI access, optional access to a large language model (LLM), or LLM access with a brief training intervention. Untrained LLM access proved counterproductive: relative to participants without any LLM access, untrained users wrote significantly shorter answers, committed more case misstatements, and scored marginally lower, though most differences fall short of conventional significance. Training reversed this pattern. Trained participants adopted the LLM at higher rates (41% vs. 26%; p = 0.044), scored 0.27 grade points higher than untrained users--roughly one fine grade--(p = 0.027), and stated applicable rules more accurately (p = 0.014). Principal stratification analysis suggests training operates primarily through adoption rather than effectiveness--the adoption lower bound (1.06) exceeds the effectiveness upper bound (0.42) at strict mean dominance--though confidence intervals are wide. Training also shifted who adopted: top-quartile students went from 0% adoption to 42%. More broadly, these findings challenge the view that GenAI primarily benefits lower-skilled workers: without training, higher-ability practitioners opt out while lower-ability users adopt but unproductively. Realizing GenAI's productivity gains requires investment in both access and instruction.
翻译:针对性的用户培训能否释放生成式人工智能在专业环境中的生产潜力?我们通过一项随机实验研究这一问题,164名法学院学生在三种条件下完成一项问题识别考试:无GenAI访问权限、选择性访问大型语言模型(LLM)、或LLM访问配合简短培训干预。未经培训的LLM访问被证明适得其反:相比无LLM访问的参与者,未经培训的用户撰写的答案明显更短,更多出现案例陈述错误,且分数略低,尽管大多数差异未达到常规显著性水平。培训扭转了这一模式。受培训参与者以更高比例使用LLM(41%对26%;p=0.044),得分比未经培训用户高出0.27个绩点——约一个精细等级——(p=0.027),且更准确地陈述适用规则(p=0.014)。主分层分析表明,培训主要通过采纳而非有效性发挥作用——在严格均值优势下,采纳下限(1.06)超过有效性上限(0.42),尽管置信区间较宽。培训还改变了谁采纳:前四分之一学生从0%采纳变为42%。更广泛而言,这些发现挑战了GenAI主要惠及低技能劳动者的观点:未经培训时,高能力从业者选择退出,而低能力用户虽采纳但无生产性。实现GenAI的生产力提升需要同时在访问和指导方面进行投资。