Can targeted user training unlock the productive potential of generative artificial intelligence (GenAI) in professional settings? We investigate this question using a randomized study involving 164 law students completing an issue-spotting examination. Participants were assigned to one of three conditions: no GenAI access, optional access to a large language model (LLM), or optional access accompanied by an approximately ten-minute training intervention. Training significantly increased LLM adoption--the usage rate rose from 26% to 41%--and improved examination performance. Students with trained access scored 0.27 grade points higher than those with untrained access (p = 0.027), equivalent to roughly one-third of a letter grade. By contrast, access to an LLM without training did not improve performance and was associated with shorter answers relative to no access. Using principal stratification, we decompose the overall effect into adoption and effectiveness channels. Point estimates are consistent with training operating primarily by expanding the scope of GenAI use rather than by enhancing effectiveness among existing users, though confidence intervals are wide. Overall, our findings provide evidence that complementary investments in user training are critical for realizing GenAI productivity gains in knowledge-intensive fields where concerns about reliability may inhibit adoption.
翻译:针对性的用户培训能否在专业环境中释放生成式人工智能(GenAI)的生产力潜力?我们通过一项随机研究探讨了这一问题,该研究涉及164名法学院学生完成一项问题识别测试。参与者被分配到以下三种条件之一:无GenAI访问权限、可选择访问大型语言模型(LLM),或可选择访问并辅以约十分钟的培训干预。培训显著提高了LLM的采纳率——使用率从26%上升至41%——并提升了测试表现。接受培训后获得访问权限的学生比未受培训即获得访问权限的学生成绩高出0.27个等级分(p = 0.027),约相当于一个字母等级的三分之一。相比之下,未经培训直接使用LLM不仅未能提升表现,且其答案长度相较于无访问权限组更短。通过主分层分析,我们将整体效应分解为采纳渠道和效能渠道。点估计结果表明,培训主要通过扩大GenAI的使用范围而非提升现有用户的使用效能来发挥作用,尽管置信区间较宽。总体而言,我们的研究结果证明,在那些因可靠性顾虑而可能抑制技术采纳的知识密集型领域,对用户培训的配套投入对于实现GenAI的生产力增益至关重要。