As large language models (LLMs) increasingly shape how users form, refine, and extend their goals, attributing contributions in human-AI collaboration becomes critical for users calibrating their own reliance and for evaluators assessing AI-assisted work. Yet existing methods focus on final artifacts, missing the process through which goals themselves are jointly shaped. We introduce a goal-level attribution framework, CoTrace, that decomposes explicit goals into verifiable requirements and traces both direct contributions and indirect influences across dialogue turns. Applying CoTrace to 638 real-world collaboration logs, we find that while models account for only 11-26% of goal-shaping contribution, they contribute substantially more on introducing lower-level concrete requirements, and make various kinds of indirect contributions. Through controlled simulations, we show that interaction design choices significantly affect model goal-shaping behavior. In a user study, exposing participants to goal-level analyses shifts their perceived contributions by nearly 2 points on a 5-point scale, revealing systematic miscalibration in how users understand their own AI-assisted work.
翻译:随着大型语言模型(LLM)日益影响用户形成、完善和拓展其目标的过程,在人机协作中归因贡献变得至关重要——这既有助于用户校准自身的依赖程度,也有助于评估者对AI辅助工作进行评判。然而,现有方法仅关注最终成果,忽略了目标本身被共同塑造的过程。我们提出了一种目标层面归因框架CoTrace,它将显式目标分解为可验证的需求,并追踪对话轮次中的直接贡献与间接影响。将CoTrace应用于638个真实协作日志后,我们发现:尽管模型仅贡献了11-26%的目标塑造份额,但在引入更具体的低层级需求方面贡献显著,并产生了多种间接影响。通过受控模拟实验,我们证明交互设计选择会显著影响模型的目标塑造行为。在一项用户研究中,让参与者接触目标层面分析后,其对自身贡献的感知在5分量表上偏移了近2分,揭示了用户理解自身AI辅助工作时存在的系统性校准偏差。