Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time). While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users. No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scaffolding mechanisms.

翻译：语言建模的进步为新型人机共写体验铺平了道路。本文探讨了大语言模型（LLMs）提供的不同支架级别如何影响共写过程。采用基于拉丁方设计的受试者内现场实验，我们要求参与者（N=131）在三种随机排序条件下回应议论文写作任务：无AI辅助（对照组）、下一句建议（低支架）和下一段建议（高支架）。研究结果揭示，支架对写作质量和生产力（字数/时间）呈U型影响。低支架未显著提升写作质量或生产力，而高支架则带来显著改善，尤其惠及非经常性写作者和技术熟练度较低的用户。使用支架式写作工具时未观察到显著认知负担，但文本所有权和满意度出现适度下降。我们的研究结果对AI写作工具的设计具有广泛启示，包括需要个性化支架机制。