Computational notebooks have become the preferred tool of choice for data scientists and practitioners to perform analyses and share results. Notebooks uniquely combine scripts with documentation. With the emergence of generative AI (GenAI) technologies, it is increasingly important, especially in competitive settings, to distinguish the characteristics of human-written versus GenAI. In this study, we present three case studies to explore potential strengths of both humans and GenAI through the coding and documenting activities in notebooks. We first characterize differences between 25 code and documentation features in human-written, medal-winning Kaggle notebooks. We find that gold medalists are primarily distinguished by longer and more detailed documentation. Second, we analyze the distinctions between human-written and GenAI notebooks. Our results show that while GenAI notebooks tend to achieve higher code quality (as measured by metrics like code smells and technical debt), human-written notebooks display greater structural diversity, complexity, and innovative approaches to problem-solving. Based on these results, we envision the work as groundwork that highlight four agendas to further investigate how GenAI could be utilized in notebooks that maximizes the potential collaboration between human and AI.
翻译:计算笔记本已成为数据科学家和从业者进行分析和分享结果的首选工具。笔记本独特地将脚本与文档相结合。随着生成式人工智能技术的兴起,区分人类撰写与生成式AI产出的特征变得日益重要,在竞赛环境中尤其如此。本研究通过三项案例研究,探讨人类与生成式AI在笔记本编码和文档撰写活动中的潜在优势。我们首先分析了25个代码与文档特征在人类撰写的获奖Kaggle笔记本中的差异,发现金牌得主主要通过更长、更详细的文档脱颖而出。其次,我们比较了人类撰写与生成式AI生成的笔记本。结果表明,虽然生成式AI笔记本往往获得更高的代码质量(通过代码异味和技术债务等指标衡量),但人类撰写的笔记本展现出更强的结构多样性、复杂性以及解决问题的创新方法。基于这些发现,本研究作为基础工作,提出了四项研究议程,以进一步探索如何在笔记本中利用生成式AI,最大化人类与AI的协作潜力。