Computational notebooks have become the preferred tool of choice for data scientists and practitioners to perform analyses and share results. Notebooks uniquely combine scripts with documentation. With the emergence of generative AI (GenAI) technologies, it is increasingly important, especially in competitive settings, to distinguish the characteristics of human-written versus GenAI. In this study, we present three case studies to explore potential strengths of both humans and GenAI through the coding and documenting activities in notebooks. We first characterize differences between 25 code and documentation features in human-written, medal-winning Kaggle notebooks. We find that gold medalists are primarily distinguished by longer and more detailed documentation. Second, we analyze the distinctions between human-written and GenAI notebooks. Our results show that while GenAI notebooks tend to achieve higher code quality (as measured by metrics like code smells and technical debt), human-written notebooks display greater structural diversity, complexity, and innovative approaches to problem-solving. Based on these results, we envision the work as groundwork that highlight four agendas to further investigate how GenAI could be utilized in notebooks that maximizes the potential collaboration between human and AI.
翻译:计算笔记本已成为数据科学家和从业者进行分析与分享结果的首选工具。笔记本独特地将脚本与文档相结合。随着生成式人工智能技术的兴起,区分人类撰写与生成式人工智能生成内容的特征变得日益重要,尤其在竞赛环境中。本研究通过三个案例,探索人类与生成式人工智能在笔记本编码和文档撰写活动中的潜在优势。首先,我们基于25项代码与文档特征,对Kaggle竞赛中人类撰写且获奖的笔记本进行特征分析,发现金牌得主主要通过更长、更详细的文档脱颖而出。其次,我们比较人类撰写与生成式人工智能生成的笔记本。结果表明,尽管生成式人工智能生成的笔记本在代码质量(如代码异味和技术债务等指标)上表现更优,但人类撰写的笔记本展现出更高的结构多样性、复杂性以及问题解决的创新性方法。基于这些结果,本研究为后续探索生成式人工智能在笔记本中的有效应用奠定了基础,并提出了四项研究议程,以深入探究如何最大化人类与人工智能在笔记本环境中的协作潜力。