Computer end users have spent billions of hours completing daily tasks like tabular data processing and project timeline scheduling. Most of these tasks are repetitive and error-prone, yet most end users lack the skill of automating away these burdensome works. With the advent of large language models (LLMs), directing software with natural language user requests become a reachable goal. In this work, we propose a SheetCopilot agent which takes natural language task and control spreadsheet to fulfill the requirements. We propose a set of atomic actions as an abstraction of spreadsheet software functionalities. We further design a state machine-based task planning framework for LLMs to robustly interact with spreadsheets. We curate a representative dataset containing 221 spreadsheet control tasks and establish a fully automated evaluation pipeline for rigorously benchmarking the ability of LLMs in software control tasks. Our SheetCopilot correctly completes 44.3\% of tasks for a single generation, outperforming the strong code generation baseline by a wide margin. Our project page:https://sheetcopilot-demo.github.io/.

翻译：计算机终端用户每天花费数十亿小时完成诸如表格数据处理和项目时间表规划等日常任务。这些任务大多重复且易出错，然而大多数终端用户缺乏自动化完成这些繁琐工作的技能。随着大语言模型（LLMs）的出现，通过自然语言用户指令操控软件成为一个可实现的目标。本文提出SheetCopilot智能体，它能接收自然语言任务并操控电子表格以满足需求。我们提出一组原子操作作为电子表格软件功能的抽象。进一步，我们设计了一种基于状态机的任务规划框架，使LLMs能够稳健地与电子表格进行交互。我们整理了一个包含221个电子表格控制任务的代表性数据集，并建立了一个全自动评估流程，用于严格基准测试LLMs在软件控制任务中的能力。在单次生成条件下，我们的SheetCopilot正确完成了44.3%的任务，大幅超越了强大的代码生成基线。项目页面：https://sheetcopilot-demo.github.io/