Agents capable of carrying out general tasks on a computer can improve efficiency and productivity by automating repetitive tasks and assisting in complex problem-solving. Ideally, such agents should be able to solve new computer tasks presented to them through natural language commands. However, previous approaches to this problem require large amounts of expert demonstrations and task-specific reward functions, both of which are impractical for new tasks. In this work, we show that a pre-trained large language model (LLM) agent can execute computer tasks guided by natural language using a simple prompting scheme where the agent recursively criticizes and improves its output (RCI). The RCI approach significantly outperforms existing LLM methods for automating computer tasks and surpasses supervised learning (SL) and reinforcement learning (RL) approaches on the MiniWoB++ benchmark. RCI is competitive with the state-of-the-art SL+RL method, using only a handful of demonstrations per task rather than tens of thousands, and without a task-specific reward function. Furthermore, we demonstrate RCI prompting's effectiveness in enhancing LLMs' reasoning abilities on a suite of natural language reasoning tasks, outperforming chain of thought (CoT) prompting. We find that RCI combined with CoT performs better than either separately.

翻译：能够执行通用计算机任务的智能体可通过自动化重复性工作并协助解决复杂问题，从而提升效率与生产力。理想情况下，此类智能体应能通过自然语言指令解决新的计算机任务。然而，现有方法需要大量专家演示和任务特定的奖励函数，这对新任务而言并不实际。在本研究中，我们证明预训练大语言模型（LLM）智能体可通过一种简单的提示方案——递归批评与改进输出（RCI）——在自然语言引导下执行计算机任务。RCI方法在自动化计算机任务方面显著优于现有LLM方法，并在MiniWoB++基准测试中超越了监督学习（SL）和强化学习（RL）方法。RCI与最先进的SL+RL方法性能相当，但每个任务仅需少量演示（而非数万个），且无需任务特定奖励函数。此外，我们展示RCI提示在增强LLM自然语言推理任务能力方面的有效性，其表现优于思维链（CoT）提示。研究发现，RCI与CoT结合的效果优于单独使用其中任一方法。