Large language models (LLMs) trained on code completion have been shown to be capable of synthesizing simple Python programs from docstrings [1]. We find that these code-writing LLMs can be re-purposed to write robot policy code, given natural language commands. Specifically, policy code can express functions or feedback loops that process perception outputs (e.g.,from object detectors [2], [3]) and parameterize control primitive APIs. When provided as input several example language commands (formatted as comments) followed by corresponding policy code (via few-shot prompting), LLMs can take in new commands and autonomously re-compose API calls to generate new policy code respectively. By chaining classic logic structures and referencing third-party libraries (e.g., NumPy, Shapely) to perform arithmetic, LLMs used in this way can write robot policies that (i) exhibit spatial-geometric reasoning, (ii) generalize to new instructions, and (iii) prescribe precise values (e.g., velocities) to ambiguous descriptions ("faster") depending on context (i.e., behavioral commonsense). This paper presents code as policies: a robot-centric formulation of language model generated programs (LMPs) that can represent reactive policies (e.g., impedance controllers), as well as waypoint-based policies (vision-based pick and place, trajectory-based control), demonstrated across multiple real robot platforms. Central to our approach is prompting hierarchical code-gen (recursively defining undefined functions), which can write more complex code and also improves state-of-the-art to solve 39.8% of problems on the HumanEval [1] benchmark. Code and videos are available at https://code-as-policies.github.io

翻译：大型语言模型（LLMs）在代码补全任务上的训练已被证明能够根据文档字符串[1]合成简单的Python程序。我们发现，这些编写代码的LLMs可被重新用于根据自然语言指令编写机器人策略代码。具体而言，策略代码可表达处理感知输出（例如来自物体检测器[2]、[3]）的函数或反馈循环，并对控制基元API进行参数化。当输入若干示例性语言指令（格式化为注释）及其对应的策略代码（通过少样本提示）时，LLMs能接收新指令并自主重新组合API调用，从而生成相应新策略代码。通过串联经典逻辑结构并引用第三方库（如NumPy、Shapely）执行算术运算，由此使用的LLMs可编写出（i）展现空间几何推理能力、（ii）泛化至新指令、（iii）根据上下文（即行为常识）为模糊描述（如"更快"）指定精确值（如速度）的机器人策略。本文提出"代码即策略"：一种以机器人为中心的语言模型生成程序（LMPs）框架，可表示反应式策略（如阻抗控制器）及基于航点的策略（基于视觉的拾取与放置、基于轨迹的控制），并在多个真实机器人平台上得到验证。该方法的核心是分层代码生成提示（递归定义未命名函数），这不仅能编写更复杂的代码，还在HumanEval[1]基准测试中将当前最优水平提升至解决39.8%的问题。代码及视频见 https://code-as-policies.github.io