Large language models (LLMs) have recently been applied in software engineering to perform tasks such as translating code between programming languages, generating code from natural language, and autocompleting code as it is being written. When used within development tools, these systems typically treat each model invocation independently from all previous invocations, and only a specific limited functionality is exposed within the user interface. This approach to user interaction misses an opportunity for users to more deeply engage with the model by having the context of their previous interactions, as well as the context of their code, inform the model's responses. We developed a prototype system -- the Programmer's Assistant -- in order to explore the utility of conversational interactions grounded in code, as well as software engineers' receptiveness to the idea of conversing with, rather than invoking, a code-fluent LLM. Through an evaluation with 42 participants with varied levels of programming experience, we found that our system was capable of conducting extended, multi-turn discussions, and that it enabled additional knowledge and capabilities beyond code generation to emerge from the LLM. Despite skeptical initial expectations for conversational programming assistance, participants were impressed by the breadth of the assistant's capabilities, the quality of its responses, and its potential for improving their productivity. Our work demonstrates the unique potential of conversational interactions with LLMs for co-creative processes like software development.

翻译：大语言模型（LLMs）近期已被应用于软件工程领域，用于执行诸如编程语言间的代码翻译、从自然语言生成代码以及代码编写过程中的自动补全等任务。当这些系统集成至开发工具时，通常会将每次模型调用视为独立操作，且用户界面仅暴露有限的特定功能。这种交互方式使用户错失了通过保留先前交互上下文及代码上下文来深化模型参与度的机会。我们开发了一个原型系统——程序员助手（Programmer's Assistant）——旨在探索基于代码的会话式交互的实用性，以及软件工程师对与代码流畅型LLM进行对话（而非简单调用）这一概念的接受度。通过一项涉及42名不同编程经验参与者的评估，我们发现该系统能够进行扩展的多轮对话，并促使LLM展现出超越代码生成的知识与能力。尽管参与者初始对会话式编程辅助持怀疑态度，但他们对助手能力的广度、响应质量及其提升生产力的潜力印象深刻。我们的工作揭示了基于LLM的会话式交互在软件开发等共创过程中的独特潜力。