Developers often face challenges in code understanding, which is crucial for building and maintaining high-quality software systems. Code comments and documentation can provide some context for the code, but are often scarce or missing. This challenge has become even more pressing with the rise of large language model (LLM) based code generation tools. To understand unfamiliar code, most software developers rely on general-purpose search engines to search through various programming information resources, which often requires multiple iterations of query rewriting and information foraging. More recently, developers have turned to online chatbots powered by LLMs, such as ChatGPT, which can provide more customized responses but also incur more overhead as developers need to communicate a significant amount of context to the LLM via a textual interface. In this study, we provide the investigation of an LLM-based conversational UI in the IDE. We aim to understand the promises and obstacles for tools powered by LLMs that are contextually aware, in that they automatically leverage the developer's programming context to answer queries. To this end, we develop an IDE Plugin that allows users to query back-ends such as OpenAI's GPT-3.5 and GPT-4 with high-level requests, like: explaining a highlighted section of code, explaining key domain-specific terms, or providing usage examples for an API. We conduct an exploratory user study with 32 participants to understand the usefulness and effectiveness, as well as individual preferences in the usage of, this LLM-powered information support tool. The study confirms that this approach can aid code understanding more effectively than web search, but the degree of the benefit differed by participants' experience levels.

翻译：开发者在代码理解方面常面临挑战，而代码理解对于构建和维护高质量软件系统至关重要。代码注释和文档能为代码提供一定上下文，但往往稀缺或缺失。随着基于大型语言模型（LLM）的代码生成工具的兴起，这一挑战变得更加紧迫。为了理解不熟悉的代码，大多数软件开发者依赖通用搜索引擎来检索各种编程信息资源，这通常需要多次迭代的查询重写和信息搜寻。近年来，开发者开始转向由LLM驱动的在线聊天机器人（如ChatGPT），这类工具能提供更个性化的回答，但同时也带来更多开销——开发者需要通过文本界面将大量上下文传递给LLM。在本研究中，我们探索了在集成开发环境（IDE）中基于LLM的对话式用户界面，旨在理解具备上下文感知能力的LLM驱动工具（即能自动利用开发者编程上下文来回答问题的工具）的优势与障碍。为此，我们开发了一个IDE插件，允许用户通过高级请求（如：解释高亮代码段、解释关键领域术语、提供API使用示例）调用GPT-3.5和GPT-4等后端模型。我们开展了一项包含32名参与者的探索性用户研究，以评估该LLM驱动的信息支持工具的实用性、有效性以及个体使用偏好。研究证实，与网络搜索相比，该方法能更有效地辅助代码理解，但收益程度因参与者的经验水平而异。