Programmers often search for usage examples for API methods. A tool that could generate realistic, idiomatic, and contextual usage examples for one or more APIs would be immensely beneficial to developers. Such a tool would relieve the need for a deep understanding of the API landscape, augment existing documentation, and help discover interactions among APIs. We present CodeScholar, a tool that generates idiomatic code examples demonstrating the common usage of API methods. It includes a novel neural-guided search technique over graphs that grows the query APIs into idiomatic code examples. Our user study demonstrates that in 70% of cases, developers prefer CodeScholar generated examples over state-of-the-art large language models (LLM) like GPT3.5. We quantitatively evaluate 60 single and 25 multi-API queries from 6 popular Python libraries and show that across-the-board CodeScholar generates more realistic, diverse, and concise examples. In addition, we show that CodeScholar not only helps developers but also LLM-powered programming assistants generate correct code in a program synthesis setting.
翻译:程序员常需检索API方法的使用示例。能够为单个或多个API生成真实、地道且上下文相关使用示例的工具,将对开发者产生极大助益。此类工具可免除深入理解API体系结构的需要,增强现有文档,并帮助发现API间的交互关系。我们提出了CodeScholar——一个能够生成API方法常见使用方式的地道代码示例的工具。该工具采用基于图结构的新型神经引导搜索技术,将查询的API逐步生长为地道代码示例。用户研究表明,在70%的案例中,开发者更偏好CodeScholar生成的示例而非GPT3.5等先进大语言模型。我们对来自6个流行Python库的60个单API查询与25个多API查询进行量化评估,显示CodeScholar在整体上能生成更真实、多样且简洁的示例。此外,我们还证明CodeScholar不仅能协助开发者,还能在程序合成场景中帮助基于LLM的编程助手生成正确代码。