SAP has released its own proprietary generative model SAP Joule, intended for various generative tasks, including serving as a code assistant for software engineers. While Joule is yet not focused on SAP-specific ABAP code generation, it can be used for other common languages, including Javascript. This paper compares SAP Joules Javascript coding capabilities against a total of 29 other models using the HumanEval-X Javascript benchmark. SAP Joule achieves a strict accuracy of 80.49% as the fifth best model in our evaluation. To the best of our knowledge, this is the first comparative evaluation of SAP Joule code generation capabilities.

翻译：SAP发布了其专有的生成模型SAP Joule，旨在处理多种生成任务，包括作为软件工程师的代码助手。尽管Joule目前尚未专注于SAP特定的ABAP代码生成，但它可用于其他常见语言，包括JavaScript。本文使用HumanEval-X JavaScript基准测试，将SAP Joule的JavaScript编码能力与共计29个其他模型进行了比较。SAP Joule以80.49%的严格准确率在我们的评估中位列第五。据我们所知，这是首次对SAP Joule代码生成能力进行的比较性评估。