As LLM-based applications reach millions of customers, ensuring their scalability and continuous quality improvement is critical for success. However, the current workflows for developing, maintaining, and operating (DevOps) these applications are predominantly manual, slow, and based on trial-and-error. With this paper we introduce the Generative AI Toolkit, which automates essential workflows over the whole life cycle of LLM-based applications. The toolkit helps to configure, test, continuously monitor and optimize Generative AI applications such as agents, thus significantly improving quality while shortening release cycles. We showcase the effectiveness of our toolkit on representative use cases, share best practices, and outline future enhancements. Since we are convinced that our Generative AI Toolkit is helpful for other teams, we are open sourcing it on and hope that others will use, forward, adapt and improve

翻译：随着基于大语言模型（LLM）的应用触达数百万用户，确保其可扩展性与持续质量改进已成为成功的关键。然而，当前这些应用的开发、维护与运维（DevOps）工作流仍主要依赖人工操作，流程缓慢且基于试错方法。本文提出生成式AI工具包，该工具包能够自动化基于LLM应用全生命周期中的核心工作流。该工具包可协助配置、测试、持续监控及优化智能体等生成式AI应用，从而在缩短发布周期的同时显著提升质量。我们通过典型用例展示工具包的有效性，分享最佳实践，并展望未来改进方向。我们深信该生成式AI工具包对其他团队具有重要价值，现已在开源平台发布，期待更多开发者使用、推广、适配并完善该工具。