The development of conversational AI assistants is an iterative process with multiple components. As such, the evaluation and continual improvement of these assistants is a complex and multifaceted problem. This paper introduces the challenges in evaluating and improving a generative AI assistant for enterprises, which is under active development, and how we address these challenges. We also share preliminary results and discuss lessons learned.

翻译：对话式AI助手的开发是一个包含多个组件的迭代过程。因此，对这些助手进行评估和持续改进是一个复杂且多层面的问题。本文介绍了在评估和改进一个处于积极开发阶段的企业级生成式AI助手时所面临的挑战，以及我们如何应对这些挑战。我们还分享了初步结果并讨论了经验教训。