From movie characters to modern science fiction - bringing characters into interactive, story-driven conversations has captured imaginations across generations. Achieving this vision is highly challenging and requires much more than just language modeling. It involves numerous complex AI challenges, such as conversational AI, maintaining character integrity, managing personality and emotions, handling knowledge and memory, synthesizing voice, generating animations, enabling real-world interactions, and integration with physical environments. Recent advancements in the development of foundation models, prompt engineering, and fine-tuning for downstream tasks have enabled researchers to address these individual challenges. However, combining these technologies for interactive characters remains an open problem. We present a system and platform for conveniently designing believable digital characters, enabling a conversational and story-driven experience while providing solutions to all of the technical challenges. As a proof-of-concept, we introduce Digital Einstein, which allows users to engage in conversations with a digital representation of Albert Einstein about his life, research, and persona. While Digital Einstein exemplifies our methods for a specific character, our system is flexible and generalizes to any story-driven or conversational character. By unifying these diverse AI components into a single, easy-to-adapt platform, our work paves the way for immersive character experiences, turning the dream of lifelike, story-based interactions into a reality.

翻译：从电影角色到现代科幻作品——将角色带入互动式、故事驱动的对话中，这一愿景世代以来一直激发着人们的想象力。实现这一愿景极具挑战性，远不止语言建模那么简单。它涉及众多复杂的人工智能挑战，例如对话式AI、保持角色一致性、管理个性与情绪、处理知识与记忆、语音合成、动画生成、实现现实世界交互以及与物理环境集成。近期，基础模型的发展、提示工程以及针对下游任务的微调等方面的进步，已使研究人员能够应对这些独立的挑战。然而，将这些技术整合用于交互式角色仍然是一个悬而未决的问题。我们提出一个用于便捷设计可信数字角色的系统与平台，在提供所有技术挑战解决方案的同时，实现对话式和故事驱动的体验。作为概念验证，我们介绍了Digital Einstein，它允许用户与阿尔伯特·爱因斯坦的数字化身就其生平、研究和人格进行对话。虽然Digital Einstein展示了我们针对特定角色的方法，但我们的系统具有灵活性，可推广至任何故事驱动或对话式角色。通过将这些多样的人工智能组件统一到一个易于适配的单一平台中，我们的工作为沉浸式角色体验铺平了道路，将栩栩如生、基于故事的互动梦想变为现实。