We present Sophia-in-Audition (SiA), a new frontier in virtual production, by employing the humanoid robot Sophia within an UltraStage environment composed of a controllable lighting dome coupled with multiple cameras. We demonstrate Sophia's capability to replicate iconic film segments, follow real performers, and perform a variety of motions and expressions, showcasing her versatility as a virtual actor. Key to this process is the integration of facial motion transfer algorithms and the UltraStage's controllable lighting and multi-camera setup, enabling dynamic performances that align with the director's vision. Our comprehensive user studies indicate positive audience reception towards Sophia's performances, highlighting her potential to reduce the uncanny valley effect in virtual acting. Additionally, the immersive lighting in dynamic clips was highly rated for its naturalness and its ability to mirror professional film standards. The paper presents a first-of-its-kind multi-view robot performance video dataset with dynamic lighting, offering valuable insights for future enhancements in humanoid robotic performers and virtual production techniques. This research contributes significantly to the field by presenting a unique virtual production setup, developing tools for sophisticated performance control, and providing a comprehensive dataset and user study analysis for diverse applications.

翻译：我们提出"试镜中的索菲亚"（SiA）——通过在人形机器人索菲亚上应用由可控照明穹顶与多摄像机组成的超舞台环境，开创了虚拟制作的新前沿。我们展示了索菲亚复刻经典电影片段、跟随真人表演者以及执行多种动作与表情的能力，充分体现了其作为虚拟演员的多才多艺。该流程的关键在于面部运动迁移算法与超舞台可控照明及多摄像机设置的整合，从而实现了与导演构想相契合的动态表演。我们的综合用户研究表明，观众对索菲亚的表演反响积极，凸显了其在减少虚拟表演中恐怖谷效应方面的潜力。此外，动态片段中的沉浸式照明因其自然度及对专业电影水准的还原能力而获得高度评价。本文首次提出了包含动态照明的多视角机器人表演视频数据集，为未来人形机器人表演者与虚拟制作技术的提升提供了宝贵见解。本研究通过呈现独特的虚拟制作设置、开发精细表演控制工具，并提供适用于多样化场景的综合数据集与用户研究分析，为该领域作出了重要贡献。