3D rendering of dynamic face captures is a challenging problem, and it demands improvements on several fronts$\unicode{x2014}$photorealism, efficiency, compatibility, and configurability. We present a novel representation that enables high-quality volumetric rendering of an actor's dynamic facial performances with minimal compute and memory footprint. It runs natively on commodity graphics soft- and hardware, and allows for a graceful trade-off between quality and efficiency. Our method utilizes recent advances in neural rendering, particularly learning discrete radiance manifolds to sparsely sample the scene to model volumetric effects. We achieve efficient modeling by learning a single set of manifolds for the entire dynamic sequence, while implicitly modeling appearance changes as temporal canonical texture. We export a single layered mesh and view-independent RGBA texture video that is compatible with legacy graphics renderers without additional ML integration. We demonstrate our method by rendering dynamic face captures of real actors in a game engine, at comparable photorealism to state-of-the-art neural rendering techniques at previously unseen frame rates.

翻译：动态面部捕捉的三维渲染是一项极具挑战性的问题，需要在真实感、效率、兼容性和可配置性等多个方面取得突破。我们提出了一种新型表示方法，能够以极低的计算量和内存占用，实现演员动态面部表演的高质量体积渲染。该方法可原生运行于商用图形软硬件之上，并允许在质量与效率之间进行优雅权衡。我们的技术利用了神经渲染领域的最新进展，特别是通过学习离散辐射流形对场景进行稀疏采样以建模体积效果。通过为整个动态序列学习单一的一组流形，同时将外观变化隐式建模为时间规范纹理，我们实现了高效建模。我们导出单个分层网格及视图无关的RGBA纹理视频，可直接兼容传统图形渲染管线，无需额外集成机器学习模块。我们在游戏引擎中渲染真实演员的动态面部捕捉数据，以先前未见的高帧率实现了与最先进神经渲染技术相当的真实感效果。