Immersive telepresence aims to transform human interaction in AR/VR applications by enabling lifelike full-body holographic representations for enhanced remote collaboration. However, existing systems rely on hardware-intensive multi-camera setups and demand high bandwidth for volumetric streaming, limiting their real-time performance on mobile devices. To overcome these challenges, we propose Mon3tr, a novel Monocular 3D telepresence framework that integrates 3D Gaussian splatting (3DGS) based parametric human modeling into telepresence for the first time. Mon3tr adopts an amortized computation strategy, dividing the process into a one-time offline multi-view reconstruction phase to build a user-specific avatar and a monocular online inference phase during live telepresence sessions. A single monocular RGB camera is used to capture body motions and facial expressions in real time to drive the 3DGS-based parametric human model, significantly reducing system complexity and cost. The extracted motion and appearance features are transmitted at < 0.2 Mbps over WebRTC's data channel, allowing robust adaptation to network fluctuations. On the receiver side, e.g., Meta Quest 3, we develop a lightweight 3DGS attribute deformation network to dynamically generate corrective 3DGS attribute adjustments on the pre-built avatar, synthesizing photorealistic motion and appearance at ~ 60 FPS. Extensive experiments demonstrate the state-of-the-art performance of our method, achieving a PSNR of > 28 dB for novel poses, an end-to-end latency of ~ 80 ms, and > 1000x bandwidth reduction compared to point-cloud streaming, while supporting real-time operation from monocular inputs across diverse scenarios. Our demos can be found at https://mon3tr3d.github.io.

翻译：沉浸式远程呈现旨在通过实现逼真的全身全息表示来增强远程协作，从而变革AR/VR应用中的人际交互。然而，现有系统依赖硬件密集的多相机配置，且需要高带宽进行体积流传输，限制了其在移动设备上的实时性能。为克服这些挑战，我们提出Mon3tr——一种创新的单目三维远程呈现框架，首次将基于三维高斯溅射（3DGS）的参数化人体建模集成到远程呈现中。Mon3tr采用分期计算策略，将流程划分为一次性离线多视角重建阶段（用于构建用户专属化身）和实时远程呈现会话中的单目在线推理阶段。系统仅需单个单目RGB相机实时捕捉身体动作与面部表情来驱动基于3DGS的参数化人体模型，显著降低了系统复杂度与成本。提取的运动与外观特征通过WebRTC数据通道以<0.2 Mbps的速率传输，可稳健适应网络波动。在接收端（如Meta Quest 3），我们开发了轻量级3DGS属性变形网络，动态生成预建化身上的修正性3DGS属性调整，以约60 FPS的速率合成逼真运动与外观。大量实验证明我们的方法达到业界领先水平：对新姿态的PSNR > 28 dB，端到端延迟约80 ms，相较于点云流传输实现>1000倍的带宽降低，同时支持跨多样场景的单目输入实时操作。演示视频详见https://mon3tr3d.github.io。