Modeling, understanding, and reconstructing the real world are crucial in XR/VR. Recently, 3D Gaussian Splatting (3D-GS) methods have shown remarkable success in modeling and understanding 3D scenes. Similarly, various 4D representations have demonstrated the ability to capture the dynamics of the 4D world. However, there is a dearth of research focusing on segmentation within 4D representations. In this paper, we propose Segment Any 4D Gaussians (SA4D), one of the first frameworks to segment anything in the 4D digital world based on 4D Gaussians. In SA4D, an efficient temporal identity feature field is introduced to handle Gaussian drifting, with the potential to learn precise identity features from noisy and sparse input. Additionally, a 4D segmentation refinement process is proposed to remove artifacts. Our SA4D achieves precise, high-quality segmentation within seconds in 4D Gaussians and shows the ability to remove, recolor, compose, and render high-quality anything masks. More demos are available at: https://jsxzs.github.io/sa4d/.

翻译：建模、理解与重建真实世界在XR/VR领域至关重要。近年来，3D高斯泼溅（3D-GS）方法在三维场景建模与理解方面展现出显著成效。类似地，多种四维表征已证明能够捕捉四维世界的动态特性。然而，目前针对四维表征内部分割任务的研究仍显不足。本文提出分割任意4D高斯模型（SA4D），这是首个基于四维高斯表征实现四维数字世界中任意对象分割的框架之一。SA4D引入了高效的时序身份特征场以处理高斯漂移问题，该机制具备从含噪稀疏输入中学习精确身份特征的潜力。此外，我们提出了四维分割优化流程以消除伪影。SA4D可在数秒内实现四维高斯表征的精准高质量分割，并展现出移除、重着色、组合及渲染高质量任意对象掩码的能力。更多演示请访问：https://jsxzs.github.io/sa4d/。