Implicit Neural Representations (INRs) approximate discrete data through continuous functions and are commonly used for encoding 2D images. Traditional image-based INRs employ neural networks to map pixel coordinates to RGB values, capturing shapes, colors, and textures within the network's weights. Recently, GaussianImage has been proposed as an alternative, using Gaussian functions instead of neural networks to achieve comparable quality and compression. Such a solution obtains a quality and compression ratio similar to classical INR models but does not allow image modification. In contrast, our work introduces a novel method, MiraGe, which uses mirror reflections to perceive 2D images in 3D space and employs flat-controlled Gaussians for precise 2D image editing. Our approach improves the rendering quality and allows realistic image modifications, including human-inspired perception of photos in the 3D world. Thanks to modeling images in 3D space, we obtain the illusion of 3D-based modification in 2D images. We also show that our Gaussian representation can be easily combined with a physics engine to produce physics-based modification of 2D images. Consequently, MiraGe allows for better quality than the standard approach and natural modification of 2D images.
翻译:隐式神经表示(INRs)通过连续函数逼近离散数据,常用于编码二维图像。传统的基于图像的INRs采用神经网络将像素坐标映射到RGB值,将形状、颜色和纹理信息捕获于网络权重中。近期,GaussianImage被提出作为一种替代方案,它使用高斯函数而非神经网络,实现了可比的图像质量与压缩率。此类方法在质量与压缩比方面与经典INR模型相似,但不支持图像修改。相比之下,我们的工作提出了一种新颖方法MiraGe,该方法利用镜面反射在三维空间中感知二维图像,并采用平面控制的高斯函数实现精确的二维图像编辑。我们的方法提升了渲染质量,并支持逼真的图像修改,包括对人类在三维世界中感知照片方式的模拟。得益于在三维空间中对图像的建模,我们实现了在二维图像中进行基于三维修改的视觉错觉。我们还证明,我们的高斯表示可以轻松与物理引擎结合,实现基于物理原理的二维图像修改。因此,MiraGe在实现优于标准方法的质量的同时,支持对二维图像进行自然的修改。