Implicit Neural Representation (INR) has become a popular method for representing visual signals (e.g., 2D images and 3D scenes), demonstrating promising results in various downstream applications. Given its potential as a medium for visual signals, exploring the development of a neural blending method that utilizes INRs is a natural progression. Neural blending involves merging two INRs to create a new INR that encapsulates information from both original representations. A direct approach involves applying traditional image editing methods to the INR rendering process. However, this method often results in blending distortions, artifacts, and color shifts, primarily due to the discretization of the underlying pixel grid and the introduction of boundary conditions for solving variational problems. To tackle this issue, we introduce the Neural Poisson Solver, a plug-and-play and universally applicable framework across different signal dimensions for blending visual signals represented by INRs. Our Neural Poisson Solver offers a variational problem-solving approach based on the continuous Poisson equation, demonstrating exceptional performance across various domains. Specifically, we propose a gradient-guided neural solver to represent the solution process of the variational problem, refining the target signal to achieve natural blending results. We also develop a Poisson equation-based loss and optimization scheme to train our solver, ensuring it effectively blends the input INR scenes while preserving their inherent structure and semantic content. The lack of dependence on additional prior knowledge makes our method easily adaptable to various task categories, highlighting its versatility. Comprehensive experimental results validate the robustness of our approach across multiple dimensions and blending tasks.
翻译:隐式神经表示(INR)已成为表示视觉信号(如二维图像和三维场景)的流行方法,在各种下游应用中展现出良好前景。鉴于其作为视觉信号媒介的潜力,探索利用INR开发神经融合方法是自然的发展方向。神经融合涉及合并两个INR以创建包含原始表示信息的新INR。直接方法是在INR渲染过程中应用传统图像编辑方法,但该方法常因底层像素网格离散化及变分问题求解中边界条件的引入,导致融合失真、伪影和色彩偏移。为解决此问题,我们提出神经泊松求解器——一个即插即用且适用于不同信号维度的通用框架,用于融合INR表示的视觉信号。该求解器基于连续泊松方程提供变分问题求解方法,在多个领域展现出卓越性能。具体而言,我们提出梯度引导的神经求解器来表示变分问题的求解过程,通过优化目标信号实现自然融合效果。同时开发了基于泊松方程的损失函数与优化方案来训练求解器,确保其在有效融合输入INR场景的同时保持其固有结构与语义内容。该方法无需依赖额外先验知识,可轻松适配多种任务类别,体现了其通用性。综合实验结果验证了该方法在多维度融合任务中的鲁棒性。