Multi-view inverse rendering aims to recover geometry, materials, and illumination consistently across multiple viewpoints. When applied to multi-view images, existing single-view approaches often ignore cross-view relationships, leading to inconsistent results. In contrast, multi-view optimization methods rely on slow differentiable rendering and per-scene refinement, making them computationally expensive and hard to scale. To address these limitations, we introduce a feed-forward multi-view inverse rendering framework that directly predicts spatially varying albedo, metallic, roughness, diffuse shading, and surface normals from sequences of RGB images. By alternating attention across views, our model captures both intra-view long-range lighting interactions and inter-view material consistency, enabling coherent scene-level reasoning within a single forward pass. Due to the scarcity of real-world training data, models trained on existing synthetic datasets often struggle to generalize to real-world scenes. To overcome this limitation, we propose a consistency-based finetuning strategy that leverages unlabeled real-world videos to enhance both multi-view coherence and robustness under in-the-wild conditions. Extensive experiments on benchmark datasets demonstrate that our method achieves state-of-the-art performance in terms of multi-view consistency, material and normal estimation quality, and generalization to real-world imagery. Project page: https://maddog241.github.io/mvinverse-page/

翻译：多视角逆渲染旨在从多个视角一致地恢复几何、材质与光照。现有单视角方法应用于多视角图像时，常忽略视角间关联，导致结果不一致。相比之下，多视角优化方法依赖缓慢的可微分渲染与逐场景优化，计算成本高昂且难以扩展。为克服这些局限，我们提出一种前馈式多视角逆渲染框架，可直接从RGB图像序列预测空间变化的漫反射率、金属度、粗糙度、漫射着色与表面法线。通过跨视角交替注意力机制，我们的模型同时捕捉视角内长距离光照交互与视角间材质一致性，实现在单次前向传播中完成连贯的场景级推理。由于真实世界训练数据稀缺，基于现有合成数据集训练的模型常难以泛化至真实场景。为此，我们提出一种基于一致性的微调策略，利用未标注的真实世界视频增强多视角连贯性与野外条件下的鲁棒性。在基准数据集上的大量实验表明，本方法在多视角一致性、材质与法线估计质量、以及真实图像泛化能力方面均达到最先进性能。项目页面：https://maddog241.github.io/mvinverse-page/