Recent progress in inpainting increasingly relies on generative models, leveraging their strong generation capabilities for addressing ill-conditioned problems. However, this enhanced generation often introduces instability, leading to arbitrary object generation within masked regions. This paper proposes a balanced solution, emphasizing the importance of unmasked regions in guiding inpainting while preserving generative capacity. Our approach, Aligned Stable Inpainting with UnKnown Areas Prior (ASUKA), employs a reconstruction-based masked auto-encoder (MAE) as a stable prior. Aligned with the robust Stable Diffusion inpainting model (SD), ASUKA significantly improves inpainting stability. ASUKA further aligns masked and unmasked regions through an inpainting-specialized decoder, ensuring more faithful inpainting. To validate effectiveness across domains and masking scenarios, we evaluate on MISATO, a collection of several existing dataset. Results confirm ASUKA's efficacy in both stability and fidelity compared to SD and other inpainting algorithms.
翻译:近年来,图像修复的进展日益依赖生成模型,利用其强大的生成能力来解决病态问题。然而,这种增强的生成能力往往带来不稳定性,导致遮罩区域内生成任意对象。本文提出一种平衡解决方案,强调未遮罩区域在引导修复过程中的重要性,同时保持生成能力。我们的方法——基于未知区域先验的对齐稳定图像修复(ASUKA),采用基于重建的遮罩自编码器(MAE)作为稳定先验。通过与鲁棒的稳定扩散修复模型(SD)对齐,ASUKA显著提升了修复稳定性。ASUKA进一步通过专用修复解码器对齐遮罩与未遮罩区域,确保更保真的修复效果。为验证方法在不同领域和遮罩场景下的有效性,我们在MISATO(多个现有数据集集合)上进行评估。结果表明,相较于SD及其他修复算法,ASUKA在稳定性和保真度方面均展现出优越性能。