We present a novel framework for rectifying occlusions and distortions in degraded texture samples from natural images. Traditional texture synthesis approaches focus on generating textures from pristine samples, which necessitate meticulous preparation by humans and are often unattainable in most natural images. These challenges stem from the frequent occlusions and distortions of texture samples in natural images due to obstructions and variations in object surface geometry. To address these issues, we propose a framework that synthesizes holistic textures from degraded samples in natural images, extending the applicability of exemplar-based texture synthesis techniques. Our framework utilizes a conditional Latent Diffusion Model (LDM) with a novel occlusion-aware latent transformer. This latent transformer not only effectively encodes texture features from partially-observed samples necessary for the generation process of the LDM, but also explicitly captures long-range dependencies in samples with large occlusions. To train our model, we introduce a method for generating synthetic data by applying geometric transformations and free-form mask generation to clean textures. Experimental results demonstrate that our framework significantly outperforms existing methods both quantitatively and quantitatively. Furthermore, we conduct comprehensive ablation studies to validate the different components of our proposed framework. Results are corroborated by a perceptual user study which highlights the efficiency of our proposed approach.

翻译：我们提出了一种新颖的框架，用于修复自然图像中退化纹理样本的遮挡与畸变。传统的纹理合成方法专注于从纯净样本生成纹理，这需要人为精心准备，而大多数自然图像往往无法提供此类样本。这些挑战源于自然图像中纹理样本因遮挡和物体表面几何变化而频繁出现的遮挡与畸变。为解决这些问题，我们提出了一种从自然图像中的退化样本合成整体纹理的框架，从而扩展了基于示例的纹理合成技术的应用范围。该框架利用条件潜在扩散模型（LDM），并配备了一种新颖的遮挡感知潜在变换器。该潜在变换器不仅能够有效编码部分观测样本中的纹理特征以支持LDM的生成过程，还能显式捕获存在大范围遮挡的样本中的长距离依赖关系。为训练我们的模型，我们引入了一种通过几何变换和自由形式掩码生成对干净纹理进行合成数据生成的方法。实验结果表明，我们的框架在定性和定量指标上均显著优于现有方法。此外，我们进行了全面的消融研究以验证所提框架中不同组件的有效性。结果通过一项感知用户研究得到佐证，突显了我们所提方法的效率。