Automatic image colorization is a challenging problem. Due to the high illness and multi-modal uncertainty, directly training a deep neural network usually leads to incorrect semantic colors and low color richness. Recent transformer-based methods can deliver better results, but they often rely on manually designed priors, which are hard to implement and suffer from poor generalization ability. Moreover, they tend to introduce serious color bleeding effects since color attention is performed on single-scale features, thus fail to exploit sufficient semantic information. To address these issues, we propose DDColor, a new end-to-end method with dual decoders for image colorization. Our approach includes a multi-scale image decoder and a transformer-based color decoder. The former restores the spatial resolution of the image, while the latter establishes the correlation between color and semantic representations via cross-attention. Rather than using additional priors, our two decoders work together to leverage multi-scale image features to guide optimization of adaptive color queries, significantly alleviating color bleeding effects. In addition, a simple yet effective colorfulness loss is introduced to further enhance the color richness of generated results. Our extensive experiments demonstrate that DDColor achieves significantly superior performance to existing state-of-the-art works both quantitatively and qualitatively. Codes will be made publicly available at https://github.com/piddnad/DDColor.

翻译：自动图像着色是一项具有挑战性的问题。由于高度病态性和多模态不确定性，直接训练深度神经网络通常会导致错误的语义颜色和较低的颜色丰富度。近年来基于Transformer的方法虽然可以取得更好的结果，但它们通常依赖人工设计的先验知识，实现困难且泛化能力较差。此外，由于颜色注意力机制在单尺度特征上执行，这些方法往往难以充分利用充分的语义信息，容易引入严重的颜色渗色效应。为解决这些问题，我们提出DDColor，一种用于图像着色的全新端到端双解码器方法。我们的方法包括多尺度图像解码器和基于Transformer的颜色解码器。前者恢复图像的空间分辨率，而后者通过交叉注意力建立颜色与语义表示之间的关联。我们的两个解码器无需额外先验知识，共同利用多尺度图像特征来指导自适应颜色查询的优化，显著减轻了颜色渗色效应。此外，我们引入了一种简单而有效的色彩丰富度损失函数，进一步增强生成结果的色彩丰富度。大量实验表明，DDColor在定量和定性指标上均显著优于现有最先进方法。代码将在https://github.com/piddnad/DDColor 公开提供。