This paper firstly presents old photo modernization using multiple references by performing stylization and enhancement in a unified manner. In order to modernize old photos, we propose a novel multi-reference-based old photo modernization (MROPM) framework consisting of a network MROPM-Net and a novel synthetic data generation scheme. MROPM-Net stylizes old photos using multiple references via photorealistic style transfer (PST) and further enhances the results to produce modern-looking images. Meanwhile, the synthetic data generation scheme trains the network to effectively utilize multiple references to perform modernization. To evaluate the performance, we propose a new old photos benchmark dataset (CHD) consisting of diverse natural indoor and outdoor scenes. Extensive experiments show that the proposed method outperforms other baselines in performing modernization on real old photos, even though no old photos were used during training. Moreover, our method can appropriately select styles from multiple references for each semantic region in the old photo to further improve the modernization performance.

翻译：本文首次提出通过统一方式实现老照片的风格化与增强，从而利用多重参考完成老照片现代化。为实现老照片现代化，我们提出一种新型的基于多重参考的老照片现代化（MROPM）框架，该框架包含一个MROPM-Net网络和一种新颖的合成数据生成方案。MROPM-Net通过逼真风格迁移（PST）利用多重参考对老照片进行风格化处理，并进一步增强结果以生成具有现代感的图像。与此同时，合成数据生成方案训练网络有效利用多重参考执行现代化操作。为评估性能，我们提出一个包含多样化自然室内外场景的新型老照片基准数据集（CHD）。大量实验表明，尽管训练过程中未使用任何真实老照片，所提方法在真实老照片现代化任务上仍优于其他基线方法。此外，我们的方法能够从多重参考中为老照片的每个语义区域合理选择风格，从而进一步提升现代化性能。