We consider the problem of realistic bokeh rendering from a single all-in-focus image. Bokeh rendering mimics aesthetic shallow depth-of-field (DoF) in professional photography, but these visual effects generated by existing methods suffer from simple flat background blur and blurred in-focus regions, giving rise to unrealistic rendered results. In this work, we argue that realistic bokeh rendering should (i) model depth relations and distinguish in-focus regions, (ii) sustain sharp in-focus regions, and (iii) render physically accurate Circle of Confusion (CoC). To this end, we present a Defocus to Focus (D2F) framework to learn realistic bokeh rendering by fusing defocus priors with the all-in-focus image and by implementing radiance priors in layered fusion. Since no depth map is provided, we introduce defocus hallucination to integrate depth by learning to focus. The predicted defocus map implies the blur amount of bokeh and is used to guide weighted layered rendering. In layered rendering, we fuse images blurred by different kernels based on the defocus map. To increase the reality of the bokeh, we adopt radiance virtualization to simulate scene radiance. The scene radiance used in weighted layered rendering reassigns weights in the soft disk kernel to produce the CoC. To ensure the sharpness of in-focus regions, we propose to fuse upsampled bokeh images and original images. We predict the initial fusion mask from our defocus map and refine the mask with a deep network. We evaluate our model on a large-scale bokeh dataset. Extensive experiments show that our approach is capable of rendering visually pleasing bokeh effects in complex scenes. In particular, our solution receives the runner-up award in the AIM 2020 Rendering Realistic Bokeh Challenge.

翻译：本文探讨从单张全焦点图像生成真实感景深效果的问题。景深渲染旨在模拟专业摄影中的美学性浅景深效果，但现有方法生成的视觉效果存在简单的平面背景模糊及焦点区域模糊问题，导致渲染结果失真。本文论证真实感景深渲染应满足：(i) 建模深度关系并区分焦点区域，(ii) 保持焦点区域清晰，(iii) 渲染物理精确的弥散圆。为此，我们提出散焦至聚焦（D2F）框架，通过融合散焦先验与全焦点图像，并在分层融合中引入辐射先验，学习真实感景深渲染。由于未提供深度图，我们引入散焦幻觉机制，通过聚焦学习整合深度信息。预测的散焦图隐含景深模糊量，用于引导加权分层渲染。在分层渲染中，我们基于散焦图融合经不同模糊核处理的图像。为增强景深真实感，采用辐射虚拟化技术模拟场景辐射。场景辐射在加权分层渲染中用于重新分配软盘核的权重以生成弥散圆。为保障焦点区域锐度，提出融合上采样景深图与原始图像，从散焦图预测初始融合掩模并通过深度网络优化。我们在大规模景深数据集上评估模型，大量实验表明该方法能渲染复杂场景中令人愉悦的景深效果。特别地，本方案在AIM 2020真实感景深渲染挑战赛中获得亚军。