The bokeh effect is an artistic technique that blurs out-of-focus areas in a photograph and has gained interest due to recent developments in text-to-image synthesis and the ubiquity of smart-phone cameras and photo-sharing apps. Prior work on rendering bokeh effects have focused on post hoc image manipulation to produce similar blurring effects in existing photographs using classical computer graphics or neural rendering techniques, but have either depth discontinuity artifacts or are restricted to reproducing bokeh effects that are present in the training data. More recent diffusion based models can synthesize images with an artistic style, but either require the generation of high-dimensional masks, expensive fine-tuning, or affect global image characteristics. In this paper, we present GBSD, the first generative text-to-image model that synthesizes photorealistic images with a bokeh style. Motivated by how image synthesis occurs progressively in diffusion models, our approach combines latent diffusion models with a 2-stage conditioning algorithm to render bokeh effects on semantically defined objects. Since we can focus the effect on objects, this semantic bokeh effect is more versatile than classical rendering techniques. We evaluate GBSD both quantitatively and qualitatively and demonstrate its ability to be applied in both text-to-image and image-to-image settings.

翻译：散景效果是一种对照片中焦外区域进行模糊处理的艺术手法，随着文本到图像合成技术的进展以及智能手机和照片分享应用的普及，该技术日益受到关注。现有散景效果渲染研究主要聚焦于利用经典计算机图形学或神经渲染技术对现有照片进行事后图像操作来产生类似模糊效果，但此类方法存在深度不连续伪影或受限于对训练数据中散景效果的复现。近年来基于扩散的模型虽能合成具有艺术风格的图像，但或需生成高维掩码、进行昂贵微调，或会影响图像全局特征。本文提出GBSD——首个能合成具有散景风格逼真图像的生成式文本到图像模型。受扩散模型图像渐进合成机制启发，本方法将潜在扩散模型与两阶段条件算法相结合，在语义定义对象上渲染散景效果。由于可将效果聚焦于特定对象，这种语义散景效果比经典渲染技术更具通用性。我们通过定量与定性评估验证了GBSD的性能，并展示其在文本到图像和图像到图像场景中的应用能力。