Smartphone cameras today are increasingly approaching the versatility and quality of professional cameras through a combination of hardware and software advancements. However, fixed aperture remains a key limitation, preventing users from controlling the depth of field (DoF) of captured images. At the same time, many smartphones now have multiple cameras with different fixed apertures - specifically, an ultra-wide camera with wider field of view and deeper DoF and a higher resolution primary camera with shallower DoF. In this work, we propose $\text{DC}^2$, a system for defocus control for synthetically varying camera aperture, focus distance and arbitrary defocus effects by fusing information from such a dual-camera system. Our key insight is to leverage real-world smartphone camera dataset by using image refocus as a proxy task for learning to control defocus. Quantitative and qualitative evaluations on real-world data demonstrate our system's efficacy where we outperform state-of-the-art on defocus deblurring, bokeh rendering, and image refocus. Finally, we demonstrate creative post-capture defocus control enabled by our method, including tilt-shift and content-based defocus effects.
翻译:如今,智能手机摄像头通过软硬件协同进步,正日益接近专业相机的多功能性与成像质量。然而,固定光圈仍是关键限制,导致用户无法控制拍摄图像的景深。与此同时,许多智能手机已配备多个固定光圈摄像头——具体而言,包括广视角、大景深的超广角摄像头,以及窄景深、高分辨率的主摄像头。本文提出 $\text{DC}^2$ 系统,通过融合此类双摄像头系统的信息,实现合成可变光圈、对焦距离及任意散焦效果的散焦控制。我们的核心洞察在于利用真实智能手机摄像头数据集,将图像重对焦作为学习散焦控制的代理任务。基于真实数据的定性与定量评估表明,本系统在散焦去模糊、散景渲染及图像重对焦任务上均超越现有最优方法。最后,我们展示了该方法所实现的创意性后期散焦控制,包括移轴与基于内容的散焦效果。