Passive, compact, single-shot 3D sensing is useful in many application areas such as microscopy, medical imaging, surgical navigation, and autonomous driving where form factor, time, and power constraints can exist. Obtaining RGB-D scene information over a short imaging distance, in an ultra-compact form factor, and in a passive, snapshot manner is challenging. Dual-pixel (DP) sensors are a potential solution to achieve the same. DP sensors collect light rays from two different halves of the lens in two interleaved pixel arrays, thus capturing two slightly different views of the scene, like a stereo camera system. However, imaging with a DP sensor implies that the defocus blur size is directly proportional to the disparity seen between the views. This creates a trade-off between disparity estimation vs. deblurring accuracy. To improve this trade-off effect, we propose CADS (Coded Aperture Dual-Pixel Sensing), in which we use a coded aperture in the imaging lens along with a DP sensor. In our approach, we jointly learn an optimal coded pattern and the reconstruction algorithm in an end-to-end optimization setting. Our resulting CADS imaging system demonstrates improvement of >1.5dB PSNR in all-in-focus (AIF) estimates and 5-6% in depth estimation quality over naive DP sensing for a wide range of aperture settings. Furthermore, we build the proposed CADS prototypes for DSLR photography settings and in an endoscope and a dermoscope form factor. Our novel coded dual-pixel sensing approach demonstrates accurate RGB-D reconstruction results in simulations and real-world experiments in a passive, snapshot, and compact manner.

翻译：被动式、紧凑型单次三维传感在存在尺寸、时间和功率约束的诸多应用领域（如显微镜、医学成像、手术导航和自动驾驶）中具有重要价值。在超紧凑外形下以被动、快照方式获取短距离RGB-D场景信息极具挑战性。双像素（DP）传感器是解决该问题的潜在方案。DP传感器通过两个交错像素阵列分别收集来自镜头两个半部分的光线，从而像立体相机系统一样捕获场景的两个略微不同的视角。然而，使用DP传感器成像意味着散焦模糊尺寸与视图间的视差呈正比，这导致视差估计与去模糊精度之间存在权衡。为改善这一权衡效应，我们提出CADS（编码孔径双像素传感），在成像镜头中采用编码孔径与DP传感器协同工作。该方法通过端到端联合优化学习最优编码图案与重建算法。实验表明，与原始DP传感相比，我们的CADS成像系统在全聚焦（AIF）估计上PSNR提升超过1.5dB，在多种光圈设置下的深度估计质量提升5-6%。此外，我们构建了适用于单反摄影设置的CADS原型，并开发了内窥镜和皮肤镜形态的样机。这种创新的编码双像素传感方法在模拟与真实实验中，以被动、快照和紧凑的方式实现了精准的RGB-D重建结果。