The joint design of the optical system and the downstream algorithm is a challenging and promising task. Due to the demand for balancing the global optimal of imaging systems and the computational cost of physical simulation, existing methods cannot achieve efficient joint design of complex systems such as smartphones and drones. In this work, starting from the perspective of the optical design, we characterize the optics with separated aberrations. Additionally, to bridge the hardware and software without gradients, an image simulation system is presented to reproduce the genuine imaging procedure of lenses with large field-of-views. As for aberration correction, we propose a network to perceive and correct the spatially varying aberrations and validate its superiority over state-of-the-art methods. Comprehensive experiments reveal that the preference for correcting separated aberrations in joint design is as follows: longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, field curvature, and coma, with astigmatism coming last. Drawing from the preference, a 10% reduction in the total track length of the consumer-level mobile phone lens module is accomplished. Moreover, this procedure spares more space for manufacturing deviations, realizing extreme-quality enhancement of computational photography. The optimization paradigm provides innovative insight into the practical joint design of sophisticated optical systems and post-processing algorithms.

翻译：光学系统与下游算法的联合设计是一项具有挑战性且前景广阔的任务。由于需要在成像系统全局最优与物理仿真计算成本之间寻求平衡，现有方法无法实现智能手机、无人机等复杂系统的高效联合设计。本研究从光学设计视角出发，采用分离像差对光学系统进行表征。为建立无梯度传播的软硬件桥梁，我们提出了一种图像仿真系统，可复现大视场镜头真实成像过程。针对像差校正问题，我们设计了一种能够感知并校正空间变化像差的网络，并验证了其相较于现有最优方法的优越性。综合实验表明，联合设计中分离像差的校正偏好顺序为：纵向色差、横向色差、球差、场曲、彗差，而像散位列最后。基于此偏好，消费级手机镜头模组的总光路长度缩减了10%。此外，该流程为制造偏差预留了更多空间，实现了计算摄影的极致画质提升。该优化范式为复杂光学系统与后处理算法的实用化联合设计提供了创新思路。