The joint design of the optical system and the downstream algorithm is a challenging and promising task. Due to the demand for balancing the global optimal of imaging systems and the computational cost of physical simulation, existing methods cannot achieve efficient joint design of complex systems such as smartphones and drones. In this work, starting from the perspective of the optical design, we characterize the optics with separated aberrations. Additionally, to bridge the hardware and software without gradients, an image simulation system is presented to reproduce the genuine imaging procedure of lenses with large field-of-views. As for aberration correction, we propose a network to perceive and correct the spatially varying aberrations and validate its superiority over state-of-the-art methods. Comprehensive experiments reveal that the preference for correcting separated aberrations in joint design is as follows: longitudinal chromatic aberration, lateral chromatic aberration, spherical aberration, field curvature, and coma, with astigmatism coming last. Drawing from the preference, a 10% reduction in the total track length of the consumer-level mobile phone lens module is accomplished. Moreover, this procedure spares more space for manufacturing deviations, realizing extreme-quality enhancement of computational photography. The optimization paradigm provides innovative insight into the practical joint design of sophisticated optical systems and post-processing algorithms.

翻译：光学系统与下游算法的联合设计是一项具有挑战性且前景广阔的任务。由于需要平衡成像系统的全局最优性与物理仿真的计算成本，现有方法无法实现如智能手机和无人机等复杂系统的高效联合设计。本研究从光学设计角度出发，以分离像差表征光学系统。此外，为在不依赖梯度的情况下衔接硬件与软件，提出了一种图像仿真系统，用于复现大视场镜头的真实成像过程。针对像差校正，我们提出了一种能够感知并校正空间变化像差的网络，并验证其相较于现有最优方法的优越性。综合实验揭示了联合设计中分离像差校正的偏好顺序：纵向色差、横向色差、球差、场曲、彗差，而像散位列最后。基于此偏好，消费级手机镜头模组的总轨迹长度减少了10%。此外，该流程为制造偏差留出更多余量，实现了计算摄影的极致质量提升。该优化范式为复杂光学系统与后处理算法的实际联合设计提供了创新性见解。