Correcting the optical aberrations and the manufacturing deviations of cameras is a challenging task. Due to the limitation on volume and the demand for mass production, existing mobile terminals cannot rectify optical degradation. In this work, we systematically construct the perturbed lens system model to illustrate the relationship between the deviated system parameters and the spatial frequency response measured from photographs. To further address this issue, an optimization framework is proposed based on this model to build proxy cameras from the machining samples' SFRs. Engaging with the proxy cameras, we synthetic data pairs, which encode the optical aberrations and the random manufacturing biases, for training the learning-based algorithms. In correcting aberration, although promising results have been shown recently with convolutional neural networks, they are hard to generalize to stochastic machining biases. Therefore, we propose a dilated Omni-dimensional dynamic convolution and implement it in post-processing to account for the manufacturing degradation. Extensive experiments which evaluate multiple samples of two representative devices demonstrate that the proposed optimization framework accurately constructs the proxy camera. And the dynamic processing model is well-adapted to manufacturing deviations of different cameras, realizing perfect computational photography. The evaluation shows that the proposed method bridges the gap between optical design, system machining, and post-processing pipeline, shedding light on the joint of image signal reception (lens and sensor) and image signal processing.
翻译:校正相机光学像差和制造偏差是一项具有挑战性的任务。由于体积限制和大规模生产需求,现有移动终端无法校正光学退化。本文系统构建了扰动透镜系统模型,以阐述偏差系统参数与从照片中测量的空间频率响应之间的关系。为进一步解决该问题,基于该模型提出了一种优化框架,用于从加工样本的SFR中构建代理相机。通过与代理相机交互,我们合成了编码光学像差和随机制造偏差的数据对,用于训练基于学习算法。在像差校正方面,尽管近期卷积神经网络展现了有希望的结果,但难以泛化至随机加工偏差。因此,我们提出了一种膨胀全维度动态卷积,并将其应用于后处理以补偿制造退化。针对两款代表性设备的多个样本进行的大量实验表明,所提出的优化框架能准确构建代理相机。同时,该动态处理模型能良好适应不同相机的制造偏差,实现了完美的计算摄影。评估结果显示,所提方法弥合了光学设计、系统加工和后处理流水线之间的鸿沟,为图像信号接收(镜头与传感器)与图像信号处理的联合提供了启示。