High-quality panoramic images with a Field of View (FoV) of 360-degree are essential for contemporary panoramic computer vision tasks. However, conventional imaging systems come with sophisticated lens designs and heavy optical components. This disqualifies their usage in many mobile and wearable applications where thin and portable, minimalist imaging systems are desired. In this paper, we propose a Panoramic Computational Imaging Engine (PCIE) to address minimalist and high-quality panoramic imaging. With less than three spherical lenses, a Minimalist Panoramic Imaging Prototype (MPIP) is constructed based on the design of the Panoramic Annular Lens (PAL), but with low-quality imaging results due to aberrations and small image plane size. We propose two pipelines, i.e. Aberration Correction (AC) and Super-Resolution and Aberration Correction (SR&AC), to solve the image quality problems of MPIP, with imaging sensors of small and large pixel size, respectively. To provide a universal network for the two pipelines, we leverage the information from the Point Spread Function (PSF) of the optical system and design a PSF-aware Aberration-image Recovery Transformer (PART), in which the self-attention calculation and feature extraction are guided via PSF-aware mechanisms. We train PART on synthetic image pairs from simulation and put forward the PALHQ dataset to fill the gap of real-world high-quality PAL images for low-level vision. A comprehensive variety of experiments on synthetic and real-world benchmarks demonstrates the impressive imaging results of PCIE and the effectiveness of plug-and-play PSF-aware mechanisms. We further deliver heuristic experimental findings for minimalist and high-quality panoramic imaging. Our dataset and code will be available at https://github.com/zju-jiangqi/PCIE-PART.

翻译：具有360度视场角的高质量全景图像对当代全景计算机视觉任务至关重要。然而，传统成像系统需要复杂的光学设计和沉重的光学组件，这限制了其在诸多追求轻薄便携、最小化成像系统的移动与可穿戴设备中的应用。本文提出全景计算成像引擎（PCIE），旨在实现最小化高质量全景成像。基于全景环形透镜（PAL）设计，采用少于三个球面透镜构建最小化全景成像原型（MPIP），但受像差和小像面尺寸影响，成像质量较低。为此，我们提出两条处理流水线：针对小像素尺寸成像传感器的像差校正（AC）流水线，以及针对大像素尺寸成像传感器的超分辨率与像差校正（SR&AC）流水线。为构建通用网络，我们利用光学系统的点扩散函数（PSF）信息，设计PSF感知的像差图像恢复Transformer（PART），其中自注意力计算和特征提取通过PSF感知机制进行引导。我们基于仿真生成的合成图像对训练PART，并构建PALHQ数据集以填补真实高质量PAL图像在底层视觉领域的空白。在合成与真实基准上的大量实验表明，PCIE具有卓越的成像效果，且即插即用的PSF感知机制有效。我们进一步为最小化高质量全景成像提供启发式实验发现。数据集和代码将发布在https://github.com/zju-jiangqi/PCIE-PART。