For improving image composition and aesthetic quality, most existing methods modulate the captured images by striking out redundant content near the image borders. However, such image cropping methods are limited in the range of image views. Some methods have been suggested to extrapolate the images and predict cropping boxes from the extrapolated image. Nonetheless, the synthesized extrapolated regions may be included in the cropped image, making the image composition result not real and potentially with degraded image quality. In this paper, we circumvent this issue by presenting a joint framework for both unbounded recommendation of camera view and image composition (i.e., UNIC). In this way, the cropped image is a sub-image of the image acquired by the predicted camera view, and thus can be guaranteed to be real and consistent in image quality. Specifically, our framework takes the current camera preview frame as input and provides a recommendation for view adjustment, which contains operations unlimited by the image borders, such as zooming in or out and camera movement. To improve the prediction accuracy of view adjustment prediction, we further extend the field of view by feature extrapolation. After one or several times of view adjustments, our method converges and results in both a camera view and a bounding box showing the image composition recommendation. Extensive experiments are conducted on the datasets constructed upon existing image cropping datasets, showing the effectiveness of our UNIC in unbounded recommendation of camera view and image composition. The source code, dataset, and pretrained models is available at https://github.com/liuxiaoyu1104/UNIC.
翻译:为提升图像构图与美学质量,现有方法大多通过裁剪图像边界附近冗余内容来调整拍摄图像。然而,这类图像裁剪方法在图像视角范围上存在局限。部分研究提出通过外推图像并预测裁剪框来改进,但合成外推区域可能被包含于裁剪图像中,导致构图结果不真实且图像质量下降。本文提出联合框架UNIC(无界相机视角与图像构图推荐),规避上述问题。该方法使裁剪图像始终为预测视角下拍摄图像的子区域,从而保证图像真实性及质量一致性。具体而言,本框架以当前相机预览帧为输入,推荐视角调整方案(包含变焦、相机移动等不受图像边界限制的操作)。为提升视角调整预测精度,我们通过特征外推扩展视野。经过一次或多次视角调整,模型收敛后可同时输出相机视角参数与表示构图推荐的边界框。基于现有图像裁剪数据集构建的数据集上的大量实验表明,UNIC在无界相机视角与图像构图推荐中具备有效性。源代码、数据集及预训练模型已开源至 https://github.com/liuxiaoyu1104/UNIC。