With the advent of virtual reality technology, omnidirectional image (ODI) rescaling techniques are increasingly embraced for reducing transmitted and stored file sizes while preserving high image quality. Despite this progress, current ODI rescaling methods predominantly focus on enhancing the quality of images in equirectangular projection (ERP) format, which overlooks the fact that the content viewed on head mounted displays (HMDs) is actually a rendered viewport instead of an ERP image. In this work, we emphasize that focusing solely on ERP quality results in inferior viewport visual experiences for users. Thus, we propose ResVR, which is the first comprehensive framework for the joint Rescaling and Viewport Rendering of ODIs. ResVR allows obtaining LR ERP images for transmission while rendering high-quality viewports for users to watch on HMDs. In our ResVR, a novel discrete pixel sampling strategy is developed to tackle the complex mapping between the viewport and ERP, enabling end-to-end training of ResVR pipeline. Furthermore, a spherical pixel shape representation technique is innovatively derived from spherical differentiation to significantly improve the visual quality of rendered viewports. Extensive experiments demonstrate that our ResVR outperforms existing methods in viewport rendering tasks across different fields of view, resolutions, and view directions while keeping a low transmission overhead.

翻译：随着虚拟现实技术的发展，全景图像缩放技术因其在保持高图像质量的同时减少传输和存储文件大小的能力而日益受到青睐。然而，现有全景图像缩放方法主要致力于提升等矩形投影格式的图像质量，这忽略了头戴式显示器上显示的内容实际上是渲染后的视口而非等矩形投影图像这一事实。本工作强调，仅关注等矩形投影图像质量会导致用户视口视觉体验下降。为此，我们提出ResVR——首个面向全景图像联合缩放与视口渲染的综合框架。ResVR支持在传输低分辨率等矩形投影图像的同时，为头戴式显示器用户渲染高质量视口。该框架开发了一种新颖的离散像素采样策略，用于解决视口与等矩形投影之间的复杂映射关系，从而实现ResVR流水线的端到端训练。此外，通过球形微分创新性地推导出球形像素形状表示技术，显著提升了渲染视口的视觉质量。大量实验表明，在不同视场角、分辨率和视角的视口渲染任务中，ResVR在保持低传输开销的同时性能优于现有方法。