Volumetric video emerges as a new attractive video paradigm in recent years since it provides an immersive and interactive 3D viewing experience with six degree-of-freedom (DoF). Unlike traditional 2D or panoramic videos, volumetric videos require dense point clouds, voxels, meshes, or huge neural models to depict volumetric scenes, which results in a prohibitively high bandwidth burden for video delivery. Users' behavior analysis, especially the viewport and gaze analysis, then plays a significant role in prioritizing the content streaming within users' viewport and degrading the remaining content to maximize user QoE with limited bandwidth. Although understanding user behavior is crucial, to the best of our best knowledge, there are no available 3D volumetric video viewing datasets containing fine-grained user interactivity features, not to mention further analysis and behavior prediction. In this paper, we for the first time release a volumetric video viewing behavior dataset, with a large scale, multiple dimensions, and diverse conditions. We conduct an in-depth analysis to understand user behaviors when viewing volumetric videos. Interesting findings on user viewport, gaze, and motion preference related to different videos and users are revealed. We finally design a transformer-based viewport prediction model that fuses the features of both gaze and motion, which is able to achieve high accuracy at various conditions. Our prediction model is expected to further benefit volumetric video streaming optimization. Our dataset, along with the corresponding visualization tools is accessible at https://cuhksz-inml.github.io/user-behavior-in-vv-watching/

翻译：三维视频作为一种新兴的吸引人的视频范式近年来逐渐兴起，它通过六自由度（DoF）提供了沉浸式、交互式的3D观看体验。与传统的2D或全景视频不同，三维视频需要密集的点云、体素、网格或庞大的神经模型来描绘三维场景，这给视频传输带来了极高的带宽负担。因此，用户行为分析，尤其是视口和注视分析，在优先传输用户视口内的内容并降级其余内容以在有限带宽下最大化用户QoE方面发挥着重要作用。尽管理解用户行为至关重要，但据我们所知，目前尚无可用的包含细粒度用户交互特征的三维视频观看数据集，更不用说进一步的分析和行为预测了。在本文中，我们首次发布了一个大规模、多维度、多样化的三维视频观看行为数据集。我们进行了深入分析，以理解用户在观看三维视频时的行为。揭示了不同视频和用户相关的视口、注视和运动偏好的有趣发现。最后，我们设计了一个基于Transformer的视口预测模型，该模型融合了注视和运动特征，能够在各种条件下实现高精度。我们的预测模型有望进一步促进三维视频流传输的优化。我们的数据集以及相应的可视化工具可在https://cuhksz-inml.github.io/user-behavior-in-vv-watching/获取。