We propose to build realistic virtual worlds, called 360RVW, for large urban environments directly from 360{\deg} videos. We provide an interface for interactive exploration, where users can freely navigate via their own avatars. 360{\deg} videos record the entire environment of the shooting location simultaneously leading to highly realistic and immersive representations. Our system uses 360{\deg} videos recorded along streets and builds a 360RVW through four main operations: video segmentation by intersection detection, video completion to remove the videographer, semantic segmentation for virtual collision detection with the avatar, and projection onto a distorted sphere that moves along the camera trajectory following the avatar's movements. Our interface allows users to explore large urban environments by changing their walking direction at intersections or choosing a new location by clicking on a map. Even without a 3D model, the users can experience collision with buildings using metadata produced by semantic segmentation. Furthermore, we stream the 360{\deg} videos so users can directly access 360RVW via their web browser. We fully evaluate our system, including a perceptual experiment comparing our approach to previous exploratory interfaces. The results confirm the quality of our system, especially regarding the presence of users and the interactive exploration, making it most suitable for a virtual tour of urban environments.

翻译：我们提出直接从360度视频为大型城市环境构建真实虚拟世界，称为360RVW。我们提供交互式探索界面，用户可通过自身虚拟化身自由导航。360度视频同步记录拍摄位置的完整环境，从而实现高度真实且沉浸式的场景再现。我们的系统利用沿街道录制的360度视频，通过四项主要操作构建360RVW：基于交叉口检测的视频分割、移除拍摄者的视频补全、用于虚拟化身碰撞检测的语义分割，以及沿相机轨迹跟随化身运动向变形球面的投影。用户可通过我们的界面在交叉口改变行走方向，或通过点击地图选择新位置，从而探索大型城市环境。即使没有三维模型，用户仍能借助语义分割生成的元数据体验与建筑物的碰撞。此外，我们采用360度视频流式传输技术，用户可直接通过网页浏览器访问360RVW。我们对系统进行了全面评估，包括将本方法与先前探索式界面进行比较的感知实验。结果证实了系统的优越性，特别是在用户临场感与交互探索方面，使其成为城市环境虚拟导览的理想解决方案。