Telecommunications and computer vision have evolved separately so far. Yet, with the shift to sub-terahertz (sub-THz) and terahertz (THz) radio communications, there is an opportunity to explore computer vision technologies together with radio communications, considering the dependency of both technologies on Line of Sight. The combination of radio sensing and computer vision can address challenges such as obstructions and poor lighting. Also, machine learning algorithms, capable of processing multimodal data, play a crucial role in deriving insights from raw and low-level sensing data, offering a new level of abstraction that can enhance various applications and use cases such as beamforming and terminal handovers. This paper introduces CONVERGE, a pioneering vision-radio paradigm that bridges this gap by leveraging Integrated Sensing and Communication (ISAC) to facilitate a dual "View-to-Communicate, Communicate-to-View" approach. CONVERGE offers tools that merge wireless communications and computer vision, establishing a novel Research Infrastructure (RI) that will be open to the scientific community and capable of providing open datasets. This new infrastructure will support future research in 6G and beyond concerning multiple verticals, such as telecommunications, automotive, manufacturing, media, and health.

翻译：电信与计算机视觉领域迄今为止各自独立发展。然而，随着通信向亚太赫兹（sub-THz）和太赫兹（THz）频段迁移，鉴于二者均依赖于视距传输，探索计算机视觉技术与无线电通信的融合机遇已然显现。无线电感知与计算机视觉的结合能够解决诸如遮挡和弱光照等难题。同时，具备多模态数据处理能力的机器学习算法，在从原始低层感知数据中提取洞见方面发挥着关键作用，提供了全新的抽象层级，可增强波束赋形、终端切换等多种应用与用例。本文提出CONVERGE，一种开创性的视觉-无线电范式，通过集成感知与通信（ISAC）技术，架起二者之间的桥梁，实现“视以通信，通以视”的双向协同方法。CONVERGE提供融合无线通信与计算机视觉的工具，构建面向科学界开放、能提供开放数据集的新型研究基础设施（RI）。这一基础设施将支撑6G及未来通信技术在电信、汽车、制造、媒体、健康等垂直领域的深入研究。