With the development of VR technology, especially the emergence of the metaverse concept, the integration of visual and tactile perception has become an expected experience in human-machine interaction. Therefore, achieving spatial-temporal consistency of visual and tactile information in VR applications has become a necessary factor for realizing this experience. The state-of-the-art vibrotactile datasets generally contain temporal-level vibrotactile information collected by randomly sliding on the surface of an object, along with the corresponding image of the material/texture. However, they lack the position/spatial information that corresponds to the signal acquisition, making it difficult to achieve spatiotemporal alignment of visual-tactile data. Therefore, we develop a new data acquisition system in this paper which can collect visual and vibrotactile signals of different textures/materials with spatial and temporal consistency. In addition, we develop a VR-based application call "V-Touching" by leveraging the dataset generated by the new acquisition system, which can provide pixel-to-taxel joint visual-tactile perception when sliding over the surface of objects in the virtual environment with distinct vibrotactile feedback of different textures/materials.

翻译：随着VR技术的发展，尤其是元宇宙概念的兴起，视觉与触觉感知的融合已成为人机交互中备受期待的体验。因此，实现VR应用中视觉与触觉信息的时空一致性已成为实现这一体验的必要条件。现有最先进的振动触觉数据集通常包含通过在物体表面随机滑动采集的时间级振动触觉信息，以及对应的材质/纹理图像。然而，这些数据集缺乏与信号采集对应的位置/空间信息，导致难以实现视觉-触觉数据的时空对齐。为此，本文开发了一种新型数据采集系统，能够以时空一致的方式采集不同纹理/材质的视觉与振动触觉信号。此外，我们利用新采集系统生成的数据集，开发了名为"V-Touching"的VR应用。该应用可在虚拟环境中滑动物体表面时，提供像素级与触觉单元级联合的视觉-触觉感知，并针对不同纹理/材质产生差异化的振动触觉反馈。