With the development of VR technology, especially the emergence of the metaverse concept, the integration of visual and tactile perception has become an expected experience in human-machine interaction. Therefore, achieving spatial-temporal consistency of visual and tactile information in VR applications has become a necessary factor for realizing this experience. The state-of-the-art vibrotactile datasets generally contain temporal-level vibrotactile information collected by randomly sliding on the surface of an object, along with the corresponding image of the material/texture. However, they lack the position/spatial information that corresponds to the signal acquisition, making it difficult to achieve spatiotemporal alignment of visual-tactile data. Therefore, we develop a new data acquisition system in this paper which can collect visual and vibrotactile signals of different textures/materials with spatial and temporal consistency. In addition, we develop a VR-based application call "V-Touching" by leveraging the dataset generated by the new acquisition system, which can provide pixel-to-taxel joint visual-tactile perception when sliding over the surface of objects in the virtual environment with distinct vibrotactile feedback of different textures/materials. Our data and code are available at \url{https://github.com/wmtlab/Pixel2Taxel}.
翻译:随着VR技术的发展,特别是元宇宙概念的兴起,视觉与触觉感知的融合已成为人机交互中的预期体验。因此,在VR应用中实现视觉与触觉信息的时空一致性已成为实现该体验的必要因素。现有最先进的振动触觉数据集通常包含通过随机滑动物体表面采集的时域振动触觉信息,以及对应的材质/纹理图像,但缺乏与信号采集对应的位置/空间信息,难以实现视觉-触觉数据的时空对齐。为此,本文开发了一套新型数据采集系统,能够以时空一致的方式采集不同纹理/材质的视觉与振动触觉信号。此外,我们利用该新采集系统生成的数据库,开发了名为"V-Touching"的VR应用,可在虚拟环境中滑动物体表面时提供像素-触觉单元联合感知,并针对不同纹理/材质产生差异化的振动触觉反馈。我们的数据与代码已开源至:\url{https://github.com/wmtlab/Pixel2Taxel}。