Current Virtual Reality (VR) environments lack the rich haptic signals that humans experience during real-life interactions, such as the sensation of texture during lateral movement on a surface. Adding realistic haptic textures to VR environments requires a model that generalizes to variations of a user's interaction and to the wide variety of existing textures in the world. Current methodologies for haptic texture rendering exist, but they usually develop one model per texture, resulting in low scalability. We present a deep learning-based action-conditional model for haptic texture rendering and evaluate its perceptual performance in rendering realistic texture vibrations through a multi part human user study. This model is unified over all materials and uses data from a vision-based tactile sensor (GelSight) to render the appropriate surface conditioned on the user's action in real time. For rendering texture, we use a high-bandwidth vibrotactile transducer attached to a 3D Systems Touch device. The result of our user study shows that our learning-based method creates high-frequency texture renderings with comparable or better quality than state-of-the-art methods without the need for learning a separate model per texture. Furthermore, we show that the method is capable of rendering previously unseen textures using a single GelSight image of their surface.

翻译：当前的虚拟现实（VR）环境缺乏人类在真实交互中体验到的丰富触觉信号，例如在表面横向移动时感受到的纹理质感。为VR环境添加逼真的触觉纹理需要一种模型，该模型能够泛化用户交互的变化以及现实世界中存在的多种纹理。目前存在触觉纹理渲染的方法，但这些方法通常为每种纹理单独开发一个模型，导致可扩展性较低。我们提出了一种基于深度学习的动作条件模型，用于触觉纹理渲染，并通过多部分人类用户研究评估了其在渲染真实纹理振动方面的感知性能。该模型对所有材料统一，并使用基于视觉的触觉传感器（GelSight）的数据，根据用户的实时动作渲染相应的表面纹理。为了渲染纹理，我们使用一个连接到3D Systems Touch设备的高带宽振动触觉换能器。用户研究结果表明，我们的基于学习的方法能够生成高频纹理渲染，其质量与最先进方法相当或更优，且无需为每种纹理单独学习一个模型。此外，我们证明了该方法能够仅通过单张GelSight表面图像渲染从未见过的纹理。