Haptic feedback contributes to immersive virtual reality (VR) experiences. However, designing such feedback at scale for all objects within a VR scene remains time-consuming. We present Scene2Hap, an LLM-centered system that automatically designs object-level vibrotactile feedback for entire VR scenes based on the objects' semantic attributes and physical context. Scene2Hap employs a multimodal large language model to estimate each object's semantics and physical context, including its material properties and vibration behavior, from multimodal information in the VR scene. These estimated attributes are then used to generate or retrieve audio signals, subsequently converted into plausible vibrotactile signals. For more realistic spatial haptic rendering, Scene2Hap estimates vibration propagation and attenuation from vibration sources to neighboring objects, considering the estimated material properties and spatial relationships of virtual objects in the scene. Three user studies confirm that Scene2Hap successfully estimates the vibration-related semantics and physical context of VR scenes and produces realistic vibrotactile signals.

翻译：触觉反馈有助于提升虚拟现实（VR）体验的沉浸感。然而，为VR场景中的所有对象大规模设计此类反馈仍然耗时。我们提出了Scene2Hap，这是一个以大语言模型（LLM）为中心的系统，能够根据对象的语义属性与物理上下文，为整个VR场景自动设计对象级的振动触觉反馈。Scene2Hap采用多模态大语言模型，从VR场景的多模态信息中估计每个对象的语义和物理上下文，包括其材料属性与振动行为。这些估计的属性随后被用于生成或检索音频信号，并进一步转换为合理的振动触觉信号。为了实现更真实的空间触觉渲染，Scene2Hap会估计振动从振源到邻近对象的传播与衰减，该过程考虑了场景中虚拟对象的估计材料属性及其空间关系。三项用户研究证实，Scene2Hap能够成功估计VR场景中与振动相关的语义和物理上下文，并生成逼真的振动触觉信号。