Virtual reality has proved to be useful in applications in several fields ranging from gaming, medicine, and training to development of interfaces that enable human-robot collaboration. It empowers designers to explore applications outside of the constraints posed by the real world environment and develop innovative solutions and experiences. Hand gestures recognition which has been a topic of much research and subsequent commercialization in the real world has been possible because of the creation of large, labelled datasets. In order to utilize the power of natural and intuitive hand gestures in the virtual domain for enabling embodied teleoperation of collaborative robots, similarly large datasets must be created so as to keep the working interface easy to learn and flexible enough to add more gestures. Depending on the application, this may be computationally or economically prohibitive. Thus, the adaptation of trained deep learning models that perform well in the real environment to the virtual may be a solution to this challenge. This paper presents a systematic framework for the real to virtual adaptation using limited size of virtual dataset along with guidelines for creating a curated dataset. Finally, while hand gestures have been considered as the communication mode, the guidelines and recommendations presented are generic. These are applicable to other modes such as body poses and facial expressions which have large datasets available in the real domain which must be adapted to the virtual one.

翻译：虚拟现实已在游戏、医学、训练以及开发人机协作接口等多个领域的应用中展现出其价值。它使设计者能够在超越真实环境限制的条件下探索应用，开发创新解决方案与体验。手部手势识别在现实世界中已成为大量研究及后续商业化的主题，其实现得益于大规模标注数据集的创建。为了在虚拟域中利用自然直观的手势力量实现具身遥操作协作机器人，必须创建类似规模的数据集，以保持工作接口易于学习且具备足够灵活性以扩展更多手势。然而，根据具体应用，这可能在计算或经济上存在限制。因此，将已在真实环境中表现良好的训练有素的深度学习模型适配到虚拟环境，或可成为解决该挑战的方案。本文提出了一种系统性框架，利用有限规模的虚拟数据集实现从真实到虚拟的适配，并附带了创建精选数据集的指导原则。最后，尽管本文以手部手势作为通信模式进行探讨，但所提出的指导原则和建议具有通用性，适用于其他模式，例如在真实域中已有大规模数据集的肢体姿态与面部表情，这些同样需要适配到虚拟域。