Remote collaboration is a common reality of spatial design processes, but tools for computer aided design were made for single users. Via TeamCAD, we introduce a user experience where online remote collaboration experience is more like working on a table. Using speech and gesture recognition based on state of the art machine learning through webcam and microphone input, TeamCAD plugs into existing software through API's, keybindings, and mouse input. We share results from user studies conducted on graduate students from <removed for double blind review>. Our user tests were run on Blender animation software, making simultaneous use of both modalities for given tasks. We mitigated challenges in terms of robustness and latency in readily available voice recognition models. Our prototype has proven to be an intuitive interface, providing a suitable denominator for collaborators with or without previous experience in three-dimensional modeling applications.
翻译:远程协作是空间设计过程中的普遍现实,但计算机辅助设计工具却为单用户设计。通过TeamCAD,我们引入了一种用户体验,使在线远程协作体验更像是共同在桌面上工作。该系统基于通过摄像头和麦克风输入的先进机器学习技术实现语音和手势识别,并通过API、快捷键绑定及鼠标输入接入现有软件。我们分享了在(为双盲评审已删除)研究生群体中进行的用户研究结果。用户测试在Blender动画软件上执行,针对特定任务同时使用两种交互模态。我们克服了现成语音识别模型在鲁棒性和延迟方面的挑战。原型已被证明是一种直观的界面,为有或无三维建模应用经验的协作者提供了合适的共同基础。