We introduce DrawTalking, an approach to building and controlling interactive worlds by sketching and speaking. It emphasizes user control and flexibility, and gives programming-like capability without requiring code. We built a prototype to demonstrate it. An early open-ended study shows the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.

翻译：我们提出了DrawTalking方法，通过涂鸦和语音来构建并操控交互世界。该方法强调用户自主性与灵活性，无需编写代码即可实现类似编程的功能。我们构建了原型系统进行验证。一项早期开放性研究表明，该交互机制能够引发用户共鸣，并适用于多种创造性探索场景，有望启发并推动未来面向创意探索与内容创作的自然交互界面研究。