With the recent surge in the use of touchscreen devices, free-hand sketching has emerged as a promising modality for human-computer interaction. While previous research has focused on tasks such as recognition, retrieval, and generation of familiar everyday objects, this study aims to create a Sketch Input Method Editor (SketchIME) specifically designed for a professional C4I system. Within this system, sketches are utilized as low-fidelity prototypes for recommending standardized symbols in the creation of comprehensive situation maps. This paper also presents a systematic dataset comprising 374 specialized sketch types, and proposes a simultaneous recognition and segmentation architecture with multilevel supervision between recognition and segmentation to improve performance and enhance interpretability. By incorporating few-shot domain adaptation and class-incremental learning, the network's ability to adapt to new users and extend to new task-specific classes is significantly enhanced. Results from experiments conducted on both the proposed dataset and the SPG dataset illustrate the superior performance of the proposed architecture. Our dataset and code are publicly available at https://github.com/Anony517/SketchIME.
翻译:随着触摸屏设备的广泛应用,徒手草图已成为人机交互中一种颇具前景的交互方式。尽管先前的研究聚焦于常见日常物体的识别、检索与生成等任务,本研究旨在构建一种专为专业C4I系统设计的草图输入法编辑器(SketchIME)。在该系统中,草图作为低保真原型,用于推荐标准化符号以生成综合态势地图。本文同时提出一个包含374种专业草图类别的系统性数据集,并设计了一种具备识别与分割协同监督机制的同步识别与分割架构,从而提升性能与可解释性。通过引入少样本领域自适应与类增量学习,网络对新用户的适应能力及对新任务特定类别的扩展能力显著增强。在本文提出的数据集与SPG数据集上的实验结果表明,该架构具有优越性能。我们的数据集与代码已公开于https://github.com/Anony517/SketchIME。