An increasing number of studies have utilized interactive deep learning as the analytic model of visual analytics systems for complex sensemaking tasks. In these systems, traditional interactive dimensionality reduction (DR) models are commonly utilized to build a bi-directional bridge between high-dimensional deep learning representations and low-dimensional visualizations. While these systems better capture analysts' intents in the context of human-in-the-loop interactive deep learning, traditional DR cannot support several desired properties for visual analytics, including out-of-sample extensions, stability, and real-time inference. To avoid this issue, we propose the neural design framework of semantic interaction for interactive deep learning. In our framework, we replace the traditional DR with a neural projection network and append it to the deep learning model as the task-specific output layer. Therefore, the analytic model (deep learning) and visualization method (interactive DR) form one integrated end-to-end trainable deep neural network. In order to understand the performance of the neural design in comparison to the state-of-the-art, we systematically performed two complementary studies, a human-centered qualitative case study and an algorithm-centered simulation-based quantitative experiment. The results of these studies indicate that the neural design can give semantic interaction systems substantial advantages while still keeping comparable inference ability compared to the state-of-the-art model.

翻译：越来越多的研究将交互式深度学习作为复杂意义构建任务中可视分析系统的分析模型。这些系统通常采用传统的交互式降维模型，在高维深度学习表示与低维可视化之间构建双向桥梁。虽然这类系统在人在回路的交互式深度学习情境中能更好地捕捉分析师意图，但传统降维方法无法支持可视分析中所需的若干关键特性，包括样本外扩展、稳定性及实时推理。为规避这一问题，我们提出了面向交互式深度学习的语义交互神经设计框架。在该框架中，我们采用神经投影网络替代传统降维方法，将其作为任务特定输出层附加至深度学习模型。由此，分析模型（深度学习）与可视化方法（交互式降维）构成一个端到端可训练的统一深度神经网络。为对比神经设计与现有最优方法的性能，我们系统性地开展了两项互补研究：以人为中心的定性案例研究，以及以算法为中心的仿真定量实验。研究结果表明，与现有最优模型相比，神经设计在保持可比推理能力的同时，能为语义交互系统带来显著优势。