Visualization for machine learning (VIS4ML) research aims to help experts apply their prior knowledge to develop, understand, and improve the performance of machine learning models. In conceiving VIS4ML systems, researchers characterize the nature of human knowledge to support human-in-the-loop tasks, design interactive visualizations to make ML components interpretable and elicit knowledge, and evaluate the effectiveness of human-model interchange. We survey recent VIS4ML papers to assess the generalizability of research contributions and claims in enabling human-in-the-loop ML. Our results show potential gaps between the current scope of VIS4ML research and aspirations for its use in practice. We find that while papers motivate that VIS4ML systems are applicable beyond the specific conditions studied, conclusions are often overfitted to non-representative scenarios, are based on interactions with a small set of ML experts and well-understood datasets, fail to acknowledge crucial dependencies, and hinge on decisions that lack justification. We discuss approaches to close the gap between aspirations and research claims and suggest documentation practices to report generality constraints that better acknowledge the exploratory nature of VIS4ML research.

翻译：面向机器学习的可视化（VIS4ML）研究旨在帮助专家应用其先验知识来开发、理解并提升机器学习模型的性能。在设计VIS4ML系统时，研究者需明确人类知识的特性以支持人机协同任务，设计交互式可视化组件使机器学习模型可解释并激发知识，同时评估人机交互的有效性。我们通过梳理近期VIS4ML论文，评估其研究贡献及主张在赋能人机协同机器学习中的可推广性。结果显示，当前VIS4ML研究范围与其实际应用愿景之间可能存在差距。我们发现，尽管论文强调VIS4ML系统可适用于特定研究情境之外的场景，但结论常过度拟合于非代表性场景，基于与少量机器学习专家及标准数据集的交互，未能承认关键依赖性，且依赖缺乏论证的决策。我们讨论如何弥合愿景与研究主张之间的差距，并提出以文档化实践报告通用性约束的建议，以更充分承认VIS4ML研究的探索性本质。