Visualisations drive all aspects of the Machine Learning (ML) Development Cycle but remain a vastly untapped resource by the research community. ML testing is a highly interactive and cognitive process which demands a human-in-the-loop approach. Besides writing tests for the code base, bulk of the evaluation requires application of domain expertise to generate and interpret visualisations. To gain a deeper insight into the process of testing ML systems, we propose to study visualisations of ML pipelines by mining Jupyter notebooks. We propose a two prong approach in conducting the analysis. First, gather general insights and trends using a qualitative study of a smaller sample of notebooks. And then use the knowledge gained from the qualitative study to design an empirical study using a larger sample of notebooks. Computational notebooks provide a rich source of information in three formats -- text, code and images. We hope to utilise existing work in image analysis and Natural Language Processing for text and code, to analyse the information present in notebooks. We hope to gain a new perspective into program comprehension and debugging in the context of ML testing.

翻译：可视化驱动着机器学习开发周期的各个方面，但研究界仍将其视为一个巨大的未开发资源。机器学习测试是一个高度互动且依赖认知过程的任务，需要人类参与其中。除了对代码库编写测试外，大部分评估工作还需应用领域专业知识来生成和解读可视化内容。为深入理解机器学习系统的测试过程，我们提议通过挖掘Jupyter笔记本来研究机器学习管道的可视化。我们提出一种双管齐下的分析方法：首先，通过定性研究少量样本笔记本来获取通用见解与趋势；然后，利用定性研究所得知识，设计基于更大样本笔记的实证研究。计算笔记本以文本、代码和图像三种格式提供了丰富的信息源。我们期望利用现有的图像分析技术以及用于处理文本和代码的自然语言处理技术，来分析笔记本中包含的信息。我们希望能在机器学习测试的背景下，为程序理解与调试提供新的视角。