Context and motivation: The development and operation of critical software that contains machine learning (ML) models requires diligence and established processes. Especially the training data used during the development of ML models have major influences on the later behaviour of the system. Runtime monitors are used to provide guarantees for that behaviour. Question / problem: We see major uncertainty in how to specify training data and runtime monitoring for critical ML models and by this specifying the final functionality of the system. In this interview-based study we investigate the underlying challenges for these difficulties. Principal ideas/results: Based on ten interviews with practitioners who develop ML models for critical applications in the automotive and telecommunication sector, we identified 17 underlying challenges in 6 challenge groups that relate to the challenge of specifying training data and runtime monitoring. Contribution: The article provides a list of the identified underlying challenges related to the difficulties practitioners experience when specifying training data and runtime monitoring for ML models. Furthermore, interconnection between the challenges were found and based on these connections recommendation proposed to overcome the root causes for the challenges.

翻译：背景与动机：包含机器学习模型的关键软件的开发与运维需要严谨的流程与规范。尤其是机器学习模型开发过程中使用的训练数据，对系统后续行为具有重大影响。运行时监控则用于为该行为提供保障。问题/难点：在如何规约关键机器学习模型的训练数据与运行时监控，进而定义系统的最终功能方面，我们观察到显著的不确定性。本访谈研究旨在探究导致这些困难的底层挑战。主要思路/成果：通过对来自汽车和电信行业、为关键应用开发机器学习模型的十位从业者进行访谈，我们识别出与训练数据和运行时监控规约挑战相关的6个挑战组中的17项底层挑战。贡献：本文系统梳理了从业者在规约机器学习模型训练数据和运行时监控时遇到困难背后的底层挑战清单。此外，我们还发现了这些挑战之间的互相关联，并基于这些关联提出克服挑战根源的建议。