Recent work has shown that Machine Learning (ML) programs are error-prone and called for contracts for ML code. Contracts, as in the design by contract methodology, help document APIs and aid API users in writing correct code. The question is: what kinds of contracts would provide the most help to API users? We are especially interested in what kinds of contracts help API users catch errors at earlier stages in the ML pipeline. We describe an empirical study of posts on Stack Overflow of the four most often-discussed ML libraries: TensorFlow, Scikit-learn, Keras, and PyTorch. For these libraries, our study extracted 413 informal (English) API specifications. We used these specifications to understand the following questions. What are the root causes and effects behind ML contract violations? Are there common patterns of ML contract violations? When does understanding ML contracts require an advanced level of ML software expertise? Could checking contracts at the API level help detect the violations in early ML pipeline stages? Our key findings are that the most commonly needed contracts for ML APIs are either checking constraints on single arguments of an API or on the order of API calls. The software engineering community could employ existing contract mining approaches to mine these contracts to promote an increased understanding of ML APIs. We also noted a need to combine behavioral and temporal contract mining approaches. We report on categories of required ML contracts, which may help designers of contract languages.

翻译：近期研究表明，机器学习程序易出错，并呼吁为代码引入契约机制。契约（Design by Contract方法论的核心）有助于文档化API并辅助用户编写正确代码。关键在于：哪些类型的契约能为API用户提供最大帮助？我们尤其关注能帮助用户在机器学习流水线早期阶段识别错误的契约类型。本文对Stack Overflow中关于四个最常讨论的机器学习库（TensorFlow、Scikit-learn、Keras和PyTorch）的帖子进行了实证研究。研究从这些库中提取了413条非正式（英文）API规范，并据此探讨以下问题：机器学习契约违规的根本原因及其影响是什么？是否存在常见的契约违规模式？理解机器学习契约是否需要高级机器学习软件专业知识？在API层面对契约进行检验是否有助于在机器学习流水线早期阶段发现违规行为？核心发现表明：机器学习API最常需要的契约包括对API单一参数的约束检查或对API调用顺序的约束检查。软件工程界可采用现有契约挖掘方法来挖掘这些契约，以促进对机器学习API的深入理解。研究还指出需要结合行为契约挖掘与时序契约挖掘方法。我们报告了所需机器学习契约的分类体系，这可能为契约语言设计者提供参考。