Machine learning (ML) is nowadays widely used for different purposes and in several disciplines. From self-driving cars to automated medical diagnosis, machine learning models extensively support users' daily activities, and software engineering tasks are no exception. Not embracing good ML practices may lead to pitfalls that hinder the performance of an ML system and potentially lead to unexpected results. Despite the existence of documentation and literature about ML best practices, many non-ML experts turn towards gray literature like blogs and Q&A systems when looking for help and guidance when implementing ML systems. To better aid users in distilling relevant knowledge from such sources, we propose a recommender system that recommends ML practices based on the user's context. As a first step in creating a recommender system for machine learning practices, we implemented Idaka. A tool that provides two different approaches for retrieving/generating ML best practices: i) an information retrieval (IR) engine and ii) a large language model. The IR-engine uses BM25 as the algorithm for retrieving the practices, and a large language model, in our case Alpaca. The platform has been designed to allow comparative studies of best practices retrieval tools. Idaka is publicly available at GitHub: https://bit.ly/idaka. Video: https://youtu.be/cEb-AhIPxnM.
翻译:机器学习(ML)如今广泛应用于不同领域和学科。从自动驾驶汽车到自动化医疗诊断,机器学习模型广泛支持用户的日常活动,软件工程任务也不例外。未能采用良好的ML实践可能导致阻碍ML系统性能的陷阱,并可能引发意外结果。尽管存在关于ML最佳实践的文献和资料,但许多非ML专家在实施ML系统时,仍会转向博客和问答系统等灰色文献寻求帮助和指导。为了更好帮助用户从这些来源中提炼相关知识,我们提出了一种基于用户上下文推荐ML实践的推荐系统。作为创建机器学习实践推荐系统的第一步,我们实现了Idaka。该工具提供两种检索/生成ML最佳实践的方法:i) 信息检索(IR)引擎和ii) 大规模语言模型。IR引擎使用BM25算法检索实践,而大规模语言模型则采用我们的案例中的Alpaca。该平台旨在支持最佳实践检索工具的比较研究。Idaka在GitHub上公开可用:https://bit.ly/idaka。视频:https://youtu.be/cEb-AhIPxnM。