This paper presents a new Python library called Automated Learning for Insightful Comparison and Evaluation (ALICE), which merges conventional feature selection and the concept of inter-rater agreeability in a simple, user-friendly manner to seek insights into black box Machine Learning models. The framework is proposed following an overview of the key concepts of interpretability in ML. The entire architecture and intuition of the main methods of the framework are also thoroughly discussed and results from initial experiments on a customer churn predictive modeling task are presented, alongside ideas for possible avenues to explore for the future. The full source code for the framework and the experiment notebooks can be found at: https://github.com/anasashb/aliceHU

翻译：本文介绍一种名为"用于洞察性比较与评估的自动化学习"（ALICE）的新型Python库，该库以简洁易用的方式融合传统特征选择与评分者间一致性的概念，旨在揭示黑箱机器学习模型的内部机理。在概述机器学习可解释性的关键概念后，本文提出了ALICE框架，并深入讨论了其主要方法的整体架构与设计理念。通过客户流失预测建模任务的初步实验结果，展示了该框架的有效性，同时提出了未来可行的研究方向。框架的完整源代码及实验笔记均可通过以下链接获取：https://github.com/anasashb/aliceHU