In many software systems, heuristics are used to make decisions - such as cache eviction, task scheduling, and information presentation - that have a significant impact on overall system behavior. While machine learning may outperform these heuristics, replacing existing heuristics in a production system safely and reliably can be prohibitively costly. We present SmartChoices, a novel approach that reduces the cost to deploy production-ready ML solutions for contextual bandits problems. SmartChoices' interface cleanly separates problem formulation from implementation details: engineers describe their use case by defining datatypes for the context, arms, and feedback that are passed to SmartChoices APIs, while SmartChoices manages encoding & logging data and training, evaluating & deploying policies. Our implementation codifies best practices, is efficient enough for use in low-level applications, and provides valuable production features off the shelf via a shared library. Overall, SmartChoices enables non-experts to rapidly deploy production-ready ML solutions by eliminating many sources of technical debt common to ML systems. Engineers have independently used SmartChoices to improve a wide range of software including caches, batch processing workloads, and UI layouts, resulting in better latency, throughput, and click-through rates.

翻译：在许多软件系统中，启发式方法被用于做出决策——例如缓存淘汰、任务调度和信息呈现——这些决策对整体系统行为具有显著影响。虽然机器学习可能超越这些启发式方法，但在生产系统中安全可靠地替换现有启发式方法的成本可能高得令人望而却步。我们提出智能选择（SmartChoices），这是一种降低上下文赌博机问题生产就绪机器学习解决方案部署成本的新方法。智能选择的接口清晰地将问题表述与实现细节分离：工程师通过定义传递给智能选择API的上下文、臂和反馈的数据类型来描述其用例，而智能选择则管理数据编码与记录、策略训练、评估与部署。我们的实现将最佳实践规范化，其效率足以满足底层应用需求，并通过共享库提供有价值的现成生产特性。总体而言，智能选择通过消除机器学习系统常见的技术债务来源，使非专家能够快速部署生产就绪的机器学习解决方案。工程师已独立使用智能选择改进了包括缓存、批处理工作负载和用户界面布局在内的多种软件，从而获得了更优的延迟、吞吐量和点击率。