Many real-world multi-label prediction problems involve set-valued predictions that must satisfy specific requirements dictated by downstream usage. We focus on a typical scenario where such requirements, separately encoding \textit{value} and \textit{cost}, compete with each other. For instance, a hospital might expect a smart diagnosis system to capture as many severe, often co-morbid, diseases as possible (the value), while maintaining strict control over incorrect predictions (the cost). We present a general pipeline, dubbed as FavMac, to maximize the value while controlling the cost in such scenarios. FavMac can be combined with almost any multi-label classifier, affording distribution-free theoretical guarantees on cost control. Moreover, unlike prior works, FavMac can handle real-world large-scale applications via a carefully designed online update mechanism, which is of independent interest. Our methodological and theoretical contributions are supported by experiments on several healthcare tasks and synthetic datasets - FavMac furnishes higher value compared with several variants and baselines while maintaining strict cost control.

翻译：许多现实世界的多标签预测问题涉及集合值预测，这些预测必须满足下游应用所指定的特定要求。我们聚焦于一个典型场景，其中此类要求分别编码了\textit{价值}和\textit{成本}，且相互竞争。例如，医院可能期望智能诊断系统尽可能多地捕获严重的、常共病的疾病（价值），同时严格管控错误预测（成本）。我们提出了一种通用流程，称为FavMac，旨在此类场景中最大化价值的同时控制成本。FavMac几乎可与任何多标签分类器结合使用，提供无分布的、关于成本控制的理论保证。此外，与先前工作不同，FavMac通过精心设计的在线更新机制可处理现实世界的大规模应用，该机制本身具有独立的研究价值。我们在若干医疗任务和合成数据集上的实验支持了我们的方法论和理论贡献——FavMac在维持严格成本控制的同时，相比多种变体和基线方法提供了更高的价值。