The increasing adoption of Artificial Intelligence (AI) in engineering problems calls for the development of calibration methods capable of offering robust statistical reliability guarantees. The calibration of black box AI models is carried out via the optimization of hyperparameters dictating architecture, optimization, and/or inference configuration. Prior work has introduced learn-then-test (LTT), a calibration procedure for hyperparameter optimization (HPO) that provides statistical guarantees on average performance measures. Recognizing the importance of controlling risk-aware objectives in engineering contexts, this work introduces a variant of LTT that is designed to provide statistical guarantees on quantiles of a risk measure. We illustrate the practical advantages of this approach by applying the proposed algorithm to a radio access scheduling problem.

翻译：随着人工智能在工程问题中的应用日益广泛，开发能够提供稳健统计可靠性保证的标定方法变得至关重要。黑箱人工智能模型的标定通过优化决定架构、优化和/或推理配置的超参数来实现。先前研究提出了学习-测试方法，这是一种为超参数优化提供平均性能指标统计保证的标定流程。鉴于工程场景中控制风险感知目标的重要性，本研究提出一种LTT变体，旨在为风险度量的分位数提供统计保证。通过将所提算法应用于无线接入调度问题，我们展示了该方法在实际应用中的优势。