Dynamic feature selection, where we sequentially query features to make accurate predictions with a minimal budget, is a promising paradigm to reduce feature acquisition costs and provide transparency into the prediction process. The problem is challenging, however, as it requires both making predictions with arbitrary feature sets and learning a policy to identify the most valuable selections. Here, we take an information-theoretic perspective and prioritize features based on their mutual information with the response variable. The main challenge is learning this selection policy, and we design a straightforward new modeling approach that estimates the mutual information in a discriminative rather than generative fashion. Building on our learning approach, we introduce several further improvements: allowing variable feature budgets across samples, enabling non-uniform costs between features, incorporating prior information, and exploring modern architectures to handle partial input information. We find that our method provides consistent gains over recent state-of-the-art methods across a variety of datasets.

翻译：动态特征选择是一种极具前景的范式，它通过顺序查询特征，在最小预算下实现准确预测，从而降低特征获取成本并提升预测过程的透明度。然而，该问题具有挑战性，因为它既需要基于任意特征集进行预测，又需要学习识别最具价值特征的策略。本文从信息论视角出发，根据特征与响应变量的互信息对其进行优先排序。核心挑战在于学习该选择策略，我们设计了一种简洁的新型建模方法，以判别式而非生成式方式估计互信息。在该学习方法基础上，我们提出了多项改进：支持样本间可变特征预算、允许特征间非均匀成本、融入先验信息，并探索处理部分输入信息的现代架构。实验结果表明，我们的方法在多个数据集上均优于当前最先进方法，具有持续的性能提升。