Deep models produce a number of features in each internal layer. A key problem in applications such as feature compression for remote inference is determining how important each feature is for the task(s) performed by the model. The problem is especially challenging in the case of multi-task inference, where the same feature may carry different importance for different tasks. In this paper, we examine how effective is mutual information (MI) between a feature and a model's task output as a measure of the feature's importance for that task. Experiments involving hard selection and soft selection (unequal compression) based on MI are carried out to compare the MI-based method with alternative approaches. Multi-objective analysis is provided to offer further insight.

翻译：深度模型在每一内部层中生成大量特征。在远程推理的特征压缩等应用中，核心问题在于如何确定每个特征对模型所执行任务的重要性。当涉及多任务推理时，这一问题尤为棘手——同一特征对不同任务可能具有不同的重要程度。本文研究了特征与模型任务输出之间的互信息作为衡量该特征对任务重要性的有效性。通过基于互信息进行硬选择与软选择（非均匀压缩）的实验，将基于互信息的方法与其他方案进行了对比。此外，还提供了多目标分析以进一步阐明问题。