Audio-language models have recently demonstrated strong zero-shot capabilities by leveraging natural-language supervision to classify audio events without labeled training data. Yet, their performance is highly sensitive to the wording of text prompts, with small variations leading to large fluctuations in accuracy. Prior work has mitigated this issue through prompt learning or prompt ensembling. However, these strategies either require annotated data or fail to account for the fact that some prompts may negatively impact performance. In this work, we present an entropy-guided prompt weighting approach that aims to find a robust combination of prompt contributions to maximize prediction confidence. To this end, we formulate a tailored objective function that minimizes prediction entropy to yield new prompt weights, utilizing low-entropy as a proxy for high confidence. Our approach can be applied to individual samples or a batch of audio samples, requiring no additional labels and incurring negligible computational overhead. Experiments on five audio classification datasets covering environmental, urban, and vocal sounds, demonstrate consistent gains compared to classical prompt ensembling methods in a zero-shot setting, with accuracy improvements 5-times larger across the whole benchmark.

翻译：音频-语言模型近期通过利用自然语言监督在无需标注训练数据的情况下对音频事件进行分类，展现了强大的零样本能力。然而，其性能对文本提示的措辞极为敏感，微小的变动会导致准确率大幅波动。先前研究通过提示学习或提示集成缓解了这一问题。但这些策略要么需要标注数据，要么未能考虑到某些提示可能对性能产生负面影响。本研究提出一种基于熵引导的提示加权方法，旨在通过寻找提示贡献的稳健组合来最大化预测置信度。为此，我们构建了一个定制化的目标函数，通过最小化预测熵来生成新的提示权重，将低熵值作为高置信度的代理指标。该方法可应用于单个音频样本或批量样本，无需额外标注且计算开销可忽略不计。在涵盖环境声、城市声和人声的五类音频分类数据集上的实验表明，在零样本设置下，相较于传统提示集成方法，本方法取得了持续的性能提升，在整个基准测试中准确率提升幅度达到传统方法的五倍。