Automated audio captioning aims to use natural language to describe the content of audio data. This paper presents an audio captioning system with an encoder-decoder architecture, where the decoder predicts words based on audio features extracted by the encoder. To improve the proposed system, transfer learning from either an upstream audio-related task or a large in-domain dataset is introduced to mitigate the problem induced by data scarcity. Besides, evaluation metrics are incorporated into the optimization of the model with reinforcement learning, which helps address the problem of ``exposure bias'' induced by ``teacher forcing'' training strategy and the mismatch between the evaluation metrics and the loss function. The resulting system was ranked 3rd in DCASE 2021 Task 6. Ablation studies are carried out to investigate how much each element in the proposed system can contribute to final performance. The results show that the proposed techniques significantly improve the scores of the evaluation metrics, however, reinforcement learning may impact adversely on the quality of the generated captions.

翻译：自动声带字幕旨在使用自然语言描述音频数据的内容。本文展示了一个带有编码器-代码结构的音频字幕系统,解码器根据编码器提取的音频特征预测文字。为了改进拟议的系统,从上游音频相关任务或大型内域数据集中转移学习,以缓解数据稀缺引起的问题。此外,将评价指标纳入强化学习的模型优化中,这有助于解决“披露偏差”问题,因为“教师强迫”培训战略和评估指标与损失功能之间的不匹配。由此产生的系统在DCASE 2021任务6中排名第3位。进行了调整研究,以调查拟议系统中每个要素对最终性能的贡献。结果显示,拟议的技术大大改进了评价指标的分数,但强化学习可能对生成的字幕的质量产生不利影响。