With the development of internet of things technologies, tremendous sensor audio data has been produced, which poses great challenges to audio-based event detection in smart cities. In this paper, we target a challenging audio-based event detection task, namely, text-to-audio grounding. In addition to precisely localizing all of the desired on- and off-sets in the untrimmed audio, this challenging new task requires extensive acoustic and linguistic comprehension as well as the reasoning for the crossmodal matching relations between the audio and query. The current approaches often treat the query as an entire one through a global query representation in order to address those issues. We contend that this strategy has several drawbacks. Firstly, the interactions between the query and the audio are not fully utilized. Secondly, it has not distinguished the importance of different keywords in a query. In addition, since the audio clips are of arbitrary lengths, there exist many segments which are irrelevant to the query but have not been filtered out in the approach. This further hinders the effective grounding of desired segments. Motivated by the above concerns, a novel Cross-modal Graph Interaction (CGI) model is proposed to comprehensively model the relations between the words in a query through a novel language graph. To capture the fine-grained relevances between the audio and query, a cross-modal attention module is introduced to generate snippet-specific query representations and automatically assign higher weights to keywords with more important semantics. Furthermore, we develop a cross-gating module for the audio and query to weaken irrelevant parts and emphasize the important ones.

翻译：随着物联网技术的发展，海量的传感器音频数据被产生，这对智慧城市中基于音频的事件检测提出了巨大挑战。本文针对一项具有挑战性的基于音频的事件检测任务——即文本-音频对齐。除了需要精确定位未修剪音频中所有期望的起止时间点外，这一新颖任务还要求具备广泛的声学与语言理解能力，以及对音频与查询之间跨模态匹配关系的推理能力。当前方法通常将查询作为一个整体，通过全局查询表示来处理上述问题。我们认为这种策略存在若干缺陷：首先，查询与音频之间的交互未被充分利用；其次，未能区分查询中不同关键词的重要性；此外，由于音频片段长度任意，存在许多与查询无关但未被有效过滤的片段，进一步阻碍了目标片段的准确定位。基于上述问题，本文提出了一种新颖的跨模态图交互（CGI）模型，通过构建语言图全面建模查询中词语之间的关系。为捕获音频与查询之间的细粒度相关性，引入跨模态注意力模块生成片段特定的查询表示，并自动为语义更重要的关键词赋予更高权重。此外，我们开发了音频与查询的交叉门控模块，以削弱不相关部分并强化重要部分。