Streaming neural network models for fast frame-wise responses to various speech and sensory signals are widely adopted on resource-constrained platforms. Hence, increasing the learning capacity of such streaming models (i.e., by adding more parameters) to improve the predictive power may not be viable for real-world tasks. In this work, we propose a new loss, Streaming Anchor Loss (SAL), to better utilize the given learning capacity by encouraging the model to learn more from essential frames. More specifically, our SAL and its focal variations dynamically modulate the frame-wise cross entropy loss based on the importance of the corresponding frames so that a higher loss penalty is assigned for frames within the temporal proximity of semantically critical events. Therefore, our loss ensures that the model training focuses on predicting the relatively rare but task-relevant frames. Experimental results with standard lightweight convolutional and recurrent streaming networks on three different speech based detection tasks demonstrate that SAL enables the model to learn the overall task more effectively with improved accuracy and latency, without any additional data, model parameters, or architectural changes.

翻译：流式神经网络模型在资源受限平台上被广泛用于对语音和传感信号进行快速的逐帧响应。因此，通过增加模型参数来提升这类流式模型的学习能力，以增强其预测能力，在实际任务中可能并不可行。为此，本文提出一种新的损失函数——流式锚点损失（SAL），通过鼓励模型从关键帧中学习更多信息，从而更有效地利用给定的学习能力。具体而言，我们的SAL及其焦点变体根据对应帧的重要性动态调整逐帧交叉熵损失，使得在语义关键事件的时间邻近区域内的帧被赋予更高的损失惩罚。因此，我们的损失函数确保模型训练聚焦于预测那些相对稀少但与任务相关的帧。在三种基于语音的检测任务上使用标准轻量级卷积和循环流式网络进行的实验结果表明，SAL使模型能够更有效地学习整体任务，提升了准确率和延迟性能，且无需额外数据、模型参数或架构更改。