Far-field speech recognition is a challenging task that conventionally uses signal processing beamforming to attack noise and interference problem. But the performance has been found usually limited due to heavy reliance on environmental assumption. In this paper, we propose a unified multichannel far-field speech recognition system that combines the neural beamforming and transformer-based Listen, Spell, Attend (LAS) speech recognition system, which extends the end-to-end speech recognition system further to include speech enhancement. Such framework is then jointly trained to optimize the final objective of interest. Specifically, factored complex linear projection (fCLP) has been adopted to form the neural beamforming. Several pooling strategies to combine look directions are then compared in order to find the optimal approach. Moreover, information of the source direction is also integrated in the beamforming to explore the usefulness of source direction as a prior, which is usually available especially in multi-modality scenario. Experiments on different microphone array geometry are conducted to evaluate the robustness against spacing variance of microphone array. Large in-house databases are used to evaluate the effectiveness of the proposed framework and the proposed method achieve 19.26\% improvement when compared with a strong baseline.

翻译：远场语音识别是一项具有挑战性的任务，传统上采用信号处理波束成形技术应对噪声和干扰问题。然而，由于对环境假设的严重依赖，其性能通常有限。本文提出了一种统一的多通道远场语音识别系统，该系统将神经波束成形与基于Transformer的Listen, Spell, Attend (LAS)语音识别系统相结合，进一步将端到端语音识别系统扩展到语音增强领域。该框架通过联合训练以优化最终目标。具体而言，采用因子化复线性投影（fCLP）实现神经波束成形。随后比较了多种合并监听方向的池化策略，以寻找最优方法。此外，在波束成形中融入了源方向信息，以探索作为先验信息的源方向的有效性——该信息在多模态场景中通常可用。通过不同麦克风阵列几何结构的实验，评估了系统对阵列间距变化的鲁棒性。利用大规模内部数据库评估所提框架的有效性，与强基线相比，所提方法实现了19.26%的性能提升。