The human brain can easily focus on one speaker and suppress others in scenarios such as a cocktail party. Recently, researchers found that auditory attention can be decoded from the electroencephalogram (EEG) data. However, most existing deep learning methods are difficult to use prior knowledge of different views (that is attended speech and EEG are task-related views) and extract an unsatisfactory representation. Inspired by Broadbent's filter model, we decode auditory attention in a multi-view paradigm and extract the most relevant and important information utilizing the missing view. Specifically, we propose an auditory attention decoding (AAD) method based on multi-view VAE with task-related multi-view contrastive (TMC) learning. Employing TMC learning in multi-view VAE can utilize the missing view to accumulate prior knowledge of different views into the fusion of representation, and extract the approximate task-related representation. We examine our method on two popular AAD datasets, and demonstrate the superiority of our method by comparing it to the state-of-the-art method.

翻译：人脑在如鸡尾酒会等场景中能够轻松关注某一位说话者并抑制其他干扰。近年研究发现，听觉注意力可从脑电图（EEG）信号中解码。然而，现有深度学习方法大多难以利用不同视角的先验知识（即目标语音与EEG属于任务相关视角），导致提取的表征不够理想。受Broadbent滤波模型启发，我们采用多视角范式解码听觉注意力，并利用缺失视角提取最关键、最相关的信息。具体而言，我们提出基于多视角变分自编码器（VAE）与任务相关多视角对比（TMC）学习的听觉注意力解码（AAD）方法。在多视角VAE中引入TMC学习，可借助缺失视角将不同视角的先验知识累积至表征融合过程，并提取近似任务相关的表征。我们在两个典型AAD数据集上验证该方法，通过与现有最优方法的对比，证明了我们方法的优越性。