In this article, we propose a framework for contactless human-computer interaction (HCI) using novel tracking techniques based on deep learning-based super-resolution and tracking algorithms. Our system offers unprecedented high-resolution tracking of hand position and motion characteristics by leveraging spatial and temporal features embedded in the reflected radar waveform. Rather than classifying samples from a predefined set of hand gestures, as common in existing work on deep learning with mmWave radar, our proposed imager employs a regressive full convolutional neural network (FCNN) approach to improve localization accuracy by spatial super-resolution. While the proposed techniques are suitable for a host of tracking applications, this article focuses on their application as a musical interface to demonstrate the robustness of the gesture sensing pipeline and deep learning signal processing chain. The user can control the instrument by varying the position and velocity of their hand above the vertically-facing sensor. By employing a commercially available multiple-input-multiple-output (MIMO) radar rather than a traditional optical sensor, our framework demonstrates the efficacy of the mmWave sensing modality for fine motion tracking and offers an elegant solution to a host of HCI tasks. Additionally, we provide a freely available software package and user interface for controlling the device, streaming the data to MATLAB in real-time, and increasing accessibility to the signal processing and device interface functionality utilized in this article.

翻译：本文提出了一种基于深度学习的超分辨率与追踪算法的新型非接触式人机交互（HCI）框架。通过利用反射雷达波形中嵌入的空间与时间特征，本系统实现了手部位置与运动特征前所未有的高分辨率追踪。与现有基于深度学习的毫米波雷达研究中常见的从预设手势集合中分类样本不同，本文提出的成像器采用回归式全卷积神经网络（FCNN）方法，通过空间超分辨率提升定位精度。尽管所提出的技术适用于多种追踪应用，本文重点将其应用于乐器界面以验证手势感知流水线与深度学习信号处理链的鲁棒性。用户可通过在垂直朝向传感器上方改变手部位置与速度来控制乐器。通过采用商用多输入多输出（MIMO）雷达而非传统光学传感器，本框架证明了毫米波感知方式在精细运动追踪中的有效性，并为多种HCI任务提供了优雅的解决方案。此外，本文提供了免费可用的软件包与用户界面，用于实时控制设备并将数据流传输至MATLAB，进一步提高了本文所涉及的信号处理与设备接口功能的可及性。