We present GrooveMeter, a novel system that automatically detects vocal and motion reactions to music via earable sensing and supports music engagement-aware applications. To this end, we use smart earbuds as sensing devices, which are already widely used for music listening, and devise reaction detection techniques by leveraging an inertial measurement unit (IMU) and a microphone on earbuds. To explore reactions in daily music-listening situations, we collect the first kind of dataset, MusicReactionSet, containing 926-minute-long IMU and audio data with 30 participants. With the dataset, we discover a set of unique challenges in detecting music listening reactions accurately and robustly using audio and motion sensing. We devise sophisticated processing pipelines to make reaction detection accurate and efficient. We present a comprehensive evaluation to examine the performance of reaction detection and system cost. It shows that GrooveMeter achieves the macro F1 scores of 0.89 for vocal reaction and 0.81 for motion reaction with leave-one-subject-out cross-validation. More importantly, GrooveMeter shows higher accuracy and robustness compared to alternative methods. We also show that our filtering approach reduces 50% or more of the energy overhead. Finally, we demonstrate the potential use cases through a case study.

翻译：我们提出GrooveMeter——一种通过耳戴式传感自动检测音乐引发的语音与动作反应的新型系统，可支持音乐互动感知类应用。为此，我们将智能耳机（已被广泛用于音乐播放）作为传感设备，利用耳机内置的惯性测量单元（IMU）与麦克风设计反应检测技术。为探索日常音乐聆听场景中的反应模式，我们首次构建了包含30名受试者、总时长926分钟的IMU与音频数据集MusicReactionSet。基于该数据集，我们发现利用音频与运动传感精确稳健地检测音乐聆听反应需应对一系列独特挑战。我们设计了精密的处理流水线以实现高效准确的反应检测，并通过综合评估检验了检测性能与系统开销。结果表明，在留一受试者交叉验证下，GrooveMeter对语音反应的宏F1得分为0.89，对动作反应为0.81。更重要的是，相比替代方法，GrooveMeter展现出更高的准确度与鲁棒性。我们还证明了所提出的滤波方法可减少50%以上的能耗开销。最后通过案例研究展示了其潜在应用场景。