Headphones, traditionally limited to audio playback, have evolved to integrate sensors like high-definition microphones and accelerometers. While these advancements enhance user experience, they also introduce potential eavesdropping vulnerabilities, with keystroke inference being our concern in this work. To validate this threat, we developed OverHear, a keystroke inference framework that leverages both acoustic and accelerometer data from headphones. The accelerometer data, while not sufficiently detailed for individual keystroke identification, aids in clustering key presses by hand position. Concurrently, the acoustic data undergoes analysis to extract Mel Frequency Cepstral Coefficients (MFCC), aiding in distinguishing between different keystrokes. These features feed into machine learning models for keystroke prediction, with results further refined via dictionary-based word prediction methods. In our experimental setup, we tested various keyboard types under different environmental conditions. We were able to achieve top-5 key prediction accuracy of around 80% for mechanical keyboards and around 60% for membrane keyboards with top-100 word prediction accuracies over 70% for all keyboard types. The results highlight the effectiveness and limitations of our approach in the context of real-world scenarios.

翻译：传统上仅限于音频播放的耳机，现已发展为集成高保真麦克风与加速度计等传感器。这些进步在提升用户体验的同时，也引入了潜在的窃听漏洞——本工作重点关注击键推断威胁。为验证该威胁，我们开发了OverHear框架，该框架利用耳机采集的声学与加速度计数据进行击键推断。加速度计数据虽不足以单独识别单个按键，但有助于根据手部位置对按键进行聚类。与此同时，通过对声学数据进行分析提取梅尔频率倒谱系数（MFCC），以区分不同按键。这些特征被输入机器学习模型进行击键预测，并进一步通过基于词典的词汇预测方法优化结果。在实验设置中，我们测试了不同环境条件下的多种键盘类型。对于机械键盘，Top-5按键预测准确率可达约80%，对于薄膜键盘约为60%，而所有键盘类型的Top-100词汇预测准确率均超过70%。实验结果凸显了该方法在真实场景中的有效性与局限性。