Self-supervised learning (SSL) speech models generate meaningful representations of given clips and achieve incredible performance across various downstream tasks. Model extraction attack (MEA) often refers to an adversary stealing the functionality of the victim model with only query access. In this work, we study the MEA problem against SSL speech model with a small number of queries. We propose a two-stage framework to extract the model. In the first stage, SSL is conducted on the large-scale unlabeled corpus to pre-train a small speech model. Secondly, we actively sample a small portion of clips from the unlabeled corpus and query the target model with these clips to acquire their representations as labels for the small model's second-stage training. Experiment results show that our sampling methods can effectively extract the target model without knowing any information about its model architecture.

翻译：自监督学习（SSL）语音模型能够对给定音频片段生成有意义的表征，并在各类下游任务中展现出卓越性能。模型提取攻击（MEA）通常指攻击者仅通过查询访问受害者模型，窃取其功能的行为。本文针对少量查询条件下自监督语音模型的MEA问题展开研究。我们提出了一种两阶段模型提取框架：第一阶段利用大规模无标注语料库进行自监督学习，预训练一个小型语音模型；第二阶段从该无标注语料库中主动采样少量音频片段，通过查询目标模型获取这些片段的表征，作为小型模型第二阶段的训练标签。实验结果表明，所提出的采样方法无需知晓目标模型的任何架构信息，即可有效完成模型提取。