Benchmarking plays a pivotal role in assessing and enhancing the performance of compact deep learning models designed for execution on resource-constrained devices, such as microcontrollers. Our study introduces a novel, entirely artificially generated benchmarking dataset tailored for speech recognition, representing a core challenge in the field of tiny deep learning. SpokeN-100 consists of spoken numbers from 0 to 99 spoken by 32 different speakers in four different languages, namely English, Mandarin, German and French, resulting in 12,800 audio samples. We determine auditory features and use UMAP (Uniform Manifold Approximation and Projection for Dimension Reduction) as a dimensionality reduction method to show the diversity and richness of the dataset. To highlight the use case of the dataset, we introduce two benchmark tasks: given an audio sample, classify (i) the used language and/or (ii) the spoken number. We optimized state-of-the-art deep neural networks and performed an evolutionary neural architecture search to find tiny architectures optimized for the 32-bit ARM Cortex-M4 nRF52840 microcontroller. Our results represent the first benchmark data achieved for SpokeN-100.

翻译：基准测试在评估和提升面向资源受限设备（如微控制器）的紧凑型深度学习模型性能中起着关键作用。本研究引入了一个全新、完全人工生成的语音识别基准数据集，这代表了微型深度学习领域的核心挑战。SpokeN-100包含32位不同说话者在四种语言（英语、普通话、德语和法语）中0至99数字的口语录音，共生成12,800个音频样本。我们提取听觉特征，并使用UMAP（统一流形逼近与投影降维）作为降维方法来展示数据集的多样性与丰富性。为突出该数据集的应用场景，我们提出两个基准任务：给定一个音频样本，分类（i）所使用的语言和/或（ii）所念的数字。我们优化了最先进的深度神经网络，并执行进化神经架构搜索，以找到针对32位ARM Cortex-M4 nRF52840微控制器优化的微型架构。我们的结果代表了SpokeN-100的首批基准数据。