Due to increasing amounts of data and compute resources, deep learning achieves many successes in various domains. The application of deep learning on the mobile and embedded devices is taken more and more attentions, benchmarking and ranking the AI abilities of mobile and embedded devices becomes an urgent problem to be solved. Considering the model diversity and framework diversity, we propose a benchmark suite, AIoTBench, which focuses on the evaluation of the inference abilities of mobile and embedded devices. AIoTBench covers three typical heavy-weight networks: ResNet50, InceptionV3, DenseNet121, as well as three light-weight networks: SqueezeNet, MobileNetV2, MnasNet. Each network is implemented by three frameworks which are designed for mobile and embedded devices: Tensorflow Lite, Caffe2, Pytorch Mobile. To compare and rank the AI capabilities of the devices, we propose two unified metrics as the AI scores: Valid Images Per Second (VIPS) and Valid FLOPs Per Second (VOPS). Currently, we have compared and ranked 5 mobile devices using our benchmark. This list will be extended and updated soon after.

翻译：由于数据和计算资源的数量增加,深层次学习在各个领域取得了许多成功。在移动和嵌入装置上应用深层次学习越来越受到关注,对移动和嵌入装置的人工智能能力进行基准和排序成为迫切的问题。考虑到模型的多样性和框架的多样性,我们提议一个基准套,即AIoTBench,侧重于评价移动和嵌入装置的推断能力。AIoTBench涵盖三个典型的重量级网络:ResNet50, InvitionV3, DenseNet121,以及三个轻型网络:SquezeNet, MobileNetV2, MnasNet。每个网络都由三个为移动和嵌入装置设计的框架实施:Tensorflow Lite, Cafe2, Pytorch Mobile。为了比较和排序这些装置的人工智能能力,我们提议两个统一的指标作为人工智能分数:有效图像/第二(VIPS)和有效FLOPs Per II(VOPs)。目前,我们用我们的基准对5个移动装置进行了比较并排在前排第5位。这个清单将很快予以扩大和更新。