The rapid growth in the size of state-of-the-art machine-learning (ML) models has led to increasing compute demands, spurring the development of energy-efficient hardware accelerators. Physical systems trained to perform ML inference, known as physical neural networks, have been proposed as accelerators in part because of their potential to be highly energy efficient. However, the ML inference accuracies reported in most investigations of physical neural networks have been substantially below the state of the art for conventional digital neural networks. In this work, we show in simulation studies that by applying modern ML training practices, including model compression, various candidate physical neural networks based on nonlinear dynamical systems can achieve accuracies as high as 99.2% on MNIST handwritten-digit recognition. We also show for one dynamical system that using it as the core building block of a parameter-efficient ML architecture inspired by MLP-Mixers enables accuracies of up to 86.4% on CIFAR-10 image classification (92.2% with pretraining and knowledge distillation), compared with 64% for the same system without the mixer architecture. The mixer approach achieved its accuracy while using $>5\times$ fewer trainable parameters than the baseline. Finally, we present back-of-the-envelope estimates for inference energy efficiency and speed when using an example physical dynamical system, suggesting that orders of magnitude improvements may be possible over digital-electronic neural networks.

翻译：暂无翻译