Consumer electronics used to follow the miniaturization trend described by Moore's Law. Despite increased processing power in Microcontroller Units (MCUs), MCUs used in the smallest appliances are still not capable of running even moderately big, state-of-the-art artificial neural networks (ANNs) especially in time-sensitive scenarios. In this work, we present a novel method called Scattered Online Inference (SOI) that aims to reduce the computational complexity of ANNs. SOI leverages the continuity and seasonality of time-series data and model predictions, enabling extrapolation for processing speed improvements, particularly in deeper layers. By applying compression, SOI generates more general inner partial states of ANN, allowing skipping full model recalculation at each inference.

翻译：消费电子产品曾遵循摩尔定律所描述的微型化趋势。尽管微控制器单元（MCU）的处理能力有所提升，但用于最小型设备的MCU仍无法运行即使是中等规模的最先进人工神经网络（ANN），在时间敏感场景中尤为如此。本研究提出一种名为散射在线推理（SOI）的新方法，旨在降低ANN的计算复杂度。SOI利用时间序列数据与模型预测的连续性和周期性特征，通过外推机制实现处理速度的提升，在深层网络中效果尤为显著。该方法通过压缩操作生成更具泛化能力的ANN内部部分状态，从而允许在每次推理时跳过完整的模型重计算过程。