We introduce the Differentiable Weightless Neural Network (DWN), a model based on interconnected lookup tables. Training of DWNs is enabled by a novel Extended Finite Difference technique for approximate differentiation of binary values. We propose Learnable Mapping, Learnable Reduction, and Spectral Regularization to further improve the accuracy and efficiency of these models. We evaluate DWNs in three edge computing contexts: (1) an FPGA-based hardware accelerator, where they demonstrate superior latency, throughput, energy efficiency, and model area compared to state-of-the-art solutions, (2) a low-power microcontroller, where they achieve preferable accuracy to XGBoost while subject to stringent memory constraints, and (3) ultra-low-cost chips, where they consistently outperform small models in both accuracy and projected hardware area. DWNs also compare favorably against leading approaches for tabular datasets, with higher average rank. Overall, our work positions DWNs as a pioneering solution for edge-compatible high-throughput neural networks.

翻译：我们提出了可微分无权重神经网络（DWN），这是一种基于互连查找表的模型。DWN的训练通过一种新颖的扩展有限差分技术实现，该技术用于对二进制值进行近似微分。我们进一步提出了可学习映射、可学习约简和谱正则化方法，以提升这些模型的准确性和效率。我们在三种边缘计算场景下评估了DWN：（1）基于FPGA的硬件加速器，与最先进的解决方案相比，DWN在延迟、吞吐量、能效和模型面积方面均表现出优越性；（2）低功耗微控制器，在严格的内存限制下，其精度优于XGBoost；（3）超低成本芯片，在精度和预估硬件面积方面均持续优于小型模型。在表格数据集上，DWN与主流方法相比也更具优势，具有更高的平均排名。总体而言，我们的工作将DWN定位为一种适用于边缘计算的高吞吐量神经网络的先驱性解决方案。