We introduce the Differentiable Weightless Neural Network (DWN), a model based on interconnected lookup tables. Training of DWNs is enabled by a novel Extended Finite Difference technique for approximate differentiation of binary values. We propose Learnable Mapping, Learnable Reduction, and Spectral Regularization to further improve the accuracy and efficiency of these models. We evaluate DWNs in three edge computing contexts: (1) an FPGA-based hardware accelerator, where they demonstrate superior latency, throughput, energy efficiency, and model area compared to state-of-the-art solutions, (2) a low-power microcontroller, where they achieve preferable accuracy to XGBoost while subject to stringent memory constraints, and (3) ultra-low-cost chips, where they consistently outperform small models in both accuracy and projected hardware area. DWNs also compare favorably against leading approaches for tabular datasets, with higher average rank. Overall, our work positions DWNs as a pioneering solution for edge-compatible high-throughput neural networks.

翻译：本文提出可微分权重无关神经网络（DWN），这是一种基于互连查找表的模型。通过一种用于二值近似微分的新型扩展有限差分技术，实现了对DWN的训练。我们进一步提出可学习映射、可学习约简和谱正则化方法，以提升此类模型的精度与效率。我们在三种边缘计算场景下评估DWN的性能：（1）基于FPGA的硬件加速器中，相较于前沿解决方案，DWN在延迟、吞吐量、能效和模型面积方面均表现出优越性；（2）在低功耗微控制器上，DWN在严格内存限制下取得的精度优于XGBoost；（3）在超低成本芯片中，DWN在精度和预估硬件面积上均持续超越小型模型。在表格数据集上，DWN相较于主流方法也表现出更优的平均排名。总体而言，本研究将DWN确立为面向边缘兼容高吞吐量神经网络的开拓性解决方案。