Increasing demands for adaptability, privacy, and security at the edge have persistently pushed the frontiers for a new generation of machine learning (ML) algorithms with training and inference capabilities on-chip. Weightless Neural Network (WNN) is such an algorithm that is principled on lookup table based simple neuron structures. As a result, it offers architectural benefits, such as low-latency, low-complexity inference, compared to deep neural networks that depend heavily on multiply-accumulate operations. However, traditional WNNs rely on memorization-based one-shot training, which either leads to overfitting and reduced accuracy or requires tedious post-training adjustments, limiting their effectiveness for efficient on chip training. In this work, we propose TsetlinWiSARD, a training approach for WNNs that leverages Tsetlin Automata (TAs) to enable probabilistic, feedback-driven learning. It overcomes the overfitting of WiSARD's one-shot training with iterative optimization, while maintaining simple, continuous binary feedback for efficient on-chip training. Central to our approach is a field programmable gate array (FPGA)-based training architecture that delivers state-of-the-art accuracy while significantly improving hardware efficiency. Our approach provides over 1000x faster training when compared with the traditional WiSARD implementation of WNNs. Further, we demonstrate 22% reduced resource usage, 93.3% lower latency, and 64.2% lower power consumption compared to FPGA-based training accelerators implementing other ML algorithms.

翻译：随着边缘端对适应性、隐私性和安全性需求的不断增长，新一代具备片上训练与推理能力的机器学习算法持续推动技术前沿。无权重神经网络（WNN）正是此类算法之一，其基于查找表的简单神经元结构，与依赖乘加运算的深度神经网络相比，在架构上具有低延迟、低复杂度推理的优势。然而，传统WNN依赖基于记忆的单次训练，要么导致过拟合和精度下降，要么需要繁琐的训练后调整，限制了其在高效片上训练中的有效性。本研究提出TsetlinWiSARD——一种利用Tsetlin自动机（TA）实现概率驱动、反馈学习的WNN训练方法。该方法通过迭代优化克服了WiSARD单次训练的过拟合问题，同时保持简洁的连续二进制反馈机制以实现高效片上训练。其核心是现场可编程门阵列（FPGA）训练架构，在实现先进精度的同时显著提升硬件效率。与传统的WiSARD神经网络实现相比，本方法训练速度提升超过1000倍。此外，相较于其他基于FPGA的机器学习算法训练加速器，本方案在资源占用率降低22%、延迟降低93.3%、功耗降低64.2%方面展现出显著优势。