Fast and efficient machine learning is of growing interest to the scientific community and has spurred significant research into novel model architectures and hardware-aware design. Recent hard? and software co-design approaches have demonstrated impressive results with entirely multiplication-free models. Differentiable Logic Gate Networks (DLGNs), for instance, provide a gradient-based framework for learning optimal combinations of low-level logic gates, setting state-of-the-art trade-offs between accuracy, resource usage, and latency. However, these models suffer from high computational cost during training and do not generalize well to logic blocks with more inputs. In this work, we introduce Walsh-Assisted Relaxation for Probabilistic Look-Up Tables (WARP-LUTs) - a novel gradient-based method that efficiently learns combinations of logic gates with substantially fewer trainable parameters. We demonstrate that WARP-LUTs achieve significantly faster convergence on CIFAR-10 compared to DLGNs, while maintaining comparable accuracy. Furthermore, our approach suggests potential for extension to higher-input logic blocks, motivating future research on extremely efficient deployment on modern FPGAs and its real-time science applications.

翻译：快速高效的机器学习日益受到科学界的关注，并推动了对新型模型架构及硬件感知设计的深入研究。近期的软硬件协同设计方法已在完全无乘法的模型上取得了令人瞩目的成果。例如，可微分逻辑门网络（DLGNs）提供了一个基于梯度的框架，用于学习底层逻辑门的最优组合，在精度、资源占用和延迟之间实现了最先进的权衡。然而，这些模型在训练期间计算成本高昂，并且难以泛化到具有更多输入的逻辑块。在本工作中，我们提出了用于概率查找表的沃尔什辅助松弛方法（WARP-LUTs）——一种新颖的基于梯度的方法，能够以显著更少的可训练参数高效地学习逻辑门的组合。我们证明，在CIFAR-10数据集上，WARP-LUTs相比DLGNs实现了显著更快的收敛速度，同时保持了相当的精度。此外，我们的方法显示出扩展到更高输入逻辑块的潜力，这为未来在现代FPGA上实现极高效率的部署及其实时科学应用的研究提供了动力。