Fast and efficient AI inference is increasingly important, and recent models that directly learn low-level logic operations have achieved state-of-the-art performance. However, existing logic neural networks incur high training costs, introduce redundancy or rely on approximate gradients, which limits scalability. To overcome these limitations, we introduce WAlsh Relaxation for Probabilistic (WARP) logic neural networks -- a novel gradient-based framework that efficiently learns combinations of hardware-native logic blocks. We show that WARP yields the most parameter-efficient representation for exactly learning Boolean functions and that several prior approaches arise as restricted special cases. Training is improved by introducing learnable thresholding and residual initialization, while we bridge the gap between relaxed training and discrete logic inference through stochastic smoothing. Experiments demonstrate faster convergence than state-of-the-art baselines, while scaling effectively to deeper architectures and logic functions with higher input arity.
翻译:快速高效的AI推理日益重要,直接学习底层逻辑运算的最新模型已实现最先进的性能。然而,现有逻辑神经网络存在训练成本高、引入冗余或依赖近似梯度等问题,限制了其可扩展性。为克服这些局限,我们提出了Walsh松弛概率(WARP)逻辑神经网络——一种基于梯度的新型框架,可高效学习硬件原生逻辑块的组合。我们证明WARP为精确学习布尔函数提供了参数效率最高的表示,且多种现有方法可作为其受限特例出现。通过引入可学习的阈值化和残差初始化改进了训练过程,同时借助随机平滑技术弥合了松弛训练与离散逻辑推理之间的差距。实验表明,相较于最先进的基线方法,WARP实现了更快的收敛速度,并能有效扩展到更深层架构及更高输入元数的逻辑函数。