Specialized function gradient computing hardware could greatly improve the performance of state-of-the-art optimization algorithms, e.g., based on gradient descent or conjugate gradient methods that are at the core of control, machine learning, and operations research applications. Prior work on such hardware, performed in the context of the Ising Machines and related concepts, is limited to quadratic polynomials and not scalable to commonly used higher-order functions. Here, we propose a novel approach for massively parallel gradient calculations of high-degree polynomials, which is conducive to efficient mixed-signal in-memory computing circuit implementations and whose area complexity scales linearly with the number of variables and terms in the function and, most importantly, independent of its degree. Two flavors of such an approach are proposed. The first is limited to binary-variable polynomials typical in combinatorial optimization problems, while the second type is broader at the cost of a more complex periphery. To validate the former approach, we experimentally demonstrated solving a small-scale third-order Boolean satisfiability problem based on integrated metal-oxide memristor crossbar circuits, one of the most prospective in-memory computing device technologies, with a competitive heuristics algorithm. Simulation results for larger-scale, more practical problems show orders of magnitude improvements in the area, and related advantages in speed and energy efficiency compared to the state-of-the-art. We discuss how our work could enable even higher-performance systems after co-designing algorithms to exploit massively parallel gradient computation.

翻译：专用函数梯度计算硬件能够大幅提升最优化算法的性能，例如基于梯度下降或共轭梯度法的方法，这些方法位于控制、机器学习与运筹学应用的核心。此前在伊辛机及相关概念背景下的硬件研究局限于二次多项式，且无法扩展到常用的高阶函数。本文提出一种用于高次多项式的大规模并行梯度计算新方法，该方法适用于高效混合信号内存计算电路实现，其面积复杂度随函数变量数和项数线性增长，且最关键的是与多项式阶数无关。我们提出了该方法的两种变体：第一种局限于组合优化问题中典型的二元变量多项式，第二种适用范围更广但代价是外围电路更复杂。为验证前一种方法，我们基于集成金属氧化物忆阻器交叉阵列电路（最具前景的内存计算器件技术之一），结合竞争性启发式算法，实验演示了解决一个小型三阶布尔可满足性问题。针对更大规模、更实际问题的仿真结果表明，与现有技术相比，该方法在面积上实现了数量级提升，并在速度和能效方面具有相关优势。我们讨论了通过协同设计算法以利用大规模并行梯度计算，本文工作如何能够实现更高性能的系统。