Advances in artificial intelligence (AI) and deep learning have raised concerns about its increasing energy consumption, while demand for deploying AI in mobile devices and machines at the edge is growing. Binary neural networks (BNNs) have recently gained attention as energy and memory efficient models suitable for resource constrained environments; however, training BNNs exactly is computationally challenging because of its discrete characteristics. Recent work proposing a framework for training BNNs based on quadratic unconstrained binary optimisation (QUBO) and progress in the design of Ising machines for solving QUBO problems suggest a potential path to efficiently optimising discrete neural networks. In this work, we extend existing QUBO models for training BNNs to accommodate arbitrary network topologies and propose two novel methods for regularisation. The first method maximises neuron margins biasing the training process toward parameter configurations that yield larger pre-activation magnitudes. The second method employs a dropout-inspired iterative scheme in which reduced subnetworks are trained and used to adjust linear penalties on network parameters. We apply the proposed QUBO formulation to a small binary image classification problem and conduct computational experiments on a GPU-based Ising machine. The numerical results indicate that the proposed regularisation terms modify training behaviour and yield improvements in classification accuracy on data not present in the training set.

翻译：人工智能（AI）与深度学习的进展引发了对其能耗日益增长的担忧，与此同时，在移动设备及边缘端机器上部署AI的需求正不断增长。二元神经网络（BNNs）因其能源与内存高效性，近期作为适用于资源受限环境的模型受到关注；然而，由于其离散特性，精确训练BNNs在计算上具有挑战性。近期研究提出了基于二次无约束二进制优化（QUBO）的训练BNNs框架，以及用于求解QUBO问题的伊辛机设计进展，为高效优化离散神经网络提供了一条潜在路径。在本工作中，我们扩展了现有用于训练BNNs的QUBO模型，以适配任意网络拓扑结构，并提出了两种新颖的正则化方法。第一种方法最大化神经元间隔，使训练过程偏向于产生更大预激活幅度的参数配置。第二种方法采用受dropout启发的迭代方案，其中训练缩减后的子网络并用于调整网络参数的线性惩罚项。我们将所提出的QUBO公式应用于一个二元图像分类小问题，并在基于GPU的伊辛机上进行了计算实验。数值结果表明，所提出的正则化项改变了训练行为，并在训练集未出现的数据上提高了分类准确率。