Binary Neural Networks emerged as a cost-effective and energy-efficient solution for computer vision tasks by binarizing either network weights or activations. However, common binary activations, such as the Sign activation function, abruptly binarize the values with a single threshold, losing fine-grained details in the feature outputs. This work proposes an activation that applies multiple thresholds following dithering principles, shifting the Sign activation function for each pixel according to a spatially periodic threshold kernel. Unlike literature methods, the shifting is defined jointly for a set of adjacent pixels, taking advantage of spatial correlations. Experiments over the classification task demonstrate the effectiveness of the designed dithering Sign activation function as an alternative activation for binary neural networks, without increasing the computational cost. Further, DeSign balances the preservation of details with the efficiency of binary operations.
翻译:二值神经网络通过对网络权重或激活进行二值化,成为计算机视觉任务中一种成本效益高且节能的解决方案。然而,常见的二值激活函数(如符号激活函数)通过单一阈值将值进行突变二值化,导致特征输出中丢失精细细节。本文提出一种基于抖动原理、应用多重阈值的激活函数,根据空间周期性的阈值核为每个像素偏移符号激活函数。与文献中的方法不同,该偏移针对一组相邻像素共同定义,利用空间相关性。分类任务的实验表明,所设计的抖动符号激活函数可作为二值神经网络的有效替代激活函数,且不增加计算成本。此外,设计的抖动符号激活函数平衡了细节保留与二值操作效率。