Recently, there have been tremendous efforts in developing lightweight Deep Neural Networks (DNNs) with satisfactory accuracy, which can enable the ubiquitous deployment of DNNs in edge devices. The core challenge of developing compact and efficient DNNs lies in how to balance the competing goals of achieving high accuracy and high efficiency. In this paper we propose two novel types of convolutions, dubbed \emph{Pixel Difference Convolution (PDC) and Binary PDC (Bi-PDC)} which enjoy the following benefits: capturing higher-order local differential information, computationally efficient, and able to be integrated with existing DNNs. With PDC and Bi-PDC, we further present two lightweight deep networks named \emph{Pixel Difference Networks (PiDiNet)} and \emph{Binary PiDiNet (Bi-PiDiNet)} respectively to learn highly efficient yet more accurate representations for visual tasks including edge detection and object recognition. Extensive experiments on popular datasets (BSDS500, ImageNet, LFW, YTF, \emph{etc.}) show that PiDiNet and Bi-PiDiNet achieve the best accuracy-efficiency trade-off. For edge detection, PiDiNet is the first network that can be trained without ImageNet, and can achieve the human-level performance on BSDS500 at 100 FPS and with $<$1M parameters. For object recognition, among existing Binary DNNs, Bi-PiDiNet achieves the best accuracy and a nearly $2\times$ reduction of computational cost on ResNet18. Code available at \href{https://github.com/hellozhuo/pidinet}{https://github.com/hellozhuo/pidinet}.

翻译：近年来，开发兼具满意精度的轻量级深度神经网络（DNN）方面投入了大量努力，这有助于DNN在边缘设备中的广泛部署。开发紧凑高效DNN的核心挑战在于如何权衡高精度与高效率这两个相互竞争的目标。本文提出两种新型卷积，分别称为像素差分卷积（PDC）和二值化PDC（Bi-PDC），它们具备以下优势：捕获高阶局部差分信息、计算高效、且能够与现有DNN集成。基于PDC和Bi-PDC，我们进一步提出两种轻量级深度网络，即像素差分网络（PiDiNet）和二值化PiDiNet（Bi-PiDiNet），用于学习面向边缘检测和目标识别等视觉任务的高效且更精确的表征。在主流数据集（如BSDS500、ImageNet、LFW、YTF等）上的大量实验表明，PiDiNet和Bi-PiDiNet实现了精度-效率的最佳平衡。在边缘检测任务中，PiDiNet是首个无需ImageNet预训练即可训练的网络，能在BSDS500上以100 FPS的速度和低于1M参数达到人类水平性能。在目标识别任务中，现有二值化DNN中，Bi-PiDiNet在ResNet18上实现了最佳精度，同时计算成本降低近2倍。代码可访问：https://github.com/hellozhuo/pidinet。