The advent of sparsity inducing techniques in neural networks has been of a great help in the last few years. Indeed, those methods allowed to find lighter and faster networks, able to perform more efficiently in resource-constrained environment such as mobile devices or highly requested servers. Such a sparsity is generally imposed on the weights of neural networks, reducing the footprint of the architecture. In this work, we go one step further by imposing sparsity jointly on the weights and on the input data. This can be achieved following a three-step process: 1) impose a certain structured sparsity on the weights of the network; 2) track back input features corresponding to zeroed blocks of weight; 3) remove useless weights and input features and retrain the network. Performing pruning both on the network and on input data not only allows for extreme reduction in terms of parameters and operations but can also serve as an interpretation process. Indeed, with the help of data pruning, we now have information about which input feature is useful for the network to keep its performance. Experiments conducted on a variety of architectures and datasets: MLP validated on MNIST, CIFAR10/100 and ConvNets (VGG16 and ResNet18), validated on CIFAR10/100 and CALTECH101 respectively, show that it is possible to achieve additional gains in terms of total parameters and in FLOPs by performing pruning on input data, while also increasing accuracy.

翻译：稀疏性诱导技术在神经网络中的出现，在过去几年带来了巨大帮助。事实上，这些方法能够找到更轻量、更快速的网络，在移动设备或高请求服务器等资源受限环境中更高效地运行。这种稀疏性通常施加于神经网络的权重上，从而减少架构的内存占用。在本工作中，我们更进一步，同时在权重和输入数据上施加稀疏性。这可以通过三步过程实现：1) 在网络权重上施加某种结构化稀疏性；2) 追溯对应于权重零块的输入特征；3) 移除无效权重和输入特征并重新训练网络。在网络和输入数据上同时执行剪枝，不仅允许在参数和运算量上实现极致缩减，还可作为解释过程。实际上，借助数据剪枝，我们能够获知哪些输入特征对网络保持性能有用。在多种架构和数据集上进行的实验：MLP在MNIST、CIFAR10/100上验证，以及ConvNet（VGG16和ResNet18）分别在CIFAR10/100和CALTECH101上验证，结果表明，通过在输入数据上执行剪枝，可以在总参数和FLOPs方面实现额外增益，同时还能提升准确率。