Conditional computing processes an input using only part of the neural network's computational units. Learning to execute parts of a deep convolutional network by routing individual samples has several advantages: Reducing the computational burden is an obvious advantage. Furthermore, if similar classes are routed to the same path, that part of the network learns to discriminate between finer differences and better classification accuracies can be attained with fewer parameters. Recently, several papers have exploited this idea to take a particular child of a node in a tree-shaped network or to skip parts of a network. In this work, we follow a Trellis-based approach for generating specific execution paths in a deep convolutional neural network. We have designed routing mechanisms that use differentiable information gain-based cost functions to determine which subset of features in a convolutional layer will be executed. We call our method Conditional Information Gain Trellis (CIGT). We show that our conditional execution mechanism achieves comparable or better model performance compared to unconditional baselines, using only a fraction of the computational resources.

翻译：条件计算仅利用神经网络中部分计算单元处理输入。学习通过路由单个样本来执行深度卷积网络的特定部分具有多重优势：降低计算负担是显而易见的好处。此外，若将相似类别样本路由至相同路径，则该网络部分能学习区分更细微的差异，从而用更少参数获得更好分类精度。近期多篇论文利用这一思想，在树形网络中选择特定子节点或跳过网络的部分模块。本研究采用基于网格的方法来生成深度卷积神经网络中的特定执行路径。我们设计了使用可微信息增益代价函数的路由机制，以确定卷积层中哪些特征子集将被执行。我们将该方法命名为条件信息增益网格（CIGT）。实验表明，与无条件的基准方法相比，我们的条件执行机制在仅使用少量计算资源的情况下，即可达到相当或更优的模型性能。