In this work, we introduce Y-Drop, a regularization method that biases the dropout algorithm towards dropping more important neurons with higher probability. The backbone of our approach is neuron conductance, an interpretable measure of neuron importance that calculates the contribution of each neuron towards the end-to-end mapping of the network. We investigate the impact of the uniform dropout selection criterion on performance by assigning higher dropout probability to the more important units. We show that forcing the network to solve the task at hand in the absence of its important units yields a strong regularization effect. Further analysis indicates that Y-Drop yields solutions where more neurons are important, i.e have high conductance, and yields robust networks. In our experiments we show that the regularization effect of Y-Drop scales better than vanilla dropout w.r.t. the architecture size and consistently yields superior performance over multiple datasets and architecture combinations, with little tuning.

翻译：本文提出Y-Drop，一种正则化方法，其使dropout算法倾向于以更高概率丢弃更重要的神经元。该方法的核心是神经元电导——一种可解释的神经元重要性度量，用于计算每个神经元对网络端到端映射的贡献。我们通过为更重要的单元分配更高的丢弃概率，研究了均匀丢弃选择准则对性能的影响。实验表明，迫使网络在缺失其重要单元的情况下完成当前任务能产生显著的正则化效果。进一步分析指出，Y-Drop产生的解中具有高电导的重要神经元数量更多，并能构建更鲁棒的网络。实验证明，相较于标准dropout，Y-Drop的正则化效果随网络规模扩大而更具扩展性，在多种数据集和网络架构组合中仅需极少调参即可持续取得更优性能。