Neurogenesis in ANNs is an understudied and difficult problem, even compared to other forms of structural learning like pruning. By decomposing it into triggers and initializations, we introduce a framework for studying the various facets of neurogenesis: when, where, and how to add neurons during the learning process. We present the Neural Orthogonality (NORTH*) suite of neurogenesis strategies, combining layer-wise triggers and initializations based on the orthogonality of activations or weights to dynamically grow performant networks that converge to an efficient size. We evaluate our contributions against other recent neurogenesis works across a variety of supervised learning tasks.

翻译：ANNs 神经发源是一个研究不足和困难的问题,即使与诸如裁剪等其他结构学习形式相比也是如此。通过将其分解为触发和初始化,我们引入了一个框架来研究神经发源的各个方面:在学习过程中,何时、何地以及如何添加神经元。我们展示了神经异端神经发源战略套件,结合了基于电动或重量的交替调整的触发和初始化,以形成一个高效大小的动态增长性网络。我们对照其他近期神经发源在各种监管的学习任务中开展的工作,评估了我们的贡献。