Network Architecture Search and specifically Regularized Evolution is a common way to refine the structure of a deep learning model.However, little is known about how models empirically evolve over time which has design implications for designing caching policies, refining the search algorithm for particular applications, and other important use cases.In this work, we algorithmically analyze and quantitatively characterize the patterns of model evolution for a set of models from the Candle project and the Nasbench-201 search space.We show how the evolution of the model structure is influenced by the regularized evolution algorithm. We describe how evolutionary patterns appear in distributed settings and opportunities for caching and improved scheduling. Lastly, we describe the conditions that affect when particular model architectures rise and fall in popularity based on their frequency of acting as a donor in a sliding window.

翻译：网络架构搜索，特别是正则化进化，是优化深度学习模型结构的常见方法。然而，关于模型如何随时间经验性进化的研究尚不充分，这为设计缓存策略、针对特定应用优化搜索算法及其他重要用例提供了设计启示。在本工作中，我们算法性地分析并定量刻画了来自Candle项目和Nasbench-201搜索空间的一组模型的进化模式。我们展示了模型结构进化如何受正则化进化算法的影响，描述了分布式环境中进化模式的出现方式以及缓存与改进调度优化的机会。最后，我们阐述了基于滑动窗口内作为供养者频率，特定模型架构流行度起伏变化的条件。