As the complexity and computational demands of deep learning models rise, the need for effective optimization methods for neural network designs becomes paramount. This work introduces an innovative search mechanism for automatically selecting the best bit-width and layer-width for individual neural network layers. This leads to a marked enhancement in deep neural network efficiency. The search domain is strategically reduced by leveraging Hessian-based pruning, ensuring the removal of non-crucial parameters. Subsequently, we detail the development of surrogate models for favorable and unfavorable outcomes by employing a cluster-based tree-structured Parzen estimator. This strategy allows for a streamlined exploration of architectural possibilities and swift pinpointing of top-performing designs. Through rigorous testing on well-known datasets, our method proves its distinct advantage over existing methods. Compared to leading compression strategies, our approach records an impressive 20% decrease in model size without compromising accuracy. Additionally, our method boasts a 12x reduction in search time relative to the best search-focused strategies currently available. As a result, our proposed method represents a leap forward in neural network design optimization, paving the way for quick model design and implementation in settings with limited resources, thereby propelling the potential of scalable deep learning solutions.

翻译：随着深度学习模型复杂性和计算需求的增加，对神经网络设计有效优化方法的需求变得至关重要。本文引入了一种创新的搜索机制，用于自动选择单个神经网络层的最佳位宽和层宽，这显著提升了深度神经网络的效率。通过利用基于Hessian矩阵的剪枝策略，搜索域得以战略性缩减，确保移除非关键参数。随后，我们详细阐述了通过采用基于聚类的树结构帕尔森估计器来构建有利和不利结果的代理模型。该策略允许对架构可能性进行流线型探索，并快速定位性能最佳的设计。通过在知名数据集上的严格测试，我们的方法证明了其相较于现有方法的明显优势。与领先的压缩策略相比，我们的方法在保持准确率不变的情况下，实现了模型大小降低20%的显著成果。此外，与当前最优的搜索策略相比，我们的方法将搜索时间减少了12倍。因此，所提出的方法标志着神经网络设计优化的重大飞跃，为在资源受限环境中的快速模型设计与实现铺平了道路，从而推动了可扩展深度学习解决方案的潜力。