With the recent growth in demand for large-scale deep neural networks, compute in-memory (CiM) has come up as a prominent solution to alleviate bandwidth and on-chip interconnect bottlenecks that constrain Von-Neuman architectures. However, the construction of CiM hardware poses a challenge as any specific memory hierarchy in terms of cache sizes and memory bandwidth at different interfaces may not be ideally matched to any neural network's attributes such as tensor dimension and arithmetic intensity, thus leading to suboptimal and under-performing systems. Despite the success of neural architecture search (NAS) techniques in yielding efficient sub-networks for a given hardware metric budget (e.g., DNN execution time or latency), it assumes the hardware configuration to be frozen, often yielding sub-optimal sub-networks for a given budget. In this paper, we present CiMNet, a framework that jointly searches for optimal sub-networks and hardware configurations for CiM architectures creating a Pareto optimal frontier of downstream task accuracy and execution metrics (e.g., latency). The proposed framework can comprehend the complex interplay between a sub-network's performance and the CiM hardware configuration choices including bandwidth, processing element size, and memory size. Exhaustive experiments on different model architectures from both CNN and Transformer families demonstrate the efficacy of the CiMNet in finding co-optimized sub-networks and CiM hardware configurations. Specifically, for similar ImageNet classification accuracy as baseline ViT-B, optimizing only the model architecture increases performance (or reduces workload execution time) by 1.7x while optimizing for both the model architecture and hardware configuration increases it by 3.1x.

翻译：随着大规模深度神经网络需求的日益增长，存内计算（CiM）作为一种重要解决方案，旨在缓解冯·诺依曼架构中带宽和片上互连瓶颈。然而，CiM硬件构建面临挑战：不同接口的缓存大小和内存带宽等特定存储层次结构，可能与神经网络张量维度、计算强度等属性不匹配，从而导致系统性能次优。尽管神经架构搜索（NAS）技术能在给定硬件指标预算（如DNN执行时间或延迟）下高效生成子网络，但其假设硬件配置固定不变，往往无法在预算内得到最优子网络。本文提出CiMNet框架，该框架联合搜索CiM架构的最优子网络与硬件配置，形成下游任务准确率与执行指标（如延迟）的帕累托最优前沿。该框架能够理解子网络性能与CiM硬件配置选择（包括带宽、处理单元大小和内存大小）之间的复杂交互关系。在CNN和Transformer系列不同模型架构上的大量实验表明，CiMNet在寻找协同优化的子网络与CiM硬件配置方面具有有效性。具体而言，在保持与基线ViT-B相近的ImageNet分类准确率下，仅优化模型架构可使性能提升（或减少工作负载执行时间）1.7倍，而同时优化模型架构与硬件配置则可提升3.1倍。