With the recent growth in demand for large-scale deep neural networks, compute in-memory (CiM) has come up as a prominent solution to alleviate bandwidth and on-chip interconnect bottlenecks that constrain Von-Neuman architectures. However, the construction of CiM hardware poses a challenge as any specific memory hierarchy in terms of cache sizes and memory bandwidth at different interfaces may not be ideally matched to any neural network's attributes such as tensor dimension and arithmetic intensity, thus leading to suboptimal and under-performing systems. Despite the success of neural architecture search (NAS) techniques in yielding efficient sub-networks for a given hardware metric budget (e.g., DNN execution time or latency), it assumes the hardware configuration to be frozen, often yielding sub-optimal sub-networks for a given budget. In this paper, we present CiMNet, a framework that jointly searches for optimal sub-networks and hardware configurations for CiM architectures creating a Pareto optimal frontier of downstream task accuracy and execution metrics (e.g., latency). The proposed framework can comprehend the complex interplay between a sub-network's performance and the CiM hardware configuration choices including bandwidth, processing element size, and memory size. Exhaustive experiments on different model architectures from both CNN and Transformer families demonstrate the efficacy of the CiMNet in finding co-optimized sub-networks and CiM hardware configurations. Specifically, for similar ImageNet classification accuracy as baseline ViT-B, optimizing only the model architecture increases performance (or reduces workload execution time) by 1.7x while optimizing for both the model architecture and hardware configuration increases it by 3.1x.

翻译：随着大规模深度神经网络需求的日益增长，存内计算（CiM）作为缓解冯·诺依曼架构中带宽和片上互连瓶颈的关键方案应运而生。然而，CiM硬件的构建面临挑战：不同接口的缓存大小和内存带宽等特定内存层次结构，可能无法与神经网络属性（如张量维度和计算强度）实现理想匹配，从而导致系统性能次优且表现不佳。尽管神经架构搜索（NAS）技术能在给定硬件指标预算（如DNN执行时间或延迟）下高效生成子网络，但其假设硬件配置固定不变，使得生成的子网络往往在给定预算下表现欠佳。本文提出CiMNet框架，该框架能够联合搜索CiM架构中的最优子网络与硬件配置，生成涵盖下游任务精度与执行指标（如延迟）的帕累托最优前沿。所提框架可深入理解子网络性能与CiM硬件配置选择（包括带宽、处理单元规模及内存大小）间的复杂交互作用。针对CNN和Transformer系列的多种模型架构进行的大规模实验表明，CiMNet在寻找协同优化的子网络与CiM硬件配置方面具有显著效果。具体而言，在保持与基线ViT-B模型相似的ImageNet分类精度条件下，仅优化模型架构可使性能提升（或工作负载执行时间缩短）1.7倍，而同时优化模型架构与硬件配置则可使性能提升3.1倍。