We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of the representations and computations learned by very large neural networks. NNsight is an open-source system that extends PyTorch to introduce deferred remote execution. The National Deep Inference Fabric (NDIF) is a scalable inference service that executes NNsight requests, allowing users to share GPU resources and pretrained models. These technologies are enabled by the Intervention Graph, an architecture developed to decouple experimental design from model runtime. Together, this framework provides transparent and efficient access to the internals of deep neural networks such as very large language models (LLMs) without imposing the cost or complexity of hosting customized models individually. We conduct a quantitative survey of the machine learning literature that reveals a growing gap in the study of the internals of large-scale AI. We demonstrate the design and use of our framework to address this gap by enabling a range of research methods on huge models. Finally, we conduct benchmarks to compare performance with previous approaches. Code, documentation, and tutorials are available at https://nnsight.net/.

翻译：我们介绍了NNsight与NDIF两项协同工作的技术，旨在支持对超大规模神经网络习得的表征与计算进行科学研究。NNsight是一个开源系统，它通过引入延迟远程执行机制扩展了PyTorch的功能。国家深度推理架构（NDIF）则是一个可扩展的推理服务，负责执行NNsight请求，使用户能够共享GPU资源与预训练模型。这些技术的实现基础是我们开发的“干预图”架构，该架构将实验设计与模型运行环境解耦。整体框架为深度神经网络（如超大规模语言模型）的内部机制提供了透明高效的访问途径，同时避免了独立托管定制化模型所需的高昂成本与操作复杂性。我们通过对机器学习文献的量化分析，揭示了当前大规模人工智能内部机制研究领域日益扩大的空白。我们通过展示该框架支持在巨型模型上开展多种研究方法的设计与应用，论证了其填补该研究空白的能力。最后，我们通过基准测试对比了本框架与先前方法的性能表现。代码、文档及教程详见 https://nnsight.net/。