Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications. This paper evaluates Microscaling (MX) data formats that combine a per-block scaling factor with narrow floating-point and integer types for individual elements.MX formats balance the competing needs of hardware efficiency, model accuracy, and user friction. Empirical results on over two dozen benchmarks demonstrate practicality of MX data formats as a drop-in replacement for baseline FP32 for AI inference and training with low user friction. We also show the first instance of training generative language models at sub-8-bit weights, activations, and gradients with minimal accuracy loss and no modifications to the training recipe.

翻译：窄位宽数据格式对于降低现代深度学习应用的计算和存储成本至关重要。本文评估了微缩放（MX）数据格式，该格式将逐块缩放因子与单个元素的窄浮点数和整数类型相结合。MX格式在硬件效率、模型精度和用户友好性之间实现了平衡。在超过二十个基准测试上的实证结果表明，MX数据格式作为AI推理和训练中基线FP32的即插即用替代品具有实用性，且用户门槛较低。我们还首次展示了在子8位权重、激活值和梯度下训练生成语言模型的实例，仅产生极小的精度损失，且无需修改训练方案。