In this paper, we introduce MCTensor, a library based on PyTorch for providing general-purpose and high-precision arithmetic for DL training. MCTensor is used in the same way as PyTorch Tensor: we implement multiple basic, matrix-level computation operators and NN modules for MCTensor with identical PyTorch interface. Our algorithms achieve high precision computation and also benefits from heavily-optimized PyTorch floating-point arithmetic. We evaluate MCTensor arithmetic against PyTorch native arithmetic for a series of tasks, where models using MCTensor in float16 would match or outperform the PyTorch model with float32 or float64 precision.

翻译：在本文中,我们引入了MCTensor(MCTensor),这是一个以PyTorrch为基础的图书馆,为DL培训提供通用和高精度的计算方法。MCTsenor与PyTorrch Tensor一样使用:我们为MCTensor执行多种基本、矩阵级计算操作员和NN模块,这些模块与PyTorch界面相同。我们的算法实现了高精度计算,也得益于高精度优化的PyTorch浮点算法。我们评估了MCTensor算法与PyToch本地算法的一系列任务相比的MCTensor算法,在这些任务中,在浮点16中使用MCTensor的模型与浮点32或浮点64的精确度匹配或超过PyTorrch模型。