A proposition that connects randomness and compression is put forward via Gibbs entropy over set of measurement vectors associated with a compression process. The proposition states that a lossy compression process is equivalent to {\it directed randomness} that preserves information content. The proposition originated from the observed behavior in newly proposed {\it Dual Tomographic Compression} (DTC) compress-train framework. This is akin to tomographic reconstruction of layer weight matrices via building compressed sensed projections, via so-called {\it weight rays}. This tomographic approach is applied to previous and next layers in a dual fashion, that triggers neuronal-level pruning. This novel model compress-train scheme appears in iterative fashion and acts as a smart neural architecture search: also called {\it compression aware training}. The experiments demonstrated the utility of this dual-tomography during training: method accelerates and supports lottery ticket hypothesis. However, random compress-train iterations having similar performance demonstrated the connection between randomness and compression from statistical physics perspective, we formulated the so-called {\it Gibbs randomness-compression proposition}, signifying randomness-compression relationship via Gibbs entropy. The proposition is supported with the experimental evidence, resulting in very high correlation between learning performance vs. Gibbs entropy over compression ratios. Practically, the DTC framework provides a promising approach for massively energy- and resource-efficient deep learning training.

翻译：通过压缩过程相关的测量向量集合上的吉布斯熵，提出了一个连接随机性与压缩的命题。该命题指出，有损压缩过程等价于保持信息内容的**定向随机性**。该命题源于新提出的**双断层扫描压缩**（DTC）压缩-训练框架中观察到的行为。这类似于通过构建压缩感知投影（即所谓的**权重射线**）对层权重矩阵进行断层扫描重建。这种断层扫描方法以前后层双模式应用，从而触发神经元级剪枝。这种新颖的模型压缩-训练方案以迭代方式出现，并作为一种智能神经架构搜索（也称为**压缩感知训练**）。实验证明了这种双断层扫描在训练过程中的效用：该方法加速并支持彩票假设。然而，具有相似性能的随机压缩-训练迭代从统计物理角度证明了随机性与压缩之间的联系，我们由此提出了所谓的**吉布斯随机性-压缩命题**，通过吉布斯熵表征随机性与压缩的关系。该命题得到了实验证据的支持，表现为学习性能与压缩比上的吉布斯熵之间存在极高的相关性。实际上，DTC框架为大规模节能和资源高效的深度学习训练提供了一种有前景的途径。