We observe the emergence of binary encoding within the latent space of deep-neural-network classifiers. Such binary encoding is induced by introducing a linear penultimate layer, which is equipped during training with a loss function that grows as $\exp(\vec{x}^2)$, where $\vec{x}$ are the coordinates in the latent space. The phenomenon we describe represents a specific instance of a well-documented occurrence known as \textit{neural collapse}, which arises in the terminal phase of training and entails the collapse of latent class means to the vertices of a simplex equiangular tight frame (ETF). We show that binary encoding accelerates convergence toward the simplex ETF and enhances classification accuracy.

翻译：我们观察到深度神经网络分类器的潜在空间中出现二元编码现象。这种二元编码通过在训练过程中引入一个线性倒数第二层实现，该层配备的损失函数随$\exp(\vec{x}^2)$增长，其中$\vec{x}$表示潜在空间坐标。所述现象是被称为"神经崩溃"的经典现象的具体实例——该现象出现在训练末期阶段，表现为类别潜在均值向等角紧框架（ETF）的单纯形顶点坍缩。研究表明，二元编码能加速向单纯形ETF的收敛，并提升分类准确率。