This thesis examines self-attention training through the lens of Optimal Transport (OT) and develops an OT-based alternative for tabular classification. The study tracks intermediate projections of the self-attention layer during training and evaluates their evolution using discrete OT metrics, including Wasserstein distance, Monge gap, optimality, and efficiency. Experiments are conducted on classification tasks with two and three classes, as well as on a biomedical dataset. Results indicate that the final self-attention mapping often approximates the OT optimal coupling, yet the training trajectory remains inefficient. Pretraining the MLP section on synthetic data partially improves convergence but is sensitive to their initialization. To address these limitations, an OT-based algorithm is introduced: it generates class-specific dummy Gaussian distributions, computes an OT alignment with the data, and trains an MLP to generalize this mapping. The method achieves accuracy comparable to Transformers while reducing computational cost and scaling more efficiently under standardized inputs, though its performance depends on careful dummy-geometry design. All experiments and implementations are conducted in R.

翻译：本论文通过最优运输（OT）的视角研究自注意力的训练过程，并开发了一种基于OT的替代方法用于表格分类。研究追踪了训练过程中自注意力层的中间投影，并使用离散OT度量（包括Wasserstein距离、Monge间隙、最优性和效率）评估其演变过程。实验在二分类、三分类任务以及一个生物医学数据集上进行。结果表明，最终的自注意力映射通常近似于OT最优耦合，但训练轨迹仍然效率低下。在合成数据上对MLP部分进行预训练能部分改善收敛性，但对初始化敏感。为解决这些局限性，本文引入了一种基于OT的算法：该算法生成类别特定的虚拟高斯分布，计算与数据的OT对齐，并训练一个MLP来泛化此映射。该方法在达到与Transformer相当精度的同时，降低了计算成本，并在标准化输入下具有更高效的扩展性，但其性能依赖于精心的虚拟几何设计。所有实验与实现均在R语言环境中完成。