Regularized discrete optimal transport (OT) is a powerful tool to measure the distance between two discrete distributions that have been constructed from data samples on two different domains. While it has a wide range of applications in machine learning, in some cases the sampled data from only one of the domains will have class labels such as unsupervised domain adaptation. In this kind of problem setting, a group-sparse regularizer is frequently leveraged as a regularization term to handle class labels. In particular, it can preserve the label structure on the data samples by corresponding the data samples with the same class label to one group-sparse regularization term. As a result, we can measure the distance while utilizing label information by solving the regularized optimization problem with gradient-based algorithms. However, the gradient computation is expensive when the number of classes or data samples is large because the number of regularization terms and their respective sizes also turn out to be large. This paper proposes fast discrete OT with group-sparse regularizers. Our method is based on two ideas. The first is to safely skip the computations of the gradients that must be zero. The second is to efficiently extract the gradients that are expected to be nonzero. Our method is guaranteed to return the same value of the objective function as that of the original method. Experiments show that our method is up to 8.6 times faster than the original method without degrading accuracy.
翻译:正则化离散最优传输(OT)是一种强大的工具,用于测量两个离散分布之间的距离,这些分布由来自两个不同领域的数据样本构建而成。尽管它在机器学习中有广泛应用,但在某些情况下,仅从一个领域采样的数据可能带有类别标签,例如无监督领域适应。在这种问题设置中,群稀疏正则化器常被用作处理类别标签的正则化项。具体而言,它通过将具有相同类别标签的数据样本对应到同一个群稀疏正则化项,从而保留数据样本上的标签结构。因此,通过使用基于梯度的算法求解正则化优化问题,我们可以在利用标签信息的同时测量距离。然而,当类别数或数据样本数较大时,由于正则化项的数量及其各自规模也变大,梯度计算代价高昂。本文提出了一种带有群稀疏正则化器的快速离散OT方法。我们的方法基于两个想法:一是安全跳过那些必须为零的梯度计算;二是高效提取那些预期非零的梯度。我们的方法保证返回与原始方法相同目标函数值。实验表明,我们的方法在不降低精度的情况下,比原始方法快最多8.6倍。