For example, in machine translation tasks, to achieve bidirectional translation between two languages, the source corpus is often used as the target corpus, which involves the training of two models with opposite directions. The question of which one can adapt most quickly to a domain shift is of significant importance in many fields. Specifically, consider an original distribution p that changes due to an unknown intervention, resulting in a modified distribution p*. In aligning p with p*, several factors can affect the adaptation rate, including the causal dependencies between variables in p. In real-life scenarios, however, we have to consider the fairness of the training process, and it is particularly crucial to involve a sensitive variable (bias) present between a cause and an effect variable. To explore this scenario, we examine a simple structural causal model (SCM) with a cause-bias-effect structure, where variable A acts as a sensitive variable between cause (X) and effect (Y). The two models, respectively, exhibit consistent and contrary cause-effect directions in the cause-bias-effect SCM. After conducting unknown interventions on variables within the SCM, we can simulate some kinds of domain shifts for analysis. We then compare the adaptation speeds of two models across four shift scenarios. Additionally, we prove the connection between the adaptation speeds of the two models across all interventions.
翻译:例如,在机器翻译任务中,为实现两种语言间的双向翻译,通常会将源语料作为目标语料,这涉及两个方向相反的模型的训练。在诸多领域中,探讨哪个模型能最快适应领域偏移具有重要实际意义。具体而言,考虑原始分布p因未知干预而发生改变,导致修正分布p*。在将p与p*对齐的过程中,适应速度受多种因素影响,包括p中变量间的因果依赖关系。然而,在真实场景中,我们必须考虑训练过程的公平性,尤其需要在因果变量与效应变量之间引入敏感变量(偏差)。为探究此类情形,我们研究了一个具有“原因-偏差-效应”结构的简单结构因果模型(SCM),其中变量A作为原因(X)与效应(Y)之间的敏感变量。两个模型在该SCM中分别呈现一致与相反的因果方向。通过对SCM中变量实施未知干预,可模拟若干领域偏移场景并进行分析。随后,我们比较了两种模型在四种偏移场景下的适应速度,并进一步证明了所有干预情形下两种模型适应速度之间的关联。