Conflicting objectives present a considerable challenge in interleaving multi-task learning, necessitating the need for meticulous design and balance to ensure effective learning of a representative latent data space across all tasks without mutual negative impact. Drawing inspiration from the concept of marginal and conditional probability distributions in probability theory, we design a principled and well-founded approach to disentangle the original input into marginal and conditional probability distributions in the latent space of a variational autoencoder. Our proposed model, Deep Disentangled Interleaving Variational Encoding (DeepDIVE) learns disentangled features from the original input to form clusters in the embedding space and unifies these features via the cross-attention mechanism in the fusion stage. We theoretically prove that combining the objectives for reconstruction and forecasting fully captures the lower bound and mathematically derive a loss function for disentanglement using Na\"ive Bayes. Under the assumption that the prior is a mixture of log-concave distributions, we also establish that the Kullback-Leibler divergence between the prior and the posterior is upper bounded by a function minimized by the minimizer of the cross entropy loss, informing our adoption of radial basis functions (RBF) and cross entropy with interleaving training for DeepDIVE to provide a justified basis for convergence. Experiments on two public datasets show that DeepDIVE disentangles the original input and yields forecast accuracies better than the original VAE and comparable to existing state-of-the-art baselines.

翻译：交错多任务学习中存在的冲突目标构成了重大挑战，需要精心设计与平衡，以确保在所有任务上有效学习具有代表性的潜在数据空间，同时避免相互间的负面影响。受概率论中边缘概率分布与条件概率分布概念的启发，我们设计了一种原理清晰、基础坚实的方案，在变分自编码器的潜在空间中将原始输入解耦为边缘概率分布与条件概率分布。我们提出的模型——深度解耦交错变分编码（DeepDIVE）——从原始输入中学习解耦特征以在嵌入空间中形成聚类，并在融合阶段通过交叉注意力机制统一这些特征。我们从理论上证明了结合重构与预测目标能够完整捕捉下界，并基于朴素贝叶斯（Naïve Bayes）数学推导出解耦的损失函数。在假设先验分布为对数凹分布混合的前提下，我们还证明了先验与后验之间的Kullback-Leibler散度存在上界，该上界可由交叉熵损失的最小化函数控制，这为我们在DeepDIVE中采用径向基函数（RBF）与交错训练交叉熵提供了收敛的理论依据。在两个公开数据集上的实验表明，DeepDIVE能够有效解耦原始输入，其预测精度优于原始VAE，并与现有最先进基线方法相当。