We present Variational Bayesian Network (VBN) - a novel Bayesian entity representation learning model that utilizes hierarchical and relational side information and is particularly useful for modeling entities in the ``long-tail'', where the data is scarce. VBN provides better modeling for long-tail entities via two complementary mechanisms: First, VBN employs informative hierarchical priors that enable information propagation between entities sharing common ancestors. Additionally, VBN models explicit relations between entities that enforce complementary structure and consistency, guiding the learned representations towards a more meaningful arrangement in space. Second, VBN represents entities by densities (rather than vectors), hence modeling uncertainty that plays a complementary role in coping with data scarcity. Finally, we propose a scalable Variational Bayes optimization algorithm that enables fast approximate Bayesian inference. We evaluate the effectiveness of VBN on linguistic, recommendations, and medical inference tasks. Our findings show that VBN outperforms other existing methods across multiple datasets, and especially in the long-tail.

翻译：我们提出变分贝叶斯网络——一种新颖的贝叶斯实体表示学习模型，该模型利用层次化与关系型辅助信息，特别适用于数据稀疏的“长尾”实体建模。VBN通过两种互补机制实现对长尾实体的优化建模：首先，VBN采用信息丰富的层次化先验，使共享共同祖先的实体间能够进行信息传播；同时，VBN显式建模实体间的关联关系以强化互补结构与一致性，引导所学表征在空间中进行更有意义的排列。其次，VBN以密度（而非向量）表示实体，从而建模不确定性，这一特性在处理数据稀疏问题时发挥互补作用。最后，我们提出一种可扩展的变分贝叶斯优化算法，实现快速近似贝叶斯推断。我们在语言处理、推荐系统和医学推理任务上评估了VBN的有效性。实验结果表明，VBN在多个数据集上均优于现有方法，尤其在长尾场景中表现突出。