We study the generalization behavior of Markov Logic Networks (MLNs) across relational structures of different sizes. Multiple works have noticed that MLNs learned on a given domain generalize poorly across domains of different sizes. This behavior emerges from a lack of internal consistency within an MLN when used across different domain sizes. In this paper, we quantify this inconsistency and bound it in terms of the variance of the MLN parameters. The parameter variance also bounds the KL divergence between an MLN's marginal distributions taken from different domain sizes. We use these bounds to show that maximizing the data log-likelihood while simultaneously minimizing the parameter variance corresponds to two natural notions of generalization across domain sizes. Our theoretical results apply to Exponential Random Graphs and other Markov network based relational models. Finally, we observe that solutions known to decrease the variance of the MLN parameters, like regularization and Domain-Size Aware MLNs, increase the internal consistency of the MLNs. We empirically verify our results on four different datasets, with different methods to control parameter variance, showing that controlling parameter variance leads to better generalization.

翻译：我们研究了马尔可夫逻辑网络（MLNs）在不同规模关系结构间的泛化行为。多项工作已注意到，在给定域上学习的MLNs在不同规模域之间泛化能力较差。这种行为源于MLN在不同域规模下使用时缺乏内部一致性。本文量化了这种不一致性，并通过MLN参数的方差对其进行了界定。参数方差也界定了从不同域规模获取的MLN边缘分布之间的KL散度。我们利用这些界限证明，在最大化数据对数似然的同时最小化参数方差，对应着两种跨域规模泛化的自然概念。我们的理论结果适用于指数随机图及其他基于马尔可夫网络的关系模型。最后，我们观察到已知的降低MLN参数方差的解决方案（如正则化和域大小感知MLNs）能够提高MLNs的内部一致性。我们在四个不同数据集上通过不同参数方差控制方法实证验证了结论，表明控制参数方差能带来更好的泛化性能。