Relational machine learning programs like those developed in Inductive Logic Programming (ILP) offer several advantages: (1) The ability to model complex relationships amongst data instances; (2) The use of domain-specific relations during model construction; and (3) The models constructed are human-readable, which is often one step closer to being human-understandable. However, these ILP-like methods have not been able to capitalise fully on the rapid hardware, software and algorithmic developments fuelling current developments in deep neural networks. In this paper, we treat relational features as functions and use the notion of generalised composition of functions to derive complex functions from simpler ones. We formulate the notion of a set of $\text{M}$-simple features in a mode language $\text{M}$ and identify two composition operators ($\rho_1$ and $\rho_2$) from which all possible complex features can be derived. We use these results to implement a form of "explainable neural network" called Compositional Relational Machines, or CRMs, which are labelled directed-acyclic graphs. The vertex-label for any vertex $j$ in the CRM contains a feature-function $f_j$ and a continuous activation function $g_j$. If $j$ is a "non-input" vertex, then $f_j$ is the composition of features associated with vertices in the direct predecessors of $j$. Our focus is on CRMs in which input vertices (those without any direct predecessors) all have $\text{M}$-simple features in their vertex-labels. We provide a randomised procedure for constructing and learning such CRMs. Using a notion of explanations based on the compositional structure of features in a CRM, we provide empirical evidence on synthetic data of the ability to identify appropriate explanations; and demonstrate the use of CRMs as 'explanation machines' for black-box models that do not provide explanations for their predictions.

翻译：归纳逻辑编程（ILP）等关系型机器学习程序具有以下优势：（1）能够对数据实例间的复杂关系进行建模；（2）在模型构建过程中利用领域特定关系；（3）所构建的模型具有人类可读性，这通常更接近人类可理解性。然而，这类ILP风格的方法尚未能充分利用推动深度神经网络发展的硬件、软件及算法快速进步。本文中将关系特征视为函数，并利用函数广义组合的概念从简单函数推导出复杂函数。我们在模式语言M中定义了M-简单特征集的概念，并识别出两个组合算子（ρ₁和ρ₂），所有可能的复杂特征均可由此推导得出。利用这些成果，我们实现了一种称为"组合关系机器"（CRMs）的可解释神经网络形式，其表示为带标签的有向无环图。CRM中任意顶点j的顶点标签包含特征函数f_j和连续激活函数g_j。若j为非输入顶点，则f_j是j直接前驱顶点所关联特征的组合。本文重点研究输入顶点（无直接前驱顶点）的顶点标签均包含M-简单特征的CRM模型。我们提出了一种随机化过程来构建和学习此类CRM。基于特征组合结构提供解释的概念，我们在合成数据上实证证明了识别恰当解释的能力，并展示了CRM作为无法提供预测解释的黑盒模型的"解释机器"的应用价值。