This paper presents a general framework to integrate prior knowledge in the form of logic constraints among a set of task functions into kernel machines. The logic propositions provide a partial representation of the environment, in which the learner operates, that is exploited by the learning algorithm together with the information available in the supervised examples. In particular, we consider a multi-task learning scheme, where multiple unary predicates on the feature space are to be learned by kernel machines and a higher level abstract representation consists of logic clauses on these predicates, known to hold for any input. A general approach is presented to convert the logic clauses into a continuous implementation, that processes the outputs computed by the kernel-based predicates. The learning task is formulated as a primal optimization problem of a loss function that combines a term measuring the fitting of the supervised examples, a regularization term, and a penalty term that enforces the constraints on both supervised and unsupervised examples. The proposed semi-supervised learning framework is particularly suited for learning in high dimensionality feature spaces, where the supervised training examples tend to be sparse and generalization difficult. Unlike for standard kernel machines, the cost function to optimize is not generally guaranteed to be convex. However, the experimental results show that it is still possible to find good solutions using a two stage learning schema, in which first the supervised examples are learned until convergence and then the logic constraints are forced. Some promising experimental results on artificial multi-task learning tasks are reported, showing how the classification accuracy can be effectively improved by exploiting the a priori rules and the unsupervised examples.

翻译：本文提出了一种通用框架，将任务函数集合间的逻辑约束形式先验知识融入核机器中。逻辑命题提供了学习者所处环境的部分表征，学习算法能同时利用监督样本中的信息与环境表征进行学习。具体而言，我们考虑多任务学习范式：核机器需学习特征空间上的多个一元谓词，而高阶抽象表示由这些谓词间对所有输入成立的逻辑子句构成。本文提出将逻辑子句转化为连续实现的通用方法，该方法通过处理基于核的谓词输出结果实现转化。学习任务被形式化为损失函数的原始优化问题，该损失函数由三个项组合而成：监督样本拟合度度量项、正则化项、以及对监督与非监督样本施加约束的惩罚项。所提出的半监督学习框架特别适用于高维特征空间中的学习场景，这类场景中监督训练样本往往稀疏且泛化困难。与标准核机器不同，需优化的代价函数通常无法保证凸性。然而实验表明，采用两阶段学习方案仍能获得优质解：首先学习监督样本直至收敛，再施加逻辑约束。本文报告了在人工多任务学习任务中取得的初步实验结果，展示了如何通过利用先验规则与非监督样本有效提升分类精度。