Research On Integrating First-Order Logical Domain Knowledge With Machine Learning

There usually exists large amount of domain knowledge in practical machine learn-ing tasks,and the knowledge can be easily expressed by first-order logic language.This dissertation tries to investigate:1)how to exploit such kind of domain knowledge to improve machine learning;2)how to use machine learning to help with knowledge refinement;3)how to integrate knowledge-driven reasoning and data-driven machine learning in a mutually beneficial way.Particularly,four machine learning approaches and frameworks are proposed to answer the above three questions:1.A machine learning approach that augments training examples with domain knowledge.It is difficult for traditional machine learning methods to utilize infor-mation expressing relations between objects.This kind of information can be con-veniently formalized by first-order logical domain knowledge.In order to exploit this kind of domain knowledge for improving the performance of machine learn-ing,we propose the SUL(Statistical Unfolded Logical)learning approach,which transforms the domain knowledge into augmented features that could be directly exploited by machine learning algorithms.Experimental result shows the effec-tiveness of SUL learning against those learning approaches that do not consider domain knowledge.2.A machine learning approach that exploits domain knowledge as constraints.Because of the highly complexity of first-order logical domain knowledge,it is hard for ordinary machine learning techniques to use them to constrain the learning pro-cess.We propose the LASIN(Logical Abduction and Statistical INduction)method to transform the domain knowledge into constraints on hypothesis space of machine learning tasks.Therefore,LASIN can force the models learned from data to be con-sistent with the domain knowledge.Experimental results show that,comparing to the approaches that do not introducing first-order logical domain knowledge,the models learned by LASIN can generalize better in the testing data.3.A knowledge refining approach that is driven by machine learning techniques.The target of knowledge refinement is to learn comprehensible logical rules from data.Traditional knowledge refining approaches can not be applied on the data that is not formulated by logical symbols.We propose the KRL(Knowledge Re-finement by Learning)method to transform knowledge refinement problems into machine learning tasks.KRL first augments the domain knowledge by using ma-chine learning models to extract primitive logical facts,then it can apply machine learning techniques for learning first-order logical rules from the augmented do-main knowledge.Results on the experiments of computer vision tasks show that KRL is able to learn formal definition of the target concepts from images.4.A framework that integrates domain knowledge and machine learning in a mu-tually beneficial way.Integrating knowledge-driven logical reasoning and data-driven machine learning has been a key challenge in Artificial Intelligence area for a long time.In this dissertation,we propose the Abductive Learning framework to combine them in a mutually beneficial way.On the one hand,Abductive Learning transforms the domain knowledge into supervision information that can be used for training the machine learning model;on the other hand,it uses machine learning to extract primitive logical facts that augment domain knowledge for logical rea-soning and knowledge refinement.Based on this framework,we further proposed the NLM(Neural Logical Machine)algorithm that combines knowledge refinement and deep learning,the experimental results on complex learning tasks shows its ef-fectiveness.