Meta Learning Algorithm Based On PAC-bayes

Machine learning algorithm has been widely used in image recognition,natural language processing,recommendation system and other practical problems.However,the excellent performance of most machine learning algorithms depends on the support of massive data.At the same time,it is time-consuming and laborious in the process of model training.It requires users to provide enough computing resources,which results in that machine learning algorithms can only play their due role in organizations and institutions with enough resources.In addition,both parameter adjustment and model selection are tedious and inevitable operations using a machine learning algorithm,which will consume a lot of human and material resources,and these problems have become one of the most concerned areas of machine learning field.Meta learning,also known as ‘learning to learn',aims to design a kind of model that can quickly learn new skills or adapt to new environment with only a few training samples.In artificial intelligence system,meta learning can be simply defined as the multi-function of acquiring knowledge.As the smartest creature,human beings can accomplish all kinds of tasks quickly with a little information.For example,humans can recognize a new object only by looking at it once,or they can learn more complex tasks,such as learning to drive a vehicle through a driving process.Although artificial intelligence systems can accomplish complex tasks,they still perform poorly in multi task processing.Therefore,in order to achieve real ‘intelligence',it is necessary for the artificial intelligence system to ”learn how to learn”,which is the problem considered by the meta learning algorithm.In this paper,we propose a new meta learning algorithm,which is a model-agnostic meta learning algorithm.It is compatible with any model using gradient descent training,and is suitable for various machine learning problems,including classification,regression and reinforcement learning.Our algorithm is based on Bayesian framework,but different from similar work.Most of the existing researches consider merging the data independent posterior distribution information of each task into the same data-independent prior distribution,and using the prior information to accelerate the learning process of new tasks.This assumption requires that each task has a strong similarity to ensure that the public prior distribution works.However,because the data distribution is unknown in most cases,it is difficult to guarantee this hypothesis.In order to increase the flexibility of the hypothesis,this paper proposes a new meta learning generalization error bound based on PAC Bayes theorem,which allows us to use a prior distribution related to data.In addition,by setting the posterior probability as a Gibbs distribution,we introduce the optimization algorithm Entropy-SGD,which gives a method to minimize the error bound.Experiments on real datasets show that our method can show better results than the traditional ones in meta learning tasks.