Research On Convolutional Neural Network Policy For Manipulator Based On Deep Reinforcement Learning

As an important part of the applications of robotics,because the household robot always works in complex environment and must deal with unpredictable and various kinds of operation tasks,the requirements of the algorithm inside these robots are very strict,and how to improve the intelligence of the household robot through algorithms has also become the key to the development of these robots.In view of the urgent requirements of improving the intelligence of the household robot,this thesis has studied the most basic object placement tasks in housework through deep reinforcement learning algorithm based on policy search.And aiming at the problems that the parameter initialization process of current policy search algorithms based on optimal control is random,the structure design of Convolutional Neural Network(CNN)policy is difficult and the training time is very long,new methods of parameter initialization and optimization of the manipulator controller and the design and training of CNN policy are studied and verified by simulations and experiments.The main researches are as follows:First,for the problem of random parameter initialization in the current policy search algorithms based on optimal control,the idea of linear Kalman filter position-velocity prediction model is used to initialize the controller parameters based on local environment dynamics model.According to the agent of policy search algorithm,the accuracy of the controller is improved by assigning corresponding weights to the cost of each time step and the optimization of the controller expression.Simulations are performed to verify the effectiveness of the methods above and the learning ability of policy search algorithm.Then,for the problem that policy search algorithm needs to relearn the controller when the target pose changes,a CNN policy is constructed to make the manipulator have adaptability to the pose of objects and complete end-to-end control of the agent.For the problem that the structure design of CNN policy is difficult,a network with vision layer and motor control layer is constructed and the problem of feature drowning is effectively solved by adding a fully connected layer to the vision layer.For the problem of long training time of CNN policy,an effective pretraining method is proposed to shorten the training time and make the training process smoother.Simulations are also performed to verify the effectiveness of the methods above and the CNN policy constructed.Finally,the policy search algorithm and the CNN policy of manipulator are further verified through experiments.The learning ability of policy search algorithm is further tested by making the manipulator perform object placement tasks with different target poses and operation objects,and the end-to-end control effect and adaptability to the target pose of CNN policy is further verified by training and testing the CNN constructed and carrying out a “blind eye” experiment.