Research On Neural Network Control Of Space Manipulator With Deep Reinforcement Learning

As an active mechanism in spacecraft components,space manipulator is playing an important role in the construction of space station and the engineering application of lunar rover.Aiming at the autonomous control of space manipulator,this paper uses deep neural network to realize end-to-end control of space manipulator,and uses deep reinforcement learning algorithm to learn the control strategy of neural network.The main research work is as follows:1.According to the characteristics of space manipulator's operation task,the feasibility of the control method based on joint angular velocity is analyzed,and the task environment of space manipulator based on Markov model is defined,including state,action and reward.In order to improve the flexibility of interaction between agent and environment,an information exchange based on TCP is proposed.2.In order to avoid dimension disasters,the deep deterministic policy gradient algorithm is used in the control of multi-degree-of-freedom space manipulator.Aiming at the application of the algorithm in the control task of space manipulator,a reward function with soft boundary term of angular velocity is designed,which can effectively restrain the divergence of joint angular velocity,thus,the joint angular velocity can be restrained.The policy optimization process of the algorithm is improved and the control effect of the manipulator is better.3.Aiming at the problem that the dynamics of the simulation platform is different from that of the real system,which leads to the need to retrain the neural network in the real system after the strategy migration,a direct strategy migration method from the simulation platform of the manipulator to the real system of the manipulator is proposed,and the effectiveness of the method is verified by experiments.4.Aiming at the image-based end-to-end control task of space manipulator,a deep learning control method for space manipulator is proposed.This control method directly uses data-driven autonomous learning to generate control strategies.It does not need to rely on dynamic models.By dividing the depth strategy network into depth network and strategy network for training,it can be independent of the real manipulator system in the training process.The effectiveness of this method is verified by experiments.In this paper,the task environment,continuous control,strategy migration and reinforcement learning of space manipulator based on deep reinforcement learning neural network control are studied.The research results can be used for reference for space manipulator to complete on-orbit operation tasks in unknown environment.