| In recent years,with the continuous progress of satellite technology,the need and importance of satellite remote orbit change control is growing day by day,and it has been widely used in many fields.The control of satellite orbit change directly affects the orbit and operating state of satellite,which is very important for the advancement of space technology.However,in the actual satellite control,due to the complex and changeable space flight environment and the lack of satellite autonomy,it is extremely difficult to achieve the task,which increases the probability of mission failure.In addition,traditional methods such as C-W equation can not realize the relative motion control of remote satellites based on elliptical orbit.Therefore,thesis paper makes an in-depth study of the problem of single satellite and multi-satellite long-range orbit transformation to reach near the target orbit.The main work is as follows:First of all,for the problem of satellite remote orbit change to reach the target orbit under the single-satellite system,this paper proposes an improved deep reinforcement learning-based twin-delayed deep deterministic policy gradient algorithm(Twin Delayed Deep Deterministic policy gradient algorithm,TD3).By simulating the discontinuous ignition pulse of the satellite through this algorithm,the remote orbit change control of the satellite is realized.Firstly,a mathematical model and a visual vector model of satellite orbit change under the simulation environment are established for this problem.Secondly,a TD3 control algorithm under deep reinforcement learning is proposed to simulate the satellite ignition operation;during this period,Gaussian noise is introduced to increase the search range of the network.Experience and optimize the strategy,and finally reach the target orbit.In addition,the convergence speed of the algorithm is accelerated by adding Z-score dynamic data processing.Finally,the design and deep neural network deterministic policy gradient algorithm(Deep Deterministic Policy Gradient,DDPG)comparison experiment and the importance of each parameter experiment show that the proposed improved version of the TD3 algorithm can effectively control the relative distance of the satellite so that the satellite reaches the target near the track.Secondly,for the problem of satellite remote orbit change to reach the target orbit under the multi-satellite system,this paper proposes an improved Multi-Agent Twin Delayed Deep Deterministic Policy Gradient algorithm(Multi-Agent Twin Delayed Deep Deterministic Policy Gradient,MATD3)based on deep reinforcement learning.Firstly,a mathematical model of satellite orbit change is established for the multi-satellite system.Secondly,a MATD3 algorithm based on the Leader-Follower framework is proposed.In order to eliminate some unreasonable actions,the idea of pruning is introduced to prune the actions.,so as to speed up the convergence speed of the experiment;again,through the design of a reasonable reward function and the continuous information exchange and learning experience between the leader and the follower,the satellite reaches the vicinity of the target orbit;finally,the simulation experiment shows that the improved version of the method can solve multiple problems.The satellite reaches the problem near the target orbit,and it has faster convergence than the traditional method of MATD3. |
