Research On A Scheduling Method Of TT&C Resources For Multi-satellite Based On Deep Reinforcement Learning

With the rapid development of commercial aerospace,the number of satellites has shown a trend of increasing in scale.However,the number of Telemetry,Tracking and Command(TT&C)resources in the space TT&C system is relatively limited.Therefore,how to effectively schedule the existing TT&C resources is the key to complete TT&C tasks under the condition of limited TT&C resources.The completion of the TT&C task is an important part related to the normal operation of the satellite and the correct use of its purpose.Therefore,it is of great significance for the study of the scheduling problem of TT&C resources.Due to the complexity of the TT&C background,scheduling of TT&C resources for multi-satellite shows the characteristics of complexity,diversity,and dynamics,and the difficulties of dynamic TT&C scenario,complicated TT&C conflicts,and rational use of TT&C resources exists.In view of these characteristics and difficulties in the scheduling problem of TT&C resources for multi-satellite,deep reinforcement learning is introduced to make optimization decisions to deal with the challenge of massive satellites to the TT&C system.The main research contents are as follows.As for the scheduling problem of TT&C resources for multi-satellite,the essence of the TT&C resources scheduling problem is explored according to the characteristics of each element of the TT&C system.And the mathematical representation of each element in the TT&C scene is explored,so as to complete the mathematical model of the TT&C resources scheduling problem.These explorations lays the foundation for its further modeling and scheduling.As for the problem of rational use of TT&C resources in the multi-satellite TT&C system,a study on the comprehensive performance evaluation index of TT&C resources scheduling performance is launched,which combines three types of indicators: task completion,TT&C resource utilization,and TT&C resource utilization balance.The Analytic Hierarchy Process is used to form the comprehensive evaluation index which evaluates the performance of TT&C resources scheduling.The evaluation index is a standard for evaluating whether the TT&C resources scheduling method is reasonable for the use of TT&C resources,it provides a benchmark for the subsequent evaluation of the TT&C resources scheduling method and the implementation of TT&C resources scheduling.Regarding the problem of complex TT&C conflicts in the scheduling process of TT&C resources for multi-satellite,the Markov decision process model of TT&C resources scheduling is studied.The design method of action,state and reward in Markov decision process model of scheduling problem of TT&C resources for multi-satellite is explored,and its visual description is given.By studying the Markov decision process model,the description of each element in the TT&C resources scheduling problem is more reasonable.In view of the dynamic TT&C scenario in the scheduling problem of TT&C resources for multi-satellite,the frequent interaction between the agent and the environment in the deep reinforcement learning algorithm,which sensing the change of the TT&C resources scheduling scenario in real time is applied to solve the TT&C scheduling problem.By design of the TT&C resources scheduling framework based on the Asynchronous Advantage Actor-Critic(A3C)algorithm(one of the deep reinforcement learning algorithms),the reasonable setting of the TT&C scenario,and the adjustment of the relevant parameters of the algorithm,the synthetic optimization of the scheduling of TT&C resources in the completion of TT&C tasks and the use of TT&C resources is achieved,which verifies the applicability of the A3C-based scheduling method of TT&C resources for multi-satellite.And through the comparison of different scheduling algorithms,the effectiveness of the A3C-based scheduling method of TT&C resources for multi-satellite is verified.