Research And Implementation Of Satellite Onboard Scheduling Algorithm Based On Reinforcement Learning

With the advancement of imaging satellite technology,a growing number of satellites have become capable of autonomous task execution.Autonomous task planning is one of the important directions in the development of imaging satellite autonomy technology.It can improve the autonomous and emergency response capabilities of imaging satellites.Conducting research on autonomous task planning for imaging satellites is of great significance for improving satellite autonomy,task execution efficiency,data acquisition capability,and expanding satellite application fields.This thesis investigates the current research status of autonomous task planning for imaging satellites and finds that the existing research on autonomous task planning for imaging satellites is mainly based on heuristic search algorithms,which are difficult to adjust planning schemes in a timely manner based on dynamically arrived observation tasks and limited on-board computing resources.To effectively solve the autonomous task planning problem for imaging satellites,this thesis establishes imaging satellite autonomous task planning models for both single and multiple satellites.The specific work includes the following aspects:1.To address the characteristics of the single-satellite autonomous task planning problem,such as the variable number of user observation requests and the dynamic arrival of observation tasks,this thesis designs and implements an effective method,D3QN-LSTM,based on reinforcement learning and recurrent neural networks to solve the problem.This method combines the deep reinforcement learning algorithm D3 QN with the long short-term memory network LSTM to better handle dynamically arriving time-series observation tasks and make autonomous decisions based on the importance of different tasks.First,a mathematical model based on the Markov decision process is established for the single-satellite autonomous task planning problem,with the goal of finding the optimal strategy that maximizes cumulative rewards.Then,the deep reinforcement learning D3QN-LSTM algorithm is used to autonomously decide whether to execute the task.Simulation experiments are conducted and compared with the first-come-first-served algorithm and deep reinforcement learning algorithms D3 QN and A3 C.The results show that the proposed algorithm can handle sequence data,adapt to dynamic environments,and improve the accuracy and stability of decisions.2.For the multi-satellite autonomous task planning problem,which has a long task sequence and complex constraints,a solution method based on reinforcement learning and pointer networks is proposed.The multi-satellite autonomous task planning problem is transformed into a sequence-to-sequence decision problem for solution,where the input sequence is the set to be solved,and the output sequence is the solution to the multi-satellite task planning problem,with the goal of finding a relatively optimal solution in a finite set.Finally,simulation experiments are conducted to compare the proposed solution method with the optimization solver CPLEX,and the results show that the planning benefits output by the proposed method are close to the optimal planning benefits output by the CPLEX optimization solver,indicating that the proposed method can effectively solve the multi-satellite autonomous task planning problem.3.Based on the algorithms studied in this thesis,an intelligent planning system for satellite imaging tasks based on the BS architecture is designed and developed.In the system,users can select regions of interest for observation and generate observation tasks,and the system can use the trained model to plan tasks specified by users to maximize the observation benefits.