Distributed Cooperative Task Planning Research Of Earth Observing Satellites Based On Agent

Earth Observing Satellites (EOSs) obtain information of the Earth’s surface fromouter space with on-board sensors, which have the advantages of wide area coverage,long duration, unrestricted airspace, and national boundaries etc. Thus they are widelyused in diverse application fields like military reconnaissance, disaster prevention andmitigation, environmental protection. Therefore, EOSs have become an importantmeans of obtaining the remote sensing information.In recent years, the number of satellites keeps increasing meanwhile theirintelligence are being improved correspondingly. Facing the complicated and massiveobservation requests, how to collaboratively plan and schedule multiple satellites toimprove the observation efficiency has received significant attentions of satellite taskplanning community. It is of great theoretical and practical value both in military andcivilian areas. Based on satellites’ cooperation, this dissertation studies themulti-satellite cooperative task planning problem in different task planningenvironments. The main contributions of this dissertation include:(1) The key elements of multi-satellite cooperative task planning problem areanalyzed, and a distributed framework of solving the problem is presented.Starting with the basic planning problem, the key elements involved in themulti-satellite task planning problem are analyzed in depth. After that, the elementsincluding satellite resources, satellite capabilities, targets to be observed, and networkcommunications are summarized, and the formalized models are establishedrespectively. Moreover, the characteristics of the problem are analyzed, including thedispersibility of the system, collaboration of the observation resources, complexity ofnetwork topology, and uncertainty of the environment. A logical process of autonomousplanning is established based on the hierarchical physical structure of the single satellite.Then, a distributed architecture for multi-satellite cooperative task planning is proposed.Based on the above analysis, the multi-satellite cooperative task planning problem isrefined to three interrelated key sub-problems, i.e. multi-satellite cooperative taskplanning under the conditions of real-time access to information, multi-satellitecooperative decision-making optimization under the conditions of randomcommunication delay, and dynamic task oriented multi-satellite cooperative taskallocation.(2) To solve the problem of multi-satellite cooperative task planning under theconditions of real-time access to information, a Distributed Constrained OptimizationProblem (DCOP) model is presented. Moreover, a distributed iterative algorithm basedon Nash optimality and cooperative coevolution is proposed.After analyzing the distributed cooperative framework of the satellites and the constraints of each satellite, a multi-satellite distributed cooperative task planning modelis established based on DCOP. Then a multi-satellite distributed iterative algorithmbased on Nash optimality and cooperative coevolution is presented according to the“divide-and-conquer” strategy under the conditions of real-time access to information.In our method, the observation requests set is decomposed by the k nearest neighbor(kNN) algorithm, and each satellite can adequately collaborate with others by usinginformation obtained from the network. Experimental results show that the proposedalgorithm can be flexibly adapted to the cases with different scales and distributions.Although the algorithm has some differences on observation effectiveness comparedwith the centralized algorithm, it greatly decreases the optimization problem scale, thusreduces the computational complexity both in theory and practice effectively.Consequently, the proposed algorithm is both feasible and practical in improving thecapability of multi-satellite cooperative planning.(3) To solve the problem of multi-satellite cooperative decision-making under theconditions of random communication delay, a multi-satellite cooperativedecision-making model is established, a unified processing policy is proposed to adaptto communication delay, together with a method to search for the optimizing policy.A multi-satellite cooperative decision-making environment is constructed based onCollaborative Time Directed Acyclic Graph (CTDAG). Focusing on the multi-satellitecooperative decision-making system which is a dynamically decoupled system, adistributed system modeling method is discussed based on the idea of DECentralizedPartial Observable Markov Decision Process (DEC-PODMP). The centralizedoptimization problem is transformed into a limited independent distributeddecision-making problem. Then, to solve the problem, the impact of differentcommunication delay on decision-making is analyzed, which results in a unified policyto process communication delay. Moreover, a Modified Very Fast Simulated Annealingalgorithm (MVFSA) is proposed to search approximate optimal observing policy.Experimental results show that: MVFSA improves both the efficiency and effectivenessof searching process compared with the current simulated annealing algorithm. Inaddition, the performance of cooperative approximate policy iteration algorithm basedon MVFSA is superior than the value iteration based algorithm. Thus our method caneffectively improve the applicability of the decision-making system to communicationdelay.(4) A multi-satellite hybrid learning framework is established for the multi-satellitedynamic task allocation problem. A task allocation policy algorithm is proposed forstatic tasks and an incremental policy transfer learning algorithm is presented fordynamic tasks..Firstly, a decision-making model is established for the dynamic task oriented.multi-satellite cooperative task allocation. Noticing that the dynamic tasks are the extension of the historical observation target set, we present a hybrid learningframework composed of “optimizing policy-transferring policy” to improve theplanning quality. After that, the two core problems are studied under this frameworkrespectively. When the tasks are static, a Multi-Agent Cooperative NEuroevolution ofAugmenting Topology (MACoNEAT) is proposed to iteratively search for the optimaltask allocation policy. And when some new observing requests come, i.e. when tasksare dynamic, an Incremental Task Policy Transfer Learning algorithm (ITPTL) ispresented. In ITPTL algorithm, the historical learning policies are transferred to thecurrent solution space according to the similarity among the policy individuals.Experimental results show that: when the task set is static, MACoNEAT could get betterobserving efficiency in the premise of longer computing time. When new observingrequests are added, the hybrid learning algorithm composed of MACoNEAT and ITPTLcan speed up the computing convergence meanwhile assure the optimizing quality.Therefore the proposed hybrid algorithm has good adaptability to the dynamic randomtasks.