Autonomous Path Planning Method Of Stratospheric Aerostat Based On Markov Decision Process

Stratospheric aerostats are the current research focuses in the field of near space,which have great advantages of their ultra long duration,wide coverage,high cost-efficiency ratio,and have broad application prospects in the areas of earth observation,reconnaissance and early warning,communication relay and scientific exploration.Autonomous path planning capability is an important prerequisite for the rapid deployment of stratospheric aerostats to the target area to ensure the effective implementation deployment of the mission,and it is also one of the key technologies that restrict future large-scale applications.The flight processes of stratospheric aerostats are greatly influenced by the wind field,and there are uncertainties in wind fields such as the spatio-temporal variations,forecasts and measurements,which lead to a large challenge in autonomous path planning of stratospheric aerostats.This paper studies the autonomous path planning for stratospheric aerostats with vertical actuation and capability of the height adjustment in uncertain wind fields based on the related models of Markov decision process,and the main research contents of this paper are as follows.A 2D global path planning method for stratospheric aerostats based on MDP is studied.Based on the basic framework of MDP,the states space and actions space are discretized,and the 2D global path planning method is designed.Considering the errors existing between the predicted wind fields and the actual wind fields,the uncertainty of wind field is introduced into the MDP model,and the deterministic wind field and the uncertain wind field are established.2D global path planning is carried out in the two wind field scenarios respectively,and the effects of different levels of horizontal actuations and wind field models on the reachability in a given region relative to the target,the optimal path and the optimal action sequence are studied.A 3D global path planning method for stratospheric aerostats based on MDP is studied.By integrating the ascent process and stationary process,a 3D global path planning method is proposed,which is based on MDP.According to the actual dynamics of stratospheric aerostats,the basic action space,the immediate reward function and the state transfer probability function are designed.This method can realize 3D global path planning from the start position on the ground to the target position under complex constraints such as the 3D wind fields,the horizontal actuations,the altitude control capability and the flight time.As a result,this method can provide a theoretical and methodological support for the flight decisions and dynamic deployment of stratospheric aerostats.A local path planning method for stratospheric aerostats based on the continuous POMDP is studied.To address the problems of the approximation for the global path planning in the discrete states and the partial observability of wind fields,a local path planning method based on the continuous POMDP is proposed.The Gaussian distribution is used to represent the belief state including the position,velocity and wind fields of stratospheric aerostats,and the extended Kalman filtering method is used to realize the belief state update.By using the method of LQG,the local optimal value function around the initial path is calculated at each time step,the optimal actions sequence is determined based on the optimal value function,and a new path is planned on the basis of the initial path by the dynamic planning.Research results of this paper provide new ideas and methods for autonomous path planning of stratospheric aerostats in the real wind fields,provide the technical support for the rapid and autonomous deployment of stratospheric aerostats to the target position,and the application of Markov decision process in the field of intelligent autonomous system about path planning is expanded.All these have important theoretical significance and engineering application value.