Research On Q-learning Path Planning Method For Hypersonic Vechicles

In recent years,hypersonic vehicles have made breakthrough developments in the field of aerospace technology,and have become the focus of world attention with their unique advantages.Hypersonic aircraft has the characteristics of high altitude,high speed and high maneuverability.For future air and space operations and commercial applications,the intelligence based on hypersonic flight has extremely important significance.Path planning is an important technology for hypersonic aircraft to perform flight missions,especially when it needs to face unknown threats from the ground or space.The purpose of aircraft path planning is to find the optimal or sub-optimal path to effectively avoid threats under multiple constraints such as maneuverability,enemy threats and flight time.As the difficulty of air missions continues to increase,especially in an unknown environment,path planning needs to consider the impact of more uncertain factors.There is an urgent requirement for aircraft to have the learning ability to adapt to the uncertainties of environmental changes.With the development of the field of artificial intelligence,the development of reinforcement learning technology in machine learning makes path planning not only less dependent on environmental models,but also no longer requires the environment to provide prior knowledge.It is achieved through continuous trial and learning.Therefore,the use of reinforcement learning for path planning to improve the adaptability of aircraft to unknown environments has important research significance and application value.Aiming at this research direction,this article has carried out the following research:(1)Research on basic path planning method under Q-Learning algorithm.To study how the Q-Learning algorithm is applied to the path planning problem.The advantage of this algorithm in path planning lies in its own adaptability to the environment and self-learning ability.At the same time,the algorithm is not necessary for environment map modeling.These characteristics enable objects that use the algorithm for path planning to deal with unknown situations in the course of motion by themselves.Through simulation experiments,this paper proves that the Q-Learning algorithm can effectively solve the path planning problem.(2)Research on fast search Q-Learning path planning algorithm.Since the classic Q-Learning algorithm is applied to the path planning problem,there are many problems including the balance between learning and utilization,and the convergence speed of the algorithm.The fast search Q-Learning path planning algorithm is researched by combining these problems.On the basis of the classic Q-Learning algorithm,Q-Learning is improved,and the fast exploration strategy and action search strategy are introduced to improve the fast convergence of the classic algorithm.(3)Research on path planning algorithm of hypersonic vehicle.Research the aircraft’s own maneuvering constraints and flight dynamic constraints,modeling its various constraints;establish a path planning simulation scenario for hypersonic vehicle threat zone avoidance.Establish path planning simulation scenarios for hypersonic vehicle threat zone avoidance.Aiming at the path planning simulation scenario of hypersonic vehicle threat zone avoidance,the Q-Learning algorithm is researched as a path planning algorithm under dynamic constraints.Finally,a flyable path that can successfully avoid obstacles and reach the target point is generated for the aircraft.And the difference between the effects of the two algorithms is verified through simulation experiments.The experiment shows that the Q-Learning algorithm can obtain a path to avoid threats while meeting the constraints of the aircraft,and the fast search algorithm is more effective.This thesis uses reinforcement learning theory to carry out method research and simulation experiment verification on the path planning problem of hypersonic vehicles.It proves that the Q-Learnnig algorithm can effectively solve the path planning problem and meets the flight constraints of the hypersonic vehicle.The algorithm can also obtain an ideal path that can reasonably avoid threats,which provides a theoretical basis for the development of autonomous control technology for hypersonic aircraft.