Research And Implementation Of Reinforcement Learning Method About Transport Strategy Between Carrier-based Aircraft Station

At present,aircraft carrier platforms are increasingly relying on computer simulation technology to study aviation operations at home and abroad.As a carrier's direct combat capability,the carrier-based aircraft's efficiency and tactics in translating positions among combat missions have a great impact on combat effectiveness.This paper does not use the way of traditional manual planning path.In order to reduce the workload of human operations,improve the intelligence of a large military demonstration system,this paper uses deep reinforcement learning.Paper has trained a large number of algorithms,allowing the program to get the kinematics model of the carrier aircraft automatically,meeting the outbound and inbound staging requirements of the carrier aircraft.The strategy is applied in the actual combat simulation demonstration system.First of all,the paper designs and implements a two-dimensional scene modeling tool,which can migrate the original simulation deck environment into a reinforcement learning environment.It can model the scenes for algorithm and the carrier aircraft,and divide the area based on the layout of the deck entity,and design a training unit that conforms the kinematic model of the carrier aircraft;Afterwards,by constructing different ways to reinforce the basic elements of learning,two research ideas based on deep reinforcement learning are proposed,including dynamic grid straight ahead and random walk within the angular interval.The two ideas ensure the final transshipment strategy complies with the carrier-based aircraft kinematics specification by rasterizing the scene dynamically,based on the minimum turning radius of the carrier aircraft,and solving single-step steering angle intervals when the carrier aircraft transits.Next,for each research idea,the dissertation designs and implements two different reinforcement training methods for inter-station transit strategy training.In first idea,the algorithm of Q-Learning and Sarsa(lambda)are used to obtain the polyline path of the transshipment,and then one algorithm is used to obtain a better transshipment strategy.In another research idea,the paper designs and implements a deep reinforcement learning algorithm based on DQN and DDPG.Through the algorithm's extensive training on the free-walking process of the carrier aircraft in the deck scenario,a relatively good inter-stand transfer strategy is learned.The visual path assessment tool is designed to make intuitive understanding of the algorithm results.Finally,a large number of experiments are conducted to compare the training efficiency and training effectiveness of the different reinforcement learning algorithms in two research ideas.The seven factors in the transit strategy are normalized by using evaluation function of the strategy.After a large number of experiments and practical applications in the project,the design and implementation of the inter-station transfer strategy algorithm based on deep reinforcement learning satisfies the needs of all aspects of the simulation.It reduces the workload of manual layout greatly and improves the intelligence of the original simulation system effectively.