Design Of Terminal Guidance Law Based On Evolution Strategy And Actor Critic Structure

Guidance law is an algorithm that plans the optimal flight path for unmanned aerial vehicles.It is often used in the control systems of unmanned aerial vehicles and missiles.Most current guidance laws are based on proportional navigation.The basic principle of this method is to limit the rotation rate of the line-of-sight angle and make it proportional to the rotation speed of the velocity vector.Modern technology continues to improve,and more technologies are also applied to the field of aircraft design,which greatly enhances its maneuverability and its maneuvering methods.The control method based on proportional navigation exposes many shortcomings when it encounters highly maneuverable targets at close range.These methods need to be compensated according to maneuvering,and cannot adapt to the complex maneuvering method of the target,which affects the method’s ability to capture the target.Deep reinforcement learning has been successfully applied to many control problems,and it has become a popular research questions in recent years.One feature of this method is that it can omit complicated feature engineering and give an end-to-end control scheme.For the problems mentioned above,this paper attempts to apply deep reinforcement learning to design a new guidance control method.In this paper,the reinforcement learning modeling is carried out according to the characteristics of the guidance problem,and the design of the Markov decision process is given.In this paper,the evolution strategy algorithm and the DDPG algorithm are used to solve the guidance and control problem.The algorithm details and the agent training process are adjusted according to the characteristics of the task,so that it gives a control method that meets the current task.Reinforcement learning agents use a simulation environment during training.The simulation environment used in this paper has many details,close to the real world situation,which can improve the reliability of the results.At the same time,the evolution strategy algorithm and DDPG algorithm selected in this paper are both excellent methods in the field of deep reinforcement learning.In recent years,there have been many successful application examples in the field of control,which can meet the requirements of guidance problems.The guidance laws currently in use are mostly based on proportional navigation.Due to their own characteristics of proportional navigation,these methods have a low success rate when attacking targets with high maneuverability at close range.Theresearch method in this paper attempts to get rid of the limitation of proportional navigation and apply deep reinforcement learning to give an end-to-end control method.The guidance law obtained by the final guidance law learning method based on the evolution strategy in this paper can adapt to different target maneuvers faster and the miss distance is within a reasonable range.The disadvantage is that the stability of the result is slightly worse.In this paper,the final guidance law learning method based on the DDPG is better than the proportional guidance law in most experimental settings.According to the simulation experiment results,the two methods given in this paper have excellent performance in the experiment.The research method in this paper can be used as a general process of guidance law learning.Th method in this paper does not require a lot of professional knowledge during the agent training process,which reflects its potential application value,and can also provide reference for researchers when designing new guidance laws.