| Aircraft carrier and carrier-based aircraft weapon systems are the most powerful weapon systems in the Navy,but due to factors such as the narrow runway,deck movement,and complex sea conditions,carrier aircraft landing has always been regarded as a difficult and high risky task.The automatic Carrier landing system is the basic guarantee for all-weather high-intensity combat capability.It can guide and control the carrier-based aircraft to land safely under various weather and sea conditions,which greatly reduces the difficulty of landing.The research and development of carrier-based aircraft landing system is of great significance to improve China’s naval defense military strength.Reinforcement learning has inherent advantages and successful practices in solving sequence decision problems,we attempt to solve the problem of automatic landing of carrier-based aircraft through reinforcement learning.However,the traditional reinforcement learning method requires the agent to be in a stable environment,and the landing environment is often in dynamic changes,which has a great impact on the training and application of automatic landing agents.Therefore,in addition to studying the automatic landing algorithm,we also study how to improve the generalization ability of reinforcement learning agents for the landing environment that changes a lot,the main research contents and the results are as follows:Firstly,the main influencing factors of the carrier-based aircraft landing process under dynamic environment are analyzed.The analysis revealed that the movement of the aircraft carrier leading to deviations from the ideal landing point,which changes the goal of the landing mission,and atmospheric disturbances cause the aircraft position to deviate from the standard glide slope,and changes the environmental dynamics model,which affects the training process.Therefore,the aircraft carrier movement and atmospheric disturbance during landing are considered as the main factors affecting the success rate of landing,and the corresponding settings are used in the simulation experiment environment to simulate the dynamic changing landing environments.Then,an automatic landing algorithm for carrier-based aircraft based on DDPG was designed.In response to the problems caused by deck movement and atmospheric disturbances,on the one hand,an MDP model that can adapt to the movement of the aircraft carrier was established;on the other hand,stratified sampling is used to replace simple random sampling,which can reduce the changes in the data distribution caused by atmospheric disturbances and the fluctuations generated during training.Experiments are carried out in the simulated experimental environment,and it is verified that the algorithm can adapt to deck movement and atmospheric disturbances during landing.On this basis,the generalization ability of the algorithm to different landing environments is verified by changing different aircraft carrier speeds.Further,for the landing environment that is quite different from the training environment,different landing environments are regarded as different reinforcement learning tasks,and a model-based transfer reinforcement learning(MBTRL)is proposed.MBTRL learns the strategy model and the environment model in the source environment at the same time during pre-training.When it is applied in the target environment,the source environment model is transferred first,and then the environment model is used to help the source strategy model transfer quickly for the current landing environment.We conduct experiments under a unified reinforcement learning experiment platform.The experimental results show that the learning speed and jump start of MBTRL are significantly improved,and the improvement of asymptotic performance is more obvious when the environment difference is large.Then,MBTRL is applied to the landing problem,and the landing experiments in different landing environments are carried out under the simulation experimental environment,the experimental results show the good adaptability of MBTRL,and MBTRL can effectively solves the problem of lower landing success rate when the landing environment changes greatly. |
