Research On Sequence Ambiguity Function Based On Deep Reinforcement Learning

Radar and communication have been developed and studied as two independent systems for a long time.However,the traditional mode of separation of radar and communication has been unable to meet the needs of miniaturization,integration and multi-function products in today's society.So far,the integration of the radar and the communication system has been a hot research direction in information technology,and the sequence design is one of the key technologies to meet the communication and radar detection performance.The ambiguous function of sequence is an effective tool to evaluate the measurement and resolution performance of radar,and in order to improve the communication transmission efficiency,the sequence needs to have a low peak-to-average power ratio(Peak-to-Average Power Ratio,PAPR).Therefore,the ambiguous function of the sequence is studied in this thesis,aiming to construct a sequence with good ambiguity function performance and low PAPR.This thesis first briefly introduces the optimization method of sequence ambiguous function based on energy gradient method and the optimization method of sequence ambiguous function under spectrum constraint are introduced,but the optimized sequences are more or less limited by the traditional mathematical expressions.Deep reinforcement learning has strong predictive ability and has been successfully applied to the sequence design neighborhood and achieved better results than traditional methods.Therefore,in order to break the limitation of mathematical structure in traditional optimization methods,deep reinforcement learning is considered in this thesis.Among the existing deep reinforcement learning methods,this thesis mainly studies the MCTS-CNN based on monte carlo tree search(Monte Carlo Tree Search,MCTS)and the convolutional neural networks(Convolutional Neural Networks,CNN).The results of the experiment show that,compared with the sequences constructed by the traditional algorithms under the same spectrum constraints,the sequences constructed by the sequence optimization method based on deep reinforcement learning have partially improved performance in the set ambiguity function region,and the sequences are binary sequences,which are more suitable for practical application.Then,in order to improve the algorithm performance,based on this scheme,two improved schemes based on MCTS-CNN are proposed: Firstly,MCTS is improved,and partial Search trees are constructed by adding sequence samples with good performance obtained from genetic algorithm in advance.Secondly,the traditional loss function in CNN is improved.The results of the experiment show that,compared with the original sequence optimization scheme based on MCTS-CNN,the PAPR of the sequence is further reduced,and the ambiguity function performance of the sequence is improved.Finally,in order to further improve the performance of ambiguity function and reduce the PAPR of sequence,an optimization algorithm of the sequence based on generative adversarial network(Generative Adversarial Network,GAN)is proposed.Four simulation experiments were set to verify the influence of different parameters on the performance of GAN algorithm.Compared with the two improved schemes based on deep reinforcement learning,the proposed algorithm can further improve the ambiguous function performance of the sequence.