Research On Urban Moving Target Tracking Method With Unknown Movement Intentions

With the accelerating trend of global urbanization,cities(clusters)as economic and political centers will become the focus of future warfare contention.Due to the complexity and specificity of the urban environment,it is extremely necessary to develop intelligent and autonomous unmanned systems to assist in the execution of urban combat missions.Among them,the use of unmanned systems for long-term tracking of suspicious targets in the urban environment is a basic requirement for urban combat missions.However,due to the complexity of the urban environment and target movement behavior,the unmanned systems are prone to lose targets.Therefore,the ability to quickly search for lost targets is required to maintain continuous tracking of the target.In the process of target search,it is necessary to accurately predict the trajectory of the target after it is lost,and to explore efficient search behavior based on the prediction results autonomously.To this end,this paper first predicts the target’s trajectory by combining the target’s movement intentions and behavior patterns,and on this basis constructs an autonomously explorable search policy using the self-learning architecture of deep reinforcement learning(DRL)to improve the unmanned system’s ability to recognize the target’s behavior and the decision planning ability of its own policy when tracking urban moving targets.The main research contents include the following parts.With an unmanned aerial vehicle(UAV)tracking an urban ground moving target as the background of the problem,a mathematical model for tracking an urban moving target using a UAV is established and its difficulties are analyzed.First,a mathematical model of the target tracking task including urban environment,target and UAV is established,and the general objective of the target tracking task is proposed on this basis.Then,we analyze the difficulties of the moving target tracking problem in urban environments and propose a research idea to solve the problem,which decomposes the moving target tracking problem into three sub-problems: target movement intention inference,target movement trajectory prediction and UAV tracking behavior design.We further decompose the UAV tracking behavior design problem into two sub-problems: UAV position control and UAV search behavior planning.The relevant methods for solving the above problems are analyzed,and the theoretical methods used subsequently in this paper are determined.In order to improve the cognitive ability of complex movement behaviors of the urban moving target,we propose an urban moving target motion intention construction and reasoning method.First,for the situation that the set of urban moving target movement intentions is unknown,a multi-level hypothetical set of the target movement intentions is established by dividing the urban environment where the target is located into multiple levels and multiple regions.Its rationality is analyzed in terms of the number of elements in the intention set and whether it contains the real target movement intention.Then,two different inference models of target movement intention are constructed based on dynamic Bayesian networks(DBN)and deep neural networks(DNN)from two ideas:generative models and discriminative models,respectively.The intention inference model based on DBN uses the Boltzmann distribution model to construct the mapping relationship between the target movement states and movement intentions,and can use the observed target movement states to iteratively infer the target movement intentions.The DNN-based intention inference model establishes an end-to-end intention inference network based on a convolutional neural network(CNN),and obtains the target movement intentions by directly processing the discretized target movement states and the urban environment.The methods for determining the parameters of the target movement intention inference models are given simultaneously and four evaluation metrics for the performance of target movement intention inference are proposed.Aiming at the problems of multiple optional paths,strong behavioral uncertainties and difficulty in predicting movement trajectories of the moving target in complex urban environments,a movement trajectory prediction method considering target behavior patterns is proposed.First,in order to characterize the influence of the surrounding environments on the target behavior manner,a target behavior preference model and a target behavior decision model characterizing the target behavior manner are established based on CNN,which makes it possible to effectively handle the environmental features around the target.Second,in order to improve the speed and quality of the training of the models,a model pre-training strategy based on supervised learning is proposed.Then,based on the idea of inverse reinforcement learning(IRL),the learning method of the target behavioral manner is constructed,and the learning method of the target behavioral preference model based on maximum entropy IRL and the learning method of the target behavioral decision model based on DRL are proposed.The alternate learning process of the target behavior preference model and the target behavior decision model is constructed by combining the above two methods,after which the implementation of predicting target movement trajectory using the target behavior decision model is given.Aiming at the problem that it is difficult for traditional methods to independently explore efficient search behaviors under uncertain target movement trajectory prediction results,a UAV search behavior planning method based on DRL and probability distribution of target location is proposed.First,in order to provide reliable guidance for the behavior planning process when the UAV searches for the target,the probability distribution of the target’s appearance in various locations in the urban environment after the target is lost is predicted based on the inferred target movement intentions and the learned target behavior.Then,a UAV search policy construction and training method based on DRL is proposed.In order to effectively deal with the high-dimensional observation states of the UAV,a DNN is used to construct a UAV search policy.Secondly,the reward function of the UAV search behavior is designed to guide the UAV to learn an efficient target search policy under the reinforcement learning framework.Then aiming to improve the use efficiency of the interactive data and increase the stability of the UAV search policy training,the training method of the UAV search policy is designed,so that the UAV can explore efficient search policies through continuous interaction with the task environment.