Research On Cross-Resolution Visible Thermal Cross-Modal Vehicle Re-Identification Algorithm

In several fields such as intelligent transportation systems,crime monitoring,and military battlefield applications,cross-modal trajectory prediction of moving targets captured by UAVs equipped with different sensors is required to obtain the trajectory of the target,the historical trajectory of the target and the relationship between the targets,to perform reasonable situational analysis of the target situation,and to automatically sense the movement intention of the target and the existence of potential threats.Among them,the target re-identification technique has a strong advantage in facing the problem of matching the association of different simultaneous targets captured by different sensors.Due to the wide application range of visible and thermal sensors,this study focuses on cross-modal re-identification between visible and thermal modalities,with a particular emphasis on vehicle data captured by unmanned aerial vehicles.The primary objective is to investigate the effectiveness of cross-modal re-identification in the visible and thermal domains.Most of the current advanced single-mode re-identification algorithms are performed under similar high-resolution conditions.However,due to the uncertainty of UAV flight altitude and the different imaging principles of each sensor,there are large resolution differences and modal differences between the actual matching data.To address the above problems,to reduce the modal differences between the visible thermal images while resolving the resolution differences,the following research on cross-resolution visible infrared cross-modal vehicle re-identification is conducted in this paper:(1)To address the problem of a single source of cross-modal re-identification data,this paper uses a UAV equipped with visible and thermal sensor modules to manipulate the UAV to shoot at different flight heights from different viewpoints in a real traffic environment and designs a new cross-resolution visible thermal cross-modal vehicle re-identification dataset named VT-Vehicle.(2)To address the weakness of the super-resolution reconstruction network in cross-resolution re-identification research,a cross-resolution re-identification architecture based on super-resolution reconstruction is proposed.On the one hand,Trans Former and CNN are used to search contextual information by combining dynamic and static to enhance feature representation.On the other hand,the normalization strategy with different depths of the network is used to obtain the invariant information of cross-resolution images.After several experimental evaluations,Rank-1 and m AP on the MLR-Ve Ri776 dataset improve by1.4% and 0.9%,respectively,over the current state-of-the-art algorithms,which validates the effectiveness of the proposed architecture.(3)To reduce the inter-modal differences,a visible thermal cross-modal re-identification architecture based on modal cross-graph sampling is proposed.On the one hand,from data sampling,the nearest neighbor graph structure is constructed using the similarity of different modal samples to explore the potential association between modes.On the other hand,from the network structure,a novel dual-stream network is proposed,with the shallow layer retaining each mode-specific feature and the deep layer embedding multiple Trans Former and CNN joint modules,so that the overall architecture can reduce the differences between different modal images while retaining the cross-resolution capability.The evaluation results in both thermal search for visible and visible search for thermal test modes are improved compared with current advanced algorithms,which validate the superiority of the proposed architecture.