Research On Cross-modal Person Re-identification Method Based On Supervised Deep Neural Network

With the widespread application of person re-identification technology in the field of intelligent security,this research task is receiving increasing attention from scholars.Traditional person reidentification methods have mostly focused on the problem of person recognition in a single modality.And often the images are captured in well-lit situations,whereas real-life scenarios can also occur in dim light or even darkness,and these scenarios need to be recorded using infrared cameras.Thus,cross-modal person re-identification methods are proposed.Cross-modality person re-identification is performed between images of persons from visible and infrared modalities.It addresses both the modality differences between the heterogeneous data of the visible and infrared images,as well as intra-modal differences due to occlusions,changes in pedestrian poses and changes in camera views.In this paper,the following research is carried out for cross-modality person re-identification.(1)A Contrastive Learning based Tri-Modal Cross-Modal Person Re-Identification(CLTM)method is proposed in order to reduce the significant modality differences between the visible and infrared images.The method generates an auxiliary greyscale-modal image from the visible image,which retains the structural information of the visible image but approximates the image style of the infrared modality,acting as an intermediate modality between the two modalities.This method uses modality-shared single-channel network to learn modal-invariant feature representations.A multimodal contrastive loss is designed based on contrastive learning to reduce the distance between features of the same category and expand the distance between features of different categories.In addition,a multi-modal mean triplet loss is designed to further enhance the compactness between similar features and the separability of dissimilar features.The distribution consistency loss is also designed based on the Kullback-Leibler divergence,so that the differences in the distributions of the three-modal features are reduced.(2)To further address the modal discrepancy problem in cross-modality person re-identification,an Adversarial Learning Based Tri-Modal Cross-Modal Person Re-Identification(ALTM)method is proposed.The method uses the idea of adversarial learning to reduce modal differences.Modalspecific network structure is first used to learn modality-specific low-level features,and then modality-shared network structures are subsequently accessed to learn modality-shared mid-level features.A generator is designed with a feature extraction network,a multi-modal classification learning module,a multi-modal maximum triplet learning module,and a distributed consistency learning module.A modality discriminator is designed to discriminate the modalities of the features,and the minimax game idea is used to optimise the network parameters.(3)In order to fully exploit local and global features,a Global and Local Features based Adversarial Learning Cross-Modal Person Re-Identification(GLAL)method is proposed.The visible and infrared images are fed into a mode-specific two-channel network,which are subsequently accessed to a modelity-shared network for feature extraction.Local features are obtained by the Partbased Convolutional Baseline(PCB)method,and global features are obtained by a global pooling operation.Discriminative features are learned using classification loss,reducing the distance between features of the same class and expanding the distance between features of different classes using triplet loss based on heterogeneous centres.And this method reduces modal differences between visible and infrared features using adversarial learning.To verify the effectiveness of the above methods,experimental validation and analysis were carried out on two datasets,SYSU-MM01 and Reg DB,and the experimental results showed that the proposed methods achieved good recognition results.