Research On Person Re-identification Based On Interactive Information And Cross Modality

With the continuous improvement of social video surveillance system,person reidentification(Re-ID),as an important research field of video surveillance,has attracted the attention of the majority of researchers.Re-ID aims to complete the task of identifying person with the same identity in the cross-camera scene.It has important applications in video surveillance tasks such as cross-camera search,suspect recognition and target pedestrian tracking.With the development of deep learning,the method of building deep network has gradually become one of the research hotspots of person re-identification.For the traditional single-modality Re ID,there are challenges caused by different shooting angles,different light intensities and occlusion of person,so how to design a reasonable deep learning network to extract depth features with high discrimination is the direction of mainstream research.For cross-modality Re ID,infrared person images are used to match visible person images,so how to reduce the feature distance of the same person in the two modalities should also be considered.In recent years,with the maturity of person re-identification method,the deep network with multi-branch structure has shown excellent performance.The multi-branch network uses the loss function to calculate the loss value for each branch,and then back propagation and optimization are carried out.Because each branch works in different regions,different branches have their own independent features,and the feature information obtained by different branches is complementary.However,most networks with multi-branch structure ignore the information interaction of each branch in the deep network,and do not use the complementary information of different features.For making good use of the complementarity of feature information,this paper studies the application of deep-level feature complementarity in single-modality Re ID and cross-modality Re ID as follows:(1)In order to promote the information interaction between global and local branches in single modality,Global-map Attention Module(GAM)is proposed.Most multi-branch networks use baseline network to extract the depth features of person images.The global branches work on the whole depth feature area,and the local branches work on part of the depth feature area.So the global branches can pay more attention to the location of person in the image,but the extraction of details is relatively weak,while the local branches can extract the local details of person,but can not get the location information of pedestrians in the whole image.The GAM proposed in this paper determines the area of person through the heatmap generated by global branches,calculates the spatial attention according to the heatmap,and then acts the spatial attention on the local features.In the person image,the person area has more effective information,and the calculated spatial attention can make the local features in the person area get higher weight.For integrating the method of generating thermal map with the training process,this paper also proposes the segmented propagation method to generate thermal map in the process of training network,which saves the steps of generating thermal map separately.(2)In order to promote the information interaction between different global branches in single modality,this paper proposes Labeled-class Mutual Learning(LML).Most methods based on feature learning make different features learn from each other in two directions or assign a teacher network to learn in one direction.LML completes the mutual learning of different global features based on divergence loss,but the learning direction of LML is not specific.For person images,each person image has a unique label,and different global branches extract the global features of the same person image.After the global features are processed by the full connection layer,the prediction vector corresponding to the number of person categories in the dataset can be obtained.In the training process of network,LML dynamically defines the learning direction as the prediction feature with high probability of label item,and the learning direction will be different for different images.(3)In order to promote the information interaction among multiple branches in cross modality,this paper proposes a Multi-modality Feature Complement Network(MFCN).Different from single-modality Re ID,MFCN focuses on narrowing the gap between features of two different modalities.MFCN constructs a baseline with two-stream inputs and threestream outputs to extract features.The images of the two modalities are input into the corresponding input streams respectively,and then the features of the infrared person images and the visible person images are extracted by two corresponding streams respectively.The third output stream is responsible for extracting the shared features of the two modalities.The input of this stream is the shallow features of the other two streams,so two kinds of features can be extracted,which are shared features of infrared image and shared features of visible image.In the process of feature supplement,MFCN uses the characteristics of graph convolution network to specify the direction of feature supplement by designing the relationship between points and edges in graph convolution network,and completes the supplement from single-modality features to shared features,which makes the information of shared features more abundant.Overall,this paper proposes three Re ID methods based on information interaction,GAM,LML and MFCN,and verifies the excellent performance of the proposed methods and the effectiveness of the module on multiple datasets.