Research On Person Re-identification Based On Learning Multi-modal Feature Representations

The main task of person re-identification is to retrieve the specific person in the images or a series of video frames.In computer vision,person re-identification is a hot problem,which plays an important role in visual applications such as intelligent security,unmanned supermarkets and behavior analysis.In the recent years,researchers have made remarkable achievements in the person re-identification,and achieved high accuracy in specific datasets.However,due to the uncertainty of the environment,such as the influence of background clutters,illumination changes,viewpoint,occlusions and other factors,person re-identification is a challenging subject.With the rapid development of thermal infrared sensor,depth sensor and other sensors,multi-modal person re-identification has caught more attention.Therefore,this thesis focuses on the complementary advantages of multi-modal person re-identification and how to obtain the infrared information by using generative adversarial network in single modality.The specific research work and contributions are as follows:(1)Aiming at the problems of illumination limitation and environmental factors in the single modality person re-identification,this paper proposes a novel progressive fusion network which designed to utilize complementary advantages of multiple modalities including visible,near infrared and thermal infrared for robust multi-modal person re-identification.The network is designed to learn effective features from single to multiple modalities and from local to global views by using three different modalities and two fusion branches.Moreover.We contribute a comprehensive multi-modal benchmark person re-identification dataset,RGBNT201,including 201 identities captured from various challenging conditions in three modalities.The performance of our method is more effective than the state-of-the-art methods in our dataset.(2)Multi-modal person re-identification can well relieve the challenges and problems in visible person re-identification.However,most of the surveillance environments are based on single visible only,which results in the lack of thermal infrared data resource for person reidentification.Therefore,missing modalities become a serious problem for person reidentification.Aiming at the problem of utilize the multi-modal information in single modality,this thesis proposes a multi-modal generation representation learning framework for single modality person re-identification.We use the generative adversarial network to translate labeled RGB person images to thermal infrared ones,trained on existing RGB-T datasets.Then,we employ the channel-spatial attentions in our network to automatically learn the important information when fusing RGB and thermal representations for robust RGB person Re-ID,which can learn effective representations by leveraging the complementary advantages of RGB and thermal infrared modalities.Extensive experiments on prevalent RGB person Re-ID datasets including Market1501,CUHK03 and DukeMTMC-reID datasets demonstrate the effectiveness of our method.