Research On Feature Mining And Semantic Matching Based Person Re-Identification

With the development of smart cities as well as the growing popularity of the smart cameras,person re-identification(Re-ID)is gradually becoming a key technology in smart surveillance,which plays an important role in visual analysis fields such as video detection,pedestrian tracking,and behavior analysis.Re-ID aims at searching the target pedestrian across non-overlapping cameras and across different times.This task is rather more difficult in surveillance videos due to the limitations of extracting robust appearance features brought by the cross-view and cross-camera data with lower resolution,pose variance and occlusion,which make it difficult for Re-ID models to extract robust features and conduct effective feature matching.This paper focuses on how to extract more distinguishable pedestrian features and how to achieve flexible and robust feature matching.On the basis of deep learning,four novel Person Re-ID frameworks are proposed from the perspective of feature extraction and feature matching.In terms of feature extraction,the part-based convolution methods are optimized in a data-driven fashion and deeply mined of the local features of pedestrians with the help of high-order semantic information of pedestrian attributes.For feature matching,on the foundation of the proposed feature mining methods,a flexible semantic matching mechanism is introduced to address the misalignment and occlusion in Person Re-ID.To summarize,our main contributions are highlighted as follows:Considering the existing part-based Re-ID methods lack of flexibility and adaptability,an adaptive and robust partition method is proposed for person Re-ID with reinforcement learning.Existing part-based approaches either utilize fixed partition strategy or resort to other pre-processing models(such as pedestrian pose prediction,pedestrian segmentation,etc.)to guide the partition.The former is subject to excessive human prior knowledge and the fixed block strategy lacks flexibility in dealing with real-world scenarios,while the latter relies too much on the accuracy of external models and introduces external data,which brings the risk of introducing noise data and injures the stability of partition.In view of this,an adaptive block model via reinforcement learning is proposed,the agent can adaptively partition the pedestrian images according to its characters.The agent is trained on the heldout set and optimized by the Policy Gradient algorithm in the form of supervised learning.In addition,the agent and the entire Re-ID network are jointly trained to ensure the robustness and generalization ability of the model.Considering the exiting attribute-based methods lack for concise and precise feature representations of attributes and rarely considered the relations between attributes,a Re-ID method via deep attribute mining and reasoning is proposed.First,Existing attribute-based person retrieval methods use global features for attribute learning and classification.However,most of human attributes are semantic descriptions for local visual cues,the global descriptor always has noisy information from irrelevant area and channels.On the other hand,correlation and reasoning between different attributes could generate more useful information and add more robustness to the retrieval system.To this end,the proposed method makes better use of pedestrian attributes from two aspects: First,the Attribute Localization Ensemble module is introduced,which is consisted of multiple localization heads and a voting mechanism.Second,the Attribute Reasoning module is designed to correlate different attributes together with the global appearance features and discover their latent relations to generate more comprehensive descriptions of pedestrians.To deal with the misalignment issue in preson Re-ID,a coarse-to-fine attribute-aligned method is presented.The mainstream pedestrian alignment methods tend to adopt pose key points.However,even the best feature representation can hardly take its advantages in the retrieval process without a flexible and robust feature match mechanism.To deal with the misalignment issue in preson Re-ID,a coarse-to-fine attribute-aligned method is proposed.The mainstream pedestrian alignment methods tend to adopt pose key points.However,these strategies are excessive relied on highly-accurate pose estimation.Pedestrian attributes are specific descriptions of distinguishable body parts.Attribute features also contain rich spatial positioning information,so they are suitable to handle pedestrian misalignment issues.At the same time,when using pedestrian attributes for alignment,the importance of different attributes varies a lot.As a result,the proposed method takes good advantages of part-based methd and attribute-method.The part-based method is used to conduct a coarse alignment,then the attribute features are incorporated into the part features for fine alignment.In addition,an attribute selection agent with reinforcement learning is designed to select proper attributes for different pedestrians,which can further refine the attribute features for alignment.To tackle the problem of over-reliance on detection model and lack of flexible matching mechanism in occluded person Re-ID,an attribute disentanglement and registration method is proposed for Occluded Person Re-ID.Existing occluded Re-ID methods tend to use part detectors to align the non-occluded body parts.However,this group of methods are limited by detection accuracy.What's more,the part detector based methods rigidly match the body parts,which will reduce the stability of feature matching and injure the Re-ID performance.In this regard,the high-level semantic of human attributes are utilized to disentangle local features via mining the non-occlude body parts.Besides,the Attribute Registration module is proposed to learn an adaptive match between the attribute disentangled region and the localized region on occluded samples,which can perfom a robust soft alignment and avoid sensitive hard one-to-one alignment.Finally,the four proposed methods are evaluated on the 7 Person Re-ID benchmarks(Market-1501,Duke MTMC-Re ID,CUHK03,Partial-REID,Partial-REID,Partial-i LIDS,Occluded-REID,Occluded-Duke).In these experiments,the proposed method achieved Rank-1=95.8%,m AP=89.2% on Market-1501,Rank-1=87.6%,Rank-3=94.7% on PartialRe ID and Rank-1=83.7%,m AP=72.5% on Occluded-Re ID.The qualitative and quantitative experimental results verify the superiority as well as the generalization ability of the proposed methods.