Video-based Person Re-identification In Natural Scenes

In recent years,with the development of image processing and computer vision,person reidentification has gradually been applied to areas such as smart communities,public security,and intelligent transportation.On account of rich spatial-temporal information,video-based person re-identification based on convolutional neural network has attracted the attention of domestic and foreign researchers.However,most of the existing algorithms are suitable for ideal scenes,so there are realistic significance and promising application for the research of video-based person re-identification in natural scenes.There are several persons who wear similar appearance in natural scenes,causing confusion of features and reducing the accuracy of re-identification.What's more,due to crowds,persons may reappear after being temporarily occluded.The occluded frames mislead the retrieval of query features in the gallery and reduce the accuracy of re-identification.Focusing on above two issues,the main contributions are as follows.(1)To solve the difficulties of similar appearance in natural scenes,a video-based person reidentification algorithm based on pose-appearance parallel feature(PAPF)is proposed.The skeleton key-point branch based on high-resolution network(HRNet)and the appearance branch based on residual network(Res Net)are combined in parallel.Each channel of keypoint heatmap is looped through and reshaped to match the dimension of appearance feature.The 15 groups of reconstructed key-point feature and the appearance feature are multiplied element by element.Thus,the feature values near key points are enlarged to highlight the differences of appearance.After concatenating the 15 groups of corrected features with the original appearance feature,the dimensionality reduction is performed.Without increasing the appearance feature dimension,the model can robustly discriminate persons with similar appearance.Thereby the proposed PAPF can improve the accuracy of re-identification for similar persons.(2)To solve the difficulties of short-term occlusion in natural scenes,a video-based person re-identification algorithm based on the channel-temporal self-attention(CTSA)is proposed.The output after the fifth layer of the residual network(Res Net)is intercepted as the appearance feature.In order to improve the quality of feature representation,the channel self-attention module is used to adaptively calibrate the response weights of different channels,which can select out channel features with clear outlines and semantics.On the basis of global average pooling,the temporal self-attention module is used to generate a mask matrix,which stores the contextual relationships of the different frames in video sequence.Thereby,the influence of the occluded frames can be reduced and global temporal feature can be captured.Through combining high-quality appearance feature and global temporal clues,the proposed CTSA can effectively improve the accuracy of person reidentification in short-term occlusion scenes.On the basis of proposed PAPF and CTSA,the end-to-end person re-identification software is designed.The software can separately re-identify persons with similar appearance or persons in short-time occlusion,then verify the effectiveness and advancement of the proposed algorithms.