Viewpoint-robust Person Re-identification Via Deep Residual Equivariant Mapping And Fine-grained Features

Person re-identification(Person Re-ID)is attracting more and more research interest due to its importance in many fields like security as well as business.Typically,researchers call the task of image or video retrieval for specific person: “person reidentification”.As a sub-realm of image or video retrieval,given an image or a video of a particular person,Person Re-ID attempts to automatically retrieve or match other pictures or videos of the same person across devices.Different from face recognition,Person Re-ID does not involve face detail extraction,hence bringing along more application scenarios in ordinary environments.However,Person Re-id remains challenging due to a series of large variations in many aspects across cameras like illumination,resolution,viewpoint,posture and obstacle.Viewpoint variation is one of the hardest issues among Person Re-ID problems.To tackle this issue,this paper explicitly takes viewpoint information into account and proposes a novel Deep Residual Equivariant Mapping and Fine-grained Features(DREMFF)approach for viewpoint-robust person re-identification.The main contributions of this paper are as follows:(1)Person viewpont estimation based on transfer learning of deep modelThe viewpoint contains important information as it has great impact on visual appearance of a person.However,the mainstream datasets lack viewpoint annotations about persons.To handle this problem,the person viewpoint estimation module is designed in this paper based on the idea of transfer learning.After being pre-trained on a dataset which contains viewpoint annotation,the module will be fixed to estimate viewpoint on other datasets.Meanwhile,the module defines four canonical viewpoints(front,back,left and rigtht).In practice,different from manual labeling,the module will estimate the probabilities of a person belonging to each of these four canonical viewpoints simultaneously,which provides more accurate viewpoint information for subsequent tasks.(2)Person feature correction based on deep residual equivariant mappingExisting person re-identification methods usually directly calculate the similarity of person pictures regardless of their viewpoints.Nevertheless,matching persons in different viewpoints is difficult since it is intrinsically hard to directly learn a representation which is geometrically invariant to large viewpoint variations.However,studies have shown that most of the layers in deep neural networks change in an easily predictable manner with the input changes and such transformations can be learned empirically from data through a simple linear transformations.Therefore,this paper hypothesizes that there exists inherent mapping between different viewpoints of a person and propose a feature correction method.This method together with previously estimated viewpoint information will bridge the representation discrepancy of a person in different viewpoints through equivariant mapping by adaptively adding residuals to original representation according corresponding angle deviation.(3)Fine-grained feature extraction based on attention mechanismInstead of paying attention to certain details,modern person re-id works primarily by comparing pedestrian’s overall appearance.In real life,there are chances that different individuals still share a similar overall look because of dressing alike,which is likely to bring about false positives.So,based on attention mechanism,DREMFF extracts fine-grained features for each image from multiple salient regions as well as different scales.These captured information is capable of providing assistant decisionmaking at lower granularities.The mapped global features and the learned fine-grained features work collaboratively to enable viewpoint-robust person re-identification.Experiments on three public surveillance datasets consistently demonstrate the effectiveness of the proposed approach.Finally,this graduation thesis concludes the strengths and weaknesses about this work and discusses the future of the Person Re-ID.