Person Re-identification Based On Human Appearance

Person Re-identification aims to associate people with identical identities across different camera views?usually with non-overlapping field of view?.It has great application potentials in smart video surveillance,home care,intelligent traffic management and safe production supervision.Due to serious variations in camera views,illumination,poses,personal belongings and inter-person occlusions,images of the same person may encounter serious appearance changes.In particular,the illumination changes may cause serious color distortion?e.g.,the same color has different RGB values under different camera views?,and the changes in camera views,poses and inter-person occlusions may cause the spatial misalignment problem?e.g.,the same human body occurs at different locations of the image plane?.To address the aforementioned issues,the person re-identification algorithms have evolved from model optimization design to architecture design of the deep convolutional neural networks.The model optimization based algorithms usually follow the"feature extraction"+"metric learning"pipeline.They aim to learn discriminative feature subspace by the optimization of specific objective function.The deep learning based algorithms aim to improve the feature discriminability by design the end-to-end framework.This paper is devoted to the exploration of effective features and metrics for person re-identification.We study both traditional model optimization based algorithms and deep learning based ones,to jointly explore the effective solutions for person re-identification from different aspects.This paper covers the two important branches of person re-identification algorithms:traditional model optimization based ones and deep learning based ones.In the study of model optimization based Re-ID,we follow the"coarse-to-fine grained","separated-to-unified optimization"concept.Validated by extensive experiments,some important tricks summarized during the study of model optimization,then act as the domain knowledge for the design of deep architecture.The detained research contents are summarized as follows:As for the model optimization based algorithm design,We propose some solutions for the coarse-grained?global human appearance based?person re-identification.Given that this kind of algorithms need only perform model update on global features,they are simple and efficient.We make the following contributions regarding the study of global metric learning based person re-identification algorithms.?1?Considering that different cameras have different distortion transforms on the human appearances,we propose a kernelized view-adaptive subspace learning algorithm for person re-identification.In specific,we can alleviate the influence of different distortions on the re-identification performance by learning different transform matrix to model the distortions from different cameras.Besides,we adopt the kernel trick to generalize the metric learning from simple linear subspace to high-dimensional nonlinear subspace,which further improves the discriminability of the metric.?2?Considering that coarse-grained person re-identification is not flexible enough in dealing with the color and spatial distortions in person re-identification.To address this issue,we propose an instance and feature importance weighting algorithm for person re-identification.Specifically,we introduce the concept of instance weighting in the metric learning process to handle the amount bias between positive and negative image pairs.Meanwhile,we introduce the L_2,1norm as a regularization term to explore the relative importance of different local features.By imposing the L_2,1term,the norms of some rows in the learned transform matrix will shrink to zero,which corresponds to assigning a low weight to the corresponding feature dimension.Meanwhile,the feature dimensions that are more important for recognition will be assigned with larger weights.Feature selection also helps to alleviate the influence of background clutters.The global appearance based algorithms are limited in the ability to address the issue of spatial misalignment.Although to some extent,feature selection can help to mask out the influence of background.Not explicitly performing alignment,spatial misalignment caused by serious variations in poses/views remains a problem.To address this issue,we propose to model the person re-identification as a"local feature alignment+correspondence transfer+local feature distance fusion"framework.In specific,we propose to establish patch-wise semantic correspondences?e.g.,shoulder to shoulder?between positive training image pairs.During testing,the off-line learned patch-level correspondence patterns are transferred to test pairs with similar pose-pair configurations for patch-wise feature distance calculation.In this setting,the patch-wise correspondence between each test pair can be robustly and accurately inferred,alleviating the spatial misalignment problem to a large extent.Both the coarse-grained metric learning solutions and the fine-grained correspondence learning strategy,view feature extraction and metric learning as cascaded independent modules,where they are separately optimized.This two-stage formulation makes it impossible to obtain global optimal solutions.To address this issue,we propose to model the feature learning and metric learning into one unified framework to jointly learn optimal feature and metric for person re-identification.More specifically,we propose to perform dictionary learning under the learned Mahalanobis space.Through this design,representative feature as well as discriminative metric can be jointly optimized within a unified framework.The formulation is not globally convex,therefore,making it hard to directly obtain the optimal solution.Therefore,we adopt the alternative iteration algorithm for optimization,and derive closed-form solutions for each iteration,greatly improving the efficiency.Extensive experiments demonstrate the effectiveness of the proposed joint optimization framework.As for the study of deep learning based algorithms,the tricks summarized from the model optimization based algorithms can be borrowed to guide the design of the deep architecture.More specifically,inspired by the"feature importance weighting"trick,we propose a weighted bilinear coding based deep end-to-end architecture to improve the representative ability of deep local features.In the proposed model,salient part network is designed to automatically derive salient body parts.The part detection scores are then utilized to weigh the learned intermediate deep features,and the weighted features are input into the bilinear coding module to obtain the final representation.The salient part net performs coarse part-level alignment,and the weighting scheme helps to mine the relative importance of different locations.Working together,the performance is significantly improved.In summary,this paper focuses on the topic of person re-identification,which has prospect applications in practical scenarios.This work mainly covers our contributions to person re-identification from the following aspects:?1?Metric learning algorithms based on global appearance features.?2?Local patch-wise correspondence learning and transfer.?3?Joint dictionary and metric learning for global optimization.?4?Deep feature learning algorithm based on weighted bilinear coding over salient body parts.The above-mentioned contents cover the two main branches of algorithms for person re-identification.The model optimization based algorithms and the deep learning based ones are suitable for different situations:Model optimization based algorithms are simple and efficient,making them superior when data is hard to collect?e.g.,surveillance data at night?.On the other hand,deep learning based algorithms are more suitable for annotated large-scale datasets,since the deep models are data-driven.Besides,the tricks summarized from model optimization based algorithms can act as domain knowledge to guide the ddesign of deep architecture.The research content of this paper forms a systematic and structured framework.We make several contributions in the theory of person re-identification,which paves the way for the landing of person re-identification into practice.