Research On Person Search With Joint Optimization Of Detection And Re-identification

Person search aims at localizing and recognizing query persons from a gallery of scene images captured by different cameras,consisting of two sub-tasks,i.e.,pedestrian detection and person reidentification(re-ID).This task shows much potential in real-world applications such as intelligent security,object tracking,video analysis,and human-computer interaction.Despite tremendous progress achieved by recent works,this task still suffers from the issues inherited from both detection and person re-ID,e.g.,viewpoint and pose variance,occlusion,complex background,false alarms in detection,misalignment,etc.In addition,the person search task requires balancing the optimization of detection and re-identification,and burdening more expensive labeling costs.To tackle these issues,we focus on the joint optimization of subtasks for person search task,and further reduce the labeling cost.In terms of the problem that end-to-end training is inaccessible in the existing two-step person search,we propose a joint training framework of subtasks for the two-step manner.In recent works,the detected bounding boxes may be sub-optimal for the following re-ID task.To alleviate this issue,a re-ID driven localization refinement framework is proposed,which realizes the end-to-end training by developing a differentiable RoI transform layer.Thus,the box coordinates can be supervised by the re-ID training other than the original detection task.With this supervision,the detector can generate more reliable bounding boxes,and the downstream re-ID model can produce more discriminative embeddings based on the refined person localizations.Extensive experimental results demonstrate that the proposed method performs favorably against the state-of-the-art person search methods.Further,we also focus on the re-ID module designation in our framework,and propose a complementationreinforced attention network to enhance the diversity of features,thus improving the accuracy of the person search.In terms of balancing the efficiency and accuracy for the person search task,we propose an enhanced decoupled and memory-reinforced one-step framework.Previous works remain major challenges,i.e.,conflicting objectives of multiple sub-tasks under the shared feature space,inconsistent memory bank caused by the limited batch size,and underutilized unlabeled identities during the identification learning.To address these issues,we first simplify the standard tightly coupled pipelines and establish a task-decoupled framework.Then,a memory-reinforced mechanism is introduced to better encode the consistency of the memorized features.Last,considering the potential of unlabeled sam_ples,we innovatively model the recognition process as semi-supervised learning.An unlabeled-aided contrastive loss is developed to boost the identification feature learning by exploiting the aggregation of unlabeled identities.Experimentally,the performance of the proposed method can match the best two-step work with high efficiency.In terms of the difficult labeling in supervised person search,the weakly supervised setting is proposed to greatly reduce the burden on manual annotation.Supervised learning is dominant in the person search task,but it requires elaborate labeling of bounding boxes and identities.Large-scale labeled training data is often difficult to collect,especially for person identities.We present a weakly supervised setting where only bounding box annotations are available.Based on this new setting,we provide an effective baseline model termed region siamese network.By developing the instancelevel consistency learning and cluster-level contrastive learning,we enforce the aggregation of closest instances and the separation of dissimilar ones in feature space.Extensive experiments validate the utility of our weakly supervised method,which even surpasses several fully supervised methods.In summary,we focus on the joint optimization of subtasks in the person search,conducting systematic and frontier research on the supervised two-step and one-step approaches,and the weakly supervised scenario.The proposed methods can balance the accuracy and efficiency,and greatly reduce the difficulty of annotation.They are easily applied to actual monitoring scenes,and will be of great significance to improving social security and constructing the intelligent city.