Study On Pedestrian Detection And Re-identificaiton Based On Fusion Of Depth And Vision Information

In video surveillance systems, human appearing in surveillance video are the object of its focus, so intelligent monitoring systems need to have a ability of pedestrian detection, re-identification, and tracking, so as to further analyze the behavior of targeted pedestrians. This requires the monitoring system has a reliable technology for pedestrian detection and re-identification. However, due to pedestrian posture complexity and variability, scale changes, as well as the fact that application scenario is susceptible to interference from application environment, such as background, light, shadows, camera parameters, pedestrian detection and re-identification technology is still unsatisfactory in terms of reliability and speed at present.Upon this situation, applying the principle that depth image is robust against illumination changes and same object keeps consistency of space information, the thesis established pedestrian detection and re-identification model based on fusion of depth and vision information, through research on background subtraction, human body segmentation, fusion of depth and vision information, viewpoint identification, keyframe selection and other issues.Firstly, we proposed pedestrian detection based on fusion of depth and vision information, as pedestrian detection is vulnerable to interference of occlusion and illumination changes. Depth image is introduced to pedestrian detection to avoid interference of illumination changes as depth images is characterized by robustness against illumination variation. And pedestrian detection problem is transformed to detection of human head in order to eliminate the influence of occlusion and posture changes on detection result. Then, the thesis built head detector respectively for depth image and color image, and employed decision-level information fusion to obtain head detector with lower miss rate. By the light of depth information continuity of the same object surface, graph theory-based human feature extraction methods were proposed, which makes extraction of the whole body pixel possible as long as pedestrians’heads can be detected, so that pedestrian and background can be seperated. Experiments show that the method improves the ability to counter interference of occlusion and posture changes.According to invariability of geodesic distance between two points on the human body surfaces, as well as applying context and space information contained in human skeleton, we proposed spatial distance features based on human skeleton and designed a human part detection algorithm based on these feature. Finally, experiments verify the feasibility of this algorithm.After that, we built human appearance model based on fusion of depth and vision information through extracting appearance model from all of human parts and then combining it with skeleton-based spatial information, for avoiding re-identification errors existing in the current human appearance model as those models are susceptible to posture and camera view changes. The method improves robustness and discrimination of the appearance model, thus achieves enhancement on pedestrian re-identification performance. Then, training scheme of pedestrian similarity function based on maximization of re-identification probability was proposed. We used immune evolutionary algorithm to get the optimal similarity function and verify by experiment that our pedestrian re-identification scheme trained with this rule is superior to pedestrian re-identification method that are trained with other rules.At last, further analysis was made on multi-shots pedestrian re-identification. We proposed pedestrian viewpoint identification method so as to make re-identification immune from interference of viewpoint changes; Due to image redundancy problem under multi-shots, key frame extraction technology based on the human skeleton was proposed, for achieving selection of pedestrians with different postures; and we established a new human appearance model, which contains global features and cyclic local features. Experimental results demonstrate that the model can improve re-identification accuracy and robustness.In the end, we summarize the content, advantage and deficiency of the paper, and narrate further research direction.