Study On Multi-object Detection And Tracking Algorithm Based On Binocular Stereo Vision

Multi-object detection and tracking is one of the main topics in the field of computer vision,and it has been widely used in the fields of robot navigation,intelligent video surveillance,human-computer interaction and so on.Due to the complex environment of the object(such as illumination changes,shadow changes,etc.),occlusion in crowded scenes,and similar appearance characteristics of different objects,multi-object detection and tracking is still one of the most challenging tasks in the field of computer vision.Binocular stereo vision technology can obtain the three-dimensional information of the scene,which has an unparalleled advantage compared with the monocular vision.This thesis focuses on the object detection algorithm and multi-object tracking algorithm based on stereo vision in crowded scenes.The main work and innovations are as follows:Firstly,a complete set of binocular stereo vision system is set up in this thesis,and its optical imaging module includes an assembled USB binocular camera,ZED camera,and an assembled near-infrared network binocular camera.Then the system is used to build a day and night binocular stereo database,which contains more than twenty complex indoor and outdoor monitoring scenes,and each scene has about 3000 frames.In addition,for the acquisition of depth information in stereo vision,three commonly used stereo matching algorithms are studied on the dataset,and their speeds and disparity map effects are compared and analyzed.Secondly,a pedestrian detection algorithm based on stereo vision is proposed in this thesis.The algorithm consists of three parts: 1)Designing and generating a three-dimensional(3D)surface model based on stereo vision,which is used to estimate the 3D spatial structure of the scene.2)Establishing a 3D surface background model,including the initialization and updating of the background model.In the updating strategy,according to the characteristics of 3D information obtained,a free update policy for unknown points is proposed.3)Studying a foreground clustering strategy based on shadow discrimination,which improves the accuracy of foreground object segmentation by extracting the shadow.The algorithm performance test and a large number of contrast experiments show that the algorithm has real-time performance and good detection performance in various complex environments,which is superior to some classical target detection methods.Finally,this thesis presents a multi-object segmentation and tracking algorithm based on deep learning and stereo vision.The algorithm first combines the object detection results of deep learning with the 3D information of stereo vision,and proposes a object segmentation algorithm based on the depth of the detection object to obtain the precise segmentation result of the object.Then,on the basis of segmentation,three kinds of object similarity model based on stereo vision are designed,and a two-tier data association algorithm is proposed to solve the tracking problem of partially occluded object and complete occluded object.Through the performance evaluation and comparison experiments of this algorithm on the dataset,it is proved that the algorithm not only has real-time performance,but also can obtain the precise object segmentation results,real distance and robust tracking results under severe occlusion conditions,which is better than some classic multi-object tracking methods.