Moving Target Perception And Tracking Based On Fusion Of Monocular Camera And 2D Lidar

In the field of robotics and autonomous driving,environmental perception has always been an important topic,and the detection and tracking of moving objects is a huge challenge.Currently,visual object detection schemes are very mature,but relying solely on a single camera has various limitations.With the increase in the number and type of sensors,perception tasks create a need to fuse multiple data.Nowadays,most 3D fusion sensing schemes rely on expensive 3D lidars or depth cameras,and cannot be applied to ordinary low-cost platforms on a large scale.However,existing research based on 2D point clouds often ignores the three-dimensional characteristics of objects and cannot reflect the true state of the target.Based on existing visual detection and target tracking methods,thesis proposes a sensor fusion scheme,starting from point cloud feature extraction,target matching,trajectory management,and other aspects.It can infer the 3D bounding box used for target detection only through 2D lidar and monocular cameras,and proposes a dual perspective tracking scheme for the fused information.The main contents of thesis are as follows:1.Based on the visual detection and point cloud feature extraction scheme of deep learning,a method for assigning assumed height information to the features of 2D point clouds and generating a simulated 3D bounding box is proposed to solve the problem of low dimensionality of data.Through joint calibration between the camera and lidar,the generated simulated 3D bounding box is projected onto the pixel plane,establishing a mapping of target features in 3D space to 2D images.2.Aiming at the problem that the matching accuracy of traditional schemes between2 D plane candidate frames cannot meet the requirements of this system,a target matching algorithm based on the image plane is proposed.This algorithm establishes a joint Intersection Over Union index weighted by the matching probability,and feeds back the fused features to the simulated 3D bounding box,forming a fused 3D bounding box that is closer to the real target.3.A target tracking scheme based on the theory of unscented Kalman filtering has been developed.Aiming at the problem that the tracking range is limited by the sensor’s field of view,thesis proposes a dual perspective tracking scheme for tracking targets in 3D space and 2D images,respectively,to jointly maintain the life cycle of the historical trajectory of the target.To sum up,thesis has developed a perception and tracking scheme via sensor fusion aimed at how to achieve 3D perception on low-cost platforms,achieving the effect of using two types of 2D sensor to simulate 3D target detection.Experiments in real scenes have verified the effectiveness of the proposed scheme on indoor moving objects.The scheme can accurately perceive and track moving objects,reliably simulate the effects of 3D perception,and requires lower costs.