Design And Implementation Of Feature Tracking And 3D Reconstruction System Based On Event Camera DAVIS

As a new visual sensor,the DAVIS(Dynamic and Active-pixel Vision Sensor)can simultaneously output image frames and event streams under the same pixel array.Since the data format of the event stream is completely different from the image frame,the traditional feature point tracking and 3D reconstruction methods cannot be directly applied to the sensor DAVIS,so new algorithms are needed to maximize the advantages of the event camera.In this paper,a DAVIS-based feature tracking method is proposed.Based on this,a semi-dense three-dimensional reconstruction method for reconstructing edges in the scene is proposed.The feature point tracking method takes the event stream,the image frame and the IMU(Inertial Measurement Unit)data as input,and is divided into two stages of initialization and tracking.In the initialization phase,the feature points are first extracted using the Harris corner detector,and then the edges are extracted using the Canny detector,and the edges around the feature points are used as templates for the corresponding feature points.In the tracking phase,firstly,a spatio-temporal window is selected for each feature point from the event stream,and the event in the window is matched with the template edge of the corresponding feature point to calculate the optical flow of the feature point,where the EM-ICP algorithm is used for matching.Then,the calculated optical flow is used to update the position of the feature point and the position of the template edge around the feature point.Here,in order to cope with the rotation of the camera,the position of the updated template edge is corrected by the IMU data.The three-dimensional reconstruction method completes the reconstruction of the edge in the scene based on the feature point tracking.Since the feature point tracking method tracks the feature points,the edges around the feature points are also tracked as the template.Therefore,after the positions of the scenes in the pixel coordinates at different times are known,the pose transformation and the triangulation are used.The reconstruction of the edge can be achieved.The three-dimensional reconstruction method is divided into two steps.Firstly,using the matching relationship of feature points,using the polar geometry and Pn P algorithm,the pose corresponding to each side of the frame is calculated,and then the matching between the pose and the edge map corresponding to the edge map is utilized.Relationship,restores the 3D position coordinates of the edge in space in the edge map.Finally,based on the above two methods,this paper implements a system of feature point tracking and 3D reconstruction based on event camera DAVIS,and tests the system on the event camera dataset.The experimental results show that the feature point tracking method has achieved good results in both the feature point tracking duration and precision.The 3D reconstruction method also achieves a good visualization effect.