| Object tracking is essential for applications in fields such as autonomous driving to understand the surrounding environment and make accurate responses.In recent years,more and more object tracking work has shifted its focus from two-dimensional image data to threedimensional point cloud data.However,the sparsity and unstructured nature of the 3D point cloud seriously affect the performance of the tracker.Although some point cloud 3D single object tracking algorithms alleviate the defects of point cloud data to a certain extent,there are still many problems.First of all,due to the lack of geometric information and semantic information,the existing point cloud 3D single object tracking algorithm is prone to fail when tracking targets with incomplete shapes or interference from similar geometric objects.Secondly,recent methods can only be adapted to tracking scenes where the target point cloud is denser,and there is no special network design for object tracking in sparse scenes.Finally,mainstream point cloud 3D single object tracking algorithms rely too much on the correlation of semantic shape features between the template and the search area,ignoring the motion information of the target,which makes these methods unable to track severely sparse targets robustly.In this paper,aiming at the above-mentioned problems in point cloud 3D single object tracking,the following research work is carried out:(1)An end-to-end image semantic and geometry-assisted point cloud 3D single object tracking network(IFP-Net)is proposed.First,a fusion module that establishes the correspondence between image and point cloud features in a point-to-point manner is designed,and the image features weighted by the self-attention and cross-attention mechanisms are used to enrich the semantic information of the point cloud.In addition,the projection transformation is used to convert the two-dimensional geometric features inferred by the deep aggregation network to three-dimensional,and use them as additional inference clues for Hough voting.Quantitative and qualitative experiments on the KITTI tracking dataset prove the effectiveness of the proposed method for tracking in adverse scenarios.(2)An efficient point-driven to center-driven sparse scene point cloud 3D single object tracking network(EPC-Net)is proposed.The key to the network is that it gradually uses the advantages of point cloud features based on point representation,voxel representation,and Bird’s Eye View(BEV)representation to perform center-driven target location.Specifically,the relation embedding unit(REU)is first designed to establish the local and global relation between the search area and the template at the same time.Secondly,a shape constraint unit(SCU)is proposed to enrich the semantic shape information in the latent space and expand the feature gap between the foreground and the background.Finally,a multi-stage point cloud feature representation conversion network from point to voxel to BEV is proposed,and centerdriven target location is implemented.The experimental results of this method on multiple datasets are significantly improved compared to the current mainstream point cloud 3D single object tracking method.(3)A robust 3D single object tracking Siamese network guided by motion information(MIG-Net)is proposed.Aiming at the problem that the existing methods rely too much on the semantic shape cues of the target,which causes the tracker to almost fail in the scene where the target is severely sparse,this method first proposes a multi-scale feature aggregation structure,which generates densely related search areas by aggregating search areas of embedded target clues of different scales.Subsequently,a flow estimation guide unit(FEGU)is proposed,which uses the scene flow obtained by the flow estimation function to generate a guide map to correct the ambiguous heatmap response.Finally,this method designs a 3D Kalman filter as the trajectory correction unit to make the object tracking more robust.Extensive experiments on various datasets have proved the effectiveness of this method in robust tracking. |
