Target Detection And Tracking Based On Fusion Of Millimeter-wave Radar And Visual For Spherical Robot

As a new type of mobile robot,the spherical robot has a broad application prospect.Due to the limitation of hardware platform,few researchers have been able to conduct detailed research on the related technologies for this new type of platform.After comparison and selection,the platform comes equipped with the mainstream sensors of mobile robots-millimeter wave radar and vision sensors as the main source of perceptual information.Meanwhile,inertial measurement unit provides pose information,which contributes to data transformation to the robot body coordinate system.In this paper,after analyzing and comparing a variety of perceptive fusion strategies,a feature-level fusion scheme is applied,which is suitable for spherical robot and its using millimeter wave radar and vision sensors.The main research work involves three aspects:hardware construction of sensor,target detection and tracking of millimeter wave radar and vision fusion algorithm based on radar detection.Firstly,the structure of the spherical robot and the particularity of its movement is investigated and problems to solve are the shielding of the shell and the shaking of the ball.An external suspension arm is designed to carry the required sensors.With the help of the inertial measurement unit and the pan-tilt on the cross arm,it is ensured that the sensors maintain stable operation with fixed pitch angle.Secondly,an approach is achieved to detect and track algorithm of millimeter wave radar on the spherical robot by the analysis of the point cloud information and the application of extended Kalman filter algorithm.Invalid information is filtered out through the adaptive correction of the platform's motion characteristics.In the experiment,the real-time detection and tracking effect of the algorithm is demonstrated by visualization tools.The experiment verifies the feasibility and effectiveness of algorithm by statistical data analysis.Thirdly,a vision fusion algorithm based on radar information is achieved.A fusion scheme that uses radar detection instead of RPN network is designed with the inspiration of the optimization and evolution of the RCNN series of networks.The stable and reliable target information is obtained by radar detection and tracking algorithm.Stable and reliable target information can be obtained by radar detection and tracking.Through previous calibration,the representative point of radar target center is mapped into the image.According to certain rules,a candidate box is generated as the region of interest for visual detection to make further classification and bounding-box regression.This kind of program naturally provides an attention mechanism and it is helpful in improving the efficiency of visual information processing and discovering early warning information in time.With the help of the region of interest as an intermediate bridge to match radar and visual detection,the superior information of each sensor is extracted and combined to provide structured data for robot obstacle avoidance.Meanwhile,the relevant information of the object is visually displayed in the image.