Study On Information Extraction And Fusion Of Radars And Vidos

Millimeter-wave radar is a kind of high-resolution radar with the ability to work all-day and all-weather.These advantages meet the requirements of modern society.As a low cost and high performance sensor,camera has the advantages of convenient installment,multiple functions,and customized development according to requirements.As two kinds of powerful sensors,information fusion of millimeter-wave radar and camera can realize information complementation and data mutual verification,thus obtaining reliability enhancement and more comprehensive descriptions of targets.Therefore,the information extraction and fusion of millimeter-wave radar and camera were researched in this thesis.And the main objectives of research work can be summarized as follows:(1)Multiple targets resolution and motion parameter estimation using radar were researched.In consideration of complicated detection conditions in practical application and the demand of multiple targets resolution,a scheme based on joint estimation of range and pulse domain was designed for range and velocity measurement.The results of simulation and measured data validate the feasibility of the scheme and the performance under complicated multiple targets conditions.Then,a multiple targets detection algorithm based on range-velocity spectrum was proposed and a multiple targets angle estimation algorithm based on range-velocity spectrum was also introduced.(2)Moving targets detection and classification using camera were researched.In this thesis,several algorithms of motion foreground extraction in video were introduced,including frame difference method and Gaussian mixture model.In addition,the foreground image denoising method based on mathematical morphology was introduced.The procedures and working principle of image classification by convolutional neural network were elaborated.(3)Transformation from the three-dimensional spherical coordinate system to the two-dimensional pixel coordinate system was derived in details.Then,a synchronization and space calibration method between millimeter-wave radar and camera was proposed.On basis of accurate calibration,the false alarm rate was reduced and the region of interest(ROI)generated by radar and video was matched through data fusion of millimeter-wave radar and camera.Then,targets classification and recognition was performed.(4)Taking the typical traffic application as an example,the effectiveness of millimeter-wave radar and camera fusion system was verified in the real scenario.