Design And Implementation Of 360-degree Panoramic Head-up View Generating System In Assisted Driving

In recent years,intelligent assisted driving is booming,in which 360-degree reversing image uses overhead projection to generate a panoramic overhead view of the road around the car to help park safely.Overhead view panorama has a small field of vision and narrow application area.However,head-up view panorama can obtain a wider field of vision and expand the application prospect.Although 360-degree head-up view panorama has been achieved in unmanned driving,it is not conducive to vehicle-mounted equipment with limited resources due to its use of more equipment,high price and high computing cost.This thesis proposes a 360-degree panoramic head-up view generation system based on a small number of fisheye cameras to generate a head-up panorama without obvious distortion difference to expand the field of vision.Firstly,images are obtained by fisheye camera,and after distortion correction and pre-processing in the environment of this thesis,the improved feature detection and matching algorithm is used to find matching pairs.After that,flexible registration mapping is completed and the mapping template is retained,then forming the panorama after image blending.Finally,the attention window is designed.The main research contents and contributions of this paper are as follows:1.In the pre-processing stage,the deficiencies of ZHANG Zhengyou’s calibration method and the edge problem of the corrected image are studied,after that the correction method considering the influence of two kinds of distortion and higher-order distortion term and the strategy of edge barrel distortion are proposed,which maximizes the field of view after correction and reduces the edge stretching distortion.2.In the stage of feature detection and matching,firstly,the shortcomings of SIFT and the problem that the detection algorithm can not adapt to the scale change of key points are studied,and an improved feature detection and description method considering hue and scalability is proposed.Secondly,rough matching considering color space and region matching based on location prior information are used to better service image registration.3.In the registration and blending stage,firstly,the limitations of traditional registration are studied,and a registration method combining multi-type local homography matrix and global similarity transformation is proposed to flexibly align the overlapping region,conformal non-overlapping region and smooth transition region.Secondly,the feathering weighted blending using adaptive weight is proposed to realize the natural transition of image blending.4.The concept of attention window is put forward,which is convenient for driver’s observation and machine processing.Finally,a complete system is built with the real vehicle platform.Through the inspection and quality test in the real vehicle,the system proposed in this thesis can maintain the basic structural characteristics of the object,and generate a panoramic head-up view image with a wide field coverage.Although the clarity aspect needs to be improved,it can provide help for assisted driving to some extent.