Multi-parameter Grid Map Modeling For Autonomous Vehicle In Off-road Environment Based On Binocular Vision

In recent years,autonomous driving technology has been rapidly developed and widely concerned.As an indispensable approach to the realization of intelligent vehicles,the perception system needs to constantly enhance its ability to express complex scenes in the real world,so as to provide real-time,accurate and simplified local map information of the surrounding area for the subsequent planning control module.At present,the real-time positioning and mapping of indoor environment for robots has been relatively mature.The deep learning algorithm used for structured road unmanned driving can judge the size and location of surrounding vehicles and pedestrians in real time.However,the automatic driving in outdoor off-road environment often lacks standard road map,camera motion speed is fast,lacks stable feature points,and is difficult to carry out loop optimization.Therefore,there are great challenges in using cameras for off-road environment modeling.In this paper,the vision Eye binocular camera as the visual sensor,combined with the positioning trajectory in the world coordinate system obtained by the RT3002 inertial navigator,to track the vehicle movement and establish the local environment around the vehicle grid map as the research objectives,to develop the online data acquisition,replay and calculation system which can be applied to real vehicles.Based on binocular vision and inertial navigation,interframe pose transformation estimation was realized,and the estimation results were used to model grid map in local coordinate system of autonomous vehicle.The main research contents of this paper are as follows:(1)Construction of data acquisition,playback and grid map visual modeling system.In this paper,the synchronous acquisition system of binocular camera image,point cloud,chassis and inertial navigation data on the controller LAN bus(CAN)is implemented,and the elevation connection relationship of the grid map established in this paper is visualized.In addition,an online semantic segmentation module is also introduced to model the semantic information in the grid map.In terms of data acquisition time synchronization,the binocular camera image,point cloud and other signals on CAN bus are synchronized based on the computer local time,and the time alignment function of time stamp and ring buffer is realized for real-time calculation.In the aspect of data playback,it realizes the complete playback and synchronization of all the collected data from the locally stored data set and calculates the local grid map.In order to debug the program and display the grid map and the original point cloud,the cross-platform interface Library Gnome Toolkit(GTK)is adopted in this paper to realize the interactive interface,in which the Open Graphics Library(Open GL)area is embedded and the grid elevation map drawn by color point cloud and triangular wire frame is drawn.In this way,the connection relation between the map grid neighborhood can be expressed in detail and intuitively.Finally,we calibrate the binocular camera and Inertial Measurement Unit(IMU)to ensure the accuracy of camera parameters and availability of the IMU,and actually collect challenging cross-country data sets.(2)Study on binocular vision odometer scheme selection and tracking strategy.Due to the sparse sand texture in the data set,the tracking using the existing visual odometer method is easy to fail.Therefore,this paper compares the principles of the existing methods,and tests the feature point method and the direct method on the self-collected sandy land data set with difficult tracking,and carries out a mapping display and comparative analysis on the success of tracking,and then chooses the binocular direct method for the off-road condition visual odometer,and then deduces the direct method visual odometer.In the front part,the form of luminosity error and the integral relation between IMU frames are deduced.The interframe error function is established in the back-end part,and the position of the key frames in the adjacent position is estimated by sliding window method and nonlinear optimization method.In this paper,the position and pose transformation between the camera frames obtained synchronically by the inertial navigation system based on the global Positioning system(GPS)and IMU is obtained,which is connected to the direct binocular vision odometer as a priori.It helps the vision odometer obtain better initial value of iterative optimization in images with a large number of repeated textures and non-convex images,reduces the tracking failure rate in difficult off-road conditions,and improves the usability of the vision odometer for multi-frame fusion grid map modeling in off-road environments.(3)Multi-parameter grid map construction and multi-frame fusion.Since it is difficult for traditional accessibility grid maps to consider vehicle characteristics(such as driving force,speed,and passability of geometric forms),the objective of this paper is to use multi-parameter grid maps to express ground information as accurately and simply as possible,leaving accessibility to the motion planning layer of autonomous vehicle for judgment.In the establishment of a single frame map,firstly,the off-ground part of the grid is removed to ensure that the ground grid map is established.Secondly,the least square plane normal vector of the point cloud in the grid map is calculated to obtain the slope information of the ground,the unevenness information expressed by variance,and the proportion of semantic categories falling into the grid point cloud.These parameters can be used to help estimate its own dynamic response,estimate pavement switching,and help the vehicle adjust its own motion planning.In addition,wheel speedometer integrations are derived and added to improve the robustness and accuracy of interframe pose estimation.Due to the limitation of accuracy of the visual method,the parameters of the single-frame map have large errors.Therefore,this paper uses the obtained inter-frame transformation relationship,takes the latest frame of the camera as the origin of the local coordinate system,and integrates the grid map of the historical frame.Compared with the single-frame map,the multi-frame fusion map not only has more accurate parameter estimation,but also,due to the expansion of the map scope,enables it to obtain the parameters below the vehicle,such as the semantic category proportion under the vehicle,which can help the autonomous vehicle better estimate the ground state of its position and facilitate better motion planning.Finally,the results of map modeling and the change curve of the proportion of semantic categories are tested.The successful establishment of grid map is proved by three-dimensional visualization,and the validity of map semantic parameters on pavement status recognition is demonstrated by curves.