Mobile Robot Autonomous Scene Understanding In Outdoor Environments

The outdoor scene understanding is a fundamental problem in the field of mobile robots,which provides a reference for the decision-making layer of the robot.The outdoor scene understanding of the mobile robot not only solves the problem of outdoor scene description,but also solves the problem of multi-class object recognition.The description of the outdoor scene relies on the effective fusion of the multi-source sensing data acquired by the mobile robot equipped with sensors such as laser,vision and inertial navigation system.Under the influence of the scene distribution,the ability of multi-class object recognition in outdoor scenes depends on the performance of pattern recognition and machine learning algorithms.For the problem of multi-source sensing data fusion for mobile robots,the data level fusion algorithm using laser ranging,vision and inertial navigation is used in this thesis.The method combines the color information of the image(RGB three-channel),the depth information of the laser ranging(Depth)and the intensity information(Intensity),and uses the RGB-DI point cloud to describe a 3D scene.Considering the change of the running speed of the robot and the dynamic objects in the scene,the RGB-DI point cloud distribution is unbalanced.This thesis proposes two conversion algorithms to complete point cloud to image,which are based on point cloud plane fitting and point cloud projection transformation.These two algorithms design different RGB-DI point cloud transform objective functions,and the purpose is to find a plane with the optimal view angle to represent RGB-DI point cloud.The Monte Carlo optimization algorithm is used to solve the normal vector representing the transformation plane,so that the point cloud is evenly distributed on the plane,and then the RGB-DI image corresponding to the RGB-DI point cloud is generated.For the multi-class object recognition problem in outdoor scenes,since the RGB-DI point cloud and the RGB-DI image are one-to-one mapping relationship,the multi-class object recognition problem based on the RGB-DI point cloud is transformed into a multi-class object recognition problem based on the RGB-DI image.Based on the deep learning theory,this thesis proposes two multi-class object recognition models for RGB-DI images,namely FCN model and CNN-based fully connected CRF model.The FCN model solves the problem of residual loss of backpropagation by using the deconvolution layer to complete the upsampling process of the feature map and merging the intermediate convolutional layer with the final deconvolution layer.For CNN-based fully connected CRF model,CNN extracts the deep feature of RGB-DI image,and the fully connected CRF model increases the constraints of adjacent pixel properties,making the RGB-DI image recognition results more positional.Finally,the recognition result of the RGB-DI image is inversely mapped to the RGB-DI point cloud scene to realize the 3D scene understanding.The long-term outdoor scene understanding of mobile robots is a challenging research issue.Due to seasonal alternation,illumination changes,weather differences,scene changes and other factors,the robot's object recognition model will have a significant impact.As the running time continues to increase,the cumulative error of the recognition model will continue to increase,which will significantly reduce the generalization of the object recognition model.In order to improve the robustness of scene understanding,scene recognition models require a variety of large sample scene data.Currently available robot long-term databases usually contain only visual data,so this research work is only based on visual scene samples.For the real-time problem of robot visual scene understanding,this thesis proposes a fast semantic segmentation method based on superpixel CRF model.The method uses superpixel as the processing unit,which greatly reduces the prediction time of the CRF model.For the low generalization problem of robot recognition in long-term operation,a stack sparse auto-encode model is proposed.The model extracts high-dimensional features of each image from large sample image data.The Membership Kmeans high-dimensional clustering algorithm completes multi-subset partitioning of large sample image data,and trains a supeipixel CRF model in each subset data.Finally,the SoftMax subset selector is used to complete the switching of multiple superpixel CRF models,which solves the long-term autonomy problem of mobile robots.In order to verify the effectiveness of the semantic segmentation in RGB-DI point cloud,the Oxford University dataset and the Dalian University of Technology dataset were used for experimental analysis.Comparing random forest based on color moment and based on FPFH,the results show that the proposed CNN-based fully connected CRF model can effectively improve the accuracy of point cloud semantic segmentation.In addition,in order to verify the effectiveness of the long-term and real-time outdoor scene understanding of mobile robots,the Carnegie Mellon University dataset and the Dalian University of Technology dataset were used for experimental analysis.These datasets are large sample visual data.The experimental results show that the stack sparse auto-encoding model can extract high-dimensional features,and the Membership Kmeans high-dimensional clustering algorithm is statistically evaluated based on Calinski-Harabasz and Davies-Bouldin clustering evaluation criteria.The clustering results of the optimal subset are obtained.According to the data distribution of each subset,the superpixel CRF model can be trained,and the robustness of the object recognition model is improved.The prediction time of the superpixel CRF model is less than 1807ms,which can meet the real-time requirements of the mobile robot in the laboratory.