Research On Vision/Vibration Based Terrain Perception For Rovers

With the gradual development of planetary exploration missions,there is an increasing demand for detection in unknown areas.In-situ resource utilization,Moon/Mars base construction,scientific exploration and investigation have been repeatedly proposed and demonstrated.It can be seen that the ideas and directions for the detection of planetary objects are gradually evolving from the purpose-type and small-scale inspections to the task-type and large-scale synergies.This undoubtedly puts forward higher requirements for the intelligence and autonomy of the rover system.At present,the navigation mode based on visual/Inertia has been successfully applied in the actual planetary rover mission,and played an important role in the patrol process.On the one hand,it obtains a large amount of surface environmental data,on the other hand,it ensures the effective detection and identification of obstacles on the patrol path.But it still reveals some problems: First,based on the existing sensing load,the material and mechanical characteristics of the terrain environment cannot be effectively perceived,so that the patrol does not have the ability to recognize the classification of terrain with different materials.Second,existing sensing loads are susceptible to environmental changes,which make the patrol not capable of carrying out complex tasks for a long time abroad.Therefore,this paper proposes a new idea of the external environment perception based on the visual/viobration fusion.It is intended to use the fusion of the two sensing modes to achieve terrain reconstruction,classification perception and semantic mapping of the detection environment,thereby overcoming the harm caused by the above problems.The research results obtained include:(1)By analyzing the composition and function of the existing planetary rover perception system,and combining with the fault problems occurring in the actual detection process,a new terrain perception model based on visual/vibration fusion is proposed which effectively solves the problem of terrain material classification and environmental robustness.The mathematical model of the visual sensing unit and the vibration sensing unit is synchronously derived,and the test performance analysis of the sensor in the actual application environment is given.(2)In the aspect of terrain reconstruction,a probabilistic terrain estimation method based on uncertain analysis is proposed to solve the three-dimensional reconstruction of terrain by only relying on ranging information and vibration information when image information is interfered.Firstly,the coordinate transformation relationship is redefined,and the data conversion is reduced to processing.Then,considering the influence of motion uncertainty,the measurement uncertainty and motion uncertainty are analyzed separately.The terrain covariance solution derivation based on vibration/gyro information is synchronously given,thereby obtain a probability model of terrain update.Finally,combined with the unmanned vehicle platform the terrain reconstruction capability was tested and analyzed in simulation environment,indoor test environment and outdoor test environment.The effectiveness of the method is verified by accuracy co mparison.(3)In the aspect of terrain material perception,a terrain classification recognition method based on vibration is proposed,which solves the problem of inaccurate distinction between terrain materials and realizes the semantic label generation of terrain types on line.Firstly,the characteristic representation of the vibration information and the overall implementation flow are given.Then,three terrain classification recognition learning methods are designed,which are based on improved BPNN algorithm,multi-layer perceptron based deep network algorithm and CNN-LSTM based deep neural network algorithm.Finally,the above methods were compared and tested based on the physical platform and five different terrain environments.The relationship between terrain classification accuracy and algorithm,running speed and platform is analyzed,and the optimal configuration under given test environment is summarized.(4)In the aspect of terrain semantic mapping,combined with the visual semantic mapping and deep learning idea,a complex terrain semantic sensing method based on visual/vibration fusion is proposed,which solves the problem of terrain semantic cognition during the patrol process.Firstly,the problem of semantic perception of planetary terrain is expounded,and the existing deficiencies and difficulties are analyzed.Secondly,based on ORB_SLAM2,3D point cloud reconstruction and semantic segmentation,a 3D terrain semantic reconstruction method based on RGBD is designed.And then,based on the terrain semantic tag based on vibration information,a terrain semantic fusion framework based on visual/vibration fusion is given.Finally,the three-dimensional semantic terrain reconstruction test was carried out by ADE20 K public dataset,indoor comp lex environment and corridor multi-material terrain.The correctness of the proposed method was verified from the reconstruction results and running time.