Research On Visual Estimation Method Of Ground Physical Characteristics Of Field Environment Based On Deep Learning

Good visual perception provides feedforward for the legged robot to walk in the complex rugged field environment,so that the robot can obtain ground information of the untraveled path in advance.The ground information of the field environment includes not only three-dimensional topographic information,but also physical property information.The ground physical property information mainly includes the ground stiffness characteristics and friction characteristics,which can play an important role in the motion state of the legged robot.The lack of such information can lead to a robot's slipping,overturning,and excessive impact of foot and ground force.At present,there are many achievements in the use of visually perceived three-dimensional topographic information of the environment in which the legged robot is located.However,there is a great lack of research on the physical characteristics of the ground in the field using the visual perception.Therefore,this paper has studied the visual estimation methods of the ground physical properties of the field environment.The main research work is as follows:Firstly,for the lack of wild ground image datasets in semantic segmentation research for deep CNN applications,this paper uses hand-held camera equipment to capture highresolution RGB images of the typical field ground types.Then,the various ground types in these images were counted and processed,the quantity balance between various types was realized.Threefore,the original RGB image dataset was established.The original RGB image dataset was manually densely labeled with the software LabelMe,and a single-channel grayscale image with pixel values corresponding to the training ID was generated.Therefore,the groundtruth image dataset was established.The field image dataset of the field environment consisting of the original RGB image dataset and the groundtruth image dataset is constructed,which lays a good foundation for the subsequent study of the visual estimation method of ground physical properties.Secondly,because image resolution has a problem of excessive attenuation in neural networks,this paper proposes a solution of feature parallel extraction and feature fusion for high-resolution images,which compensates for the loss of information in height and width,thus ensuring maximum effect on high-resolution images.The long-skip connection from the shallow layer to the deep layer of the network is built,which improves the learning ability of the geometric texture features and improves the segmentation effect on the ground type without specific shapes and specific topologies.The pyramid pooling module is designed at the back end of the network,which improves the network's ability to comprehensively perceive image global information and local information.The design of a network for semantic segmentation of ground types in the dataset is completed.Then,based on the difference in physical characteristics,this paper has a more subtle division of the same type of ground,forming a subdivision type.Following the material perception mechanism in biology,this paper extracts the surface mechanical and optical features of each subdivision type,and fits the Gaussian distribution function used to characterize the above two features.In this paper,the two features are quantized according to the above Gaussian functions,and the feature vector dataset is obtained.The friction level inference decision tree and the stiffness level inference decision tree based on CART algorithm are established,which solves the problem of inconsistency in the physical characteristics of the same ground type class,and obtains the estimation of the physical properties of the subdivision type.Finally,the semantic segmentation network model is built using the deep CNN framework Pytorch.In this framework,a contrast experiment is set up.The two-way NVIDIA TiTan xp GPUs are used to train and test the network on the training set and the test set.The loss value and accuracy of the network are calculated,and the semantic segmentation effect is visualized to verify the effectiveness of the proposed algorithm.The decision tree that solves the inconsistency of physical properties in the class is tested on the test set,and its accuracy is calculated.Then the overall experimental verification of the visual physics estimation method of the field environment based on convolutional neural network is carried out.The estimation effect on the physical properties of the ground is visualized,and the effectiveness of the research results is verified.