Research On Stereo-vision Based 3D Environment Perception Method And Application Of Operated Mobile Robot

3D environment perception technology is the basis for the autonomous and flexible processing of mobile robots.Stereovision-based 3D environment perception technology has been widely used in industrial manufacturing due to its advantages of perceived information abundance,high stability,and low price.However,the existing stereovision 3D environment perception technology has problems such as poor robustness of 3D scene reconstruction,low perception accuracy,and limited understanding ability,which cannot meet the needs of autonomous flexible processing of mobile robots in unstructured environments.Therefore,in order to meet the needs of autonomous and flexible operation of 3D environment perception information acquisition in unstructured scenarios of mobile robots,this paper integrates deep learning technology with powerful learning,understanding and representation capabilities for the problems existing in binocular vision 3D environment perception technology.Research the 3D environment perception and recognition method of mobile operation robot based on stereovision and deep learning,take mobile operation robot parts transfer,loading,unloading and grasping as an application example,focus on the technical problems of 3D visual perception faced by mobile robots in the process of operation,and carry out in-depth development Research on methods of semantic map construction,navigation,obstacle recognition and grasping target recognition for mobile robots based on stereovision.The main contents are as follows:(1)Stereovision-based 3D scene reconstruction for mobile robot mappingAiming at the problem of low 3D scene reconstruction accuracy and inability to obtain high-precision 3D environment information due to the difficulty of dynamic scale feature extraction in the semantic mapping of stereovision-based mobile robots,a progressive feature fusion stereovision-based 3D scene reconstruction neural network model is proposed.The characteristic of common feature extraction networks that some information is lost is revealed,the reasons for the difficulty of dynamic scale feature extraction in binocular images are analyzed,and a multi-scale dynamic feature extraction network based on encoderdecoder is proposed.A progressive matching cost feature fusion strategy is proposed,and combined with a coarse-to-fine multi-scale supervised training method for training.The experimental results show that the proposed progressive feature fusion binocular vision 3D scene reconstruction network model solves the problem of acquiring high-precision 3D environmental information in the construction of semantic maps of mobile robots in unstructured scenes.(2)Stereovision-based depth estimation for mobile robot navigationTo deel with the poor-robustness problem of stereovision-based depth estimation resulted by large weak texture areas,illumination,noise,etc.in the navigation of stereovision mobile robots,a stereovision-based depth estimation neural network based on hysteresis attention mechanism and supervision cost is proposed.The redistribution characteristics of feature weights of feature extraction network based on attention mechanism are revealed,a new hysteresis attention mechanism is redesigned,and a high-quality similarity measure of stereo image feature correlation is proposed.The experiment show that our stereo visual depth estimation network model can effectively resist noise interference and estimate scene depth stably.(3)Stereovision-based three-dimensional obstacles recognition for mobile robot obstacle avoidanceAiming at the problem of ambiguous matching pixel regions in stereovision-based obstacle recognition and positioning due to obstacle occlusion in the obstacle avoidance of stereovision mobile robot,a neural network model of stereovision-based 3D obstacle recognition based on confidence propagation is proposed.A differentiable confidence propagation cost aggregation method is proposed to reduce ambiguity in matching pixel regions;a feature fusion obstacle recognition method based on attention features and matching cost features is proposed.The experimental results show that the proposed 3D obstacle recognition and localization neural network model based on stereovision can effectively improve the accuracy of binocular depth estimation and obstacle recognition.(4)Stereovision-based three-dimensional workpiece recognition for mobile robot grabbedAiming at the problem of inaccurate workpiece recognition and positioning due to the puffing of the edge contour of the grasped workpiece in mobile robot grasping with stereovision,a neural network model for target recognition and positioning of stereovisionbased grasping with joint guidance cost aggregation is proposed.A differentiable jointguided cost aggregation method based on bilateral filtering is proposed,which retains edge contour feature information and reduces edge puffing in depth discontinuous regions.A multi-modality feature fusion method is proposed for target recognition,which is based on the fusion among attention feature,matching cost feature,and disparity feature.The experimental results show that the proposed method can effectively solve the problems of edge puffing at the boundary and contour of the depth discontinuity region,and provide more accurate three-dimensional space information of the target for the mobile robot to grasp.(5)Development of Stereovision-based 3D Environment Perception System for Mobile RobotA three-dimensional environment perception system for mobile working robots based on stereo-vision is developed,and the system software algorithm composition and design are given.Taking mobile robot map construction,navigation,obstacle recognition and grasping target recognition as examples,the software implementation and workflow of stereo-vision 3D target perception algorithm are analyzed in detail.For the system algorithm proposed in this paper,the actual scene test and error analysis and comparison are carried out.The test results and error analysis show that the system designed in this paper can meet the needs of autonomous and flexible processing of mobile intelligent robots in unstructured operation environment,and solve the three-dimensional perception and recognition problems faced in the processes of mobile robot transfer,loading and unloading,and grasping processing parts in smart factories.