Robot Pose Estimation Based On Multimodal Information Fusion And Its Application

At present,industrial robots have been widely used in the field of intelligent manufacturing,and robot pose estimation is one of the most common task requirements in industrial applications.In the industrial field,machine vision is mainly used to realize robot grasping pose calculation and material sorting in simple scenarios.With the increasing requirements for accuracy and automation in industrial control,traditional visual pose estimation methods can no longer meet the increasingly complex task scenarios and requirements.In complex industrial application scenarios,objects to be grasped often have problems such as mutual stacking,cross-interference,blurred edges,and noise,making it difficult for industrial robots to obtain accurate grasping poses of the object to be measured.Therefore,how to achieve accurate and efficient pose estimation of industrial robots in complex industrial scenes has become a new research hotspot.According to the characteristics of complex industrial scenes,this paper conducts in-depth research on the pose estimation of industrial robots.It guides the pose of the robot through two-dimensional visual information and three-dimensional point cloud information,and proposes a robot pose based on multi-modal information fusion.The estimation method,the main work content is as follows:1.Aiming at the problem that the target 3D pose dataset is not easy to obtain in industrial applications,a simple and easy method for constructing a 3D pose dataset is proposed,and a dataset software platform is created to realize the automatic generation of the dataset.2.This paper proposes an image information extraction module based on improved Mask RCNN.First,Dense Net densely connected network is used to optimize the feature extraction layer to enhance the extraction ability of weak features,and then a new segmentation loss function is established by introducing a new segmentation loss function to make the extraction of object information in the image more accurate.The improved Mask RCNN network can provide accurate RGB information and depth information of the object to be captured for the pose estimation network.3.Build a pose estimation network based on multi-modal information fusion.The output of the Mask RCNN module is used to extract the RGB information and depth information of the objects to be captured in the scene,and convert the depth information into point cloud information in the camera coordinate system.Use Dense Net and Point Net++ to separately analyze the RGB information and points.The cloud information is feature extracted,and then the feature fusion network is used to perform pixel-level feature fusion of heterogeneous feature information.Finally,the relative pose rotation quaternion estimation and the quaternion to the rotation parameter matrix are estimated through the pose regression network and the ICP algorithm.Analysis,the estimation error of the pose is within 8 degrees.4.The application of robot pose estimation algorithm based on multi-modal information fusion in real industrial scenes is studied.First,the hand-eye calibration method is introduced,and the Eye to hand hand-eye calibration method is theoretically deduced.The motion planning algorithm of the industrial robot is analyzed,and the simulation of the robot arm RRT motion planning algorithm is realized by using MATLAB and Mathematica.Finally,the pose estimation algorithm proposed in this paper is applied to the actual industrial scene,and the grasping application of industrial robots in complex industrial scenes is realized.