| Industrial robots can efficiently and reliably complete various tasks to significantly reduce labor cost.In today's manufacturing process,robots are mostly used in the known environment to complete the given work through teaching programming.However,with the transformation and upgrading of the traditional manufacturing industry,flexible and automated production lines are gradually becoming the mainstream in smart manufacturing under the general situation of epidemic normalization,which requires robots with the ability to sense the environment and cloud monitoring.In this context,based on RGB-D vision sensor,the object recognition and position estimation are realized by making full use of the three-dimensional space information of the object.The visual monitoring system based on digital twin is developed,and the robot grab system platform is built.The results are verified by the corresponding experiments.The main research work is as follows:(1)The calibration of RGB-D camera and robot system is studied.Firstly,the imaging model of RGBD camera is analyzed,the depth map and color map generated by the camera are registered,and the hand eye calibration method of industrial robot is studied.Finally,the camera parameters and the conversion matrix between camera coordinate system and robot coordinate system are calibrated and verified through experiments.(2)An object recognition and location method based on instance segmentation is proposed.Aiming at the interference of industrial environment stacking cartons with complex environment,few feature points,taped surface,close connection and shadows under natural light,a new object recognition and detection method based on RGB-D images is proposed.Firstly,an instance segmentation algorithm based on convolutional neural network is used to initially segment the images to be detected under the training condition of small samples.Then an improved seed region growth algorithm is used to automatically generate seed points and fuse the depth information as the merging principle of region growth,which makes full use of the 3D information of the object and improves the accuracy and robustness of the segmentation results.(3)The pose estimation method of object based on principal component analysis(PCA)is studied.Based on the information of 2D target detection combined with depth data,a 3D point cloud of the target object is generated and segmented.The maximum plane required for grasping is extracted from the segmented point cloud using the random sampling consistency algorithm,and the principal direction of the point cloud is estimated based on the PCA principal component analysis method,and the coordinate system is established with the center of mass of the extracted plane as the origin to obtain the 3D poses of the target object.(4)A digital twin system for robots based on the Unity3 D platform is proposed.Aiming at the lack of effective means for tracking and monitoring display of existing robots,a virtual digital model of the Unity3 D platform is firstly established based on the actual physical environment.The data interaction between the physical entity and the virtual digital model is integrated to achieve the organic unification of the object environment and the virtual environment,and the human-computer interaction interface of the digital twin system is designed.(5)A platform of RGB-D vision-guided robot grasping system is built,and the proposed target detection and localization method,the positional estimation method and the digital twin system are experimentally verified.The motion planning of the robot arm is implemented based on the robot operating system ROS,and several experimental verifications of grasping are conducted for objects placed at different positions.The experiments show that the RGB-D vision-guided robotic grasping system is able to identify and locate the target and grasp it well,and the digital twin system can achieve real-time mapping of physical entities within a delay of 0.5s,which meets the system design requirements. |
PDF Full Text Request