Research Of Laser Vision Seam Tracking System Based On Deep Neural Network

Recently,with the improvement of automation level and the popularization of robot technology,manual welding has been gradually replaced by automatic welding system with robot due to its low production efficiency and poor working environment.However,limited by the hardware equipment and intelligence level of industrial robots,most welding robots mainly adopt the fixed mode of "manual teaching-memory reappearance".Since the welding model is unavoidably affected by external factors,e.g.,clamping error and thermal deformation,the actual trajectory deviates from the teaching trajectory,which results in low welding precision and quality.To get rid of the constraint of “teaching-reappearance”welding mode,this paper uses the laser vision sensor to realize environmental information perception and self-cognition functions.To achieve more convenient calibration scheme and more accurate location results,we use the current mainstream data-driven technology,e.g.,deep learning and reinforcement learning.At the same time,our paper designs more lightweight networks and deployment scheme for edge embedded device.This research is funded by National Science and Technology Major Project “development and industrial engineering of five thousand robots with completely independent intellectual property rights for machine tool automatic production”(No.2015ZX04005006)and Guangdong Province Science Technology Project “dynamic model and control method of articulated robot”(No.2019B040402006).The contents of the research mainly include:(1)To realize the environmental information perception and self-cognition function,it is important for laser vision sensor to establish an accurate conversion relationship from 2D pixel frame to 3D wrist frame of the robot.As general methods to determine the transformation relationship between pixel frame and wrist frame of the robot,structured light calibration and hand-eye calibration have strong universality.However,due to the complexity of the calibration process,it inevitably causes the accumulation of intermediate errors.To reduce the influence of calibration process errors and improve the accuracy,we proposed an end-to-end calibration method based on deep reinforcement learning.The proposed method involves two dual learning tasks,a Pixel-to-Point module and a Point-to-Wrist transformation.The Pixel-to-Point subnetwork estimates the pixel coordinates of the weld points for each weld image,while the Point-to-Wrist transformation is used to establish an accurate conversion relationship from the pixel frame to the robot wrist frame.The Point-to-Wrist transformation consists of two major components: 'Actor' and ‘Critic'.The ‘Actor' model aims to infer coordinates of the feature points in the wrist frame of robot.The ‘Critic' model is introduced to guide the learning process of the ‘Actor' model by maintaining geometric consistency between the pixel coordinate and robot coordinate.Through the interaction training with the environment in a reinforcement learning framework,the algorithm simplifies the calibration process and establishes the accurate conversion relation from the 2D pixel frame to the 3D wrist frame of the robot.(2)In the process of automatic welding,the precision of welding position greatly affects the quality of welding.Thus,it is prerequisite for high-precision automated operations to ensure the accuracy and robustness of weld tracking.However,the tracking results cannot be guaranteed in long sequence and continuous noise welding environment.In this work,s seam tracking method based on a deep learning framework,which combines visual tracking and object detection,is proposed,and it is successfully applied to weld tracking system.We aim to alleviate the problem of samples to be corrupted during tracking process and enhance tracking accuracy by tackling the problem of decontaminating the sample set.Using a welding seam detection network,we solve the problem of model drift and relocation in long tracking sequence,which improves tracking stability.(3)For actual industrial production,portable embedded devices with advantages of low cost and small size are preferred.Recently,raspberry PI has a rapid development and it has the advantages of low cost,small size,complete software system and hardware.Thus,we choose the raspberry PI as the micro-controller of our project.However,considering the large gap between embedded devices and large servers in storage space,resources and computing power,it is not practical to deploy the above methods directly.Therefore,for the two core functions: denoising and positioning,we design light denoising model and positioning model.Meanwhile,we prune the redundant channels in models and make it possible to deploy models on terminals devices like raspberry PI.To give full play to the computational power of raspberry PI and improve the speed of inference on embedded equipment,we adopt the light deep neural network inference engine,MNN,for the deployment of welding models.(4)An automatic welding platform based on six-axis robot is built and it is mainly composed of laser vision sensor,industrial computer and six-axis robot.First,laser vision sensor and industrial computer are used as environmental information sensing module to locate weld feature points in images.Second,the 3D reconstructed measurement model obtained from the calibration algorithm was used as the conversion module of frame to calculate the 3D location coordinates of weld points.Finally,real-time automatic welding is completed by the robot controller.A series of experiments are carried out to verify the accuracy and robustness of our algorithm.At the same time,this paper carries out automatic welding experiments for various types of welds,proving that the tracking system can accurately complete the extraction and conversion of feature points in the noise environment,and realize real-time automatic welding.