Research On Product Status Detection For Robotic Assembly Based On Machine Vision

With the development of the robot industry,more and more robots are involved in industrial manufacturing.The current industrial robots are usually taught and reproduced,which can only handle single tasks so that the applications are restrict.Machine vision has high degree of automation and high precision,which can assist industrial robots to achieve intelligent manufacturing.In automated assembly,the robots can perceive the workpieces’ assembly status by detecting the assembly status of the product,and the situation where the installation is not correctly will be adjusted timely.It is of great significance for automated assembly.However,in addition to the traditional manual inspection of product assembly status,most of researches use contact methods such as torque sensors to detect the status assembly of product.In the actual manufacturing process,manual inspection is time-consuming and laborious and prone to omissions.The contact methods need to arrange and disassemble multiple measuring devices,and the process is cumbersome and complicated.The current researches are mainly aimed at the situation of wrong assembly and missing workpieces,and lack the detection of assembly status of product in the entire assembly cycle.In response to the above problems,this paper deeply researches the method of product status detection for robotic assembly,and proposes a non-contact method of product status detection for robotic assembly based on machine vision.The main research works are as follows:(1)Research on the pose estimation method of workpieces based on machine vision.Take the common circular workpieces in industrial production as an example,the calibrated Kinect V2 vision sensor is used to obtain their RGB images.Considering the common situation of occlusion in the industrial environment,a new ellipse detection method is proposed based on the existing ellipse detection method and the generative adversarial network.The proposed method can improve the ellipse detection rate in different degrees of occlusion.At the same time,an adaptive repair strategy is introduced on the basis of the proposed method to reduce unnecessary repair time.After obtaining the 5D parameters of the ellipse,the circular workpieces’ pose will be estimated based on the established pose detection model for circular feature.It provides a good foundation for product status detection of robotic assembly.(2)Research on the detection method of the product assembly status based on the workpieces’ pose.According to the CAD model of the assembly product,the assembly information model will be constructed under specified assembly sequence.The object recognition method based on convolutional neural network is used to recognize the assembly substrates and workpieces.This paper proposes a method that the relative pose between assembly substrate and workpiece in actual assembly is compared to it in assembly information model,the assembly status of product will be obtained based on the comparing result.Because the pose matching needs some time,an assembly status detection method based on step-by-step strategy is proposed.In the first stage,the collision detection of the bounding box is used to determine whether the workpiece enters the assembly neighborhood,and the second stage is based on the pose matching relationship between the workpiece and the assembly substrate.The second stage detects whether the workpiece is installed correctly.This method can improve the assembly efficiency of the robot,and can detect the assembly status accurately.(3)Design and realization of prototype system for product status detection for industrial robot assembly.The prototype system of product status detection for industrial robot assembly is designed and developed based on Kinect V2 vision sensor,IRB1200 industrial robot and other equipment.The vision sensor is used to capture the information of the workpiece and the assembly substrate in real time,and then using the proposed assembly status detection method of product to detect the status of product for robotic assembly.The experiments show that the proposed method can detect the assembly status of product accurately in industrial environment,and the method can be used to guide robots to assemble products.