Research On Workpiece Detection Based On Deep Learning And Collaborative Sorting With Dual Manipulators

With the popularity of robotics in industries such as industrial production,lifestyle services,and healthcare,robots are gradually replacing humans to perform more complex tasks and expanding applications in a people-centric environment.It is foreseeable that in the new generation of Smart Manufacturing(SM)revolution,more robots will perform tasks in the form of imitating human behavior and actions,thereby adapting to new application environments.In this context,research in the field of dual-arm robots is gradually gaining popularity.In recent years,artificial intelligence algorithms represented by deep learning have been deployed in dual-arm collaboration tasks,further promoting the dual-arm collaboration toward intelligence.In view of the outstanding performance of deep learning in object detection,this thesis takes the detection and sorting of workpieces in industrial scenes as the research object,and realizes the real-time detection of workpieces and the cooperative handling and sorting of workpieces by dual manipulators in the dual manipulators collaborative experimental platform.The specific research contents are as follows:(1)In order to achieve flexible gripping of multi-objective and irregular workpieces in industrial scenarios,a workpiece contour and gripping points detection method based on fusion of deep learning and image processing is proposed.First the workpiece position bounding boxes in the workpiece image are located by deep learning,then the original image is cropped based on the bounding boxes added cropping offset,and finally image processing is performed on the cropped images to obtain the contours and the gripping points of the workpieces.Furthermore,a replacement strategy in stages of Depthwise Separable Convolutions(DSC)is designed,and the network is retrained by means of differential learning rate,so as to achieve the purpose of simplifying the structure and compressing the parameter scale of YOLOv3,and finally the YOLO-DSC network is constructed.A workpiece dataset containing 10 types of workpieces and 10488 images is created for the training,verification and testing of convolutional neural networks(CNN).The experimental data shows that when the Intersection over Union(Io U)is set to 0.5,the mean Average Precision(m AP)of YOLOv3 and YOLO-DSC on the workpiece test set reaches 98.60% and 91.43% respectively.After image processing,the m AP of workpiece contour detected based on two models reaches 98.79% and 93.95% respectively,and the m AP is attenuated slower when the Io U threshold is increased.Compared with YOLOv3,the YOLO-DSC only loses with 4.84% m AP,but gets the parameter scale compressed by 53.47%,the network inference time shortened by 30.03% and the detection time of the workpiece detection method shortened by 15.27%.For the successfully recalled workpieces,the positioning accuracies of the gripping points detected based on two models for 10 types of workpieces reach 98.23% and 97.85%.(2)In order to avoid network over-parameterization and performance overflow,an improved pruning filter strategy for the 2 types of residual structure used in CNN is proposed.The strategy prunes the 2 types of residual structure separately through synthesizing the weights of the last convolution layer in each residual block by adding weight hyperparameters to evaluate the kernel pruning order,and vertically decomposing CNN into multiple substructures.The experiment tests the strategy using Darknet-53(tiny)and Res Net-56 network on the CIFAR dataset,and shows that while the scale of network parameters is compressed,the strategy preserves network performance effectively and expands the application range of CNN pruning algorithms.The pruning strategy is further extended to the application of YOLOv3 on the workpiece dataset,and 3 models are obtained by pruning 30%,40%,and 50% of the convolution kernels in equal proportions for each layer of YOLOv3.After retrained,their m AP reaches 99.38%,97.49%,and 90.92% respectively,while the parameters are reduced by 50.89%,63.83%,and 74.93%,and the inference time is reduced by 31.13%,41.22%,and 54.62%.(3)The D-H parameters method is used for the 5-axis manipulator in the experiment to derive the kinematics and inverse kinematics analytical solutions of the single manipulator,and the constraint equations of angular velocity and angular acceleration of each joint when the end effector is in linear and curved motion.The trajectory planning rules and anti-collision rules based on Oriented Bounding Box(OBB)algorithm of dual manipulators for moving workpieces cooperatively are set,and verified in the dual manipulators collaborative simulation system based on MATLAB Robotics Toolbox.(4)A dual manipulators experimental platform is built,the construction process of its cloud,transmission and equipment layers is expounded.The geometric model of machine vision system is analyzed,and industrial camera parameters calibration is implemented.The workpiece contour and gripping points detection method based on fusion of deep learning and image processing,and the dual manipulators collaborative simulation system are deployed on the cloud service.The transmission layer realizes data exchange between the cloud layer and the equipment layer,while the equipment layer implements the dual manipulators cooperative handling and sorting of the workpieces as well as palletizing and assembly simulation.