Research Of Express Sorting Robot System Based On Machine Vision

Since the birth of e-commerce,express service volume has been increasing,express companies need to improve the sorting ability of express delivery.In this context,we designed an express sorting robot system based on machine vision,which is specially used to sort the parcels with handwritten express list.In order to achieve the design goal of the system,the following works have been done:(1)We used the IRB1410 industrial robot of the Asea Brown Boveri(ABB),monocular camera,computer,STM32 and other hardware to construct the robot sorting system.On the basis of serial and Ethernet communication,a set of communication protocol is designed,and good control effect has been obtained.(2)We combined industrial robots with machine vision,enabling robots to locate and grab express.Through the binocular calibration,stereo matching and triangulation,the system can obtain the depth information of the express image.Then,according to the mapping relationship between the coordinate systems and the result of eye-to-hand calibration,the robot coordinates corresponding to the image coordinates of the express center point are calculated,thus driving the robot to grasp correctly.(3)We extracted handwritten addresses and phone numbers from express images.Firstly,the Hof transform is used to correct the express image,then the complete express list image is extracted by using OTSU threshold segmentation,morphological processing and connected domain screening.Because the location of the handwritten text is confirmed on the delivery list,it is possible to set the fixed Region Of Interest(ROI)area to extract the text and split it into a series of character pictures.(4)We used convolutional neural network to recognize handwritten characters.For the segmented handwritten digital images,the classic LeNet-5 model is used for recognition,with an accuracy rate of 99.09%.For handwritten Chinese character recognition,we design a set of convolution neural network model and train the model by using the data which are selected from the required the Chinese Academy of Sciences(CASIA)handwritten character library.After optimizing the model,the accuracy rate of 96.54% is obtained.