Binocular Stereo Vision Fused With Deep Learning For Target Recognition And Localization

Binocular stereo vision technology,as a key technology in machine vision,can identify the target without contacting the target,and can also calculate the depth information of the target,so it is sought after by many scholars.However,the diversity of targets and the complexity of the environment make binocular vision technology unable to detect targets well,as well as accurately calculate the target distance information.At the same time,the limitation of stereo matching algorithm itself leads to the relatively slow speed of building parallax,which cannot adapt to the real-time requirements of industrial detection.In recent years,with the continuous innovation of deep learning technology,it has demonstrated its powerful ability in many fields and many previously unsolvable problems in many fields can be achieved through deep learning methods.Therefore,deep learning is integrated into binocular stereo vision technology to build a target recognition and positioning system in this thesis.The main work contents include:(1)A binocular stereo vision system is established.Firstly,28 pairs of left and right checkerboard images are calibrated using Matlab calibration toolbox,and internal and external parameters of binocular cameras are obtained.Then the idealized binocular camera is obtained by correcting the internal and external parameters and Bouguet algorithm.(2)Aiming at the low efficiency of target recognition,an improved YOLOv4 target recognition network is proposed in this thesis.First,ECA attention mechanism was added to the backbone feature extraction network to optimize the feature extraction effect.Secondly,the convolutional layer at the junction of CSPBlock is replaced by the inner volume operator to improve the accuracy of the network and reduce the memory consumption.Finally,adding residuals to PANet makes network performance more stable.The results show that the MAP value of the thesis network reaches 99.4% and the FPS reaches 36,which not only achieves the accuracy of target recognition,but also meets the real-time requirements.(3)To solve the problem of poor binocular vision target localization,a binocular localization method based on deep learning is proposed.The improved Mask RCNN network is used to segment the contour of the target to be positioned,and the Canndy algorithm is used to extract the contour of each target,and then the centroid is obtained.Finally,the pixel coordinates are converted into image coordinates by using the calibrated internal and external parameters,and the target depth information is calculated using the trigonometry principle to complete the target spatial positioning.In vehicle localization experiment,lightweight ASPP and CIo U loss functions are introduced to optimize the network structure.Using the above method,binocular positioning is carried out.The results show that ACM RCNN network can fit the mask map generated by the vehicle to the actual contour,and the vehicle positioning error is within 5%.In fruit localization experiment,the end-to-side neural network and Bifpn are introduced to enhance the repeatability of feature fusion and the accuracy of prediction frame for small target characteristics.The localization results show that GBM RCNN network can effectively generate high-precision mask map for small targets,and the localization error of fruit is less than 4%.