Research On Object Detection And Binocular Localization Method For Tea-Picking

Currently,the hand-picking method is still relied upon for the harvest of premium tea in China.However,as labor costs rise,manual harvesting has become a major bottleneck in the production of premium tea.Therefore,the study of tea tree selective harvesting machinery is crucial for the development of premium tea,and the recognition and localization of harvesting targets is one of the key technologies for achieving selective tea harvesting machinery.In tea gardens,accurate recognition and positioning of harvesting targets is greatly challenged by factors such as small target size,high background interference,and complex lighting changes.To address the problem of harvesting target recognition and localization,this thesis first adopts a deep learning-based object detection method to identify the harvesting targets,followed by a semantic segmentation method based on multi-head self-attention mechanism to extract the harvesting points.Finally,a harvesting point threedimensional coordinate calculation algorithm based on target constraints is utilized to obtain the three-dimensional coordinates of the harvesting points,which partially solves the problem of target recognition and localization.This thesis has achieved the following main results:(1)To address the problem of complex lighting and diverse target shapes in tea gardens,which make it difficult to identify harvesting targets,we designed an object detection model based on Efficient Net,called EF-FPNet,to achieve accurate recognition of harvesting targets.This model combines online and offline enhancement methods to increase the number of samples,uses Efficient Net to extract features of harvesting targets,enhances the effective feature representation ability,and fuses multi-scale features through Feature Pyramid Network(FPN)to improve the detection accuracy of the model for targets of different sizes.In addition,we introduced the complete intersection over union(CIo U)term into the loss function to enable the model to converge quickly.Experimental results show that the accuracy of the model reached 98.60%,the recall rate was 87.07%,the average precision was 92.67%,and the detection speed reached 13.86 fps.When compared with YOLOv5,SSD,and Center Net models,EF-FPNet performed better in terms of accuracy,recall rate,and average precision.The model accurately identified harvesting targets in the test set with different shapes,lighting conditions,and missing target information,avoiding the interference of environmental and target shape factors.The research results show that EFFPNet can efficiently and accurately detect harvesting targets in tea gardens with various interference conditions.(2)A semantic segmentation model named RMHSA-Ne Xt based on multi-head selfattention mechanism is proposed to efficiently segment tea-picking points in tea gardens,where the small scale of tea-picking points and large background interference pose a challenge to the segmentation task.The model first employs Conv Ne Xt convolutional neural network to extract image features.Then,the residual constructs multi-head self-attention module,which focuses the model’s attention on the segmentation target and enhances the expression of important features.Atrous spatial pyramid pooling(ASPP)is utilized to fuse features of different scales,and strip pooling is specifically designed to reduce the acquisition of useless features,taking into account the elongated shape of tea-picking points.Finally,information decoding is performed through convolution and upsampling operations to obtain the segmentation results.Experimental results show that the proposed model achieves a pixel accuracy of 75.20% and a mean intersection over union(m Io U)of 70.78%in tea garden scenes,with a running speed of 8.97 fps.Comparison with other models,including HRNet V2,Efficient UNet++,Deeplab V3+,and Bi Se Net V2,on the same test set demonstrates that RMHSA-Ne Xt outperforms them in terms of accuracy,inference speed,and parameter efficiency,striking a balance between accuracy and speed.This model can serve as a reference method for accurately locating tea-picking points in tea gardens.(3)In the context of tea plantation,it is challenging to calculate the 3D coordinates of teapicking points due to their small size,similar color features,and the interference of complex backgrounds,leading to failure in matching the feature points between the stereo images.To address this problem,we propose a target-constrained algorithm for calculating the 3D coordinates of tea-picking points,which enables accurate 3D reconstruction of tea-picking points in complex backgrounds.After obtaining the location of the tea-picking region,our algorithm first calculates the contour centerline of the tea-picking region in stereo images.Then,we utilize the epipolar constraint to determine the matched tea-picking region and avoid mismatch by applying similarity comparison algorithms.Finally,based on the principle of disparity,we calculate the depth of the target and thereby determine the 3D coordinates of the tea-picking points.In order to validate the accuracy of our proposed algorithm,we design a high-precision 3D displacement platform to conduct a verification experiment for calculating the 3D coordinates of tea-picking points.The experimental results show that the average error of the proposed algorithm along the three axes is 5.46 mm,0.84 mm,and 2.04 mm,respectively,and the root-mean-square errors are 2.63 mm,0.62 mm,and 0.93 mm,respectively,with a detection speed of 12.45 pairs per second(pps).Compared with other stereo matching algorithms,including SGM,ZED camera’s depth estimation algorithm,and CFNet,our algorithm achieves the lowest error rate and the highest detection speed.The research results demonstrate that the proposed target-constrained algorithm is capable of accurately calculating the 3D coordinates of tea-picking points in tea plantation.This thesis addresses the problem of identifying and locating picking targets in tea gardens by proposing an object detection algorithm based on Efficient Net(EF-FPNet)and a semantic segmentation algorithm based on multi-head self-attention mechanism(RMHSA-Ne Xt)to extract the picking targets in the images.The proposed target-constrained 3D coordinate calculation algorithm is then employed to calculate the 3D coordinates of the picking points.The research findings presented in this thesis have to some extent resolved the challenges of identifying picking targets in natural environments,segmenting picking points in tea trees,and computing their 3D coordinates accurately,thereby providing more precise coordinate information for selective tea picking.