Detection And Location Of Pomegranate Based On RGB-D Camera

In the fruit production process,fruit harvesting requires a large number of manual picking,and the contradiction between increased fruit production and insufficient labor is becoming more and more prominent,and automatic robot picking technology is an effective way to solve this contradiction.The fruit detection and 3D location ability of the picking robot directly affects the fruit picking success rate.In this paper,an improved Lite-Deep Labv3+ fruit detection network is proposed for pomegranate in orchard environment.This study is able to accurately segment individual pomegranate fruit point clouds after segmenting overlapping fruit point clouds.This study is capable of fast and accurate detection of pomegranate fruits and segmentation of single pomegranate fruit point cloud to complete 3D positioning of fruits and provide picking location guidance for picking robots,the main research contents are as follows:(1)In this study,Kinect V2 is used to acquire color and depth images of ripe pomegranates in orchards,and the acquired data are pre-processed.The background point cloud is filtered out using pass-through filtering according to the depth distance.Then,Gaussian filtering was used to remove the noise distributed near the leaves and in space.The preprocessed data retains the pomegranate point clouds at close distances while filtering out most of the noise points for subsequent segmentation and location of the pomegranate fruit point clouds.(2)To address the shortcomings of traditional image object detection methods and deep learning-based object detection networks in terms of detection accuracy,detection speed and generalization capability,a Lite-Deep Labv3+ pomegranate detection model is proposed for the identification and localization of target fruits in natural environments.The backbone network of the model is selected as the lightweight network Mobile Net V2,which improves the detection speed while ensuring the algorithm accuracy and meets the real-time requirements of target fruit detection by picking robots.The pomegranate detection model proposed in this paper ensures the detection performance with high detection efficiency,the accuracy and F1 values of detection are91.5% and 89.9%,respectively,which are comparable with Mask R-CNN detection network with higher detection accuracy,but the algorithm proposed in this paper has 131 ms less detection time than Mask R-CNN on average for a single image.(3)In order to realize the 3D localization of fruits,for the segmentation problem of overlapping pomegranate point clouds,the LCCP algorithm is proposed to pre-segment the fruit point clouds into point cloud data containing only a single fruit.In order to solve the problem that Point Net++ network is not effective for sparse point cloud segmentation,Point Net++pomegranate fruit point cloud segmentation network based on feature mixing is proposed to further obtain accurate point clouds of pomegranate fruits.The experimental results show that the LCCP algorithm can segment the overlapping fruit point clouds effectively,and the feature Point Net++ based on feature mixing network achieves 87.2% accuracy for fruit point cloud segmentation,and this algorithm has good robustness to sparse point clouds.(4)Due to the limitations of the orchard environment and RGB-D devices,the collected pomegranate fruit point clouds are incomplete,which leads to the degradation of the localization effect.In this thesis,a deep learning-based point cloud complementation network PF-Net is used to achieve the point cloud complementation of pomegranate fruit.And Point Net-based fruit point cloud 3D localization network is proposed.The experimental results show that the point cloud complementation effect of PF-Net for fruits is better than the effect of PCN point cloud complementation algorithm.The proposed algorithm completes the fruit point cloud first and then performs 3D location with errors less than 0.6 cm in both X-axis and Y-axis directions,and less than 0.9 cm in Z-axis,and is robust to sparse point clouds and occluded point clouds.