Research On Real-Time Recognition And Location Of Navel Orange Based On RGB-D Camera

Currently,Southern Jiangxi has emerged as the largest production area for navel oranges in the country,with an annual output of approximately 1.5 million tons,ranking first globally.However,the increasing output each year has exacerbated the challenge of declining rural labor force,as navel orange picking is a seasonal and labor-intensive process.The traditional manual picking method proves to be labor-intensive,costly,untimely,and resource-wasting.To address these issues and establish a modern agricultural production system,the development of fruit picking robots and the implementation of automated picking have become imperative for creating smart orchards.Fruit picking robots employ machine vision technology to recognize various fruit characteristics,including color,shape,and size,enabling precise fruit positioning.The effectiveness of the visual system and algorithms directly impacts the success rate of fruit picking.Traditional fruit recognition algorithms primarily rely on color images,making it challenging to overcome the complexities of the natural orchard environment.However,the demand for improved fruit recognition and positioning continues to rise,surpassing the limitations imposed by color images alone.However,the widespread availability of consumergrade RGB-D cameras has enhanced robots’ perception capabilities within their surroundings.Nevertheless,many fruit recognition algorithms based on RGB-D data suffer from computational inefficiency and low operational effectiveness.In this context,this research proposes a real-time navel orange fruit identification and positioning algorithm that utilizes deep learning technology and point cloud processing techniques,effectively meeting the realtime requirements on a common PC platform while emphasizing both speed and accuracy.This algorithm aims to provide precise operational guidance for the automatic picking of navel oranges.The main contributions of this study are as follows:(1)An analysis of the strengths and weaknesses of existing fruit recognition algorithms based on RGB-D data.Taking into account the specific requirements of navel orange fruit recognition and positioning,we propose an end-to-end navel orange fruit real-time recognition and positioning algorithm framework.We simulate the mechanical work angles for image acquisition,construct the navel orange fruit RGB-D dataset,and employ image enhancement algorithms to adjust image brightness(overexposure,underexposure),contrast,saturation,and introduce pulse noise and motion blur effects.This process serves to augment and expand the original dataset,which is made openly accessible.(2)Considering the challenges of fruit recognition in complex environments,we analyze the advantages and disadvantages of existing instance segmentation algorithms.With a focus on speed and accuracy,we investigate an improved YOLACT-Fruit algorithm based on the YOLACT algorithm,tailored to navel orange fruit recognition and segmentation.Given the position sensitivity of the backbone feature extraction network,we propose a lightweight feature extraction network based on HRNet+HRFPN to enhance speed while maintaining a certain level of accuracy.Comparative qualitative and quantitative analyses are conducted to evaluate the improved algorithm’s performance in different environments(illumination),occlusion,and adhesion.The test results demonstrate that the accuracy of YOLACT-Fruit slightly trails that of Mask R-CNN,while surpassing the latter in terms of speed.During the fruit recognition stage,the YOLACT-Fruit algorithm achieves an average detection speed of44.63 frames/s and an average accuracy of 31.15% on the self-built dataset,exhibiting improved performance compared to the original YOLACT and generating finer fruit instance masks.(3)To address the fruit positioning challenge,we fuse the instance mask obtained from navel orange fruit instance segmentation with the depth image to derive the fruit depth image containing instance information.Through projection transformation and preprocessing,we extract the fruit depth point cloud,which is subsequently subjected to minimum square fitting algorithms to determine the fruit’s radius and spatial coordinates.Moreover,we conduct comparative experiments to investigate the influence of different distances and point cloud quantities on fitting accuracy.In the fruit positioning stage,when the point cloud count ranges between 1,400 and 2,000,the fitting time amounts to 1.99 ms,with a positioning error of 0.49 cm,a root mean square error of the fitted radius of 0.43 cm,and a root mean square error of the volume of 52.6 m L.When the point cloud count exceeds 800 and the distance is within 1 m,the positioning error remains within 0.46 cm.Additionally,by implementing parallel computing,the overall processing speed of Orange Point Seg reaches 29.4 frames/s,achieving a commendable balance between accuracy and speed,and facilitating practical applications and engineering deployment.The findings of this study can be extrapolated to other fruit recognition and positioning tasks exhibiting similar morphological characteristics,providing valuable technical support for intelligent orchard management.