| Due to the shortage of agricultural labor force in China,while some high economic efficiency delicate fruits and vegetables demand a very short harvest period and sophisticated operations.Therefore,there is a great demand for efficient and practical fruit and vegetable picking robots.Achieving low damage,less missed harvesting and high-efficiency harvesting in a facility planting environment has great significance for vegetable and fruit picking robots coming into practical application,however,no universally effective solution has emerged in this field.At present,visual ability is the key factor that causes the bottleneck of harvesting success rate and quality of harvesting robots,so in this paper an efficient and non-destructive vegetable and fruit picking robot system is proposed,which mainly do improvements in term of vision system and ingenious picking action.Furthermore,verification experiment was conduct by picking tomatoes in field.Aiming at the problem that large number of overlapping and occluded tomatoes in agricultural environment are hard to pick,which cause the large positioning and picking errors and difficulty to improve the harvesting success rate,an instance segmentation algorithm based on multi-source image fusion was proposed to accurately detect the targets,and reached 98.3% detecting success rate and 91.6%detecting IOU.Then,in order to compensate for the positioning error of the occluded target,a tomato shape restoration algorithm similar to human’s associative ability was proposed to recover the shape and position information of the complete tomato.For tomatoes occluded less than 60%area,a centroid positioning error of less than 3mm was achieved by the algorithm.Secondly,in order to accurately determine the optimal harvest period and achieve high-quality harvesting,a tomato picking suitability evaluation model was proposed;Moreover,in order to achieve high-quality dexterous picking operation,a tomato pose estimation model based on the key points extraction method in deep learning was proposed,which improved the target understanding ability of the visual system.Finally,an experimental platform was set up,and experiments were carried out in the actual facility planting environment.The result of experiments showed that the average fruit picking time was 13 s,the picking success rate is 90%. |
