Pose Detection For Grasping Stacked Fruit Clusters By Parallel Robot In Large Field Of View And Multiple Evaluation Factors

The automatic sorting of fruits by robot is of great significance in the automated and intelligent development of agricultural production and agriculture product processing.The parallel robot with low degree of freedom(DOF),which possesses some inherent advantages such as high precision,speed and stiffness,is more suitable for automatic sorting of fruits.These features also place greater demand on accuracy and efficiency for detection and control.The grasping pose detection of fruits is a precondition for achieving accurate,fast and nondestructive robotic manipulation.The machine vision possesses the advantages of non-contact,powerful adaptability and high cost performance.Compare with the eye-in-hand system in which camera is mounted on robot end,the eye-to-hand system with large field of view and little limit for working speed of robot end,in which camera is mounted on a fixed position,is more suitable for sorting stacked fruit clusters.For eye-to-hand system,the precision of grasping pose detection is particularly important because it has enough time to do detection on the cycle of placing fruit cluster last time.This paper focuses on the precision research of grasping pose detection for stacked fruit clusters by low-DOF parallel robot in eye-to-hand system.Currently,there are still some difficulties.It mainly includes the accurate solution of hand-eye calibration model in motion constraint of low-DOF parallel robot,accurate recognition and classification for small objects with complex backgrounds in large field of view,evaluation for grasping prioritization of stacked fruit clusters in multiple evaluation factors with unclear boundaries,and grasping pose calculation of stacked fruit clusters with unconstrained stalk by low-DOF parallel robot with motion constraint.Therefore,we need to explore a hand-eye calibration method that can accurately calculate hand-eye calibration model of low-DOF parallel robot and a fuzzy comprehensive evaluation method for grasping prioritization based on relative hierarchy factor set,construct cascaded faster region convolutional neural network(Faster R-CNN)based on multi-scale feature maps and 3D grasping models of stacked fruit clusters for low-DOF parallel robot,study grasping pose calculation method of stacked fruit clusters with unconstrained stalks by low-DOF parallel robot with motion constraint.The main contents of this paper are as follows:(1)Proposed hand-eye calibration method with error compensation and vertical-component correction for low-DOF parallel robot.For the problem that it is difficult to accurately calculate hand-eye calibration model due to motion constraint of low-DOF parallel robot,the vertical-component of hand-eye calibration is corrected based on vertical constraint between calibration plate and clamping mechanism.In addition,an improved eye-to-hand model of stereo vision with error compensation is constructed to reduce influence of camera calibration error and robot motion error on model accuracy.The non-trivial solution constraint of eye-to-hand model is constructed and adopted to plan calibration motion of clamping mechanism.The calibration experimental results illustrated,compared with random motion,eye-to-hand model without error compensation and model solving method based on matrix kronecker product,the average time of on-line calibration based on planned motion decreased by 50.51%,the 2-norm error of pose transformation matrix in calibration based on improved model and solving method decreased by 151.293.(2)Proposed cascaded Faster R-CNN based on multi-scale feature maps in large field of view.The recognition and classification based on existing neural networks tend to be low precision for small objects with complex backgrounds in large field of view.Therefore,a convolutional neural network(CNN)with small variations in feature map resolution is constructed to retain details and local features.The multi-scale feature maps in CNN are extracted and used for Faster R-CNN,and a cascaded Faster R-CNN based on coarse-to-fine and parameter-sharing strategies is proposed to reduce background interference.The experiments of stalk region extraction were carried out on the images of White Rosa grape clusters acquired by eye-to-hand system.Experimental results illustrated,compared with Faster R-CNNs based on single-scale feature map in LeNet-5,AlexNet and VGG16,the average precision of stalk region extraction based on proposed cascaded Faster R-CNN based on multi-scale feature maps increased by 25.87%,the average miss rate decreased by 30.56%.(3)Proposed fuzzy comprehensive evaluation method for grasping prioritization of stacked fruit clusters based on relative hierarchy factor set.To improve robotic grasping success,the fruit cluster features,stalk features,grasping limitations and change difficulties of robot pose are considered as evaluation factors for grasping prioritization of stacked fruit clusters.However,it is difficult to make a quantitative comprehensive analysis for these factors with unclear boundaries.Therefore,a hierarchical tree model without cross based on subtree structure is constructed to analyze the multiple evaluation factors of grasping prioritization for stacked fruit clusters.In addition,the constraints of direction changes of contours and gradient direction changes of points are added to detect segmentation points of cluster region.The weighted clustering effectiveness evaluation function constructed by similarity index,triangle inequality criterion and multi-dimensional feature vectors in multi channels of points are used to improve the existing k-means algorithm for constructing relative factor set with positive and negative effects.The mathematical expectation is used to construct average random consistency index and consistency satisfaction value for solving the problem that the consistency verification for comparison matrix based on experiential data tend to be low precision.The experiments of grasping prioritization evaluation were carried out on the images of stacked White Rosa grape clusters acquired by eye-to-hand system.Experimental results illustrated,compared with monolayer fuzzy comprehensive evaluation,the average precision of grasping prioritization based on proposed fuzzy comprehensive evaluation method with relative hierarchy factor set increased by 27.77%.(4)Proposed grasping pose calculation method for stacked fruit clusters and low-DOF parallel robot by constructing 3D grasping models.It is difficult to accurately calculate the grasping pose of stacked fruit clusters with unconstrained stalks by low-DOF parallel robot with motion constraint.Considering the clamping mechanism motion constraint of low-DOF parallel robot and stalk features of stacked fruit clusters,a grasping pose expression by a point and multi parameters is constructed,and four 3D grasping models for the fruit clusters of long stalk with node,short stalk with node,long stalk without node and short stalk without node are constructed.By constructing discrete Gaussian-Hermit moment as feature point descriptor,selecting feature points in separated grids of images and pre-detecting provisional model,the existing SURF and RANSAC algorithms are improved to detect grasping points in unconstrained stalks.The grasping pose parameters including spatial position,rotation angle about Z-axis and finger opening width of clamping mechanism are calculated based on the 3D grasping models.The grasping experiments for White Rosa grape clusters were carried out in constructed eye-to-hand system based on 4-R(2-SS)parallel robot.Experimental results illustrated,compared with grasping pose calculation method based on planar contour,the grasping success rates for stacked fruit clusters with stalk node and without stalk node,and the average grasping success rate based on proposed grasping pose calculation method increased by 14%,12% and 13% respectively.The research of this paper can realize grasping pose detection with high accuracy for stacked fruit clusters by low-DOF parallel robot in eye-to-hand system.It lays a foundation for robot automatic sorting detection and high-performance grasping control based on non-contact and non-destructive grasping pose detection theory.