Design And Key Technologies Of The Kiwifruit Picking Platform Equipped With Dual Robotic Arms

Kiwifruit is favored by customers in the fruit market due to its unique nutritional value and delicious taste.China is the largest kiwifruit producer in the world,and kiwifruit is an important economic crop in Shaanxi Province,where its industry scale continues to grow.However,the high labor costs associated with kiwifruit harvesting severely constrain the industry’s development.With China’s kiwifruit production moving towards industrialization and standardization,there is an urgent need to achieve automation and informatization of the production process in order to improve harvesting efficiency,ensure fruit quality,and reduce labor intensity.Fruit and vegetable harvesting robots are an innovative solution to overcome the challenges brought about by labor shortages.In response to the issues of poor flexibility in operation for the current rectangular coordinate configuration of the harvesting mechanical arm and the difficulty of a single mechanical arm to meet the requirements of efficiency,this paper investigates the design and key technologies of a dual robotic arms picking platform for the scaffolded kiwifruit orchard based on a multi-joint robotic arm.The main research contents and conclusions of this paper are as follows:(1)Overall design of the kiwifruit picking platform with dual robotic arms.To ensure a continuous picking area between the dual robotic arms without fruit leakage,a theoretical model was carried out with the objective function of the maximizing the picking area,minimizing the public areas,and covering the thickness of the trellis.The spatial layout of the dual robotic arms was transformed into a multi-objective optimization problem for the operating space.The optimal relative positions of the dual robotic arms(870 mm),the installation height(1020 mm),and the intermittent forward distance of the mobile platform(450 mm)were obtained using the particle swarm algorithm,which solved the problem of discontinuous harvesting area between the two arms.The overall solution and working principle of the kiwifruit picking platform with dual robotic arms were also determined,which consists of two sets of vision systems,two harvesting robotic arms,two sets of end-effectors,a control system,and a mobile platform.(2)Fruit recognition and localization,and grasp angle estimation.To achieve accurate identification and positioning of kiwifruit targets and ensure that there is enough effective travel allowance for the gripper to accommodate fruit localization errors.Additionally,suitable grasp angle was predicted to guide the gripper in safely approaching the fruit.A kiwifruit recognition system was implemented based on the YOLOv4 deep learning model,which was tested in orchard and laboratory environments and achieved an average recognition accuracy of over 90%.Combined with the internal camera parameters and hand-eye calibration external parameters,the system could perform coordinate transformation and localization of kiwifruit.Indoor fruit localization error tests showed that the average horizontal and depth localization errors were 5.0 mm and 8.3 mm,respectively,meeting the precision requirements of the picking end-effector.The end-effector grasp angle estimation was based on the GG-CNN2 grasp detection model,which defined grasp configurations and created single-fruit,linear cluster,and other cluster grasp labels.The Focal Loss function was employed to prevent the network from generating optimal grasp configurations in the background or on the edges of the fruit.Tested in orchard environments,the model had 66.7 k parameters and an average image processing speed of58 ms,achieving an average grasp detection accuracy of 76.0%.Regarding the grasp sequence,desktop tests showed that for linear clusters and other clusters with up to four fruits,the predicted sequence from outside to inside was reasonable.However,the adjacent relationship between fruit clusters with five or more fruits changed dynamically with significant uncertainty.(3)Design of the dual robotic arms motion control model and simulation.To efficiently build a simulation and real development environment for dual robotic arms motion control,a dual robotic arms motion control system was built based on the Robot Operating System(ROS).The ROS-Move It! module was used to design the dual robotic arm motion control model,including building a dual robotic arm URDF model,configuring three motion planning groups for the left arm,right arm and dual robotic arm,and creating a dual robotic arm self-collision matrix.A Move It!+Gazebo simulation environment was established by writing a dual robotic arms trajectory controller.Single planning group obstacle avoidance trajectory planning and dual robotic arms synchronous planning group obstacle avoidance trajectory planning were conducted separately in the robotic arm joint space.The operational strategies were simulated and verified using the simulation environment,which determined a parallel operational strategy for the dual robotic arms based on synchronous planning groups for each side space,as well as an asynchronous operational strategy based on a single planning group for the shared space.Finally,the UR5Kinematics-based inverse kinematic solver for the dual robotic arm was replaced.(4)Prototype and verification experiment of kiwifruit picking platform with dual robotic arms.To verify the adaptability of the kiwifruit dual robotic arms picking space and the overall operational performance,an indoor scaffolding environment corresponding to the optimal picking space size was built and the hardware and software integration of the dual robotic arms was completed.The ROS Master distributed node management mechanism was used to implement the communication tasks of the host,the two robotic arms and the two slaves.Two sets of control hardware for picking end-effectors were equipped,and a visual recognition positioning model was deployed,along with calibration of the two robotic arms to achieve information conversion.Fruit position traversal experiments in the scaffolding space and laboratory dual robotic arms picking verification experiments were conducted,and evaluation indices included the success rate of fruit position traversal,the success rate of fruit picking,and the average picking time per fruit.Results showed that the two robotic arms had a fruit traversal capability of over 90% in the scaffolding space,and they were able to traverse the required target fruit points except for the singular points,effectively solving the problem of fruit leakage due to discontinuous working areas.The success rate of fruit picking on the platform was 75%,and the average picking time per fruit was 5.36 seconds for synchronous operation on both sides and 8.67 seconds for asynchronous operation in the shared space,validating the effectiveness of the dual robotic arms picking platform for kiwifruit.