Research And Development Of Mobile Monitoring And Harvesting Robot Within Indoor Farming Systems

Precision indoor farming systems(PIFS)present a great potential to provide strong support for improving the output rate per unit area,labor productivity,resource utilization rate,and diversified and sustainable food supply agricultural production.Replacing people with machines is a solution to problems of high operating costs and skilled labor shortage of PIFS.This research aimed to realize robotization for monitoring and harvesting tasks,two key labor and technologyintensive tasks,which would further improve the intelligent level of PIFS.Some technologies such as mobile robotics,autonomous navigation,plant phenotyping,and visual servoing were involved in this research.An autonomous mobile robot(AMR),equipped with a positioning and navigation solution,suitable for use in PIFS was developed.A mobile robotics platform(MRP)was constructed by placing a multilayer perception robot(MPR)on top of the AMR,which could provide periodical monitoring of individual strawberry plants and fruit within a PIFS.A strawberry yield monitoring algorithm was developed to survey the total number of ripe strawberry fruit from inspection videos captured by the MPR for production management and harvesting schedules.A detection algorithm for multi-level ripeness of strawberry fruit was deployed to dynamically understand plant growth and provide data support for growth model construction and production management.An integrated mobile robotics harvesting system was developed to perform the tasks of detection,localization,grasping,detachment,and placement of ripe strawberries aided by an open-loop fruit positioning and handling algorithm.A rule-based strawberry scene categorization algorithm was developed for identifying executable harvesting actions,which can reduce unnecessary harvesting attempts.A novel detachment approach,which uses a pneumatically actuated soft gripper to separate a fruit’s sepal and stalk by a drag-androtate motion,has been found to improve the effectiveness of harvesting and the quality of harvested fruit.The MRP’s functions are expected to be transferable and expandable to other crop production monitoring and cultural tasks.The outcomes of this research are valuable in the development of unmanned agricultural systems.The main outcomes of this study are described as follows:(1)The perception range of AMR is limited by the narrow aisles within PIFS.The kidnapped robot problem will occur in this highly repetitive navigation environment.An AMR and a navigation approach were developed to navigate within the repetitive and narrow structural environments of a PIFS.The proposed navigation approach was to incorporate April Tag observations(captured by a monocular camera)into an inertial navigation system(INS)to form an ATI navigation system,which could avoid positioning failures caused by Li DAR-based SLAM algorithms in plant factories with repetitive shelves and narrow aisles.The cumulative errors from wheel odometry could be reduced by the loop closure detection.The AMR performed robustly at a traveling speed of 0.4 m/s with a positioning accuracy of 13.0 mm.The navigation system ensured smooth and low-error motions over a range of traveling speeds of AMR for stable quality of video collection.(2)A mobile robotics platform(MRP)was constructed by placing a multilayer perception robot(MPR)on top of an AMR,which was capable of traveling within a PIFS to achieve autonomous data collection and detection of individual strawberry plants and fruit.A strawberry yield monitoring method that incorporated keyframes extraction,fruit detection,and post processing technologies was developed to process the video data captured by the MRP.The method of keyframe extraction was applied to determine appropriate numbers of repetitive counting of a particular fruit in video frames.The counting-from-video problem was then simplified into the statistics of fruit detection results of keyframes.The results showed that an m AP@0.5 value of 0.945 could be obtained using a YOLOv5l6 model on ripeness detection tasks.The error rate of fruit counting was computed as 3.3% when the traveling speed of MRP was 0.3 m/s.The system showed robust yield monitoring results over a range of MRP traveling speeds.The best yield monitoring performance was found to have an error rate of 6.3% when the plants were inspected at a constant MRP traveling speed of 0.2 m/s.The MRP could provide strawberry yield monitoring for harvesting schedules.(3)The MRP was capable of capturing temporal-spatial phenotypic data of individual plants and fruit within a whole PIFS to dynamically understand plant growth and provide data support for growth model construction.A multi-level ripeness classification standard for strawberry was proposed to predict harvesting schedule and manage crop production.A rule-based strawberry scene categorization algorithm was developed for identifying executable harvesting actions,which can reduce unnecessary harvesting attempts.The results showed that an m AP@0.5 value of 0.863 could be obtained using the YOLOv5l6 model on strawberry multi-level ripeness detection task.The accuracy of the proposed strawberry growth scene categorization approach was 89.1%,and the weighted-averaged F1-score was 0.7,respectively.The fine-grained data of each individual strawberry enabled the selective robotic harvesting of strawberries.(4)An integrated mobile robotics harvesting system was developed to carry out harvesting tasks in strawberry PIFS.The robotic harvester is a hybrid rigid-soft manipulator with an integrated vision system.The open-loop position control architecture used in this work can facilitate the actions of perception,reasoning/learning,task planning,and execution.A lightweight YOLOv4-tiny model was deployed on an edge computing device to achieve realtime fruit detection.The strawberry scene categorization algorithm was applied for guiding the robot to perform selective harvesting,which can reduce unnecessary harvesting attempts.A novel detachment approach,which uses a pneumatically actuated soft gripper to separate a fruit’s sepal and stalk by a drag-and-rotate motion,has been found to improve the effectiveness of harvesting and the quality of harvested fruit.The results showed that an m AP@0.5 value of 0.909 could be obtained using the lightweight YOLOv4-tiny model on ripeness detection tasks.The experiments showed an overall success rate of 78% in handling harvestable strawberries with a damage rate of 23%.The average harvesting cycle time was found to be 10.5 s.The success rate of our robotic system reached 88% when handling the clearly visible ripe fruit separated from others.The grasping and detachment tasks were affected by fruit size and stalk length.These factors together determined the overall harvesting performance of the robotic system.The success rate of robotic grasping task reached 100% when harvesting fruit of small sizes(with a width less than 31 mm or a height less than 36 mm)and medium stalk lengths(between 81 and101 mm).A higher success rate of 88% with a lower damage rate of 16% were obtained when harvesting the fruit of small size.A higher success rate of 95% with a lower damage rate of 20%were obtained when harvesting the fruit of medium stalk length.The success rate and damage rate of robotic selective harvesting need to be further optimized by introducing flexible and adaptive executions.