Study On Deep Learning Based Seedling Phenotyping Extraction And Automatic Transplanter In Plant Factory

Plant factory is a highly integrated agricultural system with important features of mechanization,informatization and intelligence.In the production process of plant factories,the process of transplanting high-density seedlings in nursery sponges to low-density cultivation boards is called transplantation.It is an important step to realize the intelligent production of modern agriculture by obtaining seedling phenotypic information,selecting normal seedlings and transplanting them to the cultivation board.In this paper,based on leafy seedlings in plant factories,deep learning-based seedling phenotypic information acquisition technology and the seedling ransplanter for plant factories were developed.The main research contents and conclusions are as follows:(1)Using the FCN(Fully convolutional network)to semantically segment the seedlings to divide each pixel of the image into stems,leaves,nodes and backgrounds.Using the deep learning network Mask R-CNN(Mask region-convolutional Neural networks)to instantly segment the seedlings to divide each pixel of the image into instances of stems,leaves and nodes.Selecting appropriate super-parameters based on experience,experiments and references.Using precision,recall and F1-value to evaluate the segmentation results of the stem,leaf and background by pixel level.The instant segmentation results of the stem and leaf were evaluated by Io U.The node detection results were evaluated based on the Euclidean distance between the predicted node and the actual node.The results show that the FCN method had a higher recall.The F1-value of the stem is 0.68,the F1-value of the leaf is 0.85,and the F1-value of the node(threshold=2 mm)is0.98.The Mask R-CNN method had a higher precision.The F1-value of the stem is 0.70,the F1-value of the leaf is 0.78,and the F1-value of the node(threshold=2 mm)is 0.94.(2)A multi-view based three-dimensional(3D)segmentation method was proposed.The two-dimensional(2D)segmentation results of seedlings were projected to the seedling 3D point cloud model and the seedling phenotypic information was extracted using the voting method.The 2D and 3D segmentation results were compared.The 3D segmentation results are superior to the 2D segmentation results in terms of semantics and instance segmentation.Compared with the multi-view segmentation method,Point Net++segmentation method and SGPN segmentation method,the results of Point Net++were the best.However,compared with the fitting results of seedling phenotype information(total leaf area,single leaf area,stem length)and actual parameters,as well as the processing time,because of the Point Net++and SGPN segmentation methods were limited by the number of input points,the multi-view semantic segmentation method has the highest degree of fitting to the total leaf area(R~2=0.9578)followed by Point Net++segmentation method(R~2=0.8251)and SGPN segmentation method(R~2=0.8849);multi-view semantic segmentation method has the highest degree of fitting for stem length(R~2=0.7696),and fitting degree of Point Net++(R~2=0.7189)is not much different from the multi-view semantic segmentation method.The SGPN method cannot correctly predict the stem length(R~2=0.0879);the degree of fitting to the single leaf area based on the multi-view instance segmentation method(mean R~2=0.8616)is higher than the Point Net++segmentation method(mean R~2=0.7970);the degree of fitting to the stem length based on the multi-view instance segmentation method(R~2=0.8711)is higher than the fitting degree of Point Net++(R~2=0.3466).Based on the actual application scenario,multi-view segmentation method is more suitable for real-time access to accurate seedling phenotypes.Finally,the seedling classification program was developed based on the extracted seedling phenotypic information.The program has the functions of user self-setting,selecting classification parameters,statistical seedling phenotypic information and seedling classification to provide data support for selecting normal seedlings for the transplanter.(3)Development of transplanter.According to the parameters and physical properties of nursery sponges and planted foam boards,the transplanter with function of seedling separation,transportation,classification and transplantation was developed.The transplanter was divided into three parts:the clamping separation part,the seedling phenotypic information acquisition part and the transplanting part.The studies focus on the design of the assistant separation mechanism,the clamping separation mechanism and the seedling phenotypic acquisition mechanism,focus on simulating the process of inserting the U-shaped sticks(in the assistant separation mechanism)into the sponges to optimize the U-shaped sticks,focus on the development of the clamps in the clamping mechanism,the design of the"half arc"fixtures and the selection of the appropriate gripping claws to separate the connected sponges by means of up-and-down staggered installation,focus on the development of the separation mechanism to complete the equidistant separation of the sponges with the seedlings.The masks,backlight source and 10 cameras are used to collect the seedling pictures for the seedling phenotypes.All the designs of the transplanter enable it to select normal seedlings based on acquired seedling phenotypic information and successfully transplanting them without destroying seedlings.(4)Operation parameters optimization and performance tests of the key mechanisms of the transplanter.The key mechanisms test platform and its control system were developed.The steps of sponge clamping,separation and release were controlled by PLC controller.The effects of the three main operating parameters of the key mechanism,air pressure of the pneumatic clamp,the insertion width of the clamp and the contact length between the clamp and the sponge block on the success rate of transplanting were studied.The effects of the above parameters on the failure rate of sponge block separation,seedling damage,sponge block damage,mechanism release failure and seedling skew were analyzed.Through a comprehensive experimental study on the above parameters of"868"cabbage seedlings cultivated on a sponge with a density of 30 kg/m~3,it was determined that when the air pressure of the pneumatic clamp was 0.6 MPa,the insertion width of the clamp was 11 mm,and the contact length between the clamp and the sponge block was 15 mm,it was the better operation parameter for transplanting.According to the transplanting experiments on"868"small green vegetables planted in the density of 20,25 and 30 kg/m~3,the"Italian"lettuce planted in the density of 30 kg/m~3 and the"new Shanghai green"small green vegetables planted in the density of 30 kg/m~3,the average success rate of transplanting is 96.8%,which proves that the designed transplanter is feasible.The operation time of transplanter is more can reduce by about 60%than manual transplanting.