High-throughput Seed Recognition And Segmentation Based On Machine Vision Coupled With Deep Learning

Seeds are known as the "chip" of agriculture,also the quality and yield of crops are directly related to the quality of seeds.Therefore,seed quality inspection is of great significance for breeding high-quality crop varieties and ensuring national food security.Measuring morphological phenotypic parameters is a necessary procedure in seed quality inspection.The manual measurement method is easily affected by the measurement environment,personnel experience,and seed differences,while the mechanical measurement method has a limited application scope,and it is easy to damage seeds during the measurement process.With the rapid development of computer and image technologies,the data measurement method based on machine vision has gradually replaced the traditional methods due to easy implementation,low cost,and high reliability.Additionally,it is especially suitable for large-scale,long-time,and highintensity seed quality inspection.In the process of high-throughput seed quality inspection,it is easy to form touching and overlapping between seeds,resulting in image processing methods cannot accurately measure the morphological phenotypic parameters of each seed.Therefore,accurate and rapid recognition and segmentation of touching and overlapping seeds is difficult to measure morphological phenotypic parameters.The current domestic and international research only focuses on seed recognition and segmentation methods in laboratory and static environments,which cannot meet the requirements of seed quality inspection in various environments,such as the pre-harvest spike in situ and post-harvest production line dynamic environments.Therefore,rapid and stable seed in situ and dynamic recognition and segmentation is a difficult problem in achieving seed quality inspection in various states.This thesis focuses on the key problems in touching seed,overlapping seed,spike seed in situ,and production line seed dynamic recognition and segmentation based on machine vision coupled with deep learning technologies.The specific research contents and progress are as follows:(1)Research on touching seed recognition and segmentation.Touching is the most common problem in the seed high-throughput quality inspection process.The morphological phenotypic parameters of each seed are seriously affected by the touching problem in the image,thus,the processing of the touching problem based on machine vision technology is the key to accurately measuring the morphological phenotypic parameters of seeds.This study collected images of ten kinds of common crop seeds and innovatively proposed a multi-level segmentation algorithm for multi-category and high-density touching seeds.The multi-level segmentation algorithm comprised K-means clustering algorithm,nested watershed algorithm,and split line detection algorithm,which could recognize and segment the non-touching,simple-touching,and complex-touching seeds in the image,respectively.In addition,under-segmentation and oversegmentation caused by multi-level segmentation algorithms were solved through correction processing to improve further the effect of touching seed segmentation.The test results showed that the segmentation accuracies of ten kinds of seeds were above 99%,and the average segmentation accuracy was 99.65%.A 1000-kernel weight measurement system set was developed based on the proposed multi-level segmentation algorithm.After the manual correction of the system,the seed segmentation accuracy could be improved to 100%,and the average 1000-kernel weight measurement time was 1.54 s.The proposed multi-level segmentation algorithm and the developed system accurately and efficiently realized the recognition and segmentation of touching seeds and the 1000-kernel weight measurement.(2)Research on overlapping seed recognition and segmentation.Flat seeds are prone to overlap each other,resulting in mutual occlusion between seeds and the inability to measure the morphological phenotypic parameters.Recognition and segmentation of overlapping seeds is the premise of accurately measuring the morphological phenotypic parameters.This study used solanaceous vegetable seeds,including eggplant,tomato and pepper seeds,as the research objects.Aiming at the annotation challenges of overlapping seeds with small size and high density,this study drew on sim2 real ideas and innovatively designed a seed image simulation method based on multi-feature randomization to rapidly synthetize high-quality vegetable seed simulation images and automatically generate seed labels.The parameters of the Mask R-CNN model were optimized based on the characteristics of seeds to improve the segmentation performance of small and irregular vegetable seeds.The test results showed that the Mask R-CNN model trained by vegetable seed simulation images had an excellent instance segmentation performance on the real images,with AP@[0.50:0.95] reaching more than 0.85.In addition,the AP@[0.50:0.95] of the Mask R-CNN model in restoring the shape of occluded eggplant,tomato and pepper seeds reached 0.81,0.79,and 0.80,respectively.The combination of the designed seed image simulation method and the optimized Mask R-CNN model solved the huge annotation cost of high-density overlapping seeds.It efficiently and accurately realized the recognition and segmentation of overlapping vegetable seeds and the shape restoration of occluded seeds.(3)Research on spike seed in situ recognition and segmentation.The morphological phenotypic parameters of pre-harvest spike seeds can directly reflect the growth status of crops.The spike seeds grew more closely,resulting in complex touching and overlapping phenomena.In situ recognition and segmentation of spike seeds based on machine vision coupled with deep learning can provide a decision basis for optimizing crop growth management.In this study,the images of three varieties of wheat in the early filling,late filling and mature stages were collected in the field environment.The image scale was uniform,and the image size was optimized through perspective transformation,and size optimization.The wheat spike image was segmented based on the Mask R-CNN model,and the wheat spikelets were obtained with AP @ [0.5:0.95] reaching 0.73.In addition,52 wheat spikelet image features,including shape,color and texture features,were extracted based on the instance segmentation results.The prediction model between image features and spikelet seed number was established through the SVM model,and the test accuracy was 85.5%.The feature selection method based on random forest removed the unimportant image features of spikelets.For predicting spikelet seed number,the performance of the SVM model with 27 selected image features as input was close to that with full-feature modeling as input.Finally,the total number of seeds in the front wheat spike was measured by adding the number of seeds in each spikelet.The average absolute error of the measurement was as low as 1.04,and the average absolute percentage error was as low as 5%.In this study,the accurate in situ recognition and segmentation of wheat spike seeds were realized based on Mask R-CNN,and SVM models measured the total number of seeds in the wheat front spike.(4)Research on dynamic seed recognition and segmentation.High-throughput dynamic measurement of the morphological phenotypic parameters of seeds on the production line is an inevitable trend in modern seed quality inspection development.The seeds on the production line are in motion,resulting in the continuous change of touching and overlapping between seeds.Real-time and online recognition and segmentation of dynamic seeds based on deep learning methods are of great significance for realizing modern seed quality inspection.This study collected soybean,peanut,and corn seed videos at the speeds of 5 cm/s,10 cm/s,and 20 cm/s and decomposed them into seed image datasets.A sparse pre-annotation strategy was proposed based on a watershed algorithm to automatically and quickly annotate seed contours.In addition,a seed contour annotation software was developed based on the proposed image pre-annotation strategy to obtain accurate seed contour labels of the test image quickly.This study adopted YOLO V8 deep learning model to recognize and segment soybean,peanut,and corn seed images.The test results showed that the YOLO V8 model had the best instance segmentation effect on the seed images collected at the speed of 5 cm/s,the AP @ [0.50: 0.95] of the predicted bounding boxes was up to 0.87,and the AP @ [0.50: 0.95] of the predicted masks was also up to 0.72.The average speed of single image segmentation was less than 20 ms.The sparse pre-annotation strategy based on the watershed algorithm and YOLO V8 deep learning model solved the heavy seed frame image annotation task and realized the dynamic recognition and segmentation of seeds on the production line with 30 frames/s.