Automatic Processing Of Close-Range Hyperspectral Images For High-Throughput Plant Phenotyping

Understanding the response dynamics of plants to biotic and abiotic stresses is essential to improve field management and achieve more sustainable agriculture.In this context,a closerange hyperspectral imaging offers a particularly promising approach by providing nondestructive measurements of plants that correlate with internal structure and biochemical compounds.It also offers a wider range of information in the spectral and spatial domains than other image-based sensors.However,extracting useful plant phenotypic traits from this technology is a major bottleneck for plant science and breeding communities.Advanced computational and programming skills are required for image analysis,and accurate machine learning models are needed for predicting plant biochemical and physiological traits.This study aimed to design an easy-to-use open-source software for high-throughput analysis of closerange hyperspectral images and evaluate it under various plant phenotypic applications.To achieve this aim,a new stand-alone Python-based software called HSI-PP(Hyperspectral imaging for plant phenotyping)was designed to better analyze hyperspectral and multispectral images easily,accurately,and without requiring any sophisticated computational or programming skills.Compared(i.e.,HYPER-Tools,and ENVI)to other software,HSI-PP achieved high-throughput analysis in 0.5 hours for a 12 GB dataset,while others took significantly longer.Features extracted with HSI-PP had a high correlation with features extracted with other professional software.HSI-PP also processed 10 GB on a normal PC in a time between 30 and 73 minutes.It is worth highlighting that the proposed software will be helpful for plant researchers,allowing scientists,especially plant geneticists,to nondestructively estimate plant stress responses.To test the applicability of the proposed HSI-PP software,it was utilized to analyze hyperspectral images in two investigations.The first investigated drought stress detection,while the second aimed to predict canopy nitrogen content(CNC).In the first investigation,HSI-PP analyzed the drought responses of 6 Arabidopsis thaliana genotypes and extracted 2600 features(morphological,spectral,and textural)from a 104 GB image dataset in 5 hours then the optimal features were selected for linear discriminant analysis(LDA).According to the results,combining proposed features achieved 94% accuracy in drought stress detection on day 4,while individual features achieved lower accuracies less than 85%.For the other investigation,the oilseed rape canopy was imaged at different nitrogen(N)rates and growth stages under different camera angles(90°,75°,and 65°).HSI-PP processed about 384 GB hyperspectral images within18 h then the extracted spectrum was utilized by four built-in regression algorithms,namely random forest(RF),support vector machine(SVM),multi-layer perceptron(MLP),and partial least square regression(PLSR).In this investigation,the PLSR model performed best at a 75°camera angle,with correlation coefficients(Rp2)of 0.82 using full-range wavelength.It achieved an Rp2 of 0.80 and a mean square error of 0.07,based on 17 effective wavelengths identified by the Genetic Algorithm(GA).Finally,the calibrated PLSR-GA model was used to assess the CNC distribution using HSI-PP.In brief,the HSI-PP can be useful for extracting various valuable phenotypic traits from hyperspectral images and has the capability of reusing pre-trained models,making it suitable for precision agriculture machinery.Preparing hyperspectral images for deep-learning analysis is a necessary step,but it can be quite complicated.To address this challenge,oilseed rape plants were grown with different nitrogen(N)treatments in greenhouse pot experiments to diagnose canopy N-status.Hyperspectral images were taken at four growth stages using a line-scan imaging system.Then,HSI-PP software was utilized to prepare hyperspectral images for deep learning models.A deep two-dimensional convolutional neural network called CND-CNN was developed and compared with other architectures(VGG19,VGG16,ResNet50,Mobile Net,and MobileNetV2)for canopy N diagnosis.Four preprocessing algorithms built-in HSI-PP named standard normal variate(SNV),multi-scatter correction(MSC),and first and second derivatives were compared to determine the best one.The results showed that ResNet50,MobileNet,and MobileNetV2 achieved classification accuracy below 75% under all preprocessing procedures,while CNDCNN,VGG19,and VGG16 achieved classification accuracy above 86%.SNV achieved the lowest accuracy(52-74%),whereas first derivative preprocessing obtained the highest one(93-95%).The proposed model(CND-CNN)was superior to all other CNN architectures with an overall classification accuracy of 95% with the lowest misclassification using first derivative preprocessing.This study verified the ability of HSI-PP in preparing the hyperspectral images for deep-learning analysis and hypercube augmentation,which is only available in this software and essential for avoiding model overfitting.The final challenge is the infield hyperspectral image calibration and segmentation.The field experiment was undertaken by applying three levels of N treatment for cultivar two genotypes of oilseed rape.Hyperspectral images were captured using a snap-scan hyperspectral imaging system,and HSI-PP software was used for automatic image calibration and segmentation.Different built-in image segmentation methods were compared,where K-means using vegetation indices performing the best in segmenting the canopy.The CND-CNN model was applied for deep feature extraction from the processed hypercube through transfer learning.Six machine learning classifiers,including traditional algorithms built-in HSI-PP named as(RF,SVM,LDA,k-Nearest Neighbor,and Back-propagation neural network),and Automated Machine Learning(Auto ML),were used for analysis these features.The results indicated that Auto ML and RF achieved the best accuracy(89%),whereas others had an accuracy of less than 86%.Meanwhile,RF achieved precision and sensitivity higher than others.These findings demonstrate that HSI-PP performed well under field conditions and is capable of fine-tuning machine learning hyperparameters similar to powerful Auto ML.Overall,this dissertation demonstrates that HSI-PP is a high-throughput hyperspectral image analysis software that can be used in different plant phenotyping applications in both laboratory and field conditions.It can effectively process hyperspectral images with various interleaved formats,including line interval or band sequential.HSI-PP can also prepare hyperspectral images for deep-learning analysis or extract useful features for analysis by implementing machine learning algorithms,which can be applied in different plant research scenarios like classification,prediction,or data reduction.HSI-PP is expected to make a large-scale analysis of vegetation dynamics easier,leading to improved phenotyping efficiency for breeding programs,functional genomics,plant physiology,and agricultural management.