Research On The Predicting Methods Of Soil Available Nutrients With Hyperspectral Data

Soil available nutrients,including available nitrogen,phosphorus,and potassium(NPK),playing an important role in enhancing soil fertility and plant productivity for the growth and development of agricultural systems.It is essential to improve the efficiency and accuracy of soil available nutrient detection for the reasonable fertilization and the sustainable development of agricultural systems.Traditional laboratory methods for quantifying NPK are expensive and time-consuming,and thus cannot meet the requirements of modern soil quality assessment and management,particularly with respect to precision agriculture.Alternatively,previous studies have suggested the reflectance spectroscopy analysis approach as a rapid,non-destructive,reproducible,and cost-effective analytical method for assessing soil properties.For example,soil water content(WC),organic matter(OM)and some other high content soil properties could be relatively well predicted using VNIR/SWIR spectroscopy benefiting from well-recognized spectral absorption signals.However,NPK do not have any obvious specific spectral feature signal,and they usually exist in low concentrations in the soil.Consequently,their identification using spectral approach(calibration and prediction)is difficult to achieve.In addition,the existence of unexpected irrelevant information in spectra also greatly affects the performance of calibration models for quantifying soil NPK.The overall aim of this current study is to improve the spectral detecting accuracy of soil NPK.To achieve this,the lime concretion black soil samples that collected from three sites in the plain of northern Anhui Province of China with laboratory VNIR(400-1000 nm)imaging hyperspectral data recorded,and the loessial soil samples that collected from four small watersheds located in the central Negev Highlands of Israel with were field VNIR/SWIR(350–2500 nm)hyperspectral data recorded,were chosen as study objects.Several calibration methods including preprocessing transformations(PPTs)and regression algorithms were optimized and innovated to improve the spectral model performance.The main research contents and results are as follows:(1)Evaluating calibration methods for predicting soil NPK using laboratory imaging hyperspectral VNIR data.The spectral data were acquired with a self-made indoor hyperspectral VNIR system,which was analyzed and transformed by 21 PPTs.Raw spectrum and 21 PPTs,combined with three RAs,for a total of 66 calibration methods,were investigated for modeling and predicting NPK in the lime concretion black soil using laboratory imaging hyperspectral VNIR data.Results show that prediction performance can be significantly improved by applying proper calibration methods.The SG-filtered PPTs,such as derivative transformations and scatter correction methods,showed higher performances both in linear and non-linear RAs,especially for PLS-R.But non-linear intelligent RAs work better than PLS-R with some specific unfiltered PPTs.As a result,the calibration method of SNV/BPNN was preferred for predicting P(RPD = 2.09,SSR/SST = 0.85)and SG+LG/PLS-R was preferred for predicting K(RPD = 1.47,SSR/SST = 0.95).However,with extremely low RPD and SSR/SST values,the prediction of N was unreliable.(2)Predicting soil P using hyperspectral data based on PLS-BPNN.PLS-R was applied to conduct dimensionality reduction and feature selection.Five latent variables(LVs)were obtained by the leave one out cross-validation and nine optimal wavelengths were selected by the variable importance in projection(VIP)scores.The BPNN regression models were built with the input of the five latent variables(LVs-BPNN),the nine optimal wavelengths(VIPs-BPNN),and the whole wavelengths(Ws-BPNN),respectively.We found that the PLS-BPNN models could significantly reduce the degree of overfitting and improve the generalization ability;moreover,the LVs-BPNN model could improve the accuracy by 9.60% comparing to Ws-BPNN model.(3)Applying linear multi-task learning for predicting soil NPK using field VNIR/SWIR hyperspectral data.In addition to NPK,soil water content(WC),pH,electrical conductivity(EC),and organic matter(OM)were applied to build the LMTL models simultaneously for enhancing model learning ability.We investigated the performance of a linear multi-task learning(LMTL)algorithm based on a regularized dirty model for modeling and predicting seven key soil properties using field VNIR/SWIR spectroscopy as an integrated approach.Comparing to the commonly used single-task learning algorithm – PLS-R,our results show that all of the LMTL models outperformed the PLS-R models.The predicting RPDs of NPK are 1.40,1.49 and 1.22,and the SSR/SSTs are 0.58,0.64 and 0.52.(4)Analyzing the characteristic of soil field VNIR/SWIR and finding out the mechanism of hyperspectral prediction of soil NPK.Our study also showed that the distributions of the important wavebands with high VIP scores for the seven key soil properties were quite similar,especially in the four feature-block regions.In addition,the features used in the LMTL models illustrated the existence of correlations between the different soil properties.The correlations were mostly attributed to the soil Fe oxides,water content,organics and clay minerals,which constitute the basis of the feature-block regions and shared features in soil spectroscopy.With the advantage of shared features,the prediction accuracies of several soil properties with low concentrations or unobvious spectral absorption signals improved.(5)Improving the prediction of soil NPK with field spectroscopy by double-shrinkage.We carried out a new analysis framework based on double shrinkage,which composed of spectral shrinkage and regression shrinkage,for improving the prediction accuracy of soil NPK using field VNIR/SWIR.First,we built a filtering model based on the Y-gradient general least square weighting(GLSW)for conducting spectral shrinkage for removing unwanted information from the field spectroscopy.Second,the regression coefficients were shrunk by Lasso algorithm for selecting relevant features for enhancing model generalization ability.Results show that the double-shrinkage models got the best prediction accuracy for all of NPK with only 34,29 and 19 wavebands were used,and the predicting RPDs of NPK are 1.42,1.65 and 1.33.That outperform PLS-R models,the single-shrinkage models(GLSW+PLS-R and Lasso),and even the LMTL models.However,the explanatory power of the double-shrinkage models was moderate due to fewer features being selected by the regularization algorithm.This research clearly defines the mechanism of hyperspectral prediction of soil NPK,which can provide new ideas for improving the hyperspectral detecting accuracy of soil NPK.Morevoer,we provided an important theoretical basis and practical basis for the rapid detection and management of soil available nutrients and for the development of modern precision agriculture technology.