The Application And Implementation Of Sparse PLS Regression In Spectroscopy

Spectroscopic method has been widely used because it is rapid,non-loss,low-cost and easy to online analysis in real time.Due to the advantages of spectral analysis,it has recently been used in many different fields,such as medical science,agricultural science and chemical industry and so on.However,most spectral data sets are characterized by a high-dimensional,low sample size setting,at the same time,there is a serious overlap between the bands of the spectrum which makes the spectral data complex and difficult to analyze.In this paper,it focuses on multi-component problems which rely on near infrared spectroscopy technology.And then make models by traditionally multivariate calibration models such as PLS and other sparse methods.Finally,we give the comparison of all multiple regression methods in this paper by experimental result.Partial Least Squares Regression is the most widely used algorithm in near infrared spectroscopy.It is a non-parametric regression method based on higher-dimensional projection.It can effectively overcome the multiple correlation problems.A disadvantage of partial least squares algorithm is that they produce calibration models that are not parsimonious;all of the regression coefficients in the calibration model are non-zero.As a result,all wavelengths are used in the prediction of unknown samples.The weight or influence from irrelevant wavelengths,however small,cans have deleterious effect on prediction,since they contribute to unwanted noise.The term "sparsity" refers to calibration models having many zero-valued regression coefficients.The research of this paper focuses on sparse model.And briefly discuss the sparse methods in wavelength selection and sample selection from three aspects of covariance estimation,reweighing and threshold constraints.The main works are as follows:1.In this paper,we propose a sparse matrix transform(SMT)method to provide an accurate data covariance estimation,which can overcome the effect of small-sample-size and benefit both the PLS weight computation and subsequent regression prediction.Then,SMT method is imbedded in PLS and SPLS to calibrate model.Finally,we compare the two algorithms by software programs.2.A sparse method based on framework is proposed.There are two parts in the framework:first,classical multivariate calibration methods can be iteratively used to generate sparse models;second,the transform matrix function can be repaired by different methods.Then,we use sparse PLS in this framework(IRLS-SPLS),and iterative process the regression coefficients by different transform matrix function.Finally,we compare the two algorithms by software programs.3.Nonlinear partial least squares method is used to automatically select the sample,and then add the weight value to the result is sparse again.In this design point,for large sample data,if the sample size is too small,it is not suitable for the sparse method.,otherwise it will make the model under-fitting.