Multi-task Learning And Its Application In Spectral Multivariate Calibration

Recently, using the ultraviolet, visible or near infrared spectra to extract the physical or chemical information of analytes becomes popular in industry. Because of the simplicity and non-destructive property of the spectroscopic method, this method is particularly suitable for the online monitoring system such as monitoring the contamination in water. The developing of the machine learning is the main reason for the widely use of spectroscopic method. Many advanced machine learning technique have been introduced in modeling the spectral data. Meanwhile, because the spectral data is high dimensional and have high correlation among each dimension, many specific approaches have been proposed in recently years. Beginning with studying the transfer of multiple spectral calibration models, this dissertation introduce the multi-task learning into the spectroscopy. Several new methods about the spectral multivariate calibration and calibration transfer have been proposed. Then these methods are extended to the general machine learning problem. The main work of this dissertation can be summarized as follows:Firstly, to analysis the analytes with spectral data, we first need collect many training samples in the laboratory, and then build a linear or nonlinear calibration model to model the relations between analytes and measured spectra. When the calibration model is used in practical, the environmental condition are often changed. The model built in laboratory cannot be used in the new condition directly. We need adjust the old calibration model by some new training samples collected in the new condition. This dissertation proposes several approaches to transfer the spectral transfer model between two conditions based on multi-task learning and Gaussian process. These methods can transfer the calibration model built in laboratory into new condition with only several new training samples.Secondly, in water monitoring system using the spectrometers, the spectral data is collected from many different conditions. Utilizing the connection of spectral calibration model among different conditions, we can build more accurate models. The multi-task learning approach are used to study the relations among multiple spectrometers. For the high dimensional spectral data, two new multi-task learning methods are proposed. These methods can simultaneously discard irrelevant features and extract the common structure of the multiple calibration models.Thirdly, a new bayesian sparse mixture of experts model is proposed. This model can build a mixture of experts model with the feature selection and need not any regularization coefficient. The mixture of experts model can divide the input data into several different class and estimate different predictive function for each class. To utilize the mixture of experts model to analysis the high dimensional data, a new method combining the sparse Bayesian and the mixture of experts is proposed. This method is proposed for model the spectral data collected from different conditions without knowing their sources.Finally, a new method using the multi-task learning for a single classification problem is proposed. Integrating the multi-task learning and the mixture of experts model, we introduced the trace norm regularization into the mixture of experts method. This method utilize the mixture of experts model to divide a single classification problem into several relative sub-problems, and then use the trace norm regularization to extract the connection of these sub-problems. When the mixture of experts model divide the training samples into different classes, we may lack enough training samples in the estimation of experts models. The method can solve the problem via the multi-task learning. Experiments on several real machine learning dataset show the good performance of our new mixture of experts model.