PLS Algorithm And Its Applications To SRM-Based Machine Learning

Machine learning is one of major fields of nonlinear science. Empirical risk minimization rule is a commonly used optimization index for large part of machine learning method for building nonlinear system model. According to statistical learning theory, how to give attention to both empirical risk and confidence interval, i.e. based on structure risk minimization rule, has attracted widely attention in the area of machine learning. Partial least-squares algorithm originated from process control achieves dimension reduction in high-dimensional data space and circumvents the collinearity problem due to highly correlated data by capture most explanatory variance in the original data. Kernel partial least-squares algorithm, support vector machine algorithm and fuzzy system modeling are all effective machine learning approaches, but they still have their own deficient in constructing nonlinear models yet.From the view point of integration of kernel partial least-squares, support vector machine, fuzzy system modeling and partial least-squares, the study in this paper is expanded to attain the goal of realization structure risk minimization rule in the process of machine learning.From the Mercer theorem, a kind of simple kernel partial least-squares algorithm is presented, and an index of risk is given at the same time, whose validity is proved in the simulation. To solve the problem of kernel function matrix expanding with the increase of the identification sample, block-wise kernel partial least-squares algorithm is described which divides the kernel function matrix and cut down the calculation burden.A subspace-partition based fuzzy system model (SPFS) and an adaptive model identification algorithm are proposed in this paper to solve the rule number's explosion problem. The algorithm partitions the discourse universe on principle of consistency and completion, and is helpful in relieving the rule number's explosion problem. In the advanced algorithm, partial least-squares algorithm is employed to pretreat identification sample and build initial model, then SPFS is built on the residuals. The balance of confidence interval and empirical risk is achieved byε-insensitive loss function and subspace-partition, and the structure risk minimization rule is realized.For the sake of poor generalization ability of partial least-squares, weighted partial least-squares algorithm are presented by integrated support vector machine and partial least-squares algorithm. Support vector machine training algorithm is used to calculate the linear regression model in outer model of weighted partial least-squared, and structure risk minimization rule is realized. Then T-S fuzzy system model based on support vector machine is proposed by applying support vector machine training algorithm in T-S fuzzy model modeling. The algorithm clusters with support vectors as centers in the discourse universe, then form the fuzzy rules: cluster center as antecedent proposition and linear partial least-squares regression model of the cluster as consequent proposition. Not only construct the T-S fuzzy system model adaptively, but also achieve the structure risk minimization rule.A robust recursive partial least-squares algorithm is proposed in this paper to solve the large computation burden problem of outlier detection algorithm in regression for time-varying system or massive data. By combination of recursive partial least-squares and robust principal components regression based on principal sensitivity vectors, settle the problem of large computation burden, as well as avoiding effectively masking and swamping with multi outlier exist. Overhead contact system detection is significant to manage high speed electrified railway safely. The nonlinear model of relation between pantograph and contact wire is studied based on the robust algorithm in this paper. After data standardization and outlier detection, input-output data is created by partial least-squares algorithm, and nonlinear model is constructed by support vector machine algorithm. Simulation result show the prediction precision of the model satisfied practical requirement.