Contributions To Several Issues Of Machine Learning Method Based On Support Vector Machine And Fuzzy System

Machine learning is one of important tasks of modern intelligence technoloy, it studies mainly how to summarize the laws which is drived from the observed data but can't be obtained by analyzing the principles at present. And use these laws to analyse objective phenomenon, predict further data or the data which can't be obseved. To be short, in order to gain the knowledge automatically, it uses computers to simulate the human learning ability. There are there kinds of basic machine learning problems, including pattern recognition, function approximation and probability density estimation.In the past decades, machine learning methods and applications which are based on Support Vector Machine (abbr. SVM) and fuzzy system are studied widely and develop rapidly. And it also achieves fruitful achievements in the fields of signal processing, intelligent control, pattern recognition, system identification, bioinformatics, medical treatment, behavioral science and business.This paper is aimed at several issues based on SVM and fuzzy system, including the learning speed and generalization ability of SVM, the interpretability of type-1 and type-2 fuzzy system, the combination of type-1 and type-2 fuzzy system and probability and calculus theory. In this paper, the creative research results are:(1) A novel method of improving the performance of a support vector machine classifier is presented by modifying kernel function. The method based on the differential approximation of metric merges data information into dynamic kernel by a positive scalar function. The separability is increased by enlarging margin around the separating hyper-plane. Example is given specifically for modifying Gaussian Radial Basis Function kernel. Simulation results for both artificial and real data show remarkable improvement of generalization ability and computational cost.(2) A kind of novel kernel functions are obtained from the reproducing kernels of Hilbert spaces associated with special inner product. The kernel functions with the advantages of both polynomial kernel and Gauss kernel, not only have a good global property, but also have a strong ability to interpolate. SVM with the proposed kernel functions only need less support vectors to construct two-class hyperplane than that with Gaussian kernel functions, so the proposed kernel functions have the better generalization capability. SVM applied to Wisconsin breast cancer data and artificial data using the proposed kernel functions, and demonstrate that it provides remarkable improvement of support vectors and training time compared with that of SVM with the Gaussian kernels. Especially, the proposed kernel functions become more and more efficient with the increase of orders of the space.(3) The conventional support vector machine classifier is a nonlinear classifier by mapping a low-dimensional data space into its high-dimensional feature space where it may become linearly separable using kernel functions. The SVM solution (an optimal hyper-plane) is obtained through maximizing the margin between the separating hyper-plane and data in the feature space. The performance of SVM depends heavily on the kernel functions. A class of new kernel functions is proposed, which are named as hyper-ellipsoid coordinate transform kernels using hyper-ellipsoid coordinate transformation formula. The performance of SVM with hyper-ellipsoid kernels may be enhanced as a result of the same dimensional mapping between input space and the feature space and the enlarged spatial resolution. Another advantage is much less support vectors to be required, resulting in faster learning and better generalization capability than the conventional SVM with Gaussian kernels. Experimental results for both artificial and real data confirm the effectiveness of the proposed hyper-ellipsoid support vector machine classifier.(4) Epanechnikov Mixture Model (abbr. EMM) can be translated to Mamdani-Larsen fuzzy system. The mathematical equivalence is proved between the conditional mean of an EMM, and the defuzzified output of a Mamdani-Larsen fuzzy system. The result provides a study of the new perspective of Mamdani-Larsen fuzzy system by interpreting it from a probabilistic viewpoint. Instead of estimating the parameters of the fuzzy rules directly, the parameters of an Epanechnikov mixture model can be firstly estimated using any popular probability density estimation algorithm, such as Expectation Maximization (abbr. EM). Simulation results show Mamdani-Larsen fuzzy system trained by the new way has high accuracy and strong anti-noise ability.(5) Generalized Epanechnikov mixture model can be translated into a rule- centered generalized fuzzy system (abbr. RCGFS) with multidimensional membership functions. The conditional mean of a generalized EMM is the defuzzified output of a RCGFS. A bridge is built between probability model and fuzzy system. The distinctive advantage of the proposed fuzzy system induced by a generalized EMM is easy to manipulate and highly interpretable, due to the fact that the coefficients in the consequent polynomials of fuzzy rules can be exactly interpreted as Taylor series coefficients. Moreover, the proposed system is also rooted at multidimensional membership functions that take into account the correlation among data components, which results in a more effective partition of the input space. The power of the proposed system is experimentally demonstrated by means of three benchmark examples: piecewise function, Mackey-Glass chaotic time series and a nonlinear dynamic system.(6) Uncertain Gaussian mixture model (abbr. UGMM) can be translated to an additive type-2 Takagi-Sugeno-Kang (abbr. TSK) fuzzy logic system. The mathematical equivalence is proved between the conditional mean of a UGMM, and the defuzzified output of a type-2 TSK fuzzy system (abbr. T2-TSK-FS). The results provide a study of the new perspective of type-2 fuzzy systems by interpreting them from a probabilistic viewpoint. T2-TSK-FS can be obtained from UGMM using any popular density estimation algorithm, such as EM. The new way of training a fuzzy model has two advantages, one is diversification of estimating parameters, and another is that T2-TSK-FS has high accuracy and stronger anti-noise ability. After comparing the simulation results with the ones obtained from other system modeling tools, it can be claimed successful results are achieved.(7) The existing methods of fuzzy system identification hardly keep good trade-off between precision and fuzzy meaning. One of the main reasons is short of systematic optimal structure identification methods.Thus, a novel fuzzy system whose fuzzy membership functions are the reproducing kernel functions is constructed based on the theorey of the reproducing kernel space. An adaptive learning algorithm for structure identification of the new fuzzy system is proposed using Fuzzy C-Means (abbr. FCM) algorithm and the approximating property of the reproducing kernel functions. Simulation results show the efficiency of the fuzzy system and its structure identification algorithm.