Font Size: a A A

Graph Based Machine Learning Algorithms Design And Its Application In Neural Network Research

Posted on:2015-11-01Degree:DoctorType:Dissertation
Country:ChinaCandidate:E M TuFull Text:PDF
GTID:1108330476453954Subject:Control Science and Engineering
Abstract/Summary:PDF Full Text Request
Machine learning is a sub?eld of arti?cial intelligence and it aims to construct a system that can learn rules automatically from collected data, meanwhile, to apply the rules to subsequential data processing. As a basic research, machine learning has its wide and important applications in many ?elds, such as bioinformatics, arti?cial intelligence, aviation & aerospace and modern medicine science, etc.As a branch of mathematics, graph shows its great potential and rapid development in machine learning research. A graph based machine learning algorithm is developed by transforming a learning problem into a graph problem and then by adopting existing graph algorithms, the learning problem is ?nally solved. Comparing with other machine learning methods, graph based machine learning method has the following advantages: 1. Graph is a branch of mathematics, and thus it has solid theoretical background. This ensures the solid theoretical foundation of graph based machine learning algorithm. 2. Graph is a simple but powerful model, and this make it have great potential in describing and solving a wide range of problems. 3. Graph model can be well described by only one matrix and most graph problems can be solved by linear algebra techniques. So graph method has a concise form and can be analyzed by matrix theory. 4. Graph spectral based algorithms often have close form solutions, or can be converted to a convex optimization problem, so the global optimal solution can be obtained and no local solution is produced.Existing graph based machine learning algorithms are mainly focus on two aspects: one is the graph based semi-supervised algorithms, including semi-supervised classi?cation and semi-supervised dimension reduction; the other is graph based unsupervised algorithms, including graph spectral clustering, unsupervised dimension reduction. Although these algorithms achieve considerable successes in many ?elds,there still have some problems. Generally speaking, the problems of existing algorithms are: ?rst, graph based semi-supervised learning has been applied to various?elds because it can classify abundant unlabeled samples using just a few number of labeled samples, which are usually obtained by hand-labeling and thus cost many time and resources. On the other side, most semi-supervised learning algorithms are tranductive and/or have polynomial computational complexity. In contrast, supervised learning can build a supervised model in the input data space and thus it can predict all seen or unseen samples directly and e?ciently, but, on the other hand, the supervised model has to been trained by plenty of well-labeled samples in order to obtain a good generalization ability. Therefore, how can we combine the advantages of semisupervised learning and supervised learning, i.e. to build a supervised model with good generalization ability in input space with just a few number of labeled samples, makes great practical value and worth attention. In this paper we do some attempt on this issue. Second, existing clustering algorithms, such as spectral clustering, can produce good results in some situations, but they still exhibit some disadvantages while processing nonlinear manifold distributed high dimensional data: they can hardly hand overlapped or severely noise-contaminated manifold distributed data or they need to solve a non-smooth optimization problem which usually scales very poorly to large data sets;furthermore, they also cannot give a representative sample in each class which can best represent the samples in the belonging class, as the representative selection is a key technique in automatically video or text abstraction. Third, Spatio-temporal data are characterized by its complexity of the evolving spatial and temporal distributed patterns and large amount of data streams. Traditional machine learning algorithms, such as support vector machine(SVM), are designed for static vector data and thus they can hardly learn and predict the spatially and temporally interrelated and interacted relations contained in spatio-temporal data. To address the above mentioned issues, our main work and contributions are as follows:I). We ?rst make some improvements to the Local and Global Consistency(LGC)algorithm, which propagates label information globally without paying attention to local sample distribution density property. The new LGC can propagate label information quickly in high sample-density region and slowly in low sample-density region and this can effectively prevent incorrect propagation. Based on this improvement, we propose a new two-step training supervised learning framework which can trained a robust supervised model in input space using only a very small number of labeled samples.II). Inspired by the LGC and manifold ranking algorithm, we extend the classic point cloud centroid concept in Euclidean geometry to manifold centroid concept in Riemannian geometry and propose new graph based k-means algorithm to cluster nonlinear manifold distributed high dimensional data. Comparing with traditional kmeans and other existing manifold clustering algorithms, the new algorithm can not only produce content clustering results on manifold data, it can also give a representative for each cluster that can be used directly for video or text automatical abstraction.III). Furthermore, the main computational cost in the above graph based algorithms(and also other graph based algorithms) lies in the various related computation of the manifold ranking matrix, so in this part we propose an e?cient algorithm which enjoys linear complexity in both space and time and thus scales well to large data set.We apply the e?cient algorithm to analyze the structure of neural network reservoir and propose a new learning framework, NeuCube, to process spatio-temporal data and to make early event prediction, as well as spatial temporal pattern recognition.IV). Besides, we also studied some other common problems in machine learning?eld. We propose a new simple but effective algorithm to remove bridging points which usually cause by class overlapping or noise contamination and can bring great negative impact on the performance of many clustering algorithms. We extend the weighted LS-SVM algorithm, which is originally proposed to regress a real value function or for binary classi?cation, to vector-valued function and thus can handle multiclass cases.Multivariate regression is an important technique in many tasks and now is commonly realized by multilayer neural network or multi-times single component function regression, but the former is usually hard to determine the network structure and size for a non-pro?cient user and the latter may cause the loss of possible interrelationships between different component functions.In order to show the validity of the proposed algorithms, we conduct experiments on both synthetic and real-world data sets to compare our proposed algorithms with the most popular and latest reported algorithms.
Keywords/Search Tags:Machine learning, Graph model, Manifold ranking matrix, Graph based k-means, Spiking neural network
PDF Full Text Request
Related items