The Research And Implementation Of Dimentionality Reduction Technology Based On Manifold Learning

With the rapid development and wide application of information technology, there has been a large number of high-dimensional data in many fields. High-dimensional data provides extremely rich information about the research object, but also has brought great challenges to the traditional data analysis and processing methods. Data dimensionality reduction has become an important step in many high-dimensional data analysis tasks. However, because of the nonlinear structure of high-dimensional data in real world, traditional linear dimensionality reduction technology can not meet the demand. Manifold learning, which is a new kind of nonlinear dimensionality reduction method for finding low-dimensional representations of high-dimensional data and exploring the inherent law and intrinsic structure of data, assumes that high-dimensional data exists on potential low-dimensional manifolds. In recent years, manifold learning has attracted more and more scholars'attention and has made great progress.However, manifold learning algorithms still have some problems.One is how to construct the neighborhood, because the neighborhood construction is directly related to the embedding effect. The other is when dealing with new data points, we must recalculate the low-dimensional embedding of all data points, which can not make full use of the existing results. In this paper, the research work on these two issues is carried out.At the same time, we explore the application of manifold learning in the field of iron and steel to expand the scope of practical problems that manifold learning can solve.The main work is as follows:1. To solve the problem of neighborhood selection, an adaptive neighborhood selection algorithm based on probability distribution is proposed and applied to ISOMAP. Finally, the validity of this adaptive neighborhood selection method is verified by the experiments.2. To solve the incremental learning problem of the t-SNE algorithm,we proposed an optimization method that combines local linear structure measurement which is based on PCA with local relation maintenance.Experiments show that, this optimization method can improve the performance of the algorithm, and the new data points can be processed more reasonably and accurately.3. The t-SNE algorithm is applied to the dimensionality reduction and visualization of steel data, and with the help of dimensionality reduction results, the corresponding data analysis with K-means algorithm is carried out which verifies the availability of the t-SNE algorithm in steel field. At last, the t-SNE algorithm is developed into a component and integrated into the steel quality analysis platform.