Research And Application Of Dimensionality Reduction Algorithm Based On Deep-learning

Dimensionality reduction aims to map high-dimensional data to low-dimensional space without losing information as much as possible.It is an important method in high-dimensional data analysis.With the development of science and technology,high-dimensional data containing rich information grows explosively,With the increase of data dimensions,how to extract effective information from high-dimensional data has become a key problem to be solved in many fields,such as machine learning,text analysis,and image processing.This paper studies some classic dimensionality reduction techniques,and improved dimensionality reduction solution based on neural networks.It introduces residual structure,discriminative networks,and adversarial training ideas.Then,it constructed a data acquisition system and a dimensionality reduction system,Finally,experiments prove that the data acquisition system is stable and reliable,and the performance of the dimensionality reduction scheme is improved.The research in this thesis mainly includes the following aspects:1.Research and learn on a variety of classic dimensional reduction algorithms,be familiar with the flow of these algorithms and analyze their application effects.At the same time,the neural network algorithm used for dimensionality reduction and its improved algorithm are studied.2.A Residual Encoding Net is proposed.Firstly,the residual block is applied to the neural network instead of the original plain network.Then,the discriminative network is introduced as the error evaluation method of the neural network,and the traditional element-level error measurement is replaced with the feature-level error measurement method.Finally,the idea of adversarial training is introduced to make the encoding/decoding network and the discriminative network perfect in the game.3.A distributed data acquisition system is proposed and applied to website content acquisition and intrusion detection log acquisition.The data acquisition system and the dimensionality reduction system are coupled.The experiment proves that the function and stability of the distributed data acquisition system are normal and can be applied in practice.4.This paper integrates some dimensionality reduction algorithms into the dimensionality reduction system,and the application method and process of the dimensional reduction system are proposed.In the system testing and analysis phase,the accuracy of the low-dimensional codes generated by different dimension reduction algorithms in multiple target dimensions was compared and classified using subsequent classifiers.In summary,this paper summarizes the mathematical methods and neural network methods of dimensionality reduction.Residual Encoding Net is proposed and its performance is improved.In addition,this paper designs and implements a dimensionality reduction system and a data acquisition system,and then couples them together.