Research On Dimensionality Reduction Method Of High Dimensional Data For Trend Prediction

With the upsurge of big data,the task of analysis becomes more and more complex.Some traditional machine learning technologies have been increasingly unable to apply to most of the current data analysis tasks.Complex phenomena often need multivariable data to describe,and these data have high dimension and large sample size.Through dimension reduction technology,we can analyze the hidden data information from these data,understand the essential characteristics of the data,and help to provide better information services in various fields.In this paper,the theory and technology of high-dimensional data dimensionality reduction and prediction are systematically studied,and the related algorithms are compared and explored.The traditional principal component analysis(PCA)algorithm is selected,although it belongs to the mainstream classical algorithm,but its linear dimension reduction effect is relatively not good,and the algorithm takes a long time,and it can not meet the prediction target when applied to the actual scene.For this reason,the notion of mutual information and entropy weight in this paper are introduced to improve the traditional PCA algorithm,and an improved PCA dimension reduction algorithm ew-pca based on the weight of entropy weight method is proposed.The improved algorithm first sets mutual information threshold for preliminary feature selection,then proposes the concept of weighted average to improve the data centralization process,and finally optimizes the dimensionality reduction process by introducing entropy weight to weighted principal components.Compared with the traditional PCA algorithm and the improved ew-pca algorithm,the latter has better dimension reduction effect.In order to further verify the quality of dimensionality reduction,the data set after dimensionality reduction is applied to the prediction process.The neural network algorithm is used for prediction analysis,and the neural network model is constructed.The experimental results show that the prediction accuracy is also higher through the mean square error,mean absolute error and other evaluation indicators.Finally,this paper applies the above method to the data collected from the Academic Affairs Office of our university,designs and realizes the dimension reduction and prediction system,and makes experimental analysis on the dimension reduction and prediction.The experimental results show that the improved dimension reduction algorithm ew-pca designed in this paper has good application in the actual data set,and can better improve the quality of data dimension reduction and improve the prediction accuracy.The results test and verify the practicability of the improved algorithm.