Feature Selection And Application Based On High-dimensional Independence Test

Effective screening and application of high-dimensional data features is a hot research topic in data preprocessing.In this paper,based on the one-dimensional data mean variance(MV)independence test,a new high-dimensional data test method is proposed(RP-MV independence test).High-dimensional data is projected to low-dimensional data by means of correlation screening algorithms and the selection of optimal projections,and carry out the test of highdimensional data.Simulate the results of multiple tests,and compare and analyze the theoretical basis,computational complexity,simulation test results,etc.with other existing high-dimensional data correlation test methods(projection correlation test,distance correlation test,etc.).And explain the rapid validity of the test proposed in this paper for high-dimensional data feature screening.On this basis,the empirical analysis of data feature screening is carried out.Using the intrusion signal characteristic data measured by the real fiber security monitoring system,the multi-dimensional independence test features are screened on the time domain features and the frequency domain features.The support vector machine classification model is used to discriminate and analyze the samples before and after the feature screening to final classification,the validity of the feature screening using this test is analyzed by using the prediction accuracy of the final classification model as a measure.The research field and specific application methods of this test are studied and prospected,which provides a new fast feature screening method for high-dimensional data analysis in different fields.