Research On Feature Processing And Model Optimization Based On Random Fourier Kernel Approximation

Effective representation of input data is the basic element of model training in supervised learning tasks.Kernel methods provide an effective method for data representation by implicitly transforming data into a new feature space and allowing non-linear data representation.Because the computing time required by the kernel method is largely affected by the size of the datasets,with the increasing size of the training data for the model,especially for the deep neural network model,more data is needed to train,the cost of training for the existing non-linear kernel model selection methods and the deep neural network model increases gradually,and there are some problems such as the difficulty of feature processing and the complexity of the model structure.Kernel approximation can solve the difficulties that kernel methods are difficult to compute in large-scale data.Kernel approximation algorithms with linear or sublinear time complexity also have the advantages of low computation overhead and easy expansion to large-scale data.Therefore,using the linear solution algorithm of kernel approximation method to solve the problem of feature processing and reduce the cost of model training has important research value.Random Fourier feature kernel approximation is one of the linear kernel approximation algorithms,which maps data features from the original feature space to another relatively low-dimensional explicit random feature space,thus turning the original non-linear problem into a linear problem.In this paper,by introducing the random Fourier feature space transformation kernel approximation method,we systematically study the problem of feature processing,complex model structure and large training time cost in large-scale data,the specific work is summarized as follows:(1)A kernel approximation model selection optimization algorithm based on random Fourier feature space is presented.In this paper,the influence of kernel approximation method of random Fourier feature transformation on two widely used learning models,Support Vector Machine and Kernel Ridge Regression,is analyzed,the upper bound of the kernel approximation error of the algorithm is given.It is proved that the accuracy of the algorithm is mainly related to the spatial dimension of the Fourier feature transformation and the parameters of the kernel function.Moreover,by efficiently selecting the model parameters in the Fourier feature space,the optimal parameter model obtained by the algorithm still has a good effect,avoiding the problem that traditional grid search method needs a lot of time to train the model in large-scale data,and provides a feasible solution for the model construction of larger-scale datasets.(2)A hybrid network model combining shallow network and deep network based on kernel approximation method is presented.This model combines shallow network based on random Fourier feature transformation with deep network.Before deep network training,the shallow network with random Fourier feature kernel approximation is used to extract and process the features of the data,and then the processed data information is trained with the deep network.An efficient hybrid network model is built with similar accuracy to a single deep neural network,and the number of layers and iterations of the model network are greatly reduced,and the floating-point computation and model structure complexity are low.In this paper,large-scale data feature processing and model optimization are studied.By introducing a kernel approximation method of random Fourier feature transformation,data features are processed,which provides an effective solution for model selection and construction.The results obtained in this paper can not only enrich the research on large-scale data feature processing,but also make a useful exploration on building a hybrid network model which combines shallow network with deep network.