Research On Autoencoder Architecture Optimization Based On Correlation Analysis

Since the concept of deep learning was proposed,it has become one of the hot research topics in machine learning field.Deep learning,as an excellent machine learning method,has been widely applied in various fields,such as natural language processing(NLP),image recognition,recommendation systems,speech recognition,and so on.The successful application of these research projects in the industry also illustrates the great advantage of deep learning.In general,deep learning is the modified and expanded model of artificial neural network(ANN)which improves certain disadvantages of ANN.And for different application scenarios,the architectures of deep learning network model are also different.In a variety of deep learning network models,the autoencoder is a relatively basic and simple network model,but the model can be well applied in the field of data compression and data dimension reduction,so it has attracted a lot of research interests.However,the dimensionality reduction performance or coding performance of the autoencoder,that is,transforming the original high-dimensional image data to the low-dimensional data(code)through the network,has a direct correlation of its network architecture.For different datasets,the optimal architecture of the autoencoder may be different in order to the optimal dimensionality reduction performance.Choosing a network architecture that does not match the input dataset,such as too many or too little network nodes,will cause the model training time is too long to waste a lot of computer resources or the dimension reduction effect is not ideal and the reconstruction error is too large.So corresponding to different datasets,the optimal network architecture selection of the autoencoder is very important.The autoencoder belongs to the feedforward neural network models and for the architecture optimization of feedforward neural networks,there are many algorithms that are proposed,such as brute-force,pruning algorithms,network construction algorithms,genetic algorithms and so on.However,these algorithms are generally applicable to the early neural network models with relatively simple network architecture,and have the disadvantages of high computational complexity or slow convergence.So how to adaptively determine the optimal architecture of deep learning network models with a large number of network nodes and multiple hidden layers like autoencoder model is still an urgent problem to be solved.In this paper,a method based on the correlation analysis of the network node weight is proposed to determine the approximate optimal network architecture of the autoencoder.The proposed method is also applicable to the architecture optimization of other deep learning models similar to the autoencoder and has a high computational efficiency.Experiments show that for different datasets the optimal architecture of the autoencoder may be different,and the proposed method can be used to obtain near optimal network architecture separately for different datasets.