Research On Autoencoder Based Unsupervised Learning Algorithms And Applications

With the continuous development of machine learning,especially deep learning theory and algorithms,in order to make full use of the increasing data,unsupervised learning algorithms have made great progress.The autoencoder based unsupervised learning algorithms can extract the common feature representations of the data and effectively compress the data by reconstructing the input distribution,and has achieved considerable research results.But there are still some key issues worth studying.Firstly,the autoencoder based unsupervised learning algorithms are susceptible to the outliers in the training data during the pre-training stage,which leads to the lack of robustness of the extracted features.Secondly,when using the autoencoder for sparse feature extraction,due to the need for adjusting many hyper parameters,which makes the training complexity high.Finally,when performing image generation tasks such as image completion,the traditional algorithms have a problem of blurring the generated image due to the insufficient resolution of the high-dimensional space.The autoencoder based unsupervised learning is a cutting-edge research topic in the field of machine learning and is the key to strong artificial intelligence.However,traditional algorithms have not achieved the expected results in terms of robust feature extraction,sparse feature extraction and image generation.Therefore,in response to these problems,this paper proposes improved algorthms based on the correntropy suppression data outliers,neuron completitive activation mechanism,and the regularization principle of linear inverse problems.These efforts make sense to the researches and applications of unsupervised learning aogorithms.The main research work and contributions of this thesis are as follows:(1)By analyzing the core technology of unsupervised learning algorithm,a theoretical framework of the autoencoder based unsupervised learning is proposed.First,the basic theory of feedforward neural network and convolutional neural network is described.Second,the basic framework and typical structure of autoencoder are analyzed in detail.Then,the basic theory of generating adversarial network,network framework,the objective function and the commonly used quantitative evaluation methods are studied.Finally,the theoretical framework of the autoencoder based unsupervised learning is proposed.(2)Aiming at the problem that the autoencoder pretraining stage is susceptible to the outliers,based on the principle of the correntropy suppressing the outliers,a correntropy-based contractive autoencoder robust feature extraction algorithm is proposed,named C-CAE,based on the ability of the correntropy loss function to suppress the outliers in the training data,and the regular effect of the contractive autoencoder on the disturbance,jointly suppressing the influence of the outliers in the data on the feature quality of the autoencoder pretraining stage It can be used in downstream tasks such as image classification and reconstruction.First,the definition and properties of the correntropy are analyzed,and the principle of the correntropy loss function to suppress training outliers is derived.Then,the network structure and optimization method of the contractive autoencoder are analyzed.Finally,the contractive autoencoder and the correntropy are combined.The loss function,the basic structure of C-CAE and the stacked C-CAE model are constructed,and the corresponding training algorithm is given.The experimental results verify that the robust feature extraction algorithm of contractive autoencoder based on correntropy can effectively suppress the influence of outliers in the training set on feature extraction(3)Aiming at the problem that traditional sparse feature extraction algorithms have many hyperparameters and high training complexity,a sparse feature extraction algorithm based on sparse target matrix generation is proposed.The algorithm constructs a sparse target matrix through neuron competitive activation mechanism,introduces a new sparse feature extraction method,reducing the complexity of unsupervised sparse feature learning algorithm.First,the related algorithms and research status of sparse feature extraction are analyzed;then,the neuron competitive activation mechanism is explored;finally,the sparse target matrix is established on the basis of sparse feature extraction and the neuron competitive activation,and minimized by the distance between the output of the competition layer and the sparse target matrix,providing a basis for solving downstream tasks such as image classification.The algorithm extracts sparse feature representations of data without explicitly modeling the data distribution by introducing the neuron competitive activation mechanism.The experimental results verify the effectiveness and universality of the sparse feature extraction algorithm based on the sparse target matrix generation.(4)Aiming at the problems of dislocation and insufficient definition in the image completion task,based on the regularization principle of the linear inverse problem,a parallel image completion framework with edge and color map,named PIC-EC,is proposed.Based on the autoencoders,this framework adopts generative adversarial network to learn the edge and color information from the dataset,and explicitly uses them as the prior knowledge for the following image completion network,and improves the performance of the image completion algorithms.Firstly,the background of the image completion and the least square generative adversarial networks are represented.Then,the mathematically model of the image completion is constructed,and the solution is analyzed.The edge and color priors play the important roles in the image completion task.Finally,the PIC-EC framework is proposed based on the analysis of the image completion task.PIC-EC is a two-stage model that can be summarized into three parts: edge map generator,color map generator and the following fine details completion network,they are all deep networks based on convolutional autoencoder.The edge and color maps should be restored firstly,because they have also been lost in the missing region.Edge and color maps provide basic structure and elements information for the image completion network.Experimental results demonstrate that PICEC method achieves superior performance on challenging cases with complex compositions and outperforms existing methods on evaluations of realism and accuracy,both quantitatively and qualitatively.