Integration Of Unsupervised Feature Learning And Neural Networks Applied To Image Recognition

Unsupervised Feature Learning (UFL) is a technique used to automatically extract hiddenfeaturesfromdata,whichiswidelyusedindeeplearningtasks. WehaveintegratedUFLintotwodifferent neural network architectures, Deep Neural Network (DNN) and Convolutional NeuralNetwork (CNN) respectively, and applied to image recognition. Firstly, we use Autoencoderas the UFL model, build a Deep Neural Network by stacking multiple Autoencoders together,unsupervisely train the deep architecture layer-wise, then supervisely fine-tune the model usingBackpropagationalgorithm. Secondly,weintegratedUFLandCNNbysamplingasetofpatchesfrom training images, train an autoencoder to extract hidden features from the patches, whichare used as the convolutional kernels of the CNN, and finally train a classifier on top of theconvolved and sampled features for image recognition. The main contributions of this thesisinclude:1) To resolve the slow convergence of gradient descent methods with fixed learning rate,weproposedadescentmethodwithadaptivelearningratetotrainneuralnetworks. Learn-ing rate is dynamically determined by the difference of the parameters and their gradientsbetween consecutive iterations during training.2) Proposed a general unsupervised pre-training method with adaptive learning rate fordeep neural networks, which can be applied to networks with arbitrary layers and sizes.3) Designed a batch training procedure to calculate the gradients of autoencoder’s param-eters to speed up convergence of gradient descent training.4) Designed a batch convolving and batch sampling procedure for convolutional neuralnetworks and applied the implemented model to various iamge recognition tasks.