Research On Deep Learning Algorithms

Deep learning has become one the most challenging research area in machine learning as its huge development in object and speech recognition since 2006. To sim-ulate the human brain better, people improve the architecture of deep neural network constantly according to the research of human brain. By now, deep learning has been considered as one of most possible way to realize artificial intelligence.In this paper, we firstly introduce three most common architecture of deep neu-ral network:Convolutional Neural Networks (CNN), Restricted Boltzmann Machines (RBM) and AutoEncoder Networks. Then, we talk about some optimize methods in deep learning. Among these methods, we highlight the stochastic gradient descent and give some useful tricks in large scale machine learning problems. After this, we intro-duce some second order optimization methods and train deep auto encoder networks via hessian free method. To reduce the training time of hessian free method, we par-alleled the algorithm based on MPI. Also we accelerate the training of convolutional neural networks according to the convolution theorem. After that, we focus on the problem of visualization of features layer wisely which is a challenging area in deep learning. We use two different methods to visualize the features of convolutional neural networks and deep autoencoders separately.Finally, we proposed two new methods for host load prediction in cloud com-puting. The first one combines the Phase Space Reconstruction (PSR) method and the Group Method of Data Handling (GMDH) based on Evolution Algorithm method, while the other uses an auto encoder network as pre-current feature layer of the Echo State Networks (ESN), which captures the similarity between host load traces better.