Research On Parallel Implementation Of Neural Networks Based On Cloud Computing And Their Learning Methods

With the rapid development of electronic technology, network technology and cloud computing technology, the current data is increasing in the way of mass, has entered the era of big data. Real world data, such as digital images, the gene expression patterns, face data set or web page text, usually have the characteristics of high dimension and large data volume. For traditional technologies of artificial intelligence and pattern recognition and so on, are all faced with the challenge of how to implement the data processing in the era of big data. For example, in the classification of a large scale of face data sets, a computer or workstation is very difficult to adapt to the actual requirements because of the lack of speed and storage capacity. Therefore, it is very necessary to study how to implement the technologies of artificial intelligence and pattern recognition based on multi-computer clusters in large data environment.When using the neural network in artificial intelligence to deal with the related data, if the number of size of training samples is not large, generalization ability and running time of single neural network are relatively ideal. However, with the increase of the identification number of categories, the structure of the neural network will also become more complex, lead to neural network training time become longer, convergence speed become slower, being easy to fall into local minimum and have the worse generalization ability and so on. In order to eliminate these problems, it can consider to design Hybrid Neural Networks(HNNs) composed of multi single neural network to replace the complex single neural network. In addition, it proposed a novel semi-supervised learning algorithm——deep learning approach using Deep Belief Network embedded with Softmax regress(DBNESR) as a classifier.This thesis’ s main contributions are as follows:(1) This thesis baseing on cloud computing clusters, proposes a novel method of parallel implementation of multilayered neural networks based on Map-Reduce. Namely in order to meet the requirements of big data processing, this thesis presents an efficient mapping scheme for a fully connected multi-layered neural network, which is trained by using error backpropagation(BP) algorithm based on Map-Reduce on cloud computing clusters(MRBP). The batch-training(or epoch-training) regimes are used by effective segmentation of samples on the clusters, and are adopted in the separated training method, weight summary to achieve convergence by iterating. For a parallel BP algorithm on the clusters and a serial BP algorithm on uniprocessor, the required time for implementing the algorithms is derived. The performance parameters, such as speed-up, optimal number and minimum of data nodes are evaluated for the parallel BP algorithm on the clusters. Experiment results demonstrate that the proposed parallel BP algorithm in this thesis has better speed-up, faster convergence rate, less iterations than that of the existed algorithms.(2) Presenting an application research of radical basis function neural networks based on Map-Reduce on cloud computing clusters and an affective computing research based on Map-Reduce on cloud computing clusters. With the advent of "big data" era, the traditional standalone serial-based training machine learning has been difficult to meet the needs of "big data" applications. To this end, this thesis discusses and has realized neural network learning algorithm based on cloud computing and an affective computing research based on cloud computing clusters. Namely with the help of a cloud computing platform through network circulation and combination to provide computing power as super computer to realize parallel training and classification recognition application of RBF neural network and the relevant algorithm, so that to make neural network and the relevant algorithm can study and process mass, high-dimensional data by cross-platform. First, several experiments have been designed to verify the feasibility of neural network machine learning based on cloud computing; secondly, being applied to specific face recognition, speech recognition, affective computing and so on issues. Finally, through experimental verification, neural network learning algorithm based on cloud computing has faster training speed, higher recognition accuracy, greater data processing capabilities than the traditional standalone serial-based training neural network learning algorithm.(3) Proposing a novel method of learning hierarchical representations for face recognition using Deep Belief Network embedded with Softmax regress and multiple neural networks. In face recognition and classi?cation, feature extraction and classi?cation based on insuf?cient labeled data is a well-known challenging problem. In this thesis, a novel semi-supervised learning algorithm named deep belief network embedded with Softmax regress(DBNESR) is proposed to address this problem. DBNESR first learns hierarchical representations of feature by deep learning and then makes more efficient classification with Softmax regress. At the same time we design many kinds of classifiers based on supervised learning: BP, HBPNNs, RBF, HRBFNNs, SVM and multiple classification decision fusion classifier(MCDFC) — —hybrid HBPNNs-HRBFNNs-SVM classifier. The conducted experiments validate: Firstly, the proposed semi-supervised deep learning algorithm DBNESR is optimal for the face recognition with the highest and most stable recognition rates; Second, the semi-supervised learning algorithm has better effect than all supervised learning algorithms; Third, hybrid neural networks has better effect than single neural network; Fourth, the average recognition rate and the variance are respectively shown as BP<HBPNNs≈RBF<HRBFNNs≈SVM< MCDFC<DBNESR and BP>RBF>HBPNNs>HRBFNNs>SVM>MCDFC>DBNESR; At last, it reflects hierarchical representations of feature by DBNESR in terms of its capability of modeling hard arti?cial intelligent tasks.