Research On Hybrid Convolution Neural Network Parallel Structure Based On Spark Distributed Platform

As the rapid development of integrated circuit,chip related for computer functions and modules are also gradually mature.The progress of science and technology makes the software to efficiently deal with intensive computing power.As a hot parallel platform,Apache Spark has shown a perfect performance for intensive computation.At the same time,we know that deep learning has already developed into all walks of life,in which convolutional neural network(CNN)is the most mature tools.However,in the era of big data,traditional platform cannot make use of CNN,such that we use Spark to speed up it.It is important to make an intensive research on CNN,which has a vital significance in the commercial sector.To realize convolutional neural network based on Apache Spark distributed platform is the focus of this research.For the convolutional neural network architecture,this paper discusses how to parallel computing module partition,how to design the topology structure and parameters,layers,etc.Through the detailed design implementation on Spark,the mainly content of the research includes following aspects:1.First of all,through detailed analysis of various network parallel convolution neural network architecture,we compare the advantages and disadvantages for these structures.Considering the characteristics of Spark cluster,we select the reasonable structure for our model.After that,we present our hardware architecture,the related parameters,layers,the convolution kernel,and so on.Meanwhile,by considering the resource constraints,we design a related calculation module to divide the dataset,which is convenient to better parallel the structure and achieves better performance;2.Afterward,we realized the convolutional neural network on Spark cluster,including interface,parameters,and some join order and so on.The network fusion is the key technique of this paper,which combines two neural network to boost the performance of the features,and then,the fusion results will be used fro classification.We present a large number of instance and a large number of analysis on the working principle of a detailed description and explanation;3.Finally,we conduct a large amount of experiments to verify our structure.Mainly including MNIST dataset.Extensive experimental results have proved that out parallel structure obtained the best performance compared with other algorithms.We do a large number of experiments to verify our model and relevant algorithms,and we use our parallel model to recognize the hand writing.Experiments show that our proposed parallel algorithm gets good performance compared with other compared algorithms.