Research On Image Big Data Classification Based On MapReduce And Convolution Neural Network

With the progress and development of science and technology,tens of thousands of data are produced every day.It is very important how to mine or discovery valuable knowledge from the massive data.Deep learning is the most popular research topic in the field of machine learning,Alpha Go which overwhelmingly impacts its influence on the world is trained with deep learning algorithms.While convolution neural network(CNN)is a model trained with deep learning algorithm,CNN is widely and successfully applied in computer version.However,it is difficult to handle the problem of big data classification in stand-alone circumstance.Hadoop is a popular big data open source framework,by using Hadoop,one can easily deploy big data classification task into different cloud computing nodes.Furthermore,the Map Reduce of Hadoop provides an easy-to-use programming interface for users.Consequently,it is of important theoretical significance and application value to investigate image big data classification based on MapReduce and convolution neural network.The main works of this paper include two parts:(1)In order to understand how Hadoop handle big data problems,we investigated the principles of big data processing by Hadoop.Taking the condensed nearest neighbors of big data as an example,we proposed a MapReduce based algorithm for parallelization of condensed nearest neighbor,and implemented the proposed algorithm in Hadoop environment.(2)In order to deal with the problem of image big data classification,based on MapReduce and convolution neural network,we proposed an algorithm for classification of image big data.The proposed algorithm employed fuzzy integral to integrate the classifiers trained on different cloud computing nodes,and to classify image data.The presented method can solve the problem of big data classification,while it is impractical in stand-alone environment.