Research On Face Recognition Technology Based On Hadoop Platform

Face recognition is one of the most important research directions in the field of computer vision and artificial intelligence.With the gradual arrival of big data era,the major media data has a blowout increase,and the demand for face recognition has also increased.The traditional face recognition technology can only solve the problem of small-scale face recognition,when facing a large number of face pictures the real-time performance is very low,and the training efficiency is low in a stand-alone environment and the storage of face data is difficult.Therefore,how to quickly and accurately identify a person's information from a large number of face images has become the main research direction of current face recognition technology.In order to solve the above problems,this thesis includes the following research:(1)The first is the theoretical research.The key technology of Hadoop big data processing architecture is analyzed in detail,such as the process of reading and writing data in HDFS,the principle of MapReduce and the software architecture of HBase.In order to solve the problem of difficult storage of unstructured face data in a stand-alone environment,a method of using HDFS and HBase to store the data was proposed.and the advantages of HBase columnar storage are fully utilized to improve storage capacity.(2)In view of the problem of low accuracy of traditional CNN-based face detection algorithm in face detection experiment,an improved MTCNN face detection algorithm was proposed.The model migration of the original MTCNN algorithm was carried out to shorten the training time,the main parameters of the MTCNN network model were adjusted continuously through experimental comparison,and the candidate box of repeated detection was simplified.According to the face accuracy score given on the candidate box,the discriminant formula of face false detection was proposed to improve the accuracy of face detection.Experiments show that compared with the traditional algorithm,the improved MTCNN algorithm improves the accuracy of face detection in the classroom environment by 3.8%.(3)As for the poor performance of Hadoop in processing massive small files,this thesis adopts the method of lossless compression of small files and merging of small files.By compressing files,reducing the pressure on the Name Node,using the Sequence File method to merge small files into large files,reducing the storage pressure caused by too many small files on HDFS.Experimental results show that reading and outputting the same number of files,the task running time after merging small files is shorter.(4)In order to solve the problem of poor real-time performance of traditional face recognition methods,this dissertation uses a method that combines PCA face recognition algorithm with Hadoop batch MapReduce.First,the PCA face recognition algorithm uses Map to calculate the Euclidean distance,and obtains the processing result,the results are summarized by Reduce,and the built-in image information corresponding to the minimum Euclidean distance is obtained as the final face recognition result and uploaded to the HBase database for storage.Experiments show that in the case of a large number of face images,the MapReduce parallel computing mode is faster than face recognition in a stand-alone environment.(5)So as to further verify the advantages of face recognition in the Hadoop cluster environment,three comparative experiments were carried out,namely the comparison of the face recognition time in the cluster mode and the stand-alone mode,the influence of the number of cluster nodes on the computing performance,and the impact of cluster mode MapReduce configures of different concurrency on computing efficiency.Through comparative experiments,the performance of the research of the whole subject was tested,and good test results were obtained.