Data Preprocessing Parallel Algorithm Implementation And Performance Optimization For Deep Learning

With the development of parallel technology and deep learning,parallel technology is used in many fields,such as scientific computation.Besides that,parallel technology is widely used in big data and deep learning based image recognition.Deep learning can improve the precision of models by enhancing the quality of sample images and highlighting the characteristics.However,when building the dataset,it's necessary to preprocess many image data.Parallel technology can be used in the preprocessing to avoid plenty processing time and speed up the process.Image processing algorithm is very complicated when applied in a project,because of the volume and complexity of data.So,parallel technology is confronted with many challenges in the fields of image processing.The first part of this thesis is preprocessing the waves data to improve the recognition precision by highlighting the characteristics of the samples.Then,preprocessing algorithm,which is suitable for deep learning,is built by modifying serial algorithm.By means of parallel domain consolidation and load balancing performance optimization,this algorithm can increase the processing efficiency and reduce the calculating time.As can be seen from the comparative observation of the monitoring of the Yangshan Port wave video,shows that the waveform is with periodic trends.The image is not distinct in the videos.In order to analyze the characteristics of waves,preprocessing is used to improve the accuracy rate of wave recognition and reduce the time.In preprocessing,the first step is to extract the keyframes of the videos.Then,mean filter is applied in the keyframes to denoise the extracted images.Finally,in order to highlight the texture characteristics of the waves,then the weighted mean filtered image is refined.However,there is a large amount of calculation in the process of wave image preprocessing,therefore,the application of parallel computing in wave image preprocessing has the significance of increasing speed and shortening time.By analyzing the serial code of the wave image preprocessing,with limited hardware resources,select multi-core CPU architecture,with the OpenMP parallel programming model as the framework,parallel algorithm for image preprocessing of ocean wave sample data sets suitable for deep learning is proposed.By means of parallel domain consolidation and load balancing performance optimization,this algorithm can increase the processing efficiency and reduce the calculating time.Its main contents include:1.Introducing the parallel architecture and applying the Multi-core parallel computing model in wave image processing.OpenMP,MPI,TBB will be included in this thesis.Result shows that the data volume is enormous and should be increased when the volume is not enough by analyzing the features of data set for deep learning.It can improve the accuracy of recognition,image quality and enhance the images.2.Wave data is acquired from Yangshan port experimental base of Shanghai Ocean University.After extracting the features of images by preprocessing,raw data and the preprocessed data are both applied in Wave-CNN to contrast their model performance.Then,preprocessed data is dealt with 3 deep learning algorithms,including Wave-CNN,IQA-CNN and Typ-CNN.After comparison,result shows that preprocess can effectively enhance the accuracy of recognition and model performance.3.After extracting the keyframes and weighted mean filter denoising,preprocessing is simulated by OpenMP in multi-core computer.By comparing the parallel algorithm and the serial algorithm,result shows that parallel algorithm can obviously increase the calculation speed and multi-core utilization.Besides,this algorithm has excellent expansibility.Then,experiments of OpenMP,MPI and TBB are conducted and show that OpenMP has more advantages over MPI and TBB in image preprocess.4.By analyzing the factors,which will affect the performance of OpenMP,Program optimization method is proposed,including merge of parallel domains and dynamic load balancing.After comparing the results of raw data and preprocessed data,the result shows this optimization method can achieve great effects when applied in projects.