Research On Parallel Optimization Technology Of Graph Search And Deep Learning Algorithms For Big Data Processing

With the rapid development of information and technology,the era of big data is coming, the strategic needs also make a great change.Data, just like natural resource and human resource,implied enormous economic value.How to organize and manipulate big data effectively will paly a vital role in social economic development.Data obtained from various complex systems can be integrated into a network through their interaction.So the essence of big data is network.How to get the relationship between data, and process the relationship rapidly is currently becoming a hot research topic.Graph search and deep learning plays an important role in big data processing. Graph search can be used to process data network with query,clustering,matching and other operations,then the data network can be divided into connected subgraph according to their relationship.After that,deep learning alogorithms can be used to extract critical information through feature extraction and classification on the connected subgraph.Conversely,deep learning algorithms can be used to process big data by data mining,feature recognition and classification,thereby,the seemingly isolated data can be gathered into different networks with various characteristic.For instance,protein data gathered into protein network,web data gathered into social network.After that,graph search can be used to collect further information on the network,such as the aggregation parameters,connected subgraph,the largest independent subset and so on.Faced with big data processing,the processing speed has become a critical issue.Therefore,the speed optimization of graph search and deep learning algorithms has become a hot research topic in current.This paper is mainly about accelerating research of graph search and deep learning algorithms on big data processing.Firstly, we study the characteristics of big data in current and investigate related applications.Secondly, we use OpenMP parallel model to optimize graph search on CPU platform.Various measures are taken to optimize the algorithm in parallel,such as program locality principle,reducing synchronization overhead and overhead balancing and so on.Thirdly, taking the irregular memory access and other features of graph search algorithm into consideration,we customized a hardware accelerator on FPGA.This paper proposed a method of pipeline and multi-PE(Processing Element) to implement the algorithm,and the PEs interact with each other by message passing.And focus on the features of hardware pipeline, we proposed a fine-grained optimization approach.Finally, this papper descriped the procedure of DBN algorithm,a kind of deep learning algorithms,and summarize the characteristic of the algorithm.Then we compiled the classification function of DBN into assemble program based on assemble.Finally we descriped a hardware accelerating tool chain for the hardware implement of deep learning algorithms.