The Research On Acceleration Systems Of Deep Belief Networks Based On FPGAs

Recent years,thanks to the concept of deep learning and the improvement of computing power,deep learning has shown significant scientific values and practical values,favored in academic and commercial communities.Deep learning,a fastly growing study field,is derived from the study of artificial neural networks.The motivation of deep learning is to build or model the human brain neural networks,for the ease of interpreting images,voice,and text data.The deep belief network,a primary deep neural network,is composed of multiple restricted Boltzmann machines.Deep belief networks have been widely employed in many fields,such as speech recognition,image recognition,and text classification.As layers among deep belief networks are fully connected,with the scale of nets increasing,the number of parameters and computing costs increase exponentially.How to process mass data quickly and effectively is an attractive research point of deep belief networks.There have been sufficient research on accelerations by multi-core processor clusters,cloud computing platforms,and general purpose graphic processing units.However,these platforms still exits remarkable drawbacks,i.e.low computing efficiency and high energy consumptions.The FPGA,field programmable gate array,is a popular means to speed up deep learning algorithms.Advantages of deep learning accelerators based on FPGAs are high performance,low power consumption,and reconfigurability.In this dissertation,we start from the inference process of deep belief networks and design an FPGA-based accelerating system,in order to study how to improve the computing performance of fully-connected neural network structures.Main contents of the dissertation includes:1.We analyze the algorithm of deep belief network inference process,which is a feed-forward process.Then we find the parallel in single-layer computing and multiple-layer computing.According to the computing and storage resources of FPGAs,we design the basic process elements for the feed-forward process and realize the single-FPGA accelerating system.All elements are pipeline designed to realize high throughput.2.We extend the accelerator to a multi-FPGA accelerating system.In order to distribute the inference computing to multiple FPGAs,we try to divide the deep belief network by layers to realize an inter-layer pipeline computing.We also try to divide the network by chunks to realize an intra-layer parallel computing.Both ways bring further performance improvements.3.For the performance of multi-FPGA accelerating systems,we extract key factors of the system performance and propose the performance model.Based on the performance model,we analyze different application scenes of different partition schemes.To verify the performance of system and the performance model,we compare the performance,power,energy consumption,and energy efficiency with CPU,GPU implementations,and a traditional FPGA accelerator.Experiment results indicate that the accelerating systems proposed in this dissertation have pretty high acceleration performance with]ow power and energy,implying the advantage of high energy efficiency.