| With the rise and success of big data and deep learning,Artificial Intelligence(AI)has made revolutionary breakthroughs in vision,speech,language understanding,and many other fields.Data,algorithms and computing power are the three key elements of AI’s great success.The data scale and algorithm scale of upper-layer applications are growing continuously,making the demand for computing power increase exponentially.However,the demise of Moore’s law and Dennard scaling has led to the end of rapid performance improvement in general-purpose processors.Therefore,AI applications are facing a huge gap between the demand and supply of hardware computing power.High performance AI computing is an active research field in academia and also an urgent demand for AI applications in the industry.The algorithm-hardware co-design method can reduce the demand for computing power on the algorithm side and improve the performance on the hardware side,which is of great significance for high performance AI computing.This thesis analyzes the characteristics of AI domain and the contradiction between algorithm design and hardware design,and proposes an algorithm-hardware co-design method for high performance AI computing,mainly including hardware-oriented algorithm optimization and algorithmoriented hardware customization.This thesis selects four specific research topics to conduct algorithm-hardware co-design from two aspects: the explosive growth of data volume in web search and the continuous growth of model size and complexity in deep learning.Specifically,the main research contents of this thesis are as follows:First of all,this thesis proposes an FPGA-based hardware accelerator system(Flex Saa S)for the selection service of web search.Search engines deploy large-scale selection services on CPUs to retrieve massive web index data and identify a set of web pages that match user queries.The selection service is both data-intensive and compute-intensive,so it is challenging to provide a low latency,high throughput,and low energy consumption solution.Flex Saa S conducts algorithm optimization and hardware customization for the matching computation and the index reading,which are the key components of the selection service.Specifically,Flex Saa S reduces the amount of matching computation and index reading based on pruning on the algorithm side,and customizes a pipelined matching processor and a high-speed index stream reader on the hardware side.Evaluated on the real index data and query logs of Bing search,Flex Saa S can significantly reduce the average latency and tail latency and improve the throughput.Second,this thesis proposes Bank-Balanced Sparsity(BBS),a new sparsity pattern for DNNs that can maintain model accuracy at a high sparsity level while still enable high speedup on hardware.In recent years,the size and computational cost of DNNs continue to grow exponentially to achieve better model quality.Model sparsification can compress the model size and computational cost,but the irregular computation and memory access caused by unstructured sparsity are not friendly to parallel computing hardware.Structured sparsity is hardware friendly,but suffers from deteriorated model accuracy or sparsity ratio.To solve the tradeoff between model effectiveness and hardware efficiency,BBS partitions each weight matrix row into banks with the same size and sparsity ratio for parallel computing,while adopts unstructured sparsity inside each bank to maintain model accuracy.We also implement a highly parallel GPU acceleration library for DNNs represented in BBS to eliminate irregular memory access and computation.Experimental results demonstrate BBS has almost the same effectiveness as unstructured sparsity and outperforms structured sparsity.Compared to other GPU libraries for various sparsity patterns,the GPU implementation for BBS can achieve significant performance improvement.Third,this thesis applies BBS to LSTM models and proposes an FPGA-based accelerator for LSTM models.Neural networks based on LSTM are widely deployed in latency-sensitive language and speech applications.The most time-consuming part of LSTM inference is matrix-vector multiplication(Mx V).As the size of the LSTM network grows,Mx V cost grows quadratically,thus significantly increasing the inference cost.By taking advantage of the intrinsic bank-balanced property in BBS,the FPGA accelerator eliminates load-imbalance,irregular memory accesses and decoding overheads.Notably,the FPGA accelerator can achieve low latency and high throughput even for inference with a batch size of 1.Compared to state-of-the-art FPGA accelerators for LSTM with different compression techniques,the proposed FPGA accelerator achieves significant latency reduction.Fourth,this thesis proposes a novel and general acceleration framework(Seer Net)for CNNs by taking advantage of feature map sparsity.The feature maps generated by convolution are highly sparse with many zero values.By skipping the unnecessary computation and memory access involving zero values,the inference cost could be significantly reduced.Seer Net utilizes a highly quantized version of the original network to predict the feature map sparsity in CNNs,and then leverages the predicted feature map sparsity as output sparsity to prune unnecessary calculation and accelerate CNN model inference.This framework is lightweight and general,and avoids the overhead of training additional auxiliary networks.Experimental results demonstrate that Seer Net is sufficient in predicting feature map sparsity accurately,and incurs negligible model accuracy drop.And the CPU implementation of Seer Net can reduce the amount of convolution computation and speed up CNN inference.In summary,this thesis proposes an algorithm-hardware co-design method for high performance AI computing,and uses them as guidelines to conduct high performance computing solutions for representative AI applications and algorithms.It is expected that this thesis can provide a reference for future research,and promote the development of high performance AI computing. |
