A System-on-Chip Design Based On RISC-V And CNN Coprocessor

In recent years,with the significant improvement of computer performance,Convolutional Neural Networks(CNN)has been widely used in many fields,such as computer vision,speech recognition,and image processing.Its application scenario has developed from server cloud to embedded terminal.However,the large number of parameters and huge calculation of the convolutional neural network are difficult to apply to embedded terminals.The CNN acceleration circuit for the embedded terminal has become a hot issue in research,the combination of general-purpose CPU and CNN accelerator has become an efficient solution.To this end,this paper presents a system-on-chip design based on RISC-V open source processor and CNN coprocessor to provide support for deploy CNN algorithms in embedded terminals.The main work of this paper are as follows:(1)Research on the sparsity of CNN convolution calculations to pruning and quantifying the model,and selecting suitable data storage formats based on the characteristics of the feature map and weight parameters,reducing the storage of parameters and the time consumption during transmission in the system.(2)Design a CNN coprocessor that supports convolutional sparse calculations,use the network's dynamic sparse and static sparse characteristics to speed up the calculation process,skip the feature map dynamic sparse to accelerate the convolution calculation,and gating the weights static sparse multiplication calculations to reduce power consumption.Design a dynamic task distribution mechanism and reconfigurable computing unit array to solve the unbalanced computing unit load caused by the sparse irregularity of the feature map,Which improves the utilization rate of the computing unit.(3)Based on the flexibility of the RISC-V instruction set architecture,an efficient extended instruction is designed to configure the coprocessor for collaborative work.The extension instructions are added to the compiler,and build a software and hardware collaborative verification platform to analyzed and verified the system,obtain the trend of storage compression rate and computing performance by different sparsity of the feature maps and weight parameter.The compressed Lenet-5 network is deployed in the system-on-chip to compare with the CPU.Experimental results show that the system-on-chip implemented in this paper can compress the parameter storage space,reduce the time and energy loss caused by data transmission,and have a good acceleration effect on sparse neural networks.The energy consumption ratio is about 20.1 times than that of CPU and 21.7 times than that of GPU.It basically meets the needs of deploying convolutional neural networks in embedded application scenarios.