Co-design And Implementation Of Hardware/Software Of Convolutional Neural Network Based On FPGA

Since the development of artificial intelligence,the design of deep learning algorithms(especially convolutional neural network,CNN)based on embedded platforms with limited resources,power and cost has always been a hot research field.As a high-performance artificial intelligence algorithm,CNN can be widely used in many fields such as face recognition,pedestrian detection and fault diagnosis.Its deployment in embedded platforms accelerates the landing of these applications,frees them from the shackles of servers,truly serves the mobile and edge markets and has high practical value.The deployment of CNN in embedded platforms not only involves hardware design but also software design.At present,most researchers mainly focus on the design of CNN hardware accelerator based on field programmable logic gate array(FPGA).From a system-level perspective,there are few studies on the software/hardware co-design of FPGA-based convolutional neural networks.Besides,if you are too obsessed with the pursuit of a high-performance hardware CNN accelerator and ignore the performance of the limited software,it is likely to cause an imbalance in the processing time of the system's software/hardware.In turn,the benefits of hardware CNN acceleration will be reduced and the waste of hardware resources will be increased.In order to solve the above problems,this paper proposes an embedded system for CNN.First of all,in view of the contradiction between the limited resources of the embedded platform and the large amount of calculation in CNN,combined with the structural similarity of CNN,an FPGA-based CNN hardware reusing strategy is proposed.This method deploys the CNN model on an embedded platform with low FPGA hardware resources.Then,based on hardware reusing,a solution of software/hardware execution time balancing is proposed.This method integrates the execution time of system's software/hardware into the CNN's hardware design and achieves the goal of balancing the execution time of system's software/hardware by optimizing the parallelism of system's software/hardware.Finally,a software/hardware co-design flow is proposed to design and implement an embedded system for CNN.This flow uses modularity to abstract the system's architecture and combines the characteristics of each module to divide the system's software/hardware.Then uses the proposed hardware reusing and software/hardware execution time balancing solution to carry out the software/hardware co-design for the divided modules.Besides,combined with effective software/hardware data interaction and control protocols,the system saves hardware resources and improves the benefits of CNN hardware acceleration on the premise of ensuring accuracy and speed.The experimental results show that the CNN-oriented embedded system designed in this paper achieves a close to 1:1 software/hardware execution time ratio under the premise of ensuring accuracy and speed,while the power consumption is only about2.4W.Compared with the work in the baseline,the system's equipment utilization has reached 95% or even 99%,an increasing of 28%?46%,and the energy efficiency has been increased from less than 10 frames/W to more than 10 frames/W.More importantly,the hardware reusing strategy saves valuable FPGA resources in the system.The software/hardware execution time balancing strategy improves the utilization of the software/hardware equipment in the system and greatly improves the revenue of CNN hardware acceleration.