Design And Implementation Of Fast Super Resolution Convolutional Neural Network (FSRCNN) Based On FPGA

In real life,we often need to process blurred pictures,whether it is caused from camera shaking,noise interference or low resolution from itself and other reasons,we need an effective method to turn the fuzzy images into clear ones.Recently,more and more convolutional neural network models have been proposed,which can realize the function of super-resolution image reconstruction quickly and efficiently.After previous literature research,we found that the FSRCNN(Fast Super-Resolution Convolutional Neural Network)model has a fine performance.Meanwhile,it has less convolutional layers compared with others,containing operations such as convolution,deconvolution,activation function,padding and bias adding.Then,we collected the open-source code and its parameters obtained by training.Lastly,we designed and implemented this CNN(Convolutional Neural Network)model consulting related literatures about realizing CNNs based on FPGAs(Field Programmable Gate Array).Aimed at FPGA structure,this paper studies how to implement the convolutional neural network model in the hardware and presents a FPGA hardware implementation method of FSRCNN.The main work contents are as follows:Implemented the whole FSRCNN network structure with hardware to reduce writing to and loading from the off-chip memory.Considering the FPGA resource limitation,it is impossible to realize each convolutional layer in parallel.This paper further divides the operations in the implementation process of each layer.If there is a multi-channel operation,the filter direction and channel direction are also split and pipelined.The specific splitting method is determined according to the relationship between the input data form of this layer and output data form of the former layer.Turned the original floating-point numbers into fixed-point numbers,the precision loss after fixed is less than 1.4%.Reused the same calculate module from the third layer to the sixth layer since they have similar structures,which saves a few computing resources without adding additional delay time.Designed the loading way of kernel parameters,saves the time needed for data preparation before convolution calculation.Carried out hardware design and implementation of the process of acquiring convolutional windows from a feature map,realizing sliding windows and padding based on layer numbers at the same time.Optimized the algorithm of deconvolution layer.The repeated access and accumulation of intermediate results is avoided thus improved the calculation efficiency.In general,the FPGA hardware architecture of the FSRCNN model proposed in this paper is mainly based on Zynq7035 FPGA.The structure design and implementation of the entire network are done in the hardware.The input image of360*202 was used for verification,which was scaled to a 1080*606 high-resolution picture.The signal noise ratio was 25.94 d B,which only decreased by about 0.21%compared with the floating-point result.Our results show that the design realized the function of FSRCNN model under the limited hardware resources.The system's operating frequency is 12.5 MHz,the number of computing cycles is about 4192.7 K,and the throughput is 5.2 GOPS(Giga Operations Per Second).