Image Signal Processor (ISP)-Convolutional Neural Network (VGG16) Joint Optimization And Key Module Design

In the 1990 s,CMOS(Complementary Metal-Oxide Semiconductor)image sensor digital camera came out,and it gradually became the mainstream of the market with its advantages of high sensitivity and large spectral response range.Traditional CMOS image sensors often have the problem of insufficient dynamic range,such as overexposure and underexposure.In addition,the output images often have relatively high noise,which may not meet the output requirements of high-quality images of display devices.These problems need to be solved by using ISP(Image Signal Processor).In the digital image system processed by ISP,some images need to be output to high-definition interfaces such as monitors,mobile phone and computer displays and other terminal devices,and some are used in the field of image recognition and target positioning,such as intelligent robots and military target positioning.Among the recognition algorithms used in ISP output images,CNN(Convolutional Neural Networks)is the most widely used.This algorithm has high recognition accuracy,precise positioning,and high real-time performance.Therefore,this algorithm has become the preferred method of image classification processing.Traditional ISPs need to perform noise reduction,detail enhancement,and visibility enhancement on images collected by image sensors,but they often do not require too much detail processing and restoration in the field of computer vision.This paper fully excavated the important ISP steps and found that demosaicing and gamma transformation have a key impact on the target recognition result.Analyze the principle of the two-step algorithm,use the line buffer method to design the hardware circuit of the demosaicing algorithm,and use the highefficiency logarithmic floating point calculation system to realize the arbitrary conversion method of the gamma value,which is better than the traditional method in terms of area and other performance.Therefore,it completes the high-efficiency ISP hardware circuit design for computer vision.This article selects VGG(Visual Geometry Group)network as the deep learning target detection scheme,uses GPU(Graphics Processing Unit)training parameters and exports,and completes the hardware construction of the network prediction part on the ZYNQ platform.Considering the on-chip BRAM resources and data reading bandwidth,a data block structure is designed to store feature data and parameters,and 4 channels of data are calculated in parallel at the same time.The pipeline method is used to complete the multiplication and addition operation,thereby realizing the convolutional layer and full connection layer of the network,pooling layer and Softmax layer are respectively proposed corresponding hardware optimization methods according to the characteristics of the algorithm.Finally,an ISPConvolutional Neural Network joint system is designed.In this paper,the designed ISP algorithm is simulated and analyzed on the MATLAB platform,and then the Verilog language is used to build the hardware module.Tools such as DC(Design Complier)and Vivado are used to synthesize and analyze the performance parameters of the designed hardware circuit.Analyze the neural network algorithm,design the hardware modules suitable for FPGA(Field Programmable Gate Array)deployment,connect each module on the ARM side and use the AXI interface to complete the transmission of image data and various parameters.It can be found that the joint system obtains a relatively ideal recognition rate,and at the same time,the efficiency is significantly improved compared with the traditional method.