ISP Architecture Optimization Under Machine Vision(YOLO)

The increasing progress in deep learning has led to breakthroughs in the development of computational vision.Target recognition has also been widely used in various fields.Research in improving efficiency has seen great developments at both the algorithmic level and the hardware acceleration level.However,less attention has been paid to the cost image acquisition and processing part of the front-end.Image sensors and image signal processing(ISP)systems are generally the most energy-intensive components of all photographic devices,and optimization of this part can also be beneficial in general.And in practice,the accuracy of machine vision recognition algorithms is not closely related to the perceived quality of the images.In view of the above,this paper studies the architecture optimization of ISP from the perspective of computer vision.Through the disassembly and analysis of ISP functions module by module,it studies the influence of different quality images,that is,each module represented on the accuracy of target recognition,so as to realize the optimization of ISP architecture.Based on the research direction,we firstly design a ISP Pipelines that meets the basic principles of all current ISP designs.Secondly,in order to simulate the effect of image processing more objectively,the RAW format images output from the sensor are chosen as the object of image processing in this paper.However,since RAW format datasets are all nonpublic,this paper firstly performs an inverse operation on the dataset that will eventually be used for target recognition to restore it to RAW format.In order to clearly investigate the impact of each module of image processing,this paper takes a separate enable and disable approach for each module to generate the processing results of different schemes,and then test them using the target recognition algorithm.The impact rate of different modules on the target recognition algorithm is inferred by comparing the test results of different scenarios with the original image test results.YOLOv3 algorithm is used in the testing part of this paper.Mean Average Precision(m AP)and Intersection of Union(Io U),the recognition efficiency parameters of YOLOv3,are used to define the recognition effect.This paper also proposes to downsampling,which can judge the accuracy of target recognition when the downscaled resolution is reduced to a fixed level,and to better enhance the scene adaptation.The experimental results show that,as far as the target recognition of the image processing process is concerned,color interpolation and tone mapping have the greatest impact on the recognition rate of the original image.The recognition rate of the original image can reach79.28% with only color interpolation,60% with only tone mapping,and more than 90% with both color interpolation and tone mapping.In terms of down-sampling,the original image can achieve more than 95% of the original image when down-sampled from 8bits to 4bits and before,and more than 85% of the original image when down-sampled to 4bits and before with only color interpolation and tone mapping enabled.