Ultra-Low Power Smart CMOS Image Sensor Of Sensing-with-Computing Method

In the era of Internet of things(IOT),intelligent visual perception devices will be everywhere to realize the seamless connection between users and the physical world;With the rapid development of Artificial Intelligence(AI)algorithm in recent years,the speed and accuracy of image recognition and processing by visual equipment have been greatly improved.In order to reduce the burden of the data center,the processing function of the visual perception device has been transferred from the cloud to the edge.The designers hope to deploy the intelligent visual algorithm on the terminal device to enable it to process and identify data locally;Furthermore,in order to avoid missing any key events,some devices for intelligent vision applications need to work in normally open mode.The huge energy consumption cost brought by this continuous sensing demand is very unfavorable to most battery driven intelligent recognition devices.Therefore,at present,new requirements are put forward for edge intelligent visual sensing devices,such as low power consumption,high precision,small area,high energy efficiency and flexible and configurable algorithms.Therefore,aiming at the above problems,this paper implements an intelligent visual perception image sensor chip based on a new "SensingComputing" fusion architecture for the energy-efficient integrated circuit in the intelligent visual perception system.The main contents of this paper are as follows:(1)A new "Sensing-Computing" fusion architecture is designed.By moving part of the computing tasks in the neural network to the inside of the sensor and using the analog circuit,we can eliminate the cost of high power consumption and data bandwidth caused by the conversion of analog image into digital image in the analog-to-digital converter.(2)The key circuit units based on direct photocurrent calculation mode are designed.The analog current generated by photoelectric conversion is directly used for calculation;Combined with the active pixel unit of the traditional image sensor,a dual resolution pixel group is further proposed.The calculation of the first layer convolution characteristic graph of binary neural network with configurable convolution kernel size is realized.(3)The intelligent image sensor chip based on the above technology is realized.Taking the dual resolution pixel group as the basic pixel unit,the connection mode and peripheral circuit are designed to form the whole image sensor chip;Two different convolution kernel sliding methods(single core tiling and multi-core tiling)are designed,which can efficiently obtain the whole convolution feature map.The correctness of its function is simulated and verified combined with the algorithm,and finally the streaming verification is realized based on TSMC65 nm CMOS process.(4)Build an intelligent visual perception chip test system.Around the image sensor chip completed by streaming,a complete set of function test and demonstration system is built.The classification accuracy of MNIST dataset is 98.3%,the calculation power consumption is in the order of 2.52μW,and the energy efficiency reaches 10.8 TOPs/w,which is 12 times higher than the calculation work in the existing typical sensors.