Research On Algorithms, Architecture Of Image Processing SoC With CMOS Image Sensor And Network-on-Chip For Complex Multimedia SoC

The mature CMOS process technology leads to the development and growth of CMOS image sensor, however, the image quality gets worse as the semiconductor technology scales down. Thus, the image processing system-on-chip (SoC) for CMOS image sensor has become more and more important. Design of image processing pipeline, the key image processing algorithms and the architecture of image processing SoC are the most critical aspects. However, with the augmentation scale of hardware system, the traditional shared bus based SoC has problems in delay, communication performance bottleneck and design efficiency as, Network-on-chip offers an efficent solution for connecting modules of an ultra-large scale hardware application. NoC emulation using FPGA is widely used for evaluating the performances of the communication, but this experimental approach suffers from scalability issue, mainly due to the resource limitation of FPGA. Multi-FPGA based NoC hardware emulation systems are proposed in this thesis to solve this problem.In this thesis, FPGA-based hardware architecture design for traditional bus shared image processing SoC with CMOS image sensor, as well as multi-FPGA based NoC hardware emulation platform design for the complex multimidea SoC application is investigated. The main works of this thesis are follows:Firstly, we proposed a fast auto exposure control method against the slow process of the conventional fixed-step auto exposure control method for CMOS image sensor. The ratio of the output signal after exposure to the product of gain value and exposure time is a constant under the same illumination. According to this characteristic, we obtained the luminous intensity using the current exposure time, gain and brightness of the object. By using the light look-up table, we got the optimal exposure time and realized the fast auto exposure control method. The proposed method can be applied to CMOS image sensor well when verified on FPGA board. As it turned out, this algorithm can ensure exposure correctly and adjust the exposure time and gain value fast.Moreover, we proposed a CFA demosaicing algorithm which improves on adaptive edge sensitive algorithm and fuzzy assignment algorithm. In order to estimate the direction of edges more accurately, it adds two adjacent pixels of the current pixel as the judgment condition and classifies the edges into strong edges and weak edges. The proposed algorithm adopts fuzzy assignment algorithm to estimate the missing green value at red/blue pixels and assigns different weighting factors according to the results of edge detection. The proposed algorithm can get similar PSNRs without affecting the quality of the image and at the same time reduce the computational cost.Finally, we proposed a multi-FPGA based scalable NoC emulation system in the context of the single FPGA's resource limitation for verification of large scale complex multimidea SoC. The proposed scalable NoC on multi-FPGA based platform from existing 2D mesh NoC using the packet-switching technique. We propose a routing algorithm considering inter-FPGA and intra-FPGA communications based on existing 2D routing algorithms. The effectivenesses of the synthesizable hardware NoC emulation system are that it can be used for any multi-FPGA based NoC with small amount extra resource consumption and can support the unicast and multicast transmission mode.