Researches On Key Technologies Of Panoramic-PTZ Camera System

Recently, a wide attention has been paid to panoramic cameras due to their abilities of monitoring an area covering 180° or 360° field-of-view. However, unlike a PTZ(pan-tilt-zoom) camera which can focus on an object rapidly, panoramic cameras usually have low resolution. This has significantly limited their promotions and applications. The panoramic-PTZ camera system combines the panoramic cameras and PTZ cameras together, which has the potential to achieve both global wide field-of-view information and local high resolution object information acquiring. Moreover, by integrating the computer vision technologies, the panoramic-PTZ camera system’s enormous potential might get excavated. This dissertation studies some key technologies of the panoramic-PTZ camera system, including panoramic imaging, joint calibration, pedestrian detection and tracking technologies.A distortion correction method of fish-eye lens is proposed which is based on the Bayer image signal. Firstly, a distortion correction method that focuses on vertical straight lines and processing delay, named the VFP projection, is proposed. Second, according to the correlation among color channels of the Bayer image, a novel Hermite interpolation method appropriate for Bayer image signal is proposed. It can make use of different color channel’s information for interpolation. Experiments show that the proposed interpolation method has obvious visual superiorities and higher PSNR compared with linear interpolation and cubic interpolation methods. Lastly, a prototype system of fish-eye-lens-based imaging is established. The real-time field-programmable gate array(FPGA) implementation of the proposed method is demonstrated. Experiments demonstrate that the proposed distortion correction is not only characteristic of real-time processing and the smaller computation amount, but also is applicable to embedded hardware.An efficient and accurate joint calibration method is proposed to achieve the calibration between the panoramic and PTZ cameras. The main purpose of the calibration is to obtain the mapping relation between two cameras. Often the mapping relation is assumed to be a pure rotation. However, such an assumption is rather ideal that cannot be applied in practice. Thus, a local rotation based model is proposed. The image feature point matching is adopted to solve the local rotation mapping model. The PTZ images are projected to the panoramic image plane for achieving better matching results. A sliding-window based RANSAC method is proposed to detect the outlier matching. The joint calibration method is experimentally analyzed in four scenes. The evaluation is based on the reference object’s position in PTZ camera’s 20× zoom(2.8° field-of-view) images. Results show that at least 95% of the referenced objects can be within the PTZ camera’s 20× zoom images. The accuracy meets the demand of most applications. The calibration process is full automatic which does not require any additional equipment, e.g., a chess board.Methods of pedestrian detection and tracking for the panoramic-PTZ camera system are studied. First a framework for the pedestrian detection and tracking is proposed. Second, based on this framework, a pedestrian detection algorithm based on scale invariant integral channel feature and resampling-embedded cascade detection is proposed. The evaluation based the INRIA pedestrian detection dataset shows that proposed detector can obtain the state of the art accuracy. Next, a tracking-by-detection algorithm based on the particle filter is proposed. In order to handle abrupt motion caused by the rapid moving of the PTZ camera and/or the focal length decreasing or increasing, a feature-driven motion model is proposed in the particle filter based tracing-by-detection algorithm. Based the framework and proposed algorithms, a panoramic-PTZ camera system for pedestrian detection and tracking is established. The experimental system can detect and tracking moving pedestrian at distance up to 100 meters, with a 180° field-of-view. The experimental results show the tracking-by-detection algorithm can improve the pedestrian detection accuracy, can obtain stable pedestrian positions. Moreover, the feature-driven motion model can handle the abrupt motion efficiently.