Motion Object Tracking By Stereoscopic Vision System Based On Thermal-visiual Cameras

With the development of social economy and the improvement of science, the video monitoringsystem implemented in living communities, stations and terminals is rapidly established. To improvethe living quality, this system must be operated stably and effectively. Therefore, researches on thedetection, tracking and behavior judgment of complex moving objects are actively carried out all overthe world. Based on these researches, excellent intelligent video monitoring system can be built. Andstereo vision has attracted great concern as one of the key points in this system.Most of the existing stereo vision systems are consisted of two visible cameras, which arehomologous sensors. Although it could reduce the impact of illuminate change and shadow, thealgorithm is relatively complex. In addition, it is a huge challenge to detect and track the movingobject in poor visibility. On the other hand, a lot of researchers have made an attempt on movingobject detection with visual-thermal fusion. However, they did not make full use of the advantages ofstereo vision to obtain three-dimensional (3-D) information of the moving object. A stereo monitoringsystem consisted of a thermal infrared camera and a visible camera is able to make full use ofgrayscale and temperature to get the motion information and3-D information of a moving object.Therefore, it contributes to continuous tracking in all weather conditions. In this paper, we focus onthe research of moving object detection and tracking with visual-thermal fusion. The primary workand remarks are as follows:1) The shortcoming of the existing two-dimensional (2-D) thresholding methods based on fourquadrants is analyzed in this paper. And it is verified through a series of testing. Therefore, anovel2-D threshold line segmentation method is proposed. A2-phase strategy is specified todetermine the2-D thresholding line based on entropy. The second threshold point is determinedin the quadrants including the edge pixels and the noise pixels. Thus the attribution of the edgepixels and the noise pixels is refined. The proposed method improves the segmentation results bymaking full use of the ignored pixels. It not only improves the2-D thresholding method, but itcan be easily carried out as well. The experiments on typical images demonstrated that theproposed method achieves very competitive segmentation results in comparison with the existingrepresentative methods.2) When the2-D histogram of an image is extremely uneven in distribution, it can not get successfulsegmentation through2-D thresholding method based on four quadrants even if the ignored pixelsare fully utilized. To solve this problem, a threshoding method using the centre of mass of the2-Dhistogram is proposed to binarize this kind of images. It not only makes use of the mass of each pixel but also fully considers the location information of each pixel. Therefore, it couldeffectively improve the segmentation result, especially for the images in which2-D histogram ispoor in distinguishing the object from the background.3) A strategy takes advantage of frame difference method and background subtraction method isadopted to detect the moving object. Because there are shortcomings when any of them is usedalone in motion detection. And corner matching is implemented to match the moving object indifferent frames. Thus it makes the moving object tracking more accurate.4) Considering the complement effect came from different source images of the same scene, aregistration method of regions of interest is provided to solve the disparity correspondence inmultimodal stereo systems. The method takes advantage of NCC and NMI. It avoids the graydifference caused by the imaging mechanism of thermal image and visible image, and meanwhileit also makes use of the position information of the pixels. The fusion of the object in differentimages reduces the impact of light and shadow, and greatly improves the object detection resultsin visible images. Even if ideal objects could not be detected from the visible images or thethermal images at the same time, this method is available. And the experiments proved itsvalidity.5) According to the feature of multimodal stereo vision, a3-D reconstruction method of point isproposed on the basis of pinhole model. The rotation centers of the pan-tilt devices are used as thereference points in the method. Since the parameters identification is complicated in stereo vision,the procedure is shown to identify the parameters used in this method. A pair of images shot bytwo cameras at the same time and four reference points are required for calibration. Even if therelative position of two cameras is changed, it can still get3-D information of the object with thecalibrated multimodal stereo system. It is appropriate to quickly get the parameters of the stereovision system in field.6) Based on the multimodal stereo vision system consisted of a thermal camera and a visible camera,the3-D information of the feature point of the moving object can be obtained. It can provideposition information in the stereo tracking system. Therefore, the motion information and3-Dinformation of the moving object in all weather conditions can be gotten.