Low and variable frame rate face tracking using an IP PTZ camera

Object tracking with pan-tilt-zoom (PTZ) cameras has various applications in different computer vision topics such as video surveillance, traffic monitoring, people monitoring and face recognition. In our work, we want to cope with the problem of large inter-frame motion of targets, low usable frame rate, background changes, and tracking with various scale changes. In addition, the tracking algorithm should handle the camera response time and zooming.;For initialization, when the camera is stationary, motion detection for a static camera is used to detect the initial location of the person face entering an area. For motion detection in the FOV of the camera, a background subtraction method is applied. Then to remove false positives, Bayesian skin classifier is applied on the detected motion region to discriminate skin regions from non skin regions. Face detection based on Viola and Jones face detector can be performed on the detected skin regions independently of their face size and position within the image.;After face detection, the tracking step is started. We propose an adaptive particle filter using optical flow based sampling (APF-OFS) method adapted to the general object tracking problem with an IP PTZ camera. Optical flow is used to extract moving pixels is combined with particle filter that has robustness to non-Gaussian distribution of target movements to extract random motion of the object. Target modeling and tracking are done based on sampling around predicted positions obtained by a position predictor and moving regions are detected by optical flow. The scoring of sample features is done with some reasonable normalization functions. The normalization functions are used to combine different measure values to standardize their magnitudes within similar ranges. Normalization functions are applied to geometric and appearance features.;In the tracking mode, once a face is detected in the first image frame, the pan and tilt of the PTZ camera is controlled to bring the face back in the image center. Depending on the speed of the person motion, the face might be near the image center; so the search area for the next frame does not need to be the whole image and should be an area around it. For processing, because of camera motion, background subtraction is not effective to remove the false positive rate. In this case, the combination of an optical flow method and particle filter tracking can solve the tracking problem. While the camera is tracking the face, it starts to zoom on the face to take a photo of that person. Generally, the camera covers a wide FOV with low resolution so in order to recognize that person, it is necessary to take an image with sufficient resolution.;Our solution consists of a system initialization phase which is the processing before camera motion and a tracker based on an Adaptive Particle Filter using Optical Flow based Sampling (APF-OFS) tracker, and camera control that are the processing after the motion of the camera. Each part requires different strategies.;The general contribution is dynamic face tracking with an IP PTZ camera, a first in the field of computer vision. The specific contributions are (1) modeling and formulating the tracking program as a servo control loop that compensates all the delays resulting from network or processing; and (2) proposing of an adaptive particle filter with optical flow samples method to cover all possible candidate regions, handle large inter-frame motion of the target, low tracking frame rate and recover the tracking target in the case of occlusion or target lost.;Results show that our algorithm can handle and overcome large motion between two consecutive frames, and the detected target location is near to the ground truth. In addition the camera can center on the target with a good precision. The target usually is located in a constant distance within 1/6 th of image diameter from the image center. (Abstract shortened by UMI.)...