Face recognition: Face in video, age invariance, and facial marks

Automatic face recognition has been extensively studied over the past decades in various domains (e.g., 2D, 3D, and video) resulting in a dramatic improvement. However, face recognition performance severely degrades under pose, lighting and expression variations, occlusion, and aging. Pose and lighting variations along with low image resolutions are major sources of degradation of face recognition performance in surveillance video.;We propose a video-based face recognition framework using 3D face modeling and Pan-Tilt-Zoom (PTZ) cameras to overcome the pose/lighting variations and low resolution problems. We propose a 3D aging modeling technique and show how it can be used to compensate for age variations to improve face recognition performance. The aging modeling technique adapts view invariant 3D face models to the given 2D face aging database. We also propose an automatic facial mark detection method and a fusion scheme that combines the facial mark matching with a commercial face recognition matcher. The proposed approach can be used (i) as an indexing scheme for a face image retrieval system and (ii) to augment global facial features to improve the recognition performance.;Experimental results show (i) high recognition accuracy (>99%) on a large scale video data (>200 subjects), (ii) ∼10% improvement in recognition accuracy using the proposed aging model, and (iii) ∼0.94% improvement in the recognition accuracy by utilizing facial marks.