From few to many: Generative appearance-based models for face recognition

A generative appearance-based method is presented for recognizing human faces under variations in lighting and pose. It exploits the fact that the set of images of an object in fixed pose, but under all possible illumination conditions, is a convex cone in the space of images. Using a small number of training images of each face taken with different lighting directions, the shape and reflectance properties of the face are reconstructed. A reconstruction is performed both under the Lambertian reflectance assumption as well as with a more general reflectance function. In turn, this reconstruction serves as a generative model used to render—or synthesize—images of the face under novel illumination conditions, accurately predicting both shading and shadows, and thus synthetically forming the illumination cone. The pose space is then sampled, and for each pose its corresponding illumination cone is approximated by a low-dimensional linear subspace whose basis vectors are estimated using the generative model. The presented recognition algorithm assigns to a test image the identity of the closest approximated illumination cone (based on Euclidean distance within the image space). We test the algorithm on 4050 images from the Yale Face Database B; these images contain 405 viewing conditions (9 poses x 45 illumination conditions) for 10 individuals. The method performs almost without error, except on the most extreme lighting directions, and significantly outperforms popular recognition methods that do not use a generative model. An improvement in recognition results is noted when the face representations are created using reconstructions from an extended reconstruction algorithm that incorporates a non-Lambertian reflectance model.