The role of illumination in face recognition: Evidence for illumination dependent representations using psychophysics and computation

Humans have the ability to identify objects under varying lighting conditions with extraordinary accuracy. The behavioral aspects of this ability were investigated and compared it to the performance of the Illumination Cones (IC) model of Belhumeur and Kriegman (1998). In five experiments, observers learned 10 faces under a small subset of illumination directions. We then tested observers' recognition ability under different illuminations. Across all experiments, recognition performance was found to be dependent on the distance between the trained and tested illumination directions. This effect was modulated by the nature of the trained illumination directions. Generalizations from frontal illuminations were different than generalizations from extreme illuminations. Similarly, the IC model was also sensitive to whether the trained images were near frontal or extreme. Thus, the nature of the images in the training set were found to affect how good an estimate of the complete illumination space is derived by both humans and the model. Beyond this general correspondence, the microstructure of the generalization patterns for both humans and the IC model were remarkably similar, suggesting that the two systems may employ related algorithms.; Lighting dependency was also examined with respect to object recognition using a method similar to one used by Bulthoff and Edelman (1992) in their investigation of viewpoint dependency. They found that object recognition was viewpoint dependent and concluded that an interpolation based model proposed by Poggio and Edelman (1990) best predicted the observed psychophysical behavior. The present studies sought to answer the question of whether illumination dependence could also be described by a computational model that predicted lighting dependence and used a mechanism similar to that of the viewpoint-interpolation model. Over three different experiments, the results indicated that object recognition was lighting dependent, and that, similar to the viewpoint dependency found by Bulthoff and Edelman (1992), the psychophysical behavior could best be described by an interpolation-like model.