Research On Occluded Facial Image Restoration Based On Deep Learning

Since ancient times,the human face has been the most direct representative of a person 's identity.The highly specificity and easy acquisition of facial images have made face recognition systems widely used in our daily life.However,the currently deployed facial recognition systems still have certain limitations in practical application when processing facial image with external occlusion or self-occlusion,these systems will struggle at extracting the identity information from the input images,thus making their performance degraded drastically.To address the aforementioned issues,an occluded facial image restoration model based on generative adversarial networks using the process of "Identification via generation" is proposed.The proposed model is able to restore facial images with both external occlusion and self-occlusion into unobstructed frontal facial images,and the generated results can effectively retain face identity information.The main research contents of this paper are as follows:1.For external occlusions on facial images,we proposed a two-stage external occlusion restoration model based on partial convolution and contextual attention mechanism.The proposed model utilizes partial convolution to effectively reduces noises introduced by external occlusion in the feature extraction stage and it enables the method to repair external occlusions of arbitrary shapes and sizes.The contextual attention mechanism lets the proposed model to establish semantic links between the unoccluded and occluded regions,allowing the network to effectively utilize prior information in the facial structure.Finally,the experiment results in terms of both image quality and identity preserving performance prove that our proposed method is able to outperform existing occlusion restoration methods.2.For self-occlusion(pose),we proposed a frontal facial image synthesis method based on multi-path encoder structure.The proposed method assigns delicate encoder path for each face organ,which effectively improves quality of the extracted features from these region.Furthermore,both symmetrical and feature point constraint are introduced based on the structure of human face and the very nature of face frontalization.These constraints constructively improves stability of generated images under large poses and the quality of facial structure.The experiment results show that the proposed method outperforms existing methods on both standalone configuration and in combination with the aforementioned external occlusion restoration model.