Research On Eyeglasses Removal Method In Face Recognition

In recent years,researchers have mostly focused on suppressing or eliminating errors caused by occlusion rather than removal to achieve better face recognition.As the most common occlusion of the face,it is of great significance to study how to remove the eyeglasses from the face image automatically.Eyeglasses removal refers to the process of automatically inferring the information of the occluded area from the face images with eyeglasses and reconstructing the eyeglasses-free face images.By removing the eyeglasses in the face image,we can reduce the interference caused by the occlusion and improve the accuracy of face recognition.Existing eyeglasses removal methods can be divided into two types.One is the traditional method,which is based on the PCA theory.The image reconstructed by this method has poor quality,with obvious eyeglasses marks and low picture clarity.The other one is based on convolution neural network,which reconstructs the face image with fewer eyeglasses and higher definition but not to the extreme.Therefore,this paper proposes a new eyeglasses removal method based on the Feature Sliced Deconvolution Neural Network,which achieves a better result.The proposed method can automatically learn to infer the characteristics of the occluded area through the end-to-end training method,and output the reconstructed eyeglasses-free face image.Specifically,the main work and innovations of this paper are as follows:(1)Considering the wide variety of eyeglasses in practical application,we divide the eyeglasses into three types: full-rim,half-rim and rimless.Eyeglasses-free face images were collected as the label and then manually superimposed with different eyeglasses to get the input data.A large-scale eyeglasses removal data set is established for the network training and testing.(2)A new eyeglasses removal method based on the Feature Sliced Convolution Neural Network is proposed.First,the operation of feature slice is used to reduce the computation and the possibility of noise introduction.Second,the Dual-Branch Convolution Pool Module is designed to improve the feature extraction ability of the network.Third,the Feature Sliced Convolution Neural Network(FSCNN)is constructed by combining the above with bilinear interpolation and convolution.Fourth,the learning strategy and the solution algorithm adopt the square loss function and the stochastic gradient descent method respectively.The qualitative and quantitative analysis show that the proposed method can effectively remove the eyeglasses in the image and obtain better results,but the transition of the upper,middle and lower parts of the output image is not natural enough.(3)In order to solve the problem that the reconstructed image is not natural enough,the bilinear interpolation in FSCNN is modified to deconvolution operation,the maximum pooling operation in the front of FSCNN is cancelled,maximum pooling and average pooling are added to the front of the Dual-Branch Convolution Pool Module instead,and a new Feature Sliced Deconvolution Neural Network(FSDNN)is constructed.The qualitative and quantitative analysis shows that the eyeglasses removal method based on FSDNN achieves better results,which can be applied to different types of eyeglasses and different face postures.The output images of FSDNN have almost no residual eyeglasses trace,and is naturally clear.(4)Furthermore,the effect of the eyeglasses removal method based on FSDNN on face recognition is analyzed.The experimental results show that the proposed method is efficient and lightweight,and can effectively improve the similarity of facial images.By applying the FSDNN to face verification and closed-set face identification,the classification ability of face recognition algorithm on face images with eyeglasses has been significantly improved.