Improvement Of Face Recognition Methods In Surveillance Videos With Deep Learning

Intelligent surveillance systems have been widely deployed for the need of protecting human self-safety and property security in both public and private venues in recent years.Many key technologies are integrated into an intelligent surveillance system and face recognition technology is one of the most important part.Most face recognition algorithms perform well in constrained scenes,but they tend to suffer a lot in real-world surveillance videos due to the variations in pose,illumination or facial expression.This thesis does some work to improve the performance of the face recognition methods in surveillance videos with deep learning.The main contents and contributions are as follows:1.To solve the problem that low quality face images decrease the accuracy of recognition algorithms,a face image quality assessment method based multi-scale convolutional neural network is proposed.The accuracy will be increased by picking out the high quality face images used for recognition.There are two points in our method.First,a convolutional neural network is utilized to predict the face image quality.As we all know.convolutional neural network is good at feature extracting.By taking full advantage of that,a quality score can be got which is the comprehensive evaluation of image clarity,illumination,pose,facial expression and so on.Second,the groundtruth of a face image quality score is labeled automatically by the recognition algorithm.Specifically,the similarity of the query image and the gallery is considered to be the groundtruth of the query face image.The similarity is measured by the consine distance between the two images' features extracted from the VGGFace model.The experimental result from COLOR FERET and Multi-PIE dataset shows that the proposed method can give an effective assessment when face images quality changes.By adding the image quality assessment before face recognition,we find that our image quality metric network well distinguish good images from bad ones and the accuracy is increased from 64.67%to 81%on the dataset collected from real-world surveillance videos by ourselves.After that,the generality of the network is verified on the Choke-Point dataset.2.About the problem that there is not enough labeled face images when fine-tuning the deep model in real-world surveillance videos to improve the performance of the model,a method of collecting and labeling a reasonable face dataset from surveillance videos automatically and incrementally.In detail,firstly,face detection and tracking is used to generate a rough dataset.Secondly,the rough dataset will be purified by the graph-clustering method based on VGGFace feature and the duplicated classed will be cleaned by measuring the similarity of the center images.Then we get the dataset from target surveillance videos while requiring no or little help of human annotation.The purity of the dataset is 97.4%.The VGGface model is fine-tuned with the dataset.The experiment shows that the network after fine-tuning achieves recognition accuracy of 93.5%.which obviously outperforms the network without fine-tuning,which returns a recognition accuracy of 83.6%.3.To validate the proposed methods,a video surveillance system for face recognition is designed and constructed.