Ear Detection And Recognition Under Uncontrolled Conditions Based On Deep Learning Algorithm

Ear based human recognition technology is one of the most common research field in biometric identification.Compared with other biometric identifiers,the ear has its distinctive advantages.The ear has a stable and rich structure that changes little with age and does not suffer from facial expressions.Moreover,the collection of ear images is deemed to be easy and non-intrusive.A robust human ear recognition system has a great application potential in various scenarios,such as entrance guard management,smart phone applications,law enforcement,criminal investigation and so on.After years of research,the human ear recognition technology has made remarkable progress.Existing algorithms had obtained satisfactory results on some well known ear databases which were photographed under controlled conditions.Meanwhile,researchers noticed that most of those approaches performed poorly when the test images were photographed under uncontrolled conditions.Nevertheless,occlusions,pose and illumination variations are very common in practical scenario.So ear detection and recognition under uncontrolled conditions is a challenging problem that must be addressed.In the last few years,deep learning algorithms have significantly advanced the performance of state-of-the-art in computer vision.Compared with face recognition and fingerprint recognition,the ear recognition research developed slowly without deep learning algorithm.The key points of applying deep learning algorithm to ear recognition field are sufficient ear training data and appropriate network architectures which are specifically appropriate for ear recognition tasks.In regard of the problems above,the work involved in this paper is as follows:1)Two ear databases which were photographed under uncontrolled conditions were proposed for the training and testing of ear detection and recognition algorithms.One of the databases called USTB-WebEar database was collected from the Internet.The images in this database with complicated backgrounds,occlusions,pose and illumination variations can be utilized for ear detection.The ear images in the other proposed database named USTB-Helloear were also photographed under uncontrolled conditions.Those ear images can be utilized to train a deep learning model of ear recognition;moreover,the proposed database,along with pair-matching tests,provides a benchmark to evaluate the performances of ear recognition and verification systems.2)This paper proposes an efficient technique involving Multiple Scale Faster Region-based Convolutional Neural Networks to detect ears from 2D profile images in natural images automatically.The threshold value part was removed from the original Faster R-CNN approach,and an ear region filtering(ERF)module was proposed.The ERF approach is utilized to distinguish the correct human ear from ear shaped objects based on the information of ear location context.The experimental result shows that the improved ear detection algorithm outperforms other ear detection algorithms.3)For the characteristics of the ear recognition task,some deep models were fine-tuned and modified on the proposed database through the ear verification and recognition experiments.Firstly,the last pooling layers were replaced by SPP layers to fit arbitrary data size and obtain multi-level features.In the training phase.the CNNs were trained both under the supervision of the softmax loss and center loss to obtain more compact and discriminative features.Finally,three CNNs with different scale of ear images were assembled.Experimental results on the USTB-Helloear,AWE and CVLED ear databases demonstrated that the VGG-Ear deep model obtained more robust and discriminative capacity of ear representations than traditional fine-tuned deep models.4)An ear key-point detection algorithm based on cascade CNN was proposed to improve the performance of the proposed approach on ear images with large pose variations and major occlusions.Then a procedure of ear data normalization is proposed to unify the ear pose and location by detecting 6 key-points on the ear.The experimental results on the three ear databases demonstrate that the procedure of ear data normalization can enhance the ear representations of CNN model.In this paper,firstly,two ear databases were proposed,and the training and testing standards were established.Then the ear detection algorithm,ear key-point detection algorithm,ear recognition and verification algorithm was proposed sequentially.A complete novel end-to-end ear recognition system based on deep learning algorithm was established in this paper.The research in this dissertation is of great significance in addressing the ear recognition under uncontrolled condition problem,and it is also valuable for other biometrics research in similar cases.The identification method proposed in this research has theoretical significance in solving practical applications,such as secret security.mobile applications and the police cracking criminal cases.