Research Of Face And Ear Multimodal Recognition Based On MARS Map

With the continuous progress of the society, information security has attracted a growing number of social concerns, and biometric identification has gained more and more attraction. Plenty of research work on biometrics has shown that monomodal biometrics in practical applications has many limitations in accuracy and robustness. Multimodal biometric identification can integrate a variety of individual features and improve the recognition robustness. Face and ear multimodal fusion recognition has become one of the hottest research topics in biometric identification owing to its friendliness and non-intrusiveness.Benefit from the development of3D data acquisition technology, biometric research has been extended to the field of using3D information. With respect to its2D counterpart,3D identification has improved robustness with variations of illumination and pose, but still has many problems such as sensitive to expression variations, high costs in storage and computing, etc. Furthermore,3D recognition also faces the problem of occlusion and missing data. In actual identification scenarios, the data acquisition is uncontrollable and the acquired biometric data is often partial because of occlusion or self-occlusion. Therefore identification in uncontrolled scenarios is often based on partial data. How to perform partial data based identification is one of the core issues in uncontrolled scenarios.To achieve more robust identifications, this dissertation perform a spherical transformation to convert the3D face and ear data to a2D map, called multimodal face and ear spherical depth map and texture map (MARS Map). MARS Map naturally integrates two modals of human face and ear, contains more complete structural information and texture information, and is helpful to overcome the monomodal problems caused by occlusions, pose variations, facial expressions and other factors. MARS Map eliminates the out-of-plane rotation, and can be used to conduct identification without the preprocessing stage of data alignment. The2D form of information storage can cut down the costs and reduce the computational complexity. The practical applications of identification usally have to use partial data, so this study focused on partial-data-based identification. There are two phases in the whole identification. During the phase of data registration, we tried to collecte a relatively more complete face and ear information by integration of multi-view3D data, and then constructed the MARS Map prototype database that contains structural information and textural information simulta-neously. And during the phase of identification, we utilsed the MARS Map database to conduct local features extraction and multitasking sparse represent-tation based multimodal face and ear recognition.The main innovation of this dissertation are as follows:First, we researched the transformation of how to convert the3D face and ear data from device-centric expression to object-centric expression, and proposed a novel data representation method called MARS Map, thus reduced storage and computation cost and helped to realise identification without data alignment. Second, we studied the method of extraction of pure human face and ear in3D point cloud and the method of registration and integration of non-rigid partial-overlapping multiview data. A face and ear extraction algorithm based on skin color classification and a novel point cloud registration method called BANICP is proposed. Third, to solve the problem of identification in uncontrolled scenarios, a method of Affine-Sift based Multitask Sparse Represent Classification (ASMSRC) is presented. The ASM-SRC method first constructs a sparse representation dictionary and then calculates optimal multi-tasking sparse representation coefficient, finally conductes classify-cation according to the average reconstruction error.The method proposed in this study uses information combined by structural information and textural information of face and ear, and is robust to variations of illumination, pose, occlusion and expression. This method can to a large extend solve the problem of partial-data-based identification in uncontrolled scenarios. This study is meaningful not only for face and ear based identification, but also for biometric recognition research.