Research On Key Issues Of Finger-Based Biometrics Using Multi-View Multi-Spectral Imaging

Biometrics is the main technology for identifying individuals in various socioeconomic activities today.It uses sensors to collect physiological,behavioral or psychophysiological traits with unique identity information from individuals,and suitable recognition algorithms are adopted to extract identity information from these modality data.Security,accuracy and convenience are the most critical issues for biometric systems.Among the numerous modalities,the finger has rich and stable traits,and finger traits is dexterous,convenient and humanized to collect,and these traits make finger biometric system widely used and researched.However,the current finger traits acquisition mainly uses single-view and single-spectral imaging methods,which makes the acquired traits vulnerable to finger posture variation and inappropriate illumination,resulting in imaging area shift,texture distortion,and poor image quality,which reduce the accuracy.Multi-modal biometric systems often fuse different modalities directly,while ignoring the relevant information among different modalities.To address these problems,we explore high accuracy and high security multi-modal finger biometric technology using multi-view and multi-spectral imaging approaches,and proposes a number of novel imaging techniques and devices,traits reconstruction algorithms,and feature extraction algorithms that significantly increase the recognition accuracy and system security.The main work and contributions include:(1)To address the problems of common vein texture loss and texture distortion caused by finger posture variation in 2D finger vein images,we propose a real-time 3D finger vein reconstruction algorithm using three-view finger vein images,the obtained 3D finger vein model contains both 3D finger shape information and vein texture information.Based on this reconstruction algorithm,we construct and release the largest 3D finger vein dataset,SCUT LFMB-3DPVFV.To improve the robustness of the authentication algorithm to finger postures,we propose a 3D finger vein authentication network based on rotational group representation,where the 3D finger vein model is represented as a rotational group and fed into a network consisting of dynamic graph convolution and multilayer perceptron to aggregate fine-grained features,and transform the rotation problem into a permutation problem,and finally rotation invariant features are extracted using global symmetry operations.The proposed authentication network achieves the best recognition results on several 3D finger vein datasets.(2)It is difficult for algorithms to effectively aware of missing texture and poor texture contrast in a single finger vein image because there is no reference image for comparison.To this end,three multi-view finger vein imaging methods are adopted in this study,and three kinds of multi-view finger vein image-sets are collected for each finger,and we explore the complementary and fusion algorithm using the finger vein image set in order to improve the recognition accuracy.We construct and release the first multi-view finger vein dataset SCUT LFMB-MVFV.To effectively mine the complementary information among different views,we propose a view attention-based multi-view convolutional neural network VWMVCNN,which embeds a view attention module into MVCNN,and dynamically predict the fusion weights of each view feature,thus achieving effective multi-view vein image fusion.Experiments are carried out on three multi-view finger vein datasets,including single view recognition,best quality image recognition and multi-view recognition,to verify the effectiveness of VW-MV CNN.(3)To address the problems of poor contrast and relatively weak identity discrimination of vein images,we introduce skin modality to improve recognition accuracy.We design a multi-view multi-spectral 3D finger acquisition system,which irradiates the finger with blue visible light and near-infrared light,and simultaneously acquires six finger surface skin images and six finger internal vein images by six cameras surrounding the finger.Moreover,a holographic 3D finger model and its reconstruction algorithm are proposed for the first time.Based on this system and reconstruction algorithm,we construct and release the first large-scale 3D finger biometric dataset SCUT LFMB-3DFB,which contains all known biometric traits on the finger,including multi-view multi-spectral 2D finger images and the reconstructed holographic 3D finger model.We also design a comprehensive and scientific benchmark and evaluation protocol to standardize the subject-independent verification task and the subject-independent close-set identification task.Based on this dataset and benchmark,extensive and rigorous experiments and analysis are conducted to verify the effectiveness and research value of the proposed system and multimodal finger features.(4)To address the issue that the correlation information between different modalities is neglected in existing multi-modal biometric systems,we propose a finger disentangled representation learning framework FDRL-Net,which maps each modality into a modalityrelated private feature subspace and a modality-unrelated shared feature subspace,and these two independent subspaces constitute a complete representation of the original data.The complementarity between the disentangled private and shared features,as well as between the private features of the two modalities is enhanced,thus improving the fusion recognition performance;the shared features are modality-invariant features extracted from each modality,and based on this shared features,we attempt cross-modal heterogeneous recognition on fingers for the first time.To solve the problems of simultaneous information flow of two modalities in a single network and inconsistent optimization directions of multiple objective functions in the network,FDRL-Net is designed as a fourstream CNN based multi-task learning network,and a loss grouping and joint optimization strategy is proposed to further solve the above problems.To maintain efficient feature disentangling performance and to be able to deploy the model to embedded platform,we propose a boat-trackers-based multi-task knowledge distillation method to obtain a single-stream,light-weight FDRL-Net.We conduct extensive comparison experiments and ablation experiments on six view subsets of the SCUT LFMB-3DFB dataset and two reconstructed full-view multi-spectral finger datasets,and in-depth visualization analysis is performed to illustrate the excellent performance of FDRL-Net.