Face Recognition By RGB-D Reconstructing And Multi-modal Features Fusioning

Thanks to the development of deep learning,face recognition technology has made significant progress.However,in complex scenes,the recognition performance of 2D face often fails to meet the requirements.Due to holding spatial information,3D face is robust to changes in lighting and posture etc.Which can improve the dilemma of 2D face recognition to a certain extent.But as a result of the high cost of 3D data acquisition equipment and the large amount of calculation,it cannot be widely used in practice.RGB-D recognition combines the advantages of both with respect to 2D and 3D recognition,which not only has spatial information but also has low acquisition cost.Nevertheless,due to the influence of the acquisition equipment,the spatial information of the RGB-D data may be lost that affects the recognition effect.Therefore,how to reconstruct the depth map to recover the spatial information is an important problem that needs to be solved urgently.At the same time,in RGB-D face recognition,the fusion of texture information and spatial information is still difficult.In response to the above problems,this thesis proposes a set of end-to-end solution SDR-Net(Share-Depth-Recognition-Networks)which is realization of high-precision RGB-D data conversion with low-precision RGB-D data and multi-modal automatic fusion recognition.In consideration of effects on recognition of the lack of spatial information of RGB-D data,this thesis designs a SD-Net(Share-Depth-Networks)that is a depth map high-precision reconstruction network.SD-Net repairs the missing parts of spatial information by learning the mapping transformation of texture information and spatial information,thereby high-precision reconstruction of low-precision depth maps is completed.Finally,the automatic conversion of low-precision RGB-D face data to high-precision RGB-D face data is realized.Aiming at how to better integrate texture feature and spatial feature,this thesis proposes a method for automatic fusion between multi-modal features.Compared with the common fusion method,the fusion method of this thesis automatically blends and learns multi-modal features under the premise of ensuring that texture feature and spatial feature do not interfere with each other,thus achieving high-precision recognition of RGB-D faces.At the same time,the fusion method and SD-Net are combined to form a set of end-to-end scheme SDR-Net.SDR-Net can realize multi-modal features fusion recognition of low-precision to high-precision RGB-D face dataFinally,through a large number of experiments,the validity of the proposed depth map reconstruction and multi-mode features fusion recognition method as well as the SDR network scheme is verified.