Probabilistic Graphical Model Based Heterogeneous Face Image Synthesis And Recognition

Heterogeneous face images refer to the images acquired from different sources,such as photos taken by cameras under normal lighting condition,hand drawn sketches by the artists,composite faces generated by software,and infr ared images captured by infrared sensors.Heterogeneous face recognition p lays a vital ro le in biom etric research and industrial application.For example,there are circumstances in law enforcement where the photo of the suspect is not available.The descrip tion from eyewitness or victim,or low quality surveillance video is the only clu e.Face sket ches generated by the forensic artist or composite generation software based on the clue above are then used for matching with mug shots.There are great differences between heterogeneous face images because of the variety of geometric and textural differences.Therefore,traditional face recognition techniques often perform poor on heterog eneous face recognition scen arios.In view of such situ ation,this thesis mainly focuses on synthesis and rec ognition problems between heterogeneous face images.In this thesis,a series of heterogeneous face synthesis and recognition methods are proposed based on probabilistic graphical m odel theory.The m ain contributions of this dissertation are summarized as follows:1.A multiple representations-based heterogeneous face synthesis m ethod is prop osed.Existing heterogeneous face synthesis m ethods usually use pixel inte nsities as the only feature when representing an image patch,which is highly ef fected by different lighting conditions,background and skin colors.Consider ing that face images can be described by different features after different image filtering techniques,a multiple representations-based method is proposed which can adap tively combine multiple representations through an alternating optimization strategy.Experimental results demonstrate that the proposed method is robust to lighting,background and skin color variations,and achieves satisfying performance on matching real-world forensic sketches with mug shots.2.A superpixel-based heterogeneous face synthesis method is proposed.Existing methods simply divide the face image into equal-size rectangular patches,which ignore the inherent facial structure.In order to exploit the inherent structu re of the face i mage,a superpixel-based method is proposed which ca n take the facial structures into consideration by using image segmentation technique.The face images are firstly divided into superpixels,which are then dilated in order to enhance the compatibility of neighboring superpixel patches and reduce blurring effect in synthesized results.The probabilistic graphical model is exploited to generate an initial synthesized image.A two-stage synthesis process is then introduced to enhance the image quality of synthesized image.Experimental result shows the superior performance of proposed method in comparison to regular rectangular patch division based methods.3.A graphical represen tation is proposed fo r heterogeneous face recognition.Existing methods usually ignore the spatial information when representing the face image,which is essential for face recog nition.Therefore,a gr aphical representation based heterogeneous face recognition method is proposed.Probabilistic graphical model is applied to explore the relationship among heterogeneous face i mage patches,by regardin g the neighboring compatibility as spatial inform ation.A coupled representation sim ilarity metric is the n designed to cater the charac ter of proposed graphical re presentations.Experiments on multiple heterogeneous face recogn ition scenarios demonstrate the effectiveness of the proposed method.4.A sparse graphical representation based di scriminant analysis method is proposed for heterogeneous face recognition.Existing probabilistic graphical model based approaches suffer the same drawback that fixed nearest neighbors of the probe image patch are selected.The hyper parameter is manually set in existing methods,which may affect the performance of these methods.A sparse graphical representation is proposed to cope with aforementioned drawback,which skips the nearest neighbor searching process and all the related patches are taken into consideration when constructing the probabilistic graphical model.A spatial partition based discrim inant analysis fram ework is then developed to improve the discrim inability of heterogene ous face im ages.A superior performance is achieved on multiple scenarios compared with existing methods.5.A fundamental study of face recognition from multiple stylistic sketches is presented.Existing research of heterogeneous face recognition mainly focuses on single sketch based scenarios.Considering the fact that multiple sketches are often available in real world scenes,multiple stylistic sketches based face recognition is proposed.Three specific scenarios with datasets are firstly in troduced according to th e types of sketches availab le in community.Corresponding protocols and benchmark performance are provided,too.Through extensive benchmark evaluations on the proposed scenarios,challenges and potential future directions that worth further studied are discusses in this dissertation.