Font Size: a A A

Heterogeneous Facial Image Synthesis And Its Applications

Posted on:2016-09-19Degree:DoctorType:Dissertation
Country:ChinaCandidate:N N WangFull Text:PDF
GTID:1108330464968963Subject:Intelligent information processing
Abstract/Summary:PDF Full Text Request
With the growing development of imaging technology, there appear diverse image modalities, namely heterogeneous images. Taking facial images as an example, existing imaging technology can produce someone’s visible image, near-infrared image, thermal infrared image, and sketch portrait or line drawings. These different types of images give different descriptions and characterizations to the same object in the different representation space. There exist both redundancy and complementation among these images, which let people comprehensively recognize the nature property of things. Effective digging and deploying of these mutual information could expand our perception and understanding to objects, which plays important role in social public security and digital media entertainment and is also a significant morphological relations among future internet of things.Different imaging sensors to faces have different usage. Conventional information processing methods mostly study on the information fusion of multiple sensors to obtain comprehensive descriptions to the object. Nevertheless, in practical, there may miss one or more images from all sensors, i.e. sensors receive only images. Then, we could consider to reproduce the missing image(s) given other obtained images, which is the problem of heterogeneous facial image synthesis. This thesis studies the synthesis problem between heterogeneous facial images taking visible images(photo) and sketch portraits as instances. The mapping relation of imaging mechanism and image representation between face photos and sketches is the research focus. The image quality of synthesized images is also investigated and we also explore the proposed synthesis methods for some applications such as face recognition. The main contributions of this dissertation are summarized as follows:1. An adaptively sparse feature selection method is proposed to determine the number of relevant features. Existing methods select fixed number() of nearest neighbors during synthesis process. While due to the limitation to the number of heterogeneous facial image pairs in the training set, there may exist samples not very relevant to the query which result in some noise. Considering this fact, a method could adaptively determine the number of relevant features is proposed based on sparse representation.Experimental results show the superior performance of this method in comparison to nearest neighbor based methods.2. A hallucination based heterogeneous facial image synthesis method is proposed. Existing heterogeneous facial image synthesis methods adopt linear combination in the final synthesis process. And linear combination or linear weight average could be deemed as a low-pass filter which filters some high-frequency detail information. Inspired by face hallucination technology, a two-step framework is proposed to enhance the image quality of existing synthesis methods: the first step is to utilize existing methods such as above sparse feature selection based method to synthesize an initial image; the second step is to simulate the high-frequency detail filtering process. The estimated detail information is superimposed to the aforementioned initial image. Experimental results illustrate that this method could synthesis more vivid details.3. A transductive learning based heterogeneous facial image synthesis algorithm is proposed. Many existing heterogeneous facial image synthesis methods have been proposed under the framework of inductive learning, and these have obtained promising performance. However, these inductive learning-based methods may result in high losses for test samples, because inductive learning minimizes the empirical loss for training samples. Yet transductive learning based methods incorporate the given test samples into the learning process and optimizes the performance on these test samples. Hence it could minimize losses for test samples. In particular, it defines a Bayes probabilistic model to optimize both the reconstruction fidelity of the input image and the target output image. Experimental results demonstrate the effectiveness of the proposed method by comparing it with representative inductive learning-based synthesis methods.4. An image quality assessment(IQA) database for facial sketches is constructed and an objective synthesized sketch quality assessment framework is proposed. Subjective quality assessment is conducted to synthesized sketches generated from five representative heterogeneous facial image synthesis methods by means of pair comparisons. Then, existing objective IQA methods are imposed to these synthesized sketches to calculate objective scores. The consistency between obtained objective scores and subjective scores are computed. Experimental results show that obtained consistency is not very high, i.e. scores from existing IQA methods consist not well with visual perception of human beings. This stimulates the demand on more rational and applicable objective synthesized IQA frameworks.
Keywords/Search Tags:Heterogeneous facial image, sketch, synthesis, image quality assessment, recognition
PDF Full Text Request
Related items