Face Sketch-photo Synthesis And Recognition

Face sketch-photo recognition aims to determine the person's identity by retrieving in the photo database using simulated sketches automatically. It is applied extensively to criminal investigation, anti-terrorism, animation design, etc.When the photos of the person in question are absent, we have to fall back on simulated sketches and then face recognition is performed by matching the sketches and photos in database. Sketches are produced by artists according to their subjective understanding, in which the thickness and density of lines are used to convey shape and texture information. Photos are obtained with optical imaging equipments or other sensors objectively, which record grayscale or color information of images. Different mechanism for generating and expressing sketches and photos leads to geometrical deformations and very different texture between them. Even though the sketch and photo of a person may be similar in geometry, their texture is always very different, which makes most of the existing significant achievements for face recognition inactive for sketch-photo recognition. Consequently, this paper delves into the perfect conversion of sketches and photos with the least cost. One of the above-mentioned two expression modalities is converted into the other and face recognition is performed in the same modality, which explores novel ways to conduct cross-media information retrieval and pattern recognition.In this dissertation, in view of essential problems of sketch-photo recognition, the mapping relationship between sketches and photos is explored, based on which sketch-photo conversion algorithms are proposed so as to transform sketch-photo recognition into the face recognition in the same modality. They are necessary preconditions to improve the sketch-photo recognition performance. Furthermore, the face recognition algorithm applied in the converted feature space is proposed, in which structure and texture informations are combined to express face images. The main contributions of this dissertation are summarized as follows:1. Machine learning based pseudo-photo synthesis algorithm is proposed. The existing sketch / photo synthesis algorithms are summarized and analyzed in this dissertation. Based on this, we propose a photo synthesis algorithm based on local strategy and embedded hidden Markov model. The embedded hidden Markov model is used to learn the mapping relationship of training sketch patch-photo patch pair, so that pseudo-photos are produced in terms of sketches. Theoretical comparison and experimental results show that the proposed algorithm derives high quality pseudo-photos with less training samples, and consequently overcomes the problem that statistics based methods require a large set of training samples to generate pseudo-photos.2. Image quilting is introduced in the synthesis of pseudo-photo / pseudo-sketch. In the existing local strategy based pseudo-photo/pseudo-sketch synthesis methods, pseudo-photo patches / pseudo-sketch patches are combined by averaging the overlapping regions, which leads to the synthesized sketches/photos with blurring effect and noticeable block edges. Aiming at this problem, pseudo-photo patches / pseudo-sketch patches are stitched into a pseudo-photo / pseudo-sketch with image quilting in this dissertation. According to the different values of corresponding elements in the overlapping areas of two patches, the minimum cost path through the overlapping area is searched for and serves as quilting edge of two overlapping patches, which combines these two patches smoothly. Experimental results show that image quilting based methods may derive pseudo-photos / pseudo-sketches with higher quality, and theoretical analysis is performed.3. Image quality assessment of the synthesized images is explored. The existing image quality assessment algorithms mainly aim at quality degradation caused by noise and artifacts. Because the synthesized sketches/photos should preserve discrimination information as the original images, a novel image quality metric is expounded. The quality of the synthesized sketches / photos is evaluated from two aspects that are visual quality of the synthesized images themselves, and the similarity of the synthesized image and original image. The effectiveness of the proposed image quality assessment metric is analysed with the performance of sketch-photo recognition.4. It is proved that the performance of sketch-photo recognition is enhanced effectively with the proposed sketch-photo synthesis algorithms. After sketches and photos are transformed into the same modality, unsupervised subspace learning methods, such as principal components analysis, independent component analysis, kernel principal component analysis, locality preserving projection, tensor subspace analysis and offline tensor analysis, are adopted for face recognition in photo space and sketch space. It is proved experimentally that the proposed sketch-photo synthesis algorithms are effective to achieve promising results of sketch-photo recognition. 5. The biview face recognition algorithm is constructed by integrating texture information with shape information. In this dissertation, the sketches are realistic and affected slightly by subjective factors. The sketch and photo of a person are similar in geometry and very different in their texture. The difference of their texture is weakened and the similarity of their structure is preserved by the transformation of two feature spaces. So biview face recognition algorithm which relies on the cooperation of texture model and structure model of face images is proposed. The influence of texture difference on face recognition is further weakened by strengthening the structure similarity in recognition. Experimental results show that the proposed face recognition algorithm is robust against variation of illumination, expression and scale, which provides sketch-photo recognition under many variations with effective ways. Besides that, two graph edit distance algorithms are proposed for measuring similarity of structure models. These two algorithms are completely independent on cost function definition, and they have the virtue of high efficiency, generability and correct rat of clustering and classifying images.6. A new face photo-sketch database and experiment testing platform are constructed with independent intellectual property rights. Drawing style of sketches in the existing photo-sketch database is simplex. Aiming at sketch-photo recognition algorithms robust to sketches with different drawing style and a universal database for examining the performance of sketch-photo recognition algorithms in the further research, a new photo-sketch database is constructed. Photos are selected from standard face databases, according to which some artists are invited to produce sketches. The eligibility of the produced photo-sketch pairs is validated by the existing sketch-photo recognition algorithms. Because of different drawing styles, sketches corresponding to a photo are distinct,which is the foundation of performing machine learning based researches on the influence of different drawing styles on sketch-photo recognition.