Research On Sketch-based Cross-modal Face Retrieval Framework

In the field of computer vision, face recognition is one of the important research fields. After nearly forty years of development, the traditional vision-based face recognition problem under controllable conditions has achieved a very mature solution. Nowadays, more and more researchers are focusing on non-controllable cross-modal face retrieval problem. Because of some practical application requirements, cross-modal face recognition has become a very important work, such as criminal image retrieval for police. However, cross-modal face retrieval is a challenging task as the significant feature heterogeneity between different modal of facial images, which is more significant in sketch. The former researchers proposed to solve this problem by designing high quality features with certain modal invariance,or learning common subspace between different modal. However, for the sketched face images, especially the abstracted sketch face images, the existing methods still cannot get a good retrieval performance. Therefore, this paper attempts to solve this problem more comprehensively and synthetically.In this paper, the sketch-based cross-modal face retrieval technology is deeply researched, and the sketch-based cross-modal face retrieval framework is studied and perfected.The main works are as follows:1. Propose to extract the image feature on the facial component. The region of the facial component is determined by facial landmark detection. This method can solve the problem of misalignment between different modal significantly. This improved feature serves as a low-level feature in our overall retrieval framework.2. Proposed to detect the facial semantic attributes on facial component, which can reduce the interference and correlation between different semantic attributes.Besides, this paper also proposes an improved detection algorithm for the geometry-based semantic attributes, so as to enhance the overall attribute prediction results. The improved face semantic attribute serves as a mid-level feature in our overall retrieval framework.3. Propose an improved projection method to improve the accuracy of sketch-based cross-modal face matching. Compared with the Canonical Correlation Analysis or Partial Least Square Regression, this method can achieve a satisfied performance compared with low-level feature solely.4. Finally propose our sketch-based cross-modal face retrieval framework,combined with low-level feature with improved projection method and mid-level facial semantic attributes. Experimental results show that our framework can achieve a perfect performance, which is robust.