Research On Exemplar-based Inpainting For Dazu Rock Carvings' Buddhist Facial Image Based On A Sample Database

As the only World Cultural Heritage in Chongqing,the Dazu Rock Carvings have been suffering from natural environment erosion for almost one thousand years,so the digital conservation of these carvings is in great demand.In order to reduce the inpainting risk of “secondary damage”,it is necessary to introduce digital image processing techniques to perform virtual inpainting initially.Inspired by the process of art inpainting,digital image inpainting is proposed to restore the damaged region according to the image priors in the remainder region of the original image,making the reconstructed part plausible in vision.Among numerous image inpainting algorithms,exemplar-based inpainting is applied for the inpainting task with 3 iterative steps: filling priority computing,match patch searching,and target patch filling.It can be employed in the virtual inpainting for cultural heritage.Exemplar-based inpainting algorithm is guided by a basic assumption that the remainder of the original image itself can provide sufficient image priors for the guess recovery of the unknown part,which is not often the case in real art image inpainting.Sometimes,the art image need to be inpainted is so badly damaged that there are little priors as a good model to infer the unknown fragment.Focusing on the lookup strategy for match patches from another sample image,this paper proposes a novel exemplarbased inpainting framework via a sample database to solve such a challenging inpainting task in the following steps: select similar images of the original image from the sample database as reference images;create a sample image based on the reference images;reconstruct the damage region according to the sample image via exemplar-based inpainting.This method has been successfully applied in the virtual inpainting for the Buddhist facial images of the Dazu Rock Carvings in following tasks.(1)Buddhist Head Images Modeling Style Recognition Based on Deep Convolutional Neural Network for Dazu Rock Carvings The critical step in exemplar-based inpainting algorithm based on a sample database is to search for the most similar images of the input fast and effectively.Modeling style recognition is therefore the basis of the carvings' exemplar-based inpainting.A two-step recognition method named Buddhist Head Images Modeling Style Recognition Based on VGGNet for Dazu Rock Carvings(MSRoVGG)is proposed: feature extraction based on VGGNet,an outstanding representative of deep convolutional network;similarity measurement and clustering by K-Means algorithm.The proposed method obtains promising results compared with 5 classical feature extraction algorithms.According to the experiment results of machine learning,combining both art archaeology and image analysis,we conclude that the modeling style of the Buddhist head images is similar if they are in the same cave or region;and the modeling style of the Buddhist head images on the same subject is also similar,even though they are in different caves or regions.This meaningful conclusion can be inferred for the other Buddhist statues,and verifies the validity of the exemplar-based inpainting based on a sample database for Dazu Rock Carvings.Moreover,the efficient recognition and search for the similar images is the basis for the next two tasks.(2)Buddhist Facial Image Inpainting Based on Image Harmonization for Dazu Rock Carvings To solve the inpainting case that the original image itself can provide insufficient image priors for the inferring of the unknown part,a method named Buddhist Facial Image Inpainting Based on a Sample Created by Image Harmonization for Dazu Rock Carvings(FIIo IH)is proposed to extend the searching region for match patch from the remainder of the input image to another sample image with 3 steps: firstly,similar images are selected as references from the database via MSRoVGG;the sample images are then created based on the reference images based on image harmonization algorithm;finally,enforce exemplar-based inpainting based on the best sample.Thanks for the extension of the searching space for candidate patches,it performs efficiently for particular inpainting case that there is little prior information existing in the remainder of the input image.Furthermore,the process of sample image creation reduces the time complexity of inpainting based on a database,and avoids the taboo of a direct duplication in art restoration as well.Moreover,Poisson blending is employed for post-processing to improve the visual harmonization between the inpainted part and the known part in both color and illumination.(3)Buddhist Facial Image Inpainting Based on Average Face for Dazu Rock Carvings There are two limitations in the sample image creation via image harmonization.The melding step is just enforced in pairs to create a set of harmonized images,among which the best has to be chosen by hand as the sample.At the same time,there are some distortion and blurs in some created samples due to the different angle of the face among the similar images,while the critical angle homogeneity is required for alignment before face image harmonization.To cope with the aforementioned two limitations,a method named Buddhist Facial Image Inpainting Based on Average Face for Dazu Rock Carvings(FIIoAF)is then proposed to improve the visual quality of the sample in the following aspects: firstly,the mirror images of the selected similar images are also added as the reference images for symmetry consideration to avoid distortion;secondly,the average face generation is introduced to create a sample image which contains the universal features of each reference image and avoids manual intervention;at last,image deblurring algorithm is applied to deblur the averaged face,as the blurs increase with the increase of the number of the reference images.The above improvements ensure both validity and plausibility of the average-face-generated sample image,and therefore the quality of the inpainted image.Various experiments on Dazu rock carvings illustrate that it is not only useful for the challenging inpainting case that there is no priors in the the original image but also efficient for the exemplar-based inpainting framework based on a sample database.