Research On Digital Image Inpainting Technologies Based On Structure Tensor

Digital image inpainting technology refers to automatically modifying a damagedimage, portions of whose information is lost or corrupted completely, by using computerprogram or software. Its essence is to rebuild complete information based on incompleteone in a non-detectable way. The technology mainly aims at retouching the damageddrawings/photographs and removing the unwished targets. In addition, the technologyhas found broad applications in image super-resolution and zooming, image interpo-lation, error concealment in wireless transmission, and so on.After several years’ development, digital image inpainting technology has formed arelatively complete system. It includes variational functional or PDE (Partial Differen-tial Equation) based inpainting models which are suitable to inpaint small-sizedscratches, exemplar-based completion methods which are very good at completing thelarge objects and wavelet inpainting models which are targeting at the case of waveletinformation loss. Although they have gained great developments, there exist someproblems which need to be solved, for example, inpainting small-scale details or textureimages by using variational functional or PDE based models, adaptively controlling thegeometric regularity in wavelet inpainting model, completing large holes, etc.According to the above issues, the dissertation mainly does researches on digitalimage inpainting taking the structure tensor (ST) as the main line and incorporatingother newly emerging theories or methods. ST is an effective tool for image analysisand plays a key and important role in image processing and computer vision. Accordingto different repair requirements, the dissertation proposes better image inpaintingalgorithms by combining ST with other theories or methods. The main research workand contributions are as follows.1. In-depth investigation is conducted on the basic principle and research statusof digital image inpainting. The main inpainting methods are classified andsummarized, and the existing problems are expounded. The basic principles ofST and TD are studied and their anisotropism and good performance ofretaining the continuity of image structures in the diffusion process areanalysed in detail. Based on the above, the overall framework of ST isestablished systematically, which outlines its theory and applications in imageprocessing and specifically highlights its application in image inpainting, thusestablishes research foundations for the later chapters.2. The abilities of the existing wavelet inpainting models for adaptive regularization and restraining noise are poor. To solve the problem, a newwavelet inpainting model based on tensor diffusion (TDWI) is proposed. Thehybrid model is built by combining structure-adaptive anisotropicregularization with wavelet representation, which controls the geometricregularity in pixel domain while restores the missing or damaged waveletcoefficients in wavelet domain. Its associated Euler-Lagrange equation isdeduced to analyze its regularity performance. Due to using matrix-valued STin the regularization term, the shape of diffusion kernel changes adaptivelyaccording to the image features, including sharp edges, corners andhomogeneous regions. Therefore, the TDWI model controls the geometricregularity in the pixel domain more adaptively and accurately and gives betterrobustness to noise. In addition, for the established hybrid model, an effectiveand proper iterative numerical solution is applied to improve the computation,and then the numerical scheme for the TDWI model is given. Experimentalresults on a variety of loss scenarios are given to demonstrate the advantagesof our proposed model.3. Fine scale features cannot be recovered satisfactorily by the traditionalinteger-order inpainting models. In order to solve this problem, depending onwhether or not noise needs to be eliminated in the image, two novel imageinpainting models based on fractional-order tensor diffusion (FTDII) arepresented. The proposed models integrates fractional calculus with tensordiffusion, not only inheriting the anisotropism of tensor diffusion, but alsodealing with better image details owing to the characteristics of fractionalcalculus. Meanwhile, according to the fractional functional theory, theirassociated Euler-Lagrange equations are deduced. In the procedure ofnumerical implementation, the discrete templates of fractional derivative infour directions are deduced according to the shifted Grümwald-Letnikov (theShifted G-L) fractional calculas definition. And then, according to the derivedEuler-Lagrange equations, a numerical scheme for the FTDII models is given.According to the experimental results on various testing images, the proposedFTDII models demonstrate superior inpainting performance to the originalinteger-order ones based on classic calculus.4. The ability of the traditional ST of extracting low-level texture feature is poor,focusing on which a texture inpainting algorithm based on an improved nonlocal ST is proposed to repair structural texture image. According to thecharacteristics of this kind of image, an improved ST with three modificationsis put forward. Firstly, fractional-order structure tensor (FST) is employed toreplace the traditional ST to deal with better complex fractal-like texturedetails. Secondly, in order to avoid aliasing, the sampling rate of FST must bedoubled. The reason is expounded from the perspective of frequency-domainanalysis and the operation is implemented by computing the fractional-orderderivative image at both integer and half-integer positions according to theshifted G-L definition. Thirdly, in consideration of the nonlocal property, anonlocal filtering is performed on the resulting oversampled FST (OFST) byutilizing the redundant infromation of the tensor data. Lastly, the resultingnonlocal filtered OFST (NOFST) is inserted into the anisotropic PDE whosenumerical implementation scheme is given to perform texture inpainting. Theexperimental results show that, for structural texture images, the improved STperforms particularly well in extracting low-level texture and structure featuresand the proposed inpainting algorithm is not only quite efficient in recoveringnon-homogeneous structures, but particularly efficient in the restitution of thetexture regions.5. In order to overcome the defects of the existing exemplar-based methods in theprocedure of searching and matching, an image completion algorithm based onweighted fractal under the structure-measurement constraint is presented.Firstly, the selected domain blocks were transformed by geometricaltransformation and isomorphic transformation. Using the resulting blocks, thecodebook was constructed and then served as the ‘enriched’ searching scope.Secondly, by computing the anisotropic nonlinear ST (NLST), the localstructure measurement was obtained, according to which the normalizedweight map was constructed. Thirdly, during the luminance transformation, itstwo parameters are derived through minimizing a constrained energy functionbetween the target patch and each codebook patch. In the constrained energyfunction, two types of constraints are introduced: one is the weightedconsistency constraint between the codebook patch and the target patch overthe already known pixels where the weight patch is obtained from the weightmap, the other is the neighborhood similarity constraint between the codebookpatch and the weighted mean of the neighboring patches over the missing pixels. Lastly, the target patch is filled in using the estimated patch with theminimum constrained energy. The experimental results show that comparedwith the existing congeneric algorithms, the proposed one preserves better thecontinuity of the broken structure and forces the newly filled area to be moreconsistent with the source area. Therefore, the restored results are improvedboth subjectively and objectively.