Research And Application Of Human-Computer Interactive Image Editing Algorithm Based On Sketch

In recent years,the demand for building interactive image editing tools on touch-screen devices has been increasing.Sketch is easy to draw and rich in meaning,and it is the most direct way for humans to display creativity and interact with images.Sketch-based image editing is an emerging research direction,and its purpose is to edit the whole or part of an image with the sketch drawn by the user as conditional information.Existing sketch-based image editing algorithms suffer from two limitations.On the one hand,users need to draw additional masks,and the interaction is complicated.On the other hand,existing methods use sketch lines as conditional information through image inpainting algorithms,and there is a lack of research on complete sketches(such as a cat)as conditional information.This thesis considers sketch-based image editing as a cross-domain image transfer problem from a style transfer perspective.This thesis first proposes a sketch-based two-stage image editing algorithm,which splits the problem into an image generation stage and a generated image rendering stage.In the image generation stage,based on the style transfer algorithm,the sketch input by the user is used for image generation,and the unsupervised training and perceptual loss function are used to deal with the problem of misalignment between the sketch and the image space structure.The UCT generator(U-Net CNN Transformer Generator),adding noise and introducing the intermediate state of the grayscale image to deal with the problem of sparse sketch features.In the rendering stage of the generated image,the main body of the generated image is separated by the semantic segmentation algorithm,and the main body is pasted into the editing area.The algorithm completes image editing based on a complete sketch without the user inputting a mask.In order to enable the algorithm to perform end-to-end training,while avoiding the limitations of semantic segmentation algorithms.This research proposes a sketch-based end-to-end image editing algorithm,and one model can complete the image editing task.Since the sketch-based image editing task is compressed into one model,the model needs to discover the edited area and complete image generation in the edited area while the image in the non-edited area remains unchanged,which brings challenges to the style transfer algorithm.Inspired by the attention mechanism,this thesis introduces the attention generator to separate the edited area from the non-edited area,designs the migration generator to complete the image migration of the edited area,and adds an attention discriminator to focus on the image generation quality of the edited area.Attention loss improves the quality of attention mask generation to minimize changes in non-edited regions,and the generative model can fully learn the sparse sketch information in edited regions to complete image editing tasks.At the same time,unsupervised training is introduced to deal with the impact of structural misalignment on image editing tasks.The end result is sketch-based end-to-end image editing.In order to verify the effectiveness of the algorithm in this thesis,experiments were carried out on the ShoeV2,Sketchy,and Sketchy COCO datasets.The qualitative effects and quantitative indicators of the algorithm experiments in this thesis were higher than the baseline method compared.At the same time,in order to verify the simplicity of the user interaction process of this research algorithm,this research designs and implements a sketch-based human-computer interaction image editing system,and engineers the research algorithm.The design and development of this research system can be used for deep learning algorithm engineering.provide some reference.