Digital Matting Based On One-shot Learning

As a classic image processing technology,digital matting is increasingly used in television and movie shooting,the goal of digital matting is to perfectly separate the foreground from the background in the image.Then you can change the virtual background for the image and so on.Since a long time ago,the main problem with digital matting its own under-constraint.Therefore,we need to mark some areas that belong to absolute foreground or absolute background to provide more information for digital matting.This kind of information includes trimap and scribble.However,design a relatively accurate trimap for every image takes a lot of manpower,scribble-base algorithm provides little information and lack of robustness.Therefore,this paper designs a model to minimize the workload of user annotation,and the model has better robustness.The one-shot learning achieves the recognition of object regions from unseen classes with only single annotated sample serves as the supervision.We development the one-shot learning techniques to formation trimap for image matting.Specifically,our model takes advantage of few user-provided clicks(absolute foreground click,absolute background click),guide the segmentation branch to obtain the binary predict masks,then the trimaps are generated by ablating the predict masks edge.It is worth noting that our algorithm has a good ability to generalize unseen classes,which means that for matting problems of the same semantic class,we only need to provide annotations for any one of them.The main work of this paper includes:(1)This paper constructs a foreground and background segmentation model based onOne-shot Learning.Firstly,the conditional branch is designed to generate semanticrepresentation(foreground representation and background representation)by combiningthe image with the consultation information provided by the user(foreground click andbackground click).Secondly,the segmentation branch is designed to combine semanticrepresentation with image segmentation to produce segmentation results of foregroundand background.(2)To solve the problem of inaccurate segmentation results,this paper further optimized thesegmentation results by conditional random field algorithm,and then processed thesegmentation results by expansion and ablation to generate trimaps.(3)In this paper,the depth network is used to predict the matting results,which improvesthe matting accuracy and speeds up the operation efficiency.(4)In order to verify the feasibility,operation efficiency and accuracy of the algorithmproposed in this paper,we carried out four experiments in this paper,includinggenerating the accuracy comparison of the trimap,algorithm operation efficiencycomparison,model guidance ability verification and model feature transfer abilityverification.And the experimental results were analyzed.The main contributions of this paper are as follows:(1)We propose the digital matting algorithm based on one-shot learning,combine one-shotlearning with trimap generation for the first time.Trimap generated by computeralgorithm instead of user hand drawing.This approach greatly reduces the interactioneffort.The interaction effort is from drawing a full trimap to a few clicks.Compare withscribble algorithm,our algorithm has better robustness.(2)The proposed method can effectively extract the task representation in the same category,the task representation is used to guide the generation of trimap.This means that for allthe images in any unknown semantic class,we only need to label one of them to get aTrimap of all the images in that semantic class.