| Artificial intelligence and deep learning have attracted more and more attention.Video object segmentation based on language description is one of the tasks.Video object segmentation based on language description is a common and intuitive video object segmentation method.When the user gives a natural language description,the object instance search segment the image or frame containing the description,and locate all the objects that appear.In order to solve the limitations of the previous methods(such as accurate mask,bounding box,graffiti or keyword search for interactive segmentation),more and more researchers begin to pay attention to video object segmentation using natural language description,which contains richer and more structured details than the previous methods.However,when the language description we input does not match the video frame information,the previous video target segmentation method based on language description still segmentation the wrong results.In order to solve the above problems,we propose a new user visible accurate video object segmentation method based on language description.As long as one line of language description is input,the framework can segment the video frame objects we are interested in and the states between objects.When the relationship between the objects described in the input language description does not match the actual video frame information,the segmentation is stopped in time.Our framework is mainly composed of two modules: video object segmentation module based on language description and language description-image matching module.The former calculates the mask of the target object in the candidate image as initialization information through the language description and the first frame of the video,and uses the above mask to propagate in the video to obtain the final prediction mask result of the language description corresponding to the video.The latter consists of two parts.The first part is used to identify all instances of the video frame and the relationship between all instances,The indicative relation text of the video frame is generated according to the target object in the input language description.The second part is used to judge whether the language description input by the user matches the indicative relationship text output by the video frame,so as to judge whether the language description input by the user matches the video frame information.Our experimental results confirm the feasibility of our research.This paper compares the proposed method with the existing video object segmentation methods based on language description on two benchmark data sets Davis and A2 D.In the experiment,we verify the matching accuracy between language description and video frame information,and compare videos of different lengths for multiple rounds.With the increase of the number of video frames,the effect of this research method is still significant. |
