Research On Weakly Supervised Image Instance Segmentation From Natural Language Expressions

The image semantic segmentation task has become one of the popular research directions of computer vision because of its wide application in scenarios such as autopilot,environment perception,human body analysis,etc.However,it can only segment the same kind of things and cannot distinguish the individual things.In some human-computer interaction application scenarios,functions that require natural language and image data to work together have emerged.For example,after people give the command "open the door" to a service robot,the robot responds by combining language information and image information captured by the camera,resulting in a novel natural language image instance segmentation task,which aims to output segmentation results that match the description based on the input image and description statement.The currently proposed natural language image instance segmentation models all use a supervised training approach,i.e.,using datasets with pixel-level annotated segmentation targets as supervised samples,however,constructing such datasets is labor-intensive and time-consuming.In response to the high cost of acquiring training datasets,this paper has conducted research on the weak supervision approach from the perspective of model training,with the following main contents and innovations.The currently proposed models of image semantic segmentation based on natural language processing use a supervised training approach,i.e.,using a high-quality datasets as supervised samples,although this training approach can effectively improve the training effectiveness of the model,constructing such high-quality datasets requires a lot of labor and time.Therefore,in response to the high cost of acquiring training datasets,this paper proposes a weakly supervised natural language instance segmentation model.The main research content and innovation points of this paper are as follows.(1)To address the problem of high cost of training data acquisition by the supervision model,a weak supervision-based natural language image instance segmentation model is proposed,which requires only the target center point and image level label to complete the training,effectively reducing the cost of training data acquisition.(2)To address the problem that weak supervisory data cannot provide accurate segmentation of target a priori information,an iterative training network model is proposed,in which weak supervisory data is used as the initial supervisory sample,and the results output from the model at the end of each training round are used as supervisory for the next round of training,resulting in a natural language image instance segmentation model that can output more finely segmented results.From the experimental results,the partitioning accuracy of the weakly supervised natural language image instance partitioning model proposed in this paper is comparable to that of the existing supervised methods and reaches the highest value on individual datasets,while effectively reducing the cost of training data acquisition and making it more convenient and efficient in practical applications.