Research On Coordination Of Object Segmentation And Recognition

It is easy for humans to parse objects in their sights,such as segmenting out the targets and identifying them.Yet for computers,image segmentation is still a challenging task due to various reasons,for example,interference,complicated background,etc.One important reason may be that the image segmentation models often only rely on the underlying image data,and barely take the specific knowledge of the target to be segmentation into consideration.In fact,for both human and computer,the object segmentation task and object recognition task are closely related.Prior information of the target object will help to guide the segmenter to obtain a better result,while an precise segmentation result can also lead to a reliable recognition result,because there is no background interference.These two tasks form a chicken-and-egg problem.Based on the analysis of related research fields at home and abroad,this thesis focuses on how to introduce the object knowledge that comes from the target recognition task into the segmentation task,to achieve the coordination between the two.The contents and contributions of this thesis are as follows:1.By imitating the human visual cognitive mechanism,we propose a framework of collab-orative object segmentation and recognition tasks,where the two tasks communicate and promote each other.Based on this framework,we implement a coordinative task model with boltzmann machine(RBM)as the core on the basis of variational segmentation,in which the object is expressed in two aspects: shape and appearance.The shape is used to express the overall structure of the object,while the appearance is used to express the local and regional color,texture and other information within the shape.The discriminative and generative processes of RBM are both utilized in the coordinative task.The former is used to extract the features of the target shape and make predictions,while the latter generates a reference shape based on the features and recognition results,to guide the segmentation pro-cess.The appearance information of the object is used to further constrain the segmentation result,so that the segmentation result is consistent with the appearance knowledge.2.Due to the limitations of RBM in learning and representation,we extend the cooperative task model to modern deep learning method,and use the encoder-decoder architecture to express the discriminative and the generative processes,so as to make the construction of coordination task model more flexible.Specifically,we use the Capsule network to learn and express prior shape knowledge of the object.It is able to disentangle the features of the object and the interference,filter out the irrelevant information,and extract the shape feature of the object more precisely.The unique design of Capsule neurons also ensures that each dimension of feature has a specific meaning,thus introducing more interpretability into the coordinative task.3.In view of the insufficiency of the variational segmentation method in representing the local regions,we use the neural network instead to automatically learn and estimate the evolu-tion direction and complete the segmentation.Similar to the variational method,the neural contour evolution method also allows the introduction of prior knowledge to guide and con-strain the contour evolution,but it is more robust,more robust to different initial contours,and more efficient in evolution.We integrated the neural contour evolution method into the coordination model based on capsule network,to enhance the robustness and execution efficiency of the coordinative task model,also make it robust to the initial values.Consider that the recognition results rely too much on the object shape,we now utilize both the im-age textures and the object shape to perform the recognition,which further improves the performance and robustness of the coordination task.