Research On Graph Theory Based Interactive Segmentation Via Feature Measurement And Information Propagation

The interactive image segmentation can be described as extracting the foreground objects of interest in the complex background environment based on certain similarity criterion according to some prior knowledge provided by the user,which is a key problem in the research for image analysis,pattern recognition,computer vision,and so on.The quality of segmentation will have a direct impact on subsequent related applications.Recently,the research about graph theory methods has attracted wide attention by scholars at home and abroad,and has become one of the most popular methods for interactive image segmentation,due to the fact that it has many excellent features,such as ability to fuse a wide range of visual cues,global optima,practically efficiency,and so on.When enough seeds are provided by the user,the conventional graph-model-based methods have good segmentation performance to segment the simple color images.However,these methods are sensitive to the seed's quantity and location,and it is hard to obtain satisfactory results when the interactive information is limited.Furthermore,only the neighboring relationships between pixels are utilized for the segmentation in the conventional graph-theory-based methods,which causes them limit to images with noise and texture due to underutilizing the structure information of the image.These defects seriously affect the accuracy and robustness of these methods,which limits their practical usability.To address the above problems,this thesis provides a deeper analysis and study about the conventional graph-theory-based interactive image segmentation methods,and introduces the non-local relationship,image patch and superpixel-based higher-order relationship,and similarity-diffusion-based image global relationship into the graph to improve the segmentation performance.By combining with the information theory,pattern classification,and numerical analytic method,and making full use of the statistical learning methods and prior heuristic information,this thesis also designs the construction and optimization of the energy functions based on the graph theory framework,and researches on how to interactively segment images with high quality and high efficiency.Our work mainly includes the following parts:(1)An interactive image segmentation algorithm combining non-local information and graph cut is proposed to extend the local pixel relationships in the conventional graph model to non-local relationships.The search region is designed for each pixel,and the non-local information is obtained based on the similarity between pairwise non-local image patches within the search region.Then the non-local constraint is added into the boundary energy term of the graph cut model to improve the accuracy of the segmentation.(2)To overcome the limitations in accuracy and algorithm complexity of the similarity computation between image patches in the non-local-based methods,an interactive image segmentation algorithm based on the feature learning of image patch is proposed,by utilizing the Gaussian mixture model to model the structure feature of image patch for the foreground and background,respectively.Furthermore,to overcome the defect of the image patch for the preservation of boundary information,the structure information and the pixel information of the image are fused,and the combination information is then introduced into the region energy term of the graph cut model to further improve the segmentation accuracy around the boundary region.The utilizing of structure information can help to segment images with noise and texture.(3)To further improve the accuracy and the robustness to the seeds,the multi-layer relationships among the pixel,the superpixel and the label are fused,and two interactive multi-label image segmentation algorithms based on multi-layer graph constraints are proposed.The first algorithm estimates the probabilities of pixels and superpixels belonging to each label,respectively,and the game-theory-based optimization stategy is utilized to update the probabilities of pixels and superpixels iteratively until convergence.The second algorithm constructs the segmentation model based on the Markov Random Field framework,and the parallel partial optimality strategy is proposed to optimize the multi-label sub-modular energy fucntions.The multi-layer graph-based relationships can help to improve the segmentation quality.The game theory and parallel partial-based methods can help to improve the optimization efficiency.(4)The conventional pixel-based methods are limited to the segmentation accuracy and the image patch,superpixel-based higher-order methods are limited to the computational complexity.To address these problems,an interactive image segmentation algorithm based on affinity diffusion is proposed.The global affinity of the image is estimated by diffusing the neighboring similarity between pairwise pixels.The diffusive affinity matrix convergences to a robust limit when the iteration step tends to infinity.The segmentation model is constructed based on the global affinity.To further improve the segmentation accuracy,the Gaussian mixture model is utilized to model the seeds to estimate the initial probabilities of pixels belonging to the labels,and the initial probabilities are introduced into the segmentation model as the soft constraints to replace the conventional seeds-based hard constraints.The algorithm is simple to implement,and can improve the segmentation accuracy while keeping low computational complexity.(5)An interactive segmentation algorithm of fluid-associated regions in retinal Spectral Domain Optical Coherence Tomography image is proposed to extend the single image segmentation to sequence image segmentation.The user only needs to interact on a key frame which can be automatically selected based on the retinal thickness,and then the algorithm can automatically finish the whole segmentation.To further improve the segmentation accuracy,the higher-order energy is introduced into the Markov Random Field framework.To maintain temporal coherence of the segmentation,the neighboring labeling flow is propagated slice by slice based on the motion estimation.The proposed higher-order function can be transformed to binary sub-module function by introducing auxiliary variables,and the max-flow/min-cut algorithm can be utilized to obtain the globally optimal solution.