The Research On Conditional Random Fields Models For Image Labelling

It is a difficulty of image labelling ensuring high precision that prevents the rapidextension to so many applications such as image retrieval, the expression and modelling ofspatial context and semantic information from a given image is a major key technology tosolve the difficulty in image labelling. Conditional Random Fields（CRFs）is a probabiliticsframework for labeling and segmenting structural data. In the application of Image processing,CRFs have the unique advantages over large scale spatial dependency, semantic informationand directly modelling posteri probability of a given image. Thus, the framework is capable toreduce uncertainty and improve the precision in image labelling.Owe to the advantages mentioned above of CRFs theory, for some problems existing inobject class image segmentation based on image labelling, the thesis develops the studies onlow and higher order CRFs model when considering multi-scale intrinsic properties andsemantic feature of image, the details and innovative studies are in the following:For the purpose of better express spatial, even semantic information of image, it isproposed that Probability Latent Semantic Analysis（PLSA）technology has been integratedinto the framework of CRFs via Support Vector Machine(SVM) classifier in order to achievean improved precision of object class segmentation in multi-label case. Details are as follows:Firstly, semantic features were derived from a PLSA clustering procedure can be an input toSVM classifier. Then, an output in the form of probability value from the well-defined SVMclassifier was used to make the associative potential function, Potts function and featurecontrast function were used to define the interactive potential function, then a PLSA-CRFscan be formed. In the process, an efficient Expectation Maximization（EM） algorithm wasintroduced to better estimate the parameters of PLSA model in order to improve less efficientproblem using standard EM algorithm. Finally, piecewise training algorithm is used toestimate the parameters of the proposed model, and Loopy Belief Propagation （LBP）algorithm is used to approximately infer the model.For the problem “superpixel-based CRFs using features only from a scale can’t expressinherent multi-level spatial structure and semantic relation of an object”, the thesis develops apairwise superpixel-based CRFs model with association term defined as an output of RandomForest classifier based on multi-level spatial context and semantic features of objects so as toacquire a high precision in object class image segmentation. Main studies include: multi-levelspatial context feature extraction and boosted feature selection method, the definition ofassociative potential using Random Forest, and interaction potential weighted by common boundary of neighbors for CRFs model. Finally, pseudo-maximum likelihood method wasused to train the model for parameters estimation, and LBP algorithm was used to infer thepiecewise model.For the problem of “the pixel-based CRFs is low efficient”,“the scale and quality ofimage over-segmentation have greater effects on superpixel-based CRFs, which have limitedability of spatial interaction” for image labeling, the thesis carried out the studies and makesimprovements on higher order CRFs model based on label consistency soft constraint andquality of image over-segmentation proposed by Kohli. Details are as follows: the model waswell studied by experiments on complex scenes at the first. A new segmentation qualitysensitive function for the purpose of image labelling (not image over-segmentation) was thenproposed, thus higher order potential was updated. Finally, the improved HoCRF in the thesisis trained by piecewise learning algorithm and inferred by transformed α-expansion based ongraph cut method.Experiments on images including natural, aerial imagery with complex scene are carriedto verify the three proposed models and some relative methods. The results have shown thosemethods are effective to object class segmentation in multi-label case for those images such ascomplex natural scene and aerial images.