Research On Image Holistic Scene Understanding Methods Basaed On Probabilistic Graphical Models

Scene understanding is an extremely important basic task in image processing and computer vision research areas, its research results have been widely used in military unmanned aerial vehicles, spacecraft navigation, web multimedia information search, intelligent monitoring, intelligent transportation and medical informationization fields. This shows that scene understanding has important academic and practical values. The research has made a breakthrough on subtasks of scene understanding such as saliency detection, image semantic segmentation, scene classification, image annotation and etc. Holistic scene understanding as the extension of basic scene understanding, its complexity and integration is much more difficult than the basic scene understanding, and still in its infant stage. In recent years, guided by the ideology of “holistic understanding”, researchers have put forward the research ideas of task integration and feature information integration. They study how to integrate these subtasks or the utilization of the holistic scene information. Based on these ideas, a variety of solutions have been proposed for the holistic scene understanding. However, the existing research results are not satisfactory. Therefore, the dissertation focuses on image saliency detection, image scene classification, image semantic segmentation, image annotation issues and their overall integration, then proposes new solutions to overcome the imperfections in current research based on probabilistic graphical model. The main contents and contributions are as follows:1. The researching background, status, major technical problems and solutions for holistic scene understanding are systematically studied. The typical holistic scene understanding models and their feature engineering are compared. Based on this, the dissertation proposes the basic frameworks for holistic scene understanding and its feature engineering based on probabilistic graphical model. The studies have shown that the holistic scene understanding is one of the hottest and most difficult topics in image understanding domains. The holistic scene understanding has a prosperous future. Although significant progress are made, there are still a number of technical difficulties to prevent the wide usage of holistic scene understanding models in practice, for example: model integration, effective feature engineering selection, the depth analysis of scene cognitive theory for holistic scene understanding, effectively matching between models and features engineering. Our research provides a theoretical basis and an important reference for further study.2. For the problem of object boundary and spatial logical relation preservation among image saliency detection and direct segmentation, we propose a direct image saliency detection and segmentation framework for further image analysis and processing. 1) The enhanced graph cut algorithm is proposed to achieve initial coarse image segmentation. The method can be used for non-interactive scenes, and also has a good advantage of keeping object edge details and the spatial dependence relationships between objects; 2) In order to eliminate some redundancy region generated after the first step,the super-pixel graph segmentation algorithm based nearest neighbor graph is proposed:(1) The intermediate image of first step is divided into 2-4 regions again;(2) A regional comparison method based on comparison of regional Weber brightness benchmark is proposed. This method effective eliminates some redundancy region to further improve the overall performance of the proposed method.3. For problem of the higher dimensional features of image classification and model parameters optimization, a framework based on feature kernel transformation and classifier with randomized hyper-parameters optimization algorithm are proposed. 1) For image features extraction and feature dimensional reduce, the method based on KPCA kernel transform of PHOW feature effectively reduces the dimension of features, while keeps high accuracy. 2) The parameters have an important effect on model performance. In order to deal with the model parameters optimization, the dissertation presents the method of model classifier with randomized hyper-parameters optimization. The experiments prove the effectiveness of the proposed method. 3)We compare two classic classifiers: support vector machines and Bernoulli Bayesian classifier. With compared to support vector machine, the classification accuracy of Bernoulli Bayesian classifier is generally lower than the support vector machine, but its execution performance is significantly better than the support vector machine.4. The dissertation proposes a holistic scene understanding framework based on global context-based features and Bayesian topic model, the model integrates three basic subtasks: the scene classification, image annotation and semantic segmentation. The model takes full advantages of global feature information in two aspects. On the one side, the performance of image scene classification and image labeling are boosted by incorporating image global context-based features; On the other side, the performance of image semantic segmentation is also boosted by new superpixel region segmentation method, new superpixel regions and patch feature representation. 1) For image scene classification and image labeling:(1) We improve the feature engineering methods by using the PHOW proposed in chapter 4;(2) Furthermore, global context-based features are learned by semantic features. 2)For semantic segmentation:(1) We improve the super-pixel segmentation method by using UCM in literature [66];(2)We proposed new feature representation for super-pixel region and patches by incorporating DSIFT, texton filter banks, RGB color, HOG, LBP and location features. The experiments testify that model performance has raised on all three sub-tasks.5. The dissertation proposes a holistic scene understanding model based on the intrinsic characteristics of the image and CRF. The model integrates image scene classification, image semantic segmentation and object detection. 1) For the scene classification, we use method of PHOW feature extraction plus KPCA dimensional reduction proposed in chapter 4 to obtain feature information for each image. 2) For object detection section, saliency detection and segmentation characteristics of the image object detection is useful. We propose the method by integrating image segmentation information got by the method proposed in the third chapter. 3) For the semantic segmentation:(1) For the unary potentials, we incorporating HOG, RGB color histogram and LBP features by the methods proposed in the literature [38];(2) The image manifold structural features can better reflect the importance between hyper-pixel regions and eventually boost accuracy. Therefore, we add the higher-order potential item to reflect inherent manifold images feature of each super pixel region. The experiments testify that model performance has raised on all three sub-tasks.