Knowledge Transfer-based Image Scene And Object Information Extraction Method

It's one of the important tasks in the field of computer vision and image processing to make the computer extract information from the image as human beings,which can provide prior information for realizing some high level tasks as automatical image understanding and analysis and is valuable to research.Image scene and object are two kinds of important information for understanding and analyzing the image.For the extraction of this two kinds of image information,we study the closely corresponding image scene classification and salient object detection tasks.Image scene classification and salient object detection focus on extracting the global scene information and local interesting object information.Under the background of massive data,it's necessary to get rid of the dependence on large scale annotated data by taking advantage of the unlabeled data in image information extraction.In this thesis,we focus on the knowledge transfer-related tasks under different situations in image scene classification,multi-spectral salient object detection and within-image co-saliency detection which are the specific tasks related to image scene and object information extraction.The main work and and contributions are as follow:(1)For the different degree of domain discrepancy in image scene classification,we first propose a subspace alignment layer-based convolutional neural network for the clear image situation to help to learn a domain alignment feature space which helps to keep the original data distribution attributes at the same time.For the domain discrepancy causing by noises,an attention-consistency constrained deep network is proposed to learn to use clear image information to direct the classification of images with noises.By the domain adaptation-based knowledge transfer method,we study the image scene information extraction under the aligned data modality situation.(2)For multi-spectral salient object detection,a generative adversarial-based deep neural network is proposed to transfer the source domain label and data modality information into the target domain task and to make the generator learn the feature expression which confusing the discriminator.The public salient object detection dataset with single regular RGB modality is taken as the source domain,and it is used to improve the multi-spectral salient object detection performance on the proposed self-collected dataset.By the domain adaptation-based knowledge transfer method,we study the image object information extraction under the unaligned data modality situation.(3)For lack of studies or difficulties in data collection and annotation in withinimage co-saliency detection,a “teacher-student” model framework is proposed.The pretrained model of other related tasks is adopted as the “teacher” model to provide pseudo labels which are used to help the “student” model to learn more fine co-saliency cues.The proposed “student” model takes use of the image “easy” or “hard” classification,the strategy of “easy to hard” learning,the multiple instance learning model,etc.to reduce the negative influence causing by the noise in pseudo labels.Based on “teacher-student”model framework,we study the image object information extraction under the different source task and target task situation.As mentioned above,we study the problem of knowledge transfer-based image scene and object information extraction,and propose suitable knowledge transfer-based methods for different modality and task attributes in source and target domain for the corresponding image scene classification and salient object detection tasks.