Weakly Supervised Object Detection And Its Application For Image Classification

Object detection,i.e.,classifying and localizing objects in natural images,is one of the most fundamental problems in computer vision.It has a wide range of applications in many pratical problems,such as camera autofocus,automatic driving,robot navigation,image retrieval,etc.Conventional object detection requires images with detailed annotations of object locations and categories for training.However,collecting such detailed annotations is very time-consuming and labor-intensive.By contrast,weakly supervised object detection only requires images with image-level annotations indicating whether an object category exists in an image or not for training.Obviously,it is much easier to obtain images with only image-level annotations,e.g.,we can use image search queries to search on the Internet.Therefore,a large number of researches focus on weakly supervised object detection.However,due to the complexity of natural images,including changes in object size,position,angle of view,shape,etc.,and the lack of object location annotations,there exist great challenges in weakly supervised object detection.Recently,promising results have been achieved by using the weakly supervised learning method multiple instance learning in weakly supervised object detection.In addition,with the development of deep learning,great advances have been observed in computer vision,and great breakthroughs have been made in weakly supervised object detection.In this dissertation,a series of studies are conducted on weakly supervised object detection based on multiple instance learning and deep learning.More specifically,three different weakly supervised object networks are proposed,where the first network focuses on how to train the last two steps of weakly supervised object detection jointly and end-to-end,the second network focuses on how to learn better weakly supervised object detectors,the third network focuses on how to integrate the region proposal extraction step into weakly supervised object detection networks,and the latter networks are the extension of their previous networks.In addition,an application of weakly supervised object detection is also explored in this dissertation.To summarize,the contributions of this dissertation include:(1)Deep patch learning,a method which implements a weakly supervised object detection network based on multiple instance learning,is proposed in this dissertation.The network aggregates region proposal classification results by an instance-space multiple instance learning method,and thus the network can be trained by image-level annotations directly.The network also introduces a new object classification task by an embedded-space multiple instance learning method.The two tasks weakly supervised object detection and object classification are trained jointly,which is benefitial to weakly supervised object detection by multi-task learning.In addition,the network trains region proposal feature extraction,region proposal classification,and object classification jointly and end-to-end.Experiments on PASCAL VOC 2007 and 2012 datasets show that,the proposed network surpasses the previous multiple instance learning based methods by more than 5% for weakly supervised object detection,and surpasses the previous state of the arts by about 2% for object classification.(2)A proposal cluster learning method is proposed in this dissertation.The method groups region proposals in each image into different clusters,where each cluster corresponds to an object.For each cluster,the method treats it as a small multiple instance learning bag,and uses an instance-space multiple instance learning method to learn weakly supervised object detector in the small bag.In addition,the method is also combined with neural networks to train region proposal feature extraction and region proposal classification jointly and end-to-end.The method alleviates the problem that previous methods always tend to detect parts of objects instead of whole objects,and is robust to changes of object size,position,angle of view,etc.Experiments on PASCAL VOC 2007/2012,ImageNet Detection,and COCO datasets show that the proposed method outperforms the previous state of the arts by 5% on average for weakly supervised object detection.(3)A weakly supervised region proposal network is proposed in this dissertation.The network trains a neural network based region proposal extractor under weak supervisions,using two stages coarse proposal extraction and region proposal re-scoring.In addition,the network and a weakly supervised object detection network are also integrated into a single weakly supervised region proposal and object detection network,making that all steps of weakly supervised object detection can be accomplished by a single network,which is of great importance in weakly supervised object detection.Experiments on PASCAL VOC 2007/2012 and ImageNet Detection datasets show that the proposed network achieves the state-of-the-art performance for weakly supervised object detection with performance gain of more than 2%.(4)The application of weakly supervised object detection in image classification is also explored in this dissertation.Specially,the single pattern learning in weakly supervised object detection is extended to multiple pattern learning in this dissertation,which learns multiple category-specific patterns for each image category.These learned patterns are used as visual words in the bag of words method.Accordingly,image representations are obtained.These image representations are used as input features of an image classifier.The proposed method obtains very promising performance on image action classification dataset Action 40,image object classification dataset Caltech 101,image scene classification datasets Scene 15,MIT-Indoor 67,and SUN 397.In particular,the proposed method surpasses the previous best-performed method by 16.41% on the image action classification dataset Action 40.In summary,based on multiple instance learning and deep learning,a series of solutions for weakly supervised object detection are proposed in this dissertation,and the application of weak supervised object detection in image classification is also explored,both of which lay the foundation of the following researches on weakly supervised object detection.