Multivariate Local Information Enhancement Based Complex Image Classification And Object Detection

Computer vision has always been a research hotspot in the field of computer science.With the emergence of deep learning technology,computer vision technology has made break-throughs,especially in the field of complex image interpretation.Many remote sensing image classification techniques based on deep learning are widely used in environmental monitoring,urban planning,disaster control,agriculture and so on.The object detection technology based on deep learning for remote sensing image is widely used in the objects recognition in large scenes,such as aircraft,ships,vehicles,and various buildings.In ad-dition,natural image object detection based on deep learning has also been widely used in national life,such as pedestrian detection,road monitoring,unmanned driving and so on.In order to better understand complex images,this paper studies the classification and object detection of various types of images.In order to make full use of the spatial information,the multi-local regions spatial infor-mation representation enhancement models are studied on the basis of multispectral and hyperspectral image classification.In order to detect multi-class object in any size scene,this paper studies object detection in optical remote sensing image based on multi-scale model and fully convolution model.In order to effectively improve the accuracy of object detection,this paper studies the object-level features and part-level features fusion based object detection algorithm.The main contents are as follows:1)In this paper,we propose a superpixel-based multiple local Convolution Neural Net-work(SML-CNN)model for panchromatic and multispectral images classification.In order to reduce the amount of input data for CNN,we extend simple linear iterative clustering(SLIC)algorithm for segmenting multispectral images and generating superpixels.Super-pixels are taken as the basic analysis unit instead of pixels.To make full advantage of the spatial-spectral and environment information of superpixels,a superpixel-based multiple lo-cal regions joint representation method is proposed.Then,a superpixel-based multiple local CNN model is established to extract an efficient joint feature representation.A softmax lay-er is used to classify these features learned by multiple local CNN into different categorise.Finally,in order to eliminate the adverse effects on the classification results within and be-tween superpixels,we propose a multi-information modification strategy that combines the detail information and semantic information to improve the classification performance.Ex-periments on the panchromatic and multispectral image data sets have demonstrated the effectiveness of the proposed approach.2)In this paper,we propose an Enhanced Spatial Representation based Multi-stage Spatial-Spectral Fusion Network(ESR-S~2FN)for HSI classification.First,the Enhanced Spatial Representation module which consists of multi-neighborhood representation strategy and variable distance down-sampling strategy is introduced to extract more stable spatial struc-tural information and more abundant detail information.Second,the spatial sub-network and spectral sub-network are applied to extract the spatial-spectral feature.Then,the designed more effective fusion model named multi-stage spatial-spectral fusion network,which in-cludes middle-level feature fusion,high-level feature fusion,semantic-level spatial-spectral co-constraint,and decision-level fusion,is used to fuse the spatial-spectral information and to obtain the final classification results.Experiments on three HSI data sets demonstrate the effectiveness of the proposed method.3)In this paper,we propose a Multi-Scale Image Block-level Fully Convolutional Neural Network(MIF-CNN)for remote sensing image object detection,which can solve the above problems to a certain degree.Firstly,the training data sets which only require class labels and don't need the bounding box label can reduce the spend of manual annotation during training.Secondly,the MIF-CNN is designed to extract the multi-scale based high-level feature which can better represents the various types of objects.The image block-level fully convolutional network contributes to improve computing efficiency and can directly detect any size of the input image including a large scene remote sensing image.In the testing phase,the large scene image is directly input to MIF-CNN model,and the detection results are generated from the MIF-CNN output maps which are improved by the proposed bound-ing boxes modification strategy and local re-recognized strategy.Experiments on NWPU VHR-10 and two Airports data sets demonstrate the effectiveness of the proposed method.4)In this paper,we proposed a Generative Adversarial Networks(GANs)based weakly su-pervised learning framework combined with multi-scale detection network for VHR optical remote sensing image object detection,which can solve these problems at a certain degree.We first use the original data set to train an unsupervised GANs model as the pre-trained de-tection model,which can reduce the demand for the extra labeled data during the pre-trained process.After that,the discrimination network of GANs model based weakly supervised learning framework is used to automatically mine the training data for reducing the human cost.We design a multi-scale detection network which combines the multi-scale structure and fully convolutional process to detect various objects in higher effectiveness,even in a large scene.Moreover,the multi-scale detection network is improved through the sharing weights,which can reduce the number of parameters and make better use of the GPU.At last,the local re-recognized strategy is applied to improve the initial detection results,and then the more accurate detection results are obtained.Experiments on NWPU VHR-10 and Airports data set demonstrate the effectiveness of the proposed method.5)In this paper,we propose an object detection algorithm named Part Information Enhance-ment Network(PIEN)which consists of key part enhanced model and parts relation model.In this work,the detection method not only considers the object-level information,but also makes use of the part information including the key part information and the parts relation-ships between key part and other parts.First,the scene-level attention model is extended to the object-level attention model for generating the importance mask of object parts,and the key part enhanced information can be obtained through the importance mask and object fea-ture maps.Then,the parts relation model is designed to extract the parts relation information including spatial relationships and visual relationships of the key part and other object parts.At last,the object-level baseline combines with the key part enhanced information and parts relation information for improving the object detection.Experiments on PASCAL VOC and MS COCO datasets indicate that the part information can improve the performance of object detection with more desirable results.