Research Of RGB-D Image Feature Learning Based On Sparse Representation

RGB-D(color and depth)image classification is one of the hot spots in the field of computer vision.The depth of the image can directly reflect the 3D feature of object surface which can overcome the disadvantage of illumination changes,occlusion,shadows and changes in the complex environment,so the color and depth image fusion can improve the accuracy of image classification.However,for large-scale and complex scene RGB-D image classification problems,the traditional feature extraction methods often require hand-design features,and it is difficult to extract and identify the high performance of sparse features,affecting the classification performance.With sparse representation used in many fields rapidly,feature representation based on sparse representation has good robustness,good generalization ability and strong anti-interference ability.Deep learning is a deep-layer nonlinear neural network,which can extract the essential feature from the input data.The appearance of new approaches provides the new direction for feature extraction and classification.With research on rapid and effective RGB-D image sparse representation combining with deep learning method,the efficient features automatic learning methods of RBG-D image are developed.The features learning methods were applied in classification of fruit recognition and have important theoretical and practical significance,which not only will greatly promote the image classification process,and also promote agriculture computing machine vision application development.The dissertation dedicates to the research of feature representation and deep learning based on RGB-D image classification and recognition for the purpose of get effective vision features.By analyzing the current feature presentation and learning methods,we proposed new feature learning approaches and applied to solve the problem of RGB-D classification and fruit maturity discrimination.The main work and innovations of thisdissertation are as follow:(1)We propose a feature extraction method for RGB-D images based on locality-constrained linear coding(LLC).Aiming at the problem of slow speed and lack of locality in classical sparse coding,we extract SIFT features from the RGB image and Depth image and conduct feature coding using LLC method respectively.Compared with traditional sparse representation,the features representation method of LLC consider the sparsity and locality of the feature,which more benefits for image classification and recognition,and the experiments on RGB-D dataset and Fruit category datasets show the efficacy of LLC method.(2)We study an improved structured group sparse representation method via combining class label of image for feature representation,which enforces discriminability of dictionary by introducing an ideal regularization term to optimization problem.To solve the corresponding optimization problems,we develop efficient algorithm based on alternating direction method of multipliers method.For distinguishing the high similarity RGB-D images,the method is utilized to incorporate the label information and it promotes sparisity at the group level which realizes the maximum distance inter-class and minimum distance inner-class.Experimental results show that our approach method achieves the state-of-the-art recognition results on a household RGB-D object dataset with high shape similarity and outperforms than the present other sparse representation methods.(3)We propose a model to learn better feature representation by combing convolutional neural networks(CNN)and block group sparse coding(BGSC).Traditional feature representation method extracts the local features of image by the manual design method generally,which depend on expert experience and knowledge and the lack of feature extraction of general.The CNN method can automatically extract the image edge,texture and color features from source images directly,and can obtain the global and local feature fusion.However,it is necessary to further use very little data to capture the important information of CNN.The group sparse representation is combined with the L1-norm constraint in sparse representation and the L2-norm constraint in ridge regression,and the inter-group of sparse representation is as sparse as possible,while keeping the error of the intra-group as little as possible.Compared with sparse representation,this method can obtain better performance of image classification by fusing the features of CNN automatic learning and group effects of groups sparse.(4)We propose a model of Deep Learning-Group Sparse Coding(DL-GSC)to learn hierarchical feature representation from raw RGB-D by combing deep learning and group sparse coding.Firstly,the method of deep learning is trained to learn features from raw RGB-D images directly by unsupervised learning algorithm.Then the higher robust hierarchical feature representations can be learned adopting group sparse coding method and improved dictionary learning.Finally,we implement RGB-D classification by softmax algorithm.The proposed model is evaluated on RGB-D dataset.Experimental results demonstrate that DL-BGC approach can distinguish the high similarity RGB-D images and achieve better accuracy compared with other feature learning algorithms,and the RGB-D introduced to record high quality RGB and depth information can dramatically improve classification accuracy.(5)The system of passion fruit target detection and classification of maturity were designed and implemented under natural scenes.Firstly,we established passion fruit RGB-D database from the natural scene by the kinect2.0.Then,we developed passion fruit detection system adopting Ho G feature and Adaboost cascade method,the detection system detection accuracy reached 82.9%.Finally,we developed the color SIFT feature fusion Sc SPM,color SIFT feature fusion LLC and CNN feature learning methods to identify different growth stages of passion fruit,and color SIFT feature fusion LLC method can obtain the best classification accuracy(91.5%),which can satisfy the demand for classifying growth stages of passion fruit under natural scenes.In conclusion,through above work,it turns out that effective feature representation methods could improve the performance of image classification and recognition greatly.The feature learning method based on CNN architectures could learn feature from the raw image data avoiding the complexity of the design of the hand-crafted features.The featureextraction method based on sparse representation and deep learning can obtain high accuracy in image recognition,and the feature learning is a very frontier research direction and has wide application prospects.