Image Semantic Analysis Based On Feature Fusion And Regularization

Image semantic analysis is a collective term for a series of methods to bridge the semantic gap between low-level features of images and high-level semantics.It is an important branch of image understanding research and one of the hot topics in current computer vision research.With the increasing ap-plication of image understanding in the current Internet,multimedia and other fields,multimedia data such as images and videos are increasing day by day,and the demand for image semantic analysis of related applications has become more prominent.The large demand for image semantic analysis comes from two aspects.First,the management and retrieval of a large amount of image data requires automatic labeling of the image.Second,to understand the content of an image,you need to perform processing such as recognition and segmenta-tion on the elements or objects in the image.Due to the different requirements of various application scenarios,the specific manifestations of image semantic analysis problems are also complex and diverse,coupled with the complexity of the image semantic analysis problems themselves,which makes it impossi-ble for existing image semantic analysis methods to be well performed in the scenarios used.Semantic labeling tasks and semantic segmentation tasks face many challenges from both an academic perspective and an industrial develop-ment direction.Therefore,although the research on image semantic analysis has been progressing,it still has many unresolved problems and huge research space.The research on image semantic analysis method has important theoret-ical significance and practical value.This thesis studies in depth the effectiveness and performance improve-ment of two types of methods in computer vision image semantic analysis,which are feature fusion method and regularization method.Focusing on the feature fusion method,feature fusion at different levels is discussed:including shallow feature fusion deep models,deep learning features and shallow model fusion.Secondly,for the problem of model generalization in image semantic analysis,the application of two types of regularization methods in image clas-sification and image segmentation is discussed.Whether it is a feature fusion method or a regularization method,the application in image semantic analysis is not achieved overnight,and there is no general paradigm to follow.Specific problems and models need to be addressed,such as the adaptation of features to models,the integration of different features,and the selection of regulariza-tion methods,as well as the design of error functions and regularization terms.In view of these specific problems and challenges,this thesis focuses on the following aspects of image semantic analysis:This thesis first discusses and analyzes the problems and challenges in image semantic analysis,summarizes the current basic framework of image semantic analysis,and lists some important research directions and application problems in image semantic analysis.In view of some key technical links in automatic image annotation and image semantic segmentation,the methods and ideas of some existing research results are analyzed.Based on the above investigation of existing work,this thesis studies an automatic image annotation method based on the fusion of MFoM and deep neural network systems.Based on the MFoM learning framework,a method for directly solving the maximization of MAP method,the AP approximation of each individual sample score is solved as a step function.Compared with the pairwise ranking approximation method,our AP gradient approximation scheme significantly reduces the computational complexity;in view of DNN classifier has a strong recognition ability in image classification.The method proposed in this thesis uses MAP as the objective function and optimizes it by training of deep neural networks.Combining the MAP method with the DNN technology and introducing non-linear elements into LDF to improve the original LDF based on MFoM training The flexibility and discrimination of the classifier.Experimental results show that this method achieves better results than other schemes.Secondly,an image annotation method based on manual design feature fusion and in-depth learning feature is proposed.Fusion of low-level color features and in-depth learning features learned from CNN from the original image.The set of these two features is used as input to train in AIA system of deep neural network.The experimental results of Cifar-10 and Corel-5K for single-label and multi-label datasets show that the proposed method can effectively integrate manual design features and in-depth learning features to improve annotation performance.Thirdly,we propose an instance-embedding regularizer that can boost the performance of both instance-and bag-embedding learning in a unified fash-ion.this method learns more robust instance and bag embeddings by considering the relationship between instances as a regularizer,i.e.,maximizing the correla-tion between instance-embedding similarities and similarities of the underlying instance labels.The proposed instance-embedding regularizer together with bag classification loss and instance classification loss were optimized using the stochastic gradient descent method in an end-to-end manner.We conducted a number of experiments on datasets of drug molecular activity prediction,image classification,text classification,and cancer prediction.The results show that the proposed method achieved a significant improvement over previous multi-instance networks.Finally,this thesis studies the image segmentation algorithm.A multi-level feature fusion image semantics segmentation method based on depth neu-ral network is proposed,which integrates CRF-based HED and global smooth-ing regularization.It is an end-to-end and pixel-to-pixel deep convolution net-work,which can get better results than HED based method and CRF inference as post-processing method.The experimental results on three sets of retinal vascu-lar image data show that the proposed multi-level feature fusion can better map the deep features,and the global regularization method of CRF is optimized.Compared with other advanced algorithms,the proposed DSSRN achieves the best performance.To sum up,this thesis proposes several image semantic analysis methods based on different feature fusion strategies and regularization methods for the two research issues of image semantic annotation and image semantic segmen-tation in image analysis.It has improved significantly and made some contri-butions to the research progress of image semantic analysis.