Multi-label Image Recognition Based On Deep Graph Convolution

The semantic information contained in multi-label images is extremely rich.When recognizing multi-label images,it is not only necessary to consider the complexity and challenge caused by the observation angle,illumination,the scale ratio of the image and the occlusion between the semantic objects in the image,but also to model the correlation between the semantic regions.In order to accomplish the task of multi-label image recognition,the implicit relationships between the category labels cannot be ignored.To address these issues,this thesis focuses on the following research work on multi-label image recognition methods based on deep graph convolution networks:Firstly,aiming at the problem that the previous multi-label image recognition methods only considered the correlation between the category labels in the classification process,and do not utilize the knowledge of the category relationships contained in the labels in the visual feature extraction process,a multi-label image recognition method based on knowledge fusion is proposed.First,a knowledge fusion module is designed,which uses a deep graph convolutional network to learn the relationships existing between labels in the process of visual feature extraction from the input image,and fuses the learned knowledge of label relationships with the visual features output from each convolutional stage of the Res Net-101 backbone.Then,the category label information contained in each node in the label relationship graph is aggregated by the deep graph convolutional network to learn category-associated classifiers.Finally,the learned classifiers are used to classify the visual features with label relation knowledge,and then multi-label image recognition is completed.Secondly,aiming at the problem of over-smoothing caused by only the first-order spectral graph convolution in the previous multi-label image recognition methods,and the ignoring of the importance of the node's own information in the label relation graph,a multi-label image recognition method based on adaptive multi-scale graph convolutional networks is proposed.First,the adaptive relation graph module is used to automatically construct the category label relation graph,while a multi-scale graph convolutional network in the form of block Krylov subspace is used to model the category correlation,so as to make full use of multi-scale information to make multilevel features complementary.Then,in order to make the model fully consider the importance of the nodes' own information when fusing the information of each node,the nodes are given greater weights in the adjacency matrix for self-connection,thus solving the over-smoothing problem and better mining the label correlation.Finally,the obtained class-related classifiers are applied to the image features extracted by the Res Net-101 backbone to complete the multi-label image recognition.In this thesis,the proposed methods are validated on PASCAL VOC 2007 dataset and MS-COCO 2014 dataset respectively,and the influence of relevant parameters on the recognition effect is investigated.Finally,the recognition effects of the proposed methods are compared with other methods,and the experimental results prove that the proposed methods have better multi-label image recognition performance.The thesis contains 20 figures,6 tables,and 87 references.