Research On Hard Attention Algorithms For Weakly Labeled Images

In order to solve the weakly-labeled image classification problem,traditional deep learning methods often need to encode the entire input image,unable to selectively focus on the key information part,and have high requirements on the data set,not only requiring manual labeling of the data set,And when the image data contains both foreground and background information,it is often necessary to manually add a bounding box to the foreground information,which undoubtedly consumes a lot of human,material and financial resources.By imitating the human visual attention mechanism,the attention mechanism in deep learning came into being.Hard attention mechanism,as a kind of attention mechanism,can selectively input some key information of the input image to the network for feature coding,reducing the cost of manually labeling the bounding box.Therefore,the research of hard attention algorithms for weakly labeled image recognition is an important subject.In this paper,a new hard attention algorithm is proposed to solve the problem of weakly labeled image recognition.However,in actual application scenarios,the training set(source domain)data and test set(target domain)data often have large distribution differences,making the trained classifier on the source domain unable to classify the target domain data well.At the same time,the amount of data in some fields is scarce or even does not have any labeling information.Domain adaptation can solve the distribution difference between the source and target domain datasets.By aligning the feature representations of the source and target domains,the classifier learned on the source domain data with rich labeled information can be applied to the target domain without any labeled information.This paper analyzes the domain adaptive algorithm and its research status,and improves the classification effect of the target domain image in the domain adaptation through the study of the adversarial domain adaptation algorithm based on hard attention transfer.Based on the hard attention mechanism,this paper has completed the following work:(1)From the perspective of weakly labeled image recognition,this paper analyzes the principle of attention mechanism and its related applications,proves the effectiveness of hard attention mechanism in weakly labeled image recognition,and introduces and deduces the optimization algorithm REINFORCE in the hard attention mechanism.Secondly,it introduces the research necessity and principle of domain adaptation work and its current research status.It focuses on the adversarial domain adaptation algorithm and analyzes the rationality of domain adaptation research based on hard attention transfer.(2)Inspired by the recurrent attention model,this paper proposes a new hard attention model algorithm for weakly labeled image recognition: pan-zoom.Under the guidance of the reinforcement learning reward function designed by us,this algorithm can gradually locate the most discriminative attention feature position in the weakly labeled image by panning or zooming two discrete action options,and finally identify the weakly labeled image category.In this paper,the structure and function of the network model we designed are described in detail,and data sets of weakly labeled images used in this paper are introduced.The effectiveness of the pan-zoom model is proved by experiments and analysis on weakly labeled images.(3)Inspired by the adversarial discriminative domain adaptation algorithm,this paper proposes an adversarial domain adaptation algorithm for hard attention migration combined with a recurrent attention model in view of the non-differentiability and lack of labels in the unsupervised domain adaptation process.This algorithm provides a unified framework for hard attention migration.Because the hard attention model contains non-differentiable networks,it needs to be optimized by the strategy gradient algorithm of reinforcement learning.On the target domain without labeled information,the adversarial reward function designed by the output of the discriminator network is used to optimize the process of extracting attention features from the hard attention in the target domain.In this paper,the hard attention model is regarded as a control problem,that is,a typical reinforcement learning problem.Therefore,the process of extracting the features of the source domain and the target domain of the hard attention model is regarded as the process of interaction between the source domain agent and the target domain agent and the corresponding data set respectively.Through the adversarial training between the target domain agent and the discriminator network,finally,the attention characteristics of the source and target domains are aligned,thereby realizing the migration of hard attention,and improving the classification effect of the target domain data in domain adaptation.The algorithm performs experiments on multiple sets of domain adaptation task datasets and shows visualization of hard attention transfer.