Research On Zero-shot Learning Algorithm Based On Semantic Relation Enhancement

Zero-shot learning(ZSL)is an important topic in the field of machine learning,which aims to train a model on seen classes to recognize objects belonged to unseen classes,with the help of semantic information shared between both the seen and the unseen classes.In recent years,as the data demand of deep learning models is rapidly increasing,the ZSL algorithms which research on how to overcome data dependency problem have become new academic hot topics.This paper mainly researches on embedding method and gating method in ZSL.To overcome the limitation existing in these methods,two ZSL algorithms based on semantic relationship enhancement are proposed.The work contents of the paper are as follows:(1)By utilizing normalization techniques in deep learning network,the embedding method CN-ZSL can suppress the fluctuations of learning signals and achieve high performance for zero-shot learning.However,it uses a simple unidirectional mapping to match the complex relationship between visual modality and semantic modality,which results in inadequate discrimination ability in embedding features.To tackle this problem,the paper proposes a novel method based on bidirectional mapping structure,named semantic regression visual embedding network(SRVE).The proposed SRVE retains the basic embedding structure of CN-ZSL,while establishes a new reverse mapping structure by utilizing a designed semantic regression network.The learning of reverse mapping structure is guided by both semantic relationship constraint and feature regression constraint,among which,the semantic relationship constraint enables the semantic regression network to contain abundant semantic information,and the feature regression constraint transfers the learned semantic information from the semantic regression network to the visual embedding network.As the semantic relationship contained in the visual embedding network is enhanced,the learned embedding features are more discriminative to be distinguished.According to the experimental results on four benchmark datasets(AWA1,AWA2,CUB,SUN),the proposed method can achieve high performance for both ZSL and GZSL tasks,and has a significant improvement compared with the baseline method.(2)By utilizing a binary discriminator as the gating mechanism,gating methods can decompose GZSL into a conventional ZSL problem and a supervised learning task,thereby leading to outstanding performance by GZSL.However,existing gating methods cannot solve the class bias problem in GZSL,prone to identify the hard visual samples of unseen classes as in-distribution samples,rather than out-of-distribution samples.To solve this problem,the paper proposes a semantic encoding out-of-distribution network(SEOOD)for GZSL.The proposed SEOOD utilizes a semantic encoding network to project visual features to their corresponding semantic attributes,to obtain the semantic-meaningful visual features for all the original visual samples.By using the semantic-meaningful visual features to learn bounded manifold for seen classes,stronger semantic relationship can be encoded to their latent distribution,thus eliminating the negative impact of class bias problem and hard samples interference in the process of out-of-distribution detection.Extensive experiments are conducted on ZSL benchmarks including AWA1,AWA2,and CUB,and the results show that the proposed SEOOD significantly improves the performance in out-of-distribution detection and achieves outstanding GZSL classification accuracy.In summary,in the field of ZSL,this paper proposes an embedding method SRVE based on bidirectional mapping structure and a gating method SEOOD based on semantic-meaningful visual features.The proposed two methods respectively utilize semantic regression network and semantic encoding network to embed stronger sematic relationship in the process of zero-shot classification.Therefore,the learned features have stronger semantic discrimination ability compared with the original features,and the proposed methods have achieved higher accuracies in benchmarks compared with the baseline methods.