Zero-shot Chinese Character Recognition With Stroke-and Radical-Level Decompositions

The rich and precise semantic information carried by Chinese characters is widely used in people’s daily life and work,and plays a very important role in scene understanding.With the development of deep learning,Chinese character recognition technology has emerged in many artificial intelligence application scenarios.In particular,zero-shot Chinese character recognition has attracted a lot of interest from researchers because of its potential applications in important fields such as autonomous driving,robot navigation,and ancient book recognition.Existing zero-shot Chinese character recognition methods mainly use the extracted Chinese character component information to achieve the recognition of unseen Chinese characters.However,the performance of existing methods for zero-shot Chinese character recognition in practical applications involving low-frequency characters,complex scenes,and scribbled handwriting is still unsatisfactory.To address the above problems,this paper proposes two effective zero-shot Chinese character recognition methods based on stroke and radical decomposition.The main contributions of this paper are as follows.(1)To address the problem of poor recognition performance of existing methods,this paper proposes an effective zero-shot Chinese character recognition method based on stroke and radical decomposition(called STAR for short),inspired by the fact that stroke level and radical level decomposition of Chinese characters can provide different levels of information.In the training phase,the proposed method obtains the stroke codes with the help of both stroke decomposition and radical decomposition information,while in the inference phase,the proposed method achieves zero-shot recognition of Chinese characters by matching the obtained stroke codes.In order to utilize the stroke decomposition and radical decomposition information more effectively,a similarity loss is introduced in the training phase to constrain the semantic consistency of the stroke and radical information representations;in order to improve the inference performance,an effective stroke revision scheme is proposed in the inference phase to expand the final candidate character set.The effectiveness of the proposed method is verified on three benchmark datasets covering handwritten,printed art and street scenes.The experimental results show that the proposed method outperforms existing methods in both the character and radical zero-shot cases,and remains quite competitive in the traditional visible character case.(2)To address the problem of insufficient utilization of stroke and radical information in the training phase and the "one-to-many" recognition problem in the inference phase of the proposed STAR method,this paper proposes an improved method based on the attention and integration mechanism(called SRAE for short).Specifically,in the training stage,we introduce an effective attention mechanism to deeply integrate the information of strokes and radicals to provide effective guidance for the decoding process of strokes and radicals,while in the inference stage,we introduce an effective integration mechanism to further improve the inference results of the strokes and radicals inference module,thus greatly alleviating the "one-to-many" recognition problem of the stroke-based decomposition method.The experimental results show that the proposed improved model outperforms the existing method in zero-shot experiments for both characters and radicals,and remains competitive in the traditional visible character case.In particular,the improved method achieves better accuracy in some challenging tasks involving cursive characters and fuzzy occlusion.