Research On Handwriting Verification Based On Multi-modal Fusion

With the continuous improvement of information technology,it is common that there are many incidents of information leakage in the construction of network space environment.It is increasingly imminent to establish a forceful protective barrier for people's information security,and the protection of identity authentication information can not be ignored.From the two aspects of fusing multimodal handwriting features and weakening the influence of handwritten content,this dissertation has researched on the identity authentication technology based on the analysis of handwritten behavior characteristics to effectively resist the skilled forgery attack in handwriting verification.Also,this dissertation considered the performance and security of practical application requirements of handwriting authentication system to protect the safety of the user au-thentication information and network application space.But there are still some prob-lems and limitations of current research of handwriting verification,such as the fact that single mode handwriting cannot fully characterize writers' handwritten behavior features of the handwritten process and the traditional handwriting verification with the name as handwritten content can easily be learned by forgers.To solve these problems,this dissertation proposes the corresponding solutions and analysis.The main contents and contributions of this dissertation are as follows:1.A dynamic and static handwriting feature fusion model is proposed.Aiming at the limitations of static handwriting and dynamic handwriting,this dissertation proposes a scheme to improve the performance of handwriting verification system by combining the characteristics of the two modal handwriting.In this dissertation,the convolutional neural network is used as the basic framework of the model,and the spatial and tem-poral correlation of dynamic and static handwriting are considered.The fusion scheme of dynamic and static handwriting features is designed from three aspects of data level,different dimension feature space of network model and high dimension embedded fea-tures.In addition,the Arcface loss function,which has been successfully applied in face recognition research,is introduced in this dissertation to further improve the per-formance of the network model.According to the experimental results on public data sets and self-collected data sets,the proposed scheme in this dissertation is verified to be able to effectively improve the performance of handwriting verification system.2.A content-independent handwriting verification method is proposed.In the practical application environment,the accuracy of identity authentication is very impor-tant.At the same time,the attacker will obtain the identity authentication information through copying and replaying attacks,thus posing as legitimate users to access the ap-plication environment and endangering the security of the network space.Therefore,it is crucial to protect the identity information.To solve the above problems,this disserta-tion designs a content-indenpendent handwriting style feature extraction model.From three perspectives of data augmentation,style attention mechanism and loss function construction,we design to reduce the influence of handwritten content,so that hand-writing feature extraction model will focus on handwriting style features which are un-related to handwritten content.Thus,the handwriting style which is not easy to be obtained and imitated is used as the main basis to identify the authenticity of handwrit-ing.The feasibility and accuracy of the proposed scheme are verified by comparative experiments on public data sets and self-collected data sets.