Design And Implementation Of Compression And Acceleration System Of Lip Reading Model Based On Knowledge Distillation

Lip recognition,also known as lip reading,refers to the technology that only relies on the moving image sequence of the speaker’s lips to decode the corresponding text sequence.It has potential application value in security monitoring,judicial trial,individual identity authentication and other fields.Due to the phenomenon of cooperative pronunciation and individual differences,some traditional techniques have encountered more or less bottlenecks in solving lip reading problems.With the explosive blowout of massive data and the improvement of parallel computing power,many researchers turn to the deep learning technology to solve the problem of lip recognition.However,in the era of mobile terminals,it is difficult to directly migrate to resource constrained devices such as smart phones.Therefore,based on the extensive research of the current deep learning based lip reading model and the deep model compression and acceleration algorithm,combined with their advantages and disadvantages,from the perspective of real-time,considering the performance of the model,this paper innovatively introduces the knowledge distillation algorithm into the field of lip recognition,designs and implements the compression and acceleration algorithm of lip reading model based on a variety of knowledge distillation methods.The main work and innovation of this paper include:(1)In this paper,knowledge distillation method is firstly transferred to lip reading field.On the basis of the standard knowledge distillation framework which mainly aims at classification problems,the creative decoding method of step by step in the process of fusion sequence labeling is proposed,and the distillation method of step by step accumulation is proposed.A variety of comparative experiments are carried out on the self recorded Chinese data set to verify the validity of the distillation scheme;(2)In this paper,the core idea of integrated learning and promotion method is used for reference.Based on the assumption that for the network model with similar structure and similar function,if the overall effect is the best,then the high probability assumes that part of the effect is also the best,a student network based on multi teacher dynamic weight distillation is designed and implemented.The dynamic weight is integrated into the overall network model in the form of parameterization In the process of training,the subjectivity and blind obedience of artificial design weight function are avoided;(3)Aiming at the problem of information overload in teacher’s model,this paper proposes a knowledge Distillation Algorithm Based on the combination of attention in the layer and attention in the layer for the first time,which makes the student’s model focus on the more critical information of lip reading task,has a certain ability of anti information overload,improves the efficiency of data processing and the accuracy of recognition.Finally,this paper designs and implements a lightweight automatic lip reading system,and carries out simulation experiments on the self recorded Chinese sentence level data set.The performance results further verify the effectiveness and reliability of knowledge distillation method.