Research On Model Simplification Based On High-Precision Deep Learning

Bio-inspired Deep Neural Network is widely used in the field of deep learning.Deep neural networks have given a variety of smart devices new development space.However,deep models have massive parameters and require expensive computing capacity,making it difficult to run on devices with limited computing,storage resources and endurance.This makes the deployment of neural networks on terminal devices a new problem.In order to solve the above problem,the principle of biological evolutionary selection was studied.A new algorithm and a optimized learning algorithm based on the existing model compression method were also proposed.The paper can be divided into the following aspects:1)The degree of freedom of parameters has a key influence on the model compression effect.Large-scale models often contain many parameters,and there is a high redundancy among these parameters.By reducing the freedom of these parameters,the volume of the network can be effectively reduced.Firstly,analyze the network weights' distribution to verify the high degree of freedom between the parameters.Secondly,cluster the parameters to find the core parameters of the network,and other parameters share the corresponding core parameters.The experimental results show that the reduction of parameter freedom can reduce the storage space occupied by the model to a greater extent.2)A new neural network algorithm BIO-NET.Inspired by biological principles,a new neural network algorithm combining "evolution","random" and "selection" was proposed.First,select a neural network with better performance to ensure the reconstructed network model has evolutionary capabilities;then,based on the cluster centroid values,the parameters share the centroid of the class,and pick out representative parameters;finally,based on the shared centroid,random perturbation was added to reconstruct parameters.This algorithm can greatly simplify the network while maintaining the original neural network infrastructure.We analyze the accuracy of the reconstruction model and verify the model stability for image classification and object detection models.The experimental results demonstrate the effectiveness and stability of the algorithm.3)A new BERT optimization algorithm(KD?DQ)for joint learning of distillation and quantification was proposed.BERT improves various downstream tasks' performance,but the massive parameters,complex calculations limit the application of BERT on terminal equipments.Knowledge distillation can well transfer the excellent performance of a huge teacher network to a lighter student network.Dynamic quantization reduces the number of bits represented by the parameter,thereby reducing the model calculation and storage space.This paper combines distillation and quantification to compress BERT model,and designs a small-scale,high-performance model.Different from existing algorithms,this paper performs quantization before the student network refining knowledge to improve the parameter expression of higher learning ability,so as to further optimize the learning effect of model distillation.Experimental result proves the usability and effectiveness of KD?DQ.