Design And Implementation Of Lightweight Learning Network Based On Attention Mechanism

With the development of scientific progress,as the main technology of artificial intelligence,deep learning has been widely concerned by researchers.However,the current traditional deep convolution neural network often has more layers,larger models and more parameters,the demand for actual computing resources is large,and the computing cost is very expensive.In addition,such a complex network is difficult to deploy on mobile devices with limited computing resources.Therefore,convolutional neural network technology is difficult to deploy to embedded mobile devices.In order to solve these problems,how to make the network smaller under the condition of ensuring the effect has become the key research direction of researchers.Aiming at the research of lightweight network design,this paper designs a new lightweight network structure,quantifies its parameters,and successfully deploys it on mobile devices.The main work of this paper is as follows:Firstly,this paper proposes a lightweight residual block network based on attention mechanism.In this algorithm,the problem of feature extraction ability is solved by adding channel attention mechanism and spatial attention mechanism,which solves the problem of the ability of residual block to capture context information.The BasicBlock which is a kind of basic module is proposed to enhance the expression ability of features.In addition,based on Mobile Net V2,the paper optimizes the number and position of the reduction sampling of pool layer and convolution layer,avoids the loss of the network features in the key position,prevents the final feature map size from too low,and constructs a new lightweight network model combined with the BasicBlock with strong feature expression ability.According to the experimental results,the accuracy of the network structure proposed in this paper can reach 93.84% on cifar-10,and the numbers of parameters is 1.43 M.Meanwhile,FLOPs is only 0.28 G,only a few parameters are used to achieve good results.At the same time,the accuracy can reach 73.51% on cifar-100,and the parameter quantity is only 1.45 M,FLOPs is only 0.28 G,and the algorithm in this paper is also excellent in 100 classification scenarios,and has certain robustness.Next,in order to further reduce the network parameters,this paper proposes a network compression algorithm based on dimension rising ghost residual block and parameter quantization to further optimize the previous network.In order to solve the problem of feature redundancy in conventional convolution,this algorithm proposes an ascending dimension ghost residual block,which greatly reduces the amount of network parameters,increases the flexibility of the module,and enhances the richness of residual block feature extraction.In addition,aiming at the problem of negative information loss of Re LU function,this paper introduces the Mish function as the activation function of the network,which stabilizes the gradient flow of the network and enhances the generalization ability and overall effect of the network.Finally,the parameters of the network are quantified,and the INT8 type is used to store and calculate the parameters of the network,so as to further reduce the memory consumption of the model.According to the experimental results,on cifar-10,the number of parameters of the optimized network is 0.55 M,FLOPs is 129 M,and the memory occupied by the model is only 2.18 M,which are less than 40% of the unoptimized model,and the accuracy of the model is only reduced by 1%;on cifar-100,the number of parameters of the optimized network is 0.56 M,FLOPs is 129 M,and the memory occupied by the model is only 2.21 M and the accuracy of the method is less decreased.The method proposed in this paper further compresses the network parameters,reduces flops,can achieve good results,and all indicators surpass Mobilenet V2.Then,this paper quantifies the network,and successfully compresses the network model to about 1/3 to 1/4 of the original model.Finally,this paper transplants the quantized network model to Android mobile phone,and compiles the Android functional interface.The real-time detection frames per second of iqoo7 mobile phone equipped with snapdragon 888 processor reach more than 80 frames,which can be applied to the real-time image classification scene of Android mobile terminal.