| With the advent of the era of big data,massive amounts of data are generated in people’s daily life.In order to process diverse data and face various complex application scenarios,the neural network model continues to deepen,which brings certain challenges to the computing and storage performance of hardware.On the one hand,the deep neural network increases the time cost of model training.On a single device,it often takes days or even weeks to complete the training of deep neural network.On the other hand,deep neural network is difficult to deploy on some resource-constrained equipment.Moreover,the reasoning speed is slow and can not meet some application scenarios with high real-time requirements.Therefore,a certain acceleration strategy is needed to accelerate the training and reasoning process of the deep neural network model.In order to speed up the training of deep neural networks,parallel training methods are often used.Common parallel methods include data parallel and model parallel.Data parallelism requires computing devices to store complete models,which has certain limitations.In addition,the traffic is large in large models.Improper model partitioning algorithm of model parallel will make the load of computing equipment unbalanced and the global traffic large.If the model parallel is used separately,the utilization rate of computing devices is also low.In order to speed up the reasoning of the model,the method of model compression is often used,but the traditional compression method will make the precision of the model lose greatly under high compression rate.Aiming at the above problems,this paper studies the parallel training and model compression of deep neural network.The main work is as follows:(1)A pipeline parallel training method for optimizing model partition.Firstly,the neural network is modeled as a directed acyclic graph.The weights of vertices and edges in the graph represent the theoretical calculation time of the model layer and the theoretical communication time between layers.Then an optimized model partitioning algorithm is proposed.From the perspective of load balancing of computing devices and global communication time,the model is divided.Then pipelined parallelism is introduced to improve the throughput of model training and speed up the training stage.Experimental results show that the optimized model partitioning algorithm proposed in this paper has better acceleration effect than other model partitioning algorithms.After the introduction of pipeline strategy,the training process of neural network model is further accelerated.(2)Model pruning method based on two-dimension attention mechanism.Firstly,the original neural network is embedded with the spatial and channel dual-dimensional attention module,and the redundant channels are removed according to the weight metric generated by the sum of the attention weights.While ensuring a certain model accuracy,the model is compressed to accelerate the model reasoning stage.Experimental results show that our model pruning method is more accurate than other pruning methods under the same compression rate.(3)The acceleration strategy proposed in this paper is extended to the application of helmet detection.Center Net algorithm is adopted as the target detection algorithm.The model is trained in parallel on the cloud server,and the compressed model is deployed on the PC.Experiments show that the acceleration strategy in this paper can accelerate the speed of model training and reasoning while ensuring a certain target detection performance. |
