| Recently,the rapid development of deep learning has boosted the performance of neural networks especially convolutional neural networks(CNNs),which demonstrate excellent per-formance in a wide range of tasks,including computer vision and natural language processing.However,the significant improvement comes at the expense of massive computation and storage consumption.With the deepening of the application field,the network that purely persue perfor-mance is no longer suitable for scenarios where limited resource,low energy consumption and real-time inference are considered.Therefore for the design and deployment of neural networks,how to reduce unnecessary parameters and computation,meet the requirement of real-time in-ference while still keeping high performance is difficult but imperative in the research of neural network.Previous works improve the efficiency of networks mainly from four different perspectives:1)Design efficient network structures,2)Apply compression methods such as pruning,quanti-zation and so on,3)Leverage dynamic pruning or quantization to exploit the input-dependent redundancy during runtime,4)Design hardware accelerator.This thesis investigates and studies techniques in prior researches,and observes some problems that should be further solved.Firstly,different compression techniques lightweight networks from variuos perspectives,but there lacks a joint-way compression method that combines those techniques to further improve the compres-sion ratio.Secondly,previous works consider the design of networks and hardware accelerator seperately,neglecting the co-optimization of software and hardware which reduce the efficiency of inference.Thirdly,structure and algorithm of compression methods are not comprehensively considered,limiting the compression ratio on efficient network structures.due to the irregular distribution of the dynamic redundancy,dynamic compression without considering the hardware architecture will result in a large gap between the realistic speedup and the theoretical compression ratio.To this end,this thesis focuses on the compression and hardware acceleration of CNNs.It first explores how to design efficient network structures and combine reasonable compression techniques.Then it studies the optimization of channel pruning from both algorithm and structure levels,how to combine various compression techniques and how to feed the hardware character-istics into the pruning.Finally,it investigates the algorithm-architecture co-design of dynamic quantization to achieve realistic speedup.This thesis includes the following three aspects:·Propose an adaptive hybrid composition-based lightweight super-resolution network(HC-SRN),which optimizes both upsampling and feature extraction modules,and utilizes chan-nel pruning for further compression.First,to compact the model,the parameter-and compute-intensive upsampling convolution layers are removed,which are replaced by the concate-nation of different level features.Second,in order to improve the efficiency of feature extraction module,the hybrid composition is proposed with multiple efficient structures,and HCSRN is built based on this basic module.Third,since manual design can not guar-antee for a given resource constraint,channel pruning is applied to derive adaptive hybrid modules with different weights on efficient structures,and a fine-grained strategy is ap-plied for residual neural networks.Evaluation results show that the proposed method can achieve a superior tradeoff between performance and compression ratio,which shows 4.3×parameters compression without significant performance loss.·Propose an acceleration aware fine-grained channel pruning framework(AFCP),which op-timizes both algorithm and structure levels,and considers hardware characteristics to im-prove the efficiency of pruning.For the algorithm level,since existing methods always adopt a single heuristic-based criterion,which neglects important channels easily,to solve this problem AFCP considers both zero-out-and similarity-based criteria,and leverages trainable parameters to learn the criterion for each channel via the residual gating func-tion.For the structure level,it is challenging to reach very deep compressions for such a coarse-grained pruning structure without significant performance degradation,and prun-ing of residual neural networks lacks an appropriate strategy for constraint channels with residual connections,thus AFCP combines decomposition and adopts a block-wise pruning strategy for residual neural networks,which extends the optimization space while main-taining regular structures.Moreover,prior works focus on eliminating ideal computations(FLOPs)without considering the hardware characteristics,which reduces the pruning effi-ciency,thus AFCP proposes the hardware predictor to feed the hardware characteristics into the procedure of pruning,which improves the efficiency of models when deployed on real hardware.Experimental results demonstrate that AFCP can achieve a favorable balance be-tween model performance and computation cost,which can achieve up to 4x compression ratio without accuracy loss.·Propose a structured precision skipping algorithm-architecture co-design SPS,which ex-ploits dynamic precision redundancy based on static quantization models,and achieves real speedup on the hardware accelerator.Due to the irregular distribution of the dynamic re-dundancy,dynamic compression without considering the hardware architecture will result in a large gap between the realistic speedup and the theoretical compression ratio.To solve this problem on the algorithm level,SPS proposes the structured dynamic block to exploit the dynamic sparsity in a structured manner.Based on the block,a budget-aware training method is proposed by inducing a budget regularization to learn the optimal model under a target resource constraint.On the hardware level,SPS presents an architecture design based on the bit-serial architecture with support for SPS models,where only a few modules with low cost are introduced,efficient scheduling and memory access strategies are pro-posed to achieve ideal speedup.Extensive evaluation results demonstrate that SPS reaches a good tradeoff between model performance and acceleration,which can achieve up to 1.5×speedup and 1.4× energy saving on various models and datasets with marginal accuracy loss. |
