| In recent years convolutional neural networks(CNNs)have become the dominant approach for a variety of computer vision tasks,such as image classification,object detection,semantic segmentation.With the growth of scale of datasets,the enhancement of computing ability of hardwares,and the publication of a series of excellent network architectures,the depth of trainable CNNs becomes larger and larger.For instance,AlexNet,VGGNet,Inception Net,ResNet,and DenseNet were proposed in succession for image classification,evolving from 8 layers to over 100 layers.Deeper CNNs,although with stronger power in feature extraction,have much more parameters and require much more computing operations.For example,to classify an image with resolution 224x224,a 152-layer ResNet has more than 60M parameters and requires more than 20G float-point-operations.As a result,it is almost unlikely to deploy deep CNNs on mobile devices or Internet of Things devices.The problem of significantly reducing the numbers of parameters and computing operations,without compromising accuracy,namely,network slimming,thus attracts more and more attentions.Quite a few approaches,including low-rank decomposition,weight quantization and sparse pruning,have been proposed.In 2017,Chollet proposed the Xception network based on Inception Net by replacing the Inception modules with depthwise separable convolutions.Xception slightly outperforms Inception V3,though they have the same number of parameters.This result indicates that the depth separable convolutions may be applicable to network slimming.The main works of this thesis include:1.Apply the depth separable convolutions to the three excellent networks,ResNet,DenseNet,and PyramidNet,and propose the so-called depth separable convolutional ResNet(SResNet),depth separable convolutional DenseNet(SDeseNet),and depth separable convolutional PyramidNet(SPyramidNet).The numbers of parameters of new networks are only 53%,75%,and 59%of the original networks,respectively.2.Apply the "Squeeze-and-Excitation" blocks to SDenseNet.This results in slight performance gains,without increasing the number of parameters.3.Demonstrate that,the DenseNet can perform better by using the combination of "Conv-ReLU-BN-ReLU-Conv-BN" in its dense blocks,instead of"Conv-ReLU-BN-ReLU-Conv-BN".On CIFAR-10,CIFAR-100,and SVHN,three datasets for image classification,a set of experiments were conducted.The results show the feasibility and superiority of applying depthwise separable convolutions to network slimming. |
