Going Deeper With Transposed Convolutions

Aimed at designing a more effective neural network structure for generative adversarial nets(GANs),new generator structures,Inception-trans-Res Nets,have been proposed,which can be used to implement GANs or GANs' derived models for generating pictures.Mainly consisting of convolutions and transposed convolutions that have sparse connection and shared weights,Inception-trans-Res Nets have more hidden layers,more transposed convolution kernels but less parameters.Inception-trans-Res Nets can recover multi-scale features of pictures in a single hidden layer through parallel transposed convolutions,which lead to the diversity of features in the generated pictures.Factorizing transposed convolutions with large filter size into smaller transposed convolutions has been used in the Inception-trans-Res Nets to decrease parameters.1×1 convolutions has been used to increase the ability of the Inception-trans-Res Nets to fit non-linear functions.Further more,1×1 convolutions are also in conformity with Hebbian principle since they connect the output that are highly correlated of each hidden layer together.As regularization,batch normalization has been used a lot in the Inception-trans-Res Nets to avert Internal Covariate Shift.Meanwhile,batch normalization can stable the training process and accelerate the convergence of training algorithm.To avert degradation problem,residual blocks have been added into the Inception-trans-Res Nets,making it easier for the networks to be optimized by stochastic gradient descent.In addition,Inception-trans-Res Nets is modularized so it is convenient for users to set hyper-parameters of the nets for generating pictures with different size or types.To exam the advantages of Inception-trans-Res Nets that have been said as above,different types of experiments has been done and the results show that though Inception-trans-Res Nets have less parameters,but they are deeper and wider and they can generate higher quality pictures or increase the Inception Score of the generated pictures without tuning hyper-parameters.Further more,in no experiment did Inceptiontrans-Res Nets be observed that they were unstable to train or mode collapse.