Research On The Optimization Algorithm Of Generative Adversarial Networks

In recent years,due to the substantial growth of data and the improvement of computer hardware level,deep learning with GAN as the main framework has gradually become one of hot issues and achieved good results in many applications.At the same time,anomaly detection of multivariate time series data plays a crucial role in intelligent operation and maintenance as well as monitoring of power and network systems.The anomaly detection based on deep learning can mine the feature representation in the data better than the traditional machine learning algorithm,so applying GAN to the anomaly detection of multivariate time series is a very important research direction.The work of this thesis is mainly composed of three aspects:Firstly,the principle of the original GAN and its variant,WGAN,is analyzed,the existence of the optimal generator and the optimal discriminator of GAN is proved,and the handwritten number generation experiment is conducted both with the original GAN and WGAN to compare the performance of the two kinds of generative adversarial networks.Secondly,the theoretical analysis of the optimization process of GAN by the simplified model shows that the original GAN and WGAN converge to the equilibrium solution oscillatively in the process of model training.Aiming to solve the problem of training instability and training crash in the process of GAN optimization,we improve the loss function and optimization algorithm in GAN based on the results of theoretical analysis.Specially,(1)an improved loss function based on gradient penalty is proposed,which can avoid the gradient clipping in WGAN and theoretically obtain the equilibrium solution of stable convergence.(2)based on sinusoidal variation,a weighted moving average stochastic gradient descent algorithm is proposed,which can play the role of₂ regularization and alleviate the instability in the optimization process.Additionally,we applied the GAN with our improved optimization algorithm to handwritten number generation experiment and experimental results show that GAN based on improved optimization algorithm performs well in handwritten number generation experiment.Thirdly,GAN and our improved GAN are applied to detect the anomaly in multivariate time series data,and the comparison with traditional machine learning anomaly detection methods is done.On the whole,the improved GAN proposed in this thesis has achieved better results in the aspects of convergence in training,handwritten numeral generation and time series anomaly detection.