Knowledge Distillation For Generative Adversarial Networks Based On Weight Decay

Generative Adversarial Networks(GAN)have been extensively researched and developed in the field of artificial intelligence content generation and are widely used for various image generation and editing tasks.However,when deploying them on devices with limited hardware resources,it is often necessary to simplify the structure of the generator and improve the generation quality through Knowledge Distillation.But previous studies mostly only improved the way of acquiring knowledge,ignoring the differences between the training objectives of Knowledge Distillation and GAN.The thesis analyzes and improves the training strategy based on the characteristics of GAN,specifically proposing a Knowledge Distillation method based on weight decay.The thesis analyzed the role of Knowledge Distillation in the training of GAN through experiments and found that Knowledge Distillation based on the similarity of generated images has certain defects.Since the similarity required by Knowledge Distillation is not completely consistent with the authenticity required by adversarial training,the effectiveness of Knowledge Distillation will decline as the performance of the student network increases.Therefore,the paper restricts the impact of Knowledge Distillation during training by timely reducing its weight in the loss function to avoid its erroneous guidance to the student network.In addition,a more efficient Style GAN generator was designed using Neural Architecture Search,and it was found that the structure of the student network largely determines the result of Knowledge Distillation and can generate higher-quality images while meeting the requirements of computational complexity.The thesis conducted multiple validation experiments on the FFHQ dataset and LSUNchurch dataset.Compared with previous knowledge distillation methods,the method proposed can achieve better image generation results.On the FFHQ dataset,the model trained in the thesis improved the FID image quality evaluation index by 26% compared to existing methods,and by 43% on the LSUN-church dataset.And the Neural Architecture Search can further improve the performance of the generator,making it close to the generation level of the original Style GAN model.