Research On Semantic Segmentation Method Based On Generative Adversarial Networks

Image Semantic Segmentation is one of the research hotspots of computer vision.It is the basis of image analysis and image understanding,and it has been used in many fields,such as car driverless(street scene recognition and understanding),drone application(landing judgment and aerial photography),wearable devices(virtual Reality,human-computer interaction,medical treatment and other practical scenarios.In recent years,with the continuous improvement of convolutional neural networks in deep learning,semantic segmentation has also made breakthrough progress.The semi-supervised/weakly supervised method based on GAN(Generative Adversarial Networks)has been widely used in the field of image segmentation because it saves the labeling cost of a large number of data sets,but the existing methods all use the fixed penalty factor to counter the loss in the discriminator.For supervised learning,the model lacks generalization ability,and the segmentation is not fine enough.It is easy to cause serious class infection and class drift when segmenting more complex scenes.Aimed on these issues,this paper has studied related research work and achieved the following results:1.Recent deep networks achieved state of the art performance on a variety of semantic segmentation tasks.Despite such progress,these models rely on supervision with pixel-level ground truth which is time-consuming and labor-intensive for labeling.And these models may not generalize well to unseen image domains.So it is very important to seek model based on domain adaption to overcome this problem without target domain labels.Semi-supervised methods based on GAN(Generative Adversarial Networks)is widely used in the field of image segmentation due to save the annotation cost on a large data set,however the fixed penalty factor is introduced to the model in the adversarial learning of different feature layers,and the FCN(Fully Convolutional Networks)is used as the basis framework of discriminators.Which will result in low generalization ability of the model and the segmentation result is not as fine as expected;furthermore,it is easy to cause the problem of class infection and class drift,which exists in the latest domain adaptive method in more complex scenarios.To address this issue,a new semi-supervised semantics segmentation method based on GAN network is proposed for urban scene segmentation in this paper.For the discriminator,we use architecture similar to SegNet and utilize all fully convolutional layers to retain the spatial information,which adopts maximum pooling method to up-sample image nonlinearly,which carries on the advantage of FCN to process any scale input image as well as to preserve refined feature correlation information.For segmentation network of generator,we use both source dataset with label and target dataset without label to train the generator segmentation network.Moreover,the distribution difference between the output feature map of the segmentation network and the Ground Truth of the source domain labels by adopting three different measures simultaneously.Among the three measures,one is to adopt the cross-entropy loss from the Ground Truth of the labeled dataset and the output of the segmentation network,another one is to add the adversarial loss from the discrimination network,and the last one is to control the adversarial loss by adjusting the adaptive learning rate dynamically.That is,an adaptive learning rate is employed to adjust deep neural network features in different layers in the processing of fusion adversarial loss and cross-entropy loss.Result On one hand,the presented GAN model improves sematic segmentation precision,which is capable of adjusting the weight of adversarial loss and cross-entropy loss through adaptive learning rate to update the segmentation net in the generator.On the other hand,the provided model refines the segmentation result,because SegNet takes the place of FCN in the discriminator part of GAN to get rid of violence pooling.Furthermore,edge information of unlabeled target dataset is imported in the networks.As a result,the margin area in the net is corrected effectively and the edge information in the image is preserved as far as possible.Conclusion The results show that the proposed discriminator can improve the performance of semantics segmentation by coupling the adversarial loss and the standard cross-entropy loss in the segmented network.In addition,the full convolution discriminator provides additional supervisory signals by discovering the credible regions in the predicted results of unlabeled images,thus realizing semi-supervised learning.Compared with the existing methods using weak labeled images,our method uses unlabeled images to enhance the segmentation model.The validation results on PASCAL VOC2012 standard data set show that the proposed model is capable of segmenting object in more complex scenarios,relieving class infections and class drift,moreover enhance the edge detail effectively.2.AdaptSegNet(Adaptive Segmentation Network)achieves good results in semantic segmentation based on semi/weak-supervisor.However,the fixed penalty factor is introduced to the model in the adversarial learning of different feature layers,and the model is trained directly based on the synthetic data set GTA5 and the real data set CityScapes which are different in distribution,so the segmentation accuracy is not so ideal.To address this problem,an adversarial network is proposed for urban scene segmentation,which is capable of adjusting deep neural network features in different layers based on adapt learning rate and domain simultaneously.Firstly,SG-GAN(Semantic-aware Grad-GAN)method is used to train the synthetic data set,which makes the newly generated synthetic data set SG-GTA5 closer to the real scene data set in color and texture,and is suitable to substitute the original data set GTA5 in AdaptSegNet.Then,adapt learning rate scheme is adopted to adjust loss value and neural network parameters of multi-layers.In addition,a convolution layer is added into the discriminator part in order to improve the discriminant ability of the GAN network.Finally,the algorithm is validated on CityScapes data set and compared with several current popular semi-supervised segmentation ones.The experimental results show that the proposed model has a 0.7% improvement in segmentation accuracy of MIoU(Mean Intersection over Union)compared with the latest AdaptSegNet algorithm.The segmentation accuracy of big objects like road,sidewalk and car is improved by nearly 5%.