| Since the 21st century,the vigorous development of artificial intelligence technology,big data technology and internet of things technology has led to great progress in all walks of life,especially in image processing,the research and development of object recognition,face recognition,image generation,3D graphics processing and other technologies has brought earth-shaking changes to human society.Behind these changes is the technical support of massive amounts of data.How to deal with these data is a hot topic for scholars.In recent years,the research enthusiasm of deep learning has remained high.Its wide application stems from the processing capabilities of massive data,the powerful feature decomposition and feature fusion capabilities of the deep network structure.According to different learning content,deep learning models can be divided into discriminant models and generative models.The discriminant model predicts results by learning the decision function（decision surface）or conditional probability distribution,and studies data differences;the generative model can reflect the distribution of data and study data similarity by learning the joint probability distribution.Research on deep generative networks has a significant impact on actual work.This is because first of all in the natural world,most of the data is unlabeled,and the generative model can extract features from unlabeled data for unsupervised learning,or for semi-supervised learning combined with data labels,thereby reducing the difficulty of obtaining data samples.Secondly,the deep generation network can learn high-level features,which can be widely used in data generation,object recognition,information acquisition,classification and regression tasks.Generally,deep generative networks include deep belief network,generative adversarial network,and variational auto-encoder.Through the unremitting efforts of scientific researchers,these models have been widely used in unsupervised and semi-supervised learning.Nevertheless,there are still a lot of unsolved problems worthy of further exploration.（1）Generative network learns a large amount of data to fit the real data distribution.The difficulty of fitting is related to the amount of data and task complexity.With limited number of data,problems such as"mode collapse"often occur,it is difficult to balance the accuracy and diversity of fitting;（2）With the continuous expansion of the number and width of the deep neural network,the number of activated neuron nodes is huge,which not only causes data redundancy and waste of resources,but also affects learning efficiency and accuracy;（3）The learning process of the deep generative network is similar to a black box operation and lacks theoretical support for interpretability.Based on the above,this paper mainly focuses on the key issues in the deep generative network.The main work and contributions are as follows:First,the"mode collapse"problem in the deep generation network is that the network does not restrict data,diversity,which leads to the collapse of the generated data to a certain part of the true distribution,which will cause the generated image duplication and lack of diversity,thereby affecting Image generation quality.In response to this problem,we have proposed two different mechanisms to modify the network structure,which greatly improved the influence of"mode collapse"problem.We propose a deep generation network structure in which collaboration and adversarial coexist.The collaboration mechanism encourages mutual cooperation between generative networks,learns different distribution patterns of real data,and ensures diversity;the adversarial mechanism continuously improves the generation and discrimination capabilities of the network.During the training process,the two mechanisms of cooperation and adversarial are alternated,thereby curbing the problem of"mode collapse",strengthening the fitting ability of the network,and further improving the quality of image generation.Furthermore,we combined with evolutionary calculation,the particle swarm optimization is used to optimize the network parameters of the generator,the inertia weight method of the particle swarm algorithm is improved,and the fitness function of particle swarm optimization is designed from the two aspects of generation quality and generation diversity.Suppress the limitations of the original GAN network,such as gradient disappearance and mode collapse,and further improve the robustness and quality of the generative adversarial network.Second,although a large number of studies have shown that with sufficient data and sufficient training,the deep generative network can perfectly fit the real data distribution,but this process lacks sufficient theoretical support.Research on generative adversarial network only explains the fitting process of the network through adversary theory and Jensen-Shannon divergence.It is almost impossible to find the optimal randomly in the entire feature space.Through the study of Bayesian harmony theory,we found that it can be introduced into the generative adversarial network,adding a process of calculating expectations in addition to the original fitting maximization process,the two processes are iterated alternately,and controlled by constructing a Bayesian harmony function.Bayesian harmony theory can not only enhance the learning ability of deep generative network,but also provide strong theoretical support for the data fitting process.Third,in the non-grid data processing,existing models generally assume that the node features can be fully observed.This may violate the fact that many real applications come with only the pairwise relationships and the corresponding node features are unavailable.In this paper,a novel graph convolutional generative model based on Bayesian framework is proposed to handle the graph node classification task without node features.Our method equips the graph node with the pseudo features generated from variational auto-encoder.When analyzing the effect that generated stochastic features cannot maintain the independent and identically distributed property between the training and testing dataset,a hidden space structure preservation term is proposed and embedded into the generation process.Moreover,when dealing with the graph node classification task with features,a concatenating strategy is constructed which extends our model to the graph node classification task with features and without features in a unified framework.Although the model inference is hard,we derive an efficient training and predication algorithm via the variational inference.Experiments on different datasets demonstrate the proposed method can significantly outperform traditional methods.Fourth,by observing the activation state of neuron nodes in the deep learning network,we found that a large number of neurons fluctuate frequently between active and inactive states,and only stabilized near the end of training.This not only slows down the network convergence,but also wastes a lot of resources.The sparsity method can effectively reduce the fire rate of network nodes and speed up the convergence.For this reason,we designed a sparsity method that combines local and global operators.The local sparse operator is used to reduce the fire rate of a single node,and the global sparse operator reduces the fire rate of the entire hidden layer to a specific value.There is an irreconcilable contradiction in the use of a single operator,but the combination of the two can solve problems.At the same time,L1/2 norms increased the sparsity of the network,which ultimately benefits network convergence,computing resources and learning accuracy. |
