Research On Construction,Inference And Applications Of Deep Dynamic Probability Models

In recent years,deep learning,a prominent representative of artificial intelligence technology,has closely borrowed from the hierarchical information processing mechanism of the human brain and made important breakthroughs in computing and intelligent simulation capabilities.At present,research on how to use deep learning to build models and learn effective features from data in various applications is of great significance to both national defense and people's livelihood.In this paper,we firstly study how to use deep learning to extract multi-layer features of SAR images for target recognition,where each sample in the data is assumed obey to independent and identically distributed.In addition,there is an obvious time series relationship between the samples in real applications,such as weather forecast,natural language processing,disease prediction,International Relations Analysis,signal generation,etc.Since a lot of valuable information is lost when the time sequence of the sample is ignored,building a deep dynamic model to characterize the time sequence relationship of the data and obtaining a more accurate description of the data is an important research direction in deep learning.This dissertation also focus on the problem of temporality of data samples,starting from deep learning,combining probability and statistical models,constructing deep probabilistic dynamic models for radar target recognition,international relations analysis,natural language processing and other applications,while developing fast inference algorithms.1 The research of RATR in high-resolution wideband radar can be divided into two categories,including RATR based on high-resolution range profile(HRRP)and that based on synthetic aperture radar(SAR)or inverse SAR(ISAR).SAR is widely used in military and civil fields for its advantages of full-time operation,all-weather ability and robustness to environmental conditions.Since SAR image contains abundant discriminative information,such as shape,scatterer distribution,automatic target recognition(ATR)of SAR images has obtained intensive attention.Feature extraction is one of the key steps for SAR image target recognition,which directly affects the recognition performance.Some researchers elaborately designed “hand-crafted”feature extractors or complicated classification systems,requiring large effort from human experts.Besides,they ignored the human learning system,which can extract hierarchical representations and be helpful for the interpretation of SAR images.In this section,to build a flexible and interpretable model for SAR image target recognition task,we apply a deep variational auto-encoding model(DVAEM)that constructs a multi-stochastic-layer generative network(decoder)and variational inference network(encoder).It is scalable in training phase and fast in testing stage,and has the ability to extract the hierarchical structured and interpretable features from SAR images.To jointly model SAR images and their corresponding labels,we further propose supervised DVAEM with Euclidean distance restriction,which enhances the discriminative power of its latent representations.Experimental results on moving and stationary target acquisition and recognition(MSTAR)public dataset demonstrate the effectiveness of the proposed method.2 The need to model time-varying count vectors appears in a wide variety of settings,such as text analysis,international relation study,social interaction understanding,and natural language processing.To model these count data,it is important to not only consider the sparsity of high-dimensional data and robustness to over-dispersed temporal patterns,but also capture complex dependencies both within and across time steps.In this section,We develop deep Poisson-gamma dynamical systems(DPGDS)to model sequentially observed multivariate count data,improving previously proposed models by not only mining deep hierarchical latent structure from the data,but also capturing both first-order and long-range temporal dependencies.Using sophisticated but simple-to-implement data augmentation techniques,we derived closed-form Gibbs sampling update equations by first backward and upward propagating auxiliary latent counts,and then forward and downward sampling latent variables.Moreover,we develop stochastic gradient MCMC inference that is scalable to very long multivariate count time series.Experiments on both synthetic and a variety of real-world data demonstrate that the proposed model not only has excellent predictive performance,but also provides highly interpretable multilayer latent structure to represent hierarchical and temporal information propagation.3 In reality,a lot of information is included in each corpus in the form of text.At present,topic model and language model are widely used in text analysis.Both topic and language models are widely used for text analysis.While having semantically meaningful latent representation,they typically treat each document as a bag of words(Bo W),ignoring word order.Language models have become key components of various natural language processing tasks,such as text summarization,speech recognition,machine translation,and image captioning.While RNN-based language models do not ignore word order,they often assume that the sentences of a document are independent of each other.This simplifies the modeling task to independently assigning probabilities to individual sentences,ignoring their orders and document context.Such language models may consequently fail to capture the longrange dependencies and global semantic meaning of a document.In this section,we propose a new larger-context recurrent neural network(RNN)-based language model,which extracts recurrent hierarchical semantic structure via a dynamic deep topic model to guide natural language generation.Moving beyond a conventional RNN-based language model that ignores long-range word dependencies and sentence order,the proposed model captures not only intra-sentence word dependencies,but also temporal transitions between sentences and inter-sentence topic dependences.For inference,we develop a hybrid of stochastic-gradient Markov chain Monte Carlo and recurrent autoencoding variational Bayes.Experimental results on a variety of real-world text corpora demonstrate that the proposed model not only outperforms larger-context RNN-based language models,but also learns interpretable recurrent multilayer topics and generates diverse sentences and paragraphs that are syntactically correct and semantically coherent.4 Compared with SAR and ISAR,HRRP owns the obvious advantage for being processed directly without first forming an image.Since HRRP contains abundant discriminative information,such as the target size and scatterer distribution,HRRP based RATR has received intensive attention.We develop recurrent gamma belief network(r GBN)for radar automatic target recognition(RATR)based on high-resolution range profile(HRRP),which characterizes the temporal dependence across the range cells of HRRP.The proposed r GBN adopts a hierarchy of gamma distributions to build its temporal deep generative model.For scalable training and fast out-of-sample prediction,we propose the hybrid of a stochastic-gradient Markov chain Monte Carlo(MCMC)and a recurrent variational inference model to perform posterior inference.To utilize the label information to extract more discriminative latent representations,we further propose supervised r GBN to jointly model the HRRP samples and their corresponding labels.Experimental results on synthetic and measured HRRP data show that the proposed models are efficient in computation,have good classification accuracy and generalization ability,and provide highly interpretable multi-stochastic-layer latent structure.