Some Information Entropy Theorems For Nonhomogeneous Markov Chains And The Estimation Of Rényi Entropy

The entropy was originally derived from thermodynamics and subsequently evolved into disciplines such as statistical physics and information theory,and played an important role in these fields.The fundamental entropy so-called Shannon entropy has been studied extensively.The Rényi entropy,as the generalization of Shannon entropy,has been widely applied in computer science,finance,cryptography and information theory.In this dissertation,we will mainly study some information entropy theorems for nonhomogeneous Markov chains and the Rényi entropy estimation.Based on the study of Shannon entropy and Rényi entropy,various conditional Rényi entropies were discussed in more detail,and the relationship among them was compared.By using the analysis method of probability theory of limit theory,the generalized Shannon McMillan-Breiman theorem and Rényi entropy rate of a class of non-homogeneous Markov sources-finite asymptotically cyclic Markov chains were studied.And the Monte Carlo method was used to estimate the Rényi entropy.This thesis includes five chapters.The chapter 1 is the introduction of some research background of entropy.In chapter 2,some definitions and properties of various conditional Rényi entropies are summarized and the relationships among them are discussed.In chapter 3,we prove some limit theorems on the generalized averaging of finite asymptotically cyclic Markov chains,which generalizes Shannon McMillan-Breiman theorem for finite asymptotically cyclic Markov chains.In chapter 4,we focus on the study of the Rényi entropy rate of nonhomogeneous Markov chains,and obtain the expression of the Rényi entropy rate of the asymptotically cyclic Markov chain.In the last chapter,we briefly discuss the nonparametric estimation of the Rényi entropy.