Statistical Identification For Several Types Of Discrete-time Markov Chains

The theoretical study of Markov chains is usually based on a natural assumption of a known transition probability matrix(TPM).However,the TPM of a Markov chain in realistic systems might be unknown and might even need to be identified by partially observable data.Thus,an issue on how to identify the TPM of the Markov chain by partially observable information is derived from the great significance in applications.That is what we call the statistical identification of Markov chain(SIMA).Firstly,find the constrained relationship between the hitting-time and the sojourn-time probability distribution of a certain set of states and transition matrix with taking full advantage of the intrinsic properties of Markov chains and matrix analysis,then identify the transition matrix according to the necessary constrained relationship.This method is called Markov chain inversion approach(MCIA).The MCIA has been used to solve the SIMC of discrete-time birth-death chains,star-graph branch chains(Markov chains on two special types of trees)and most of continuous-time basic Markov models.In the current letter,we continue using MCIA to solve the SIMC of discrete-time loop chains,chains on general trees and some special trees.Firstly,for reversible discrete-time Markov chains,it is proved that the probability distributions of hitting time and sojourn time of a certain set of states are mixtures of geometric distributions,and the constrained relation between the distribution and the TPM is given;Secondly,it is proved that the TPM of a loop Markov chain can be uniquely determined by the distributions of hitting time and sojourn time at arbitrary two adjacent states and the corresponding algorithm is included;Then,it is proved that the TPM of a Markov chain on trees can be uniquely determined by the distributions of hitting time and sojourn time at all leaves and the corresponding algorithm is included;Finally,Markov chains on three special types(double-star graph,banana tree graph,firecracker graph)of trees are considered,the corresponding SIMCs and algorithms are given.