Bayes Estimation Of Generalized Entropies

In 1948, C. E. Shannon introduced the concept of entropy in the thermodynamic system into the information theory, and that marked the birth of modern information theory. Then, estimation of the Shannon entropy is applied in engineering, biomedicine and biochemistry etc..To solve the practical problems in life, scholars put forward the generalized entropy of different forms, such as Tsallis entropy and Renyi entropy. In the statistical analysis, the application of generalized entropy is more concerned by scholars. This article mainly discusses the generalized entropy of a discrete random variable, and extends estimation of Shannon entropy to Tsallis entropy and Renyi entropy. Based on the maximum likelihood estimation of Shannon entropy, this article proposes Bayes estimation method of Shannon entropy, and compares the Bayes estimation method with the maximum likelihood estimation. Further more, this article presents the Bayes estimation both of Renyi entropy and Tsallis entropy which are representative of the generalized entropy.Random sample of size N with probability(0.1,0.4,0.5) is got by the use of Matlab software. Traditional method of maximum likelihood estimation of entropy estimation is got by the use of frequencies to replace probability for each number. Substituting frequency of each number into the Bayes estimation. The value of Bayes estimation of the sample of size N is calculated by Matlab. A series value of maximum likelihood estimation and Bayesian estimation are obtained by changing N and multiple tests, and the two methods are compared. By the sum of squared error, the effect of Bayes estimation of is better than maximum likelihood estimation. However, Bayes estimation of Tsallis entropy does not appear to be a better effect. Changing the value of q and operating a large number of randomized trials can be considered, and comparing the Bayes estimation of Tsallis entropy with the maximum likelihood estimation.Although Bayes estimation method of generalized entropy is not necessarily the best, this article still has some practical significance through the application of the generalized entropy.Finally, this article utilizes order statistics to further discuss the Bayes estimation of continuous random variables,and gives the Shannon entropy and Renyi entropy of continuous random variables.