Research On Parameter Estimation Method Of Normal Distribution Based On Multiparticle System

For a given probability model,researchers need to accurately estimate its parameters and make statistical inference according to the parameters.In order to obtain correct statistical inference conclusions,it is necessary to use more accurate parameter estimation methods.However,the accuracy of the parameter estimation method relates to the dimension and structure of the model,and there is no accurate method applicable to all models.Maximum likelihood estimation is a parameter estimation method based on optimization algorithm.This method belongs to point estimation method,and it is difficult to reflect the overall information of parameter distribution.This paper focuses on the exploratory research of parameter estimation of normal distribution.Firstly,the parameters of multivariate normal distribution are estimated by the Black-Box Variational Inference method.This method can estimate the approximate range of parameters and make up for the deficiency of maximum likelihood method.Later,further research revealed that the classical Black-Box Variational Inference is essentially an approximate method using multiple univariate normal distributions,which cannot accurately estimate the parameters of multivariate normal distributions.Finally,based on the Hamilton Monte Carlo method,it is found that the acceptance probability of this method is constant 1 in theory,and there is the possibility of divergence in operation,and its runtime parameters are difficult to adjust.In order to achieve stable sampling,an improved algorithm of multi-particle system based on energy conservation is proposed.Since the improved scheme still cannot effectively improve the sampling accuracy,further research shows that the sampling accuracy is mainly affected by the trajectory shape,which is determined by kinetic energy.After summarizing the trajectory of Hamilton Monte Carlo and Riemannian Manifold Hamilton Monte Carlo,the "optimal" kinetic energy for positive definite,negative definite and indefinite Hessian matrices is proposed.The sampling method using this kinetic energy is more stable and accurate,and can be applied to multivariate normal distribution.The research contents and achievements of this paper mainly include the following aspects:(1)An improved Black-Box Variational Inference method based on multivariate normal distribution and Laplace distribution is proposed,which extends the application range of this method.This paper discusses the essential idea of the classical Black-Box Variational Inference from the point of view of information theory,which shows that this method can not obtain the overall information of the parameters of multivariate normal distribution.(2)The single particle system that does not comply with the conservation of energy adopted by the classical method is replaced by the multi-particle system that complies with the conservation of energy,and a method of exchanging particle energy through virtual collision is proposed.This method uses general kinetic energy and Mahalanobis distance to expand the Maxwell-Boltzmann distribution,which can ensure the overall stability of the system and the strong exploration ability of particles at the same time.A scheme is designed to adjust the simulation step parameters and the total energy of the system according to the overall operation state of all particles,which ensures the stability of the algorithm operation process and creates conditions for accurate sampling.The approximate inverse relationship between kinetic energy and acceptance probability is revealed,and a method for automatically adjusting the acceptance probability of Markov Chain Monte Carlo is constructed.(3)It is found that the kinetic energy of Hamilton Monte Carlo is only applicable to the positive definite Hessian matrix,and the optimal kinetic energy function corresponding to the multivariate normal distribution is constructed,which is applicable to the positive definite and non-positive definite Hessian matrix.The quantitative experiment based on multivariate normal distribution reveals the special phenomenon that the parameter range of Hamilton Monte Carlo estimation is overlarge,and the parameter range of Riemannian Manifold Hamilton Monte Carlo estimation is oversmall.The proposed method can estimate the parameters of the probability model accurately and stably,and its convergence effect is good,and the sampling efficiency is high.(4)Realized parameter estimation and statistical inference of various probability models,explored the influence of various kinetic energy functions on convergence and sampling results,and revealed that the form of kinetic energy is related to the width of confidence interval,thus affecting the conclusion of statistical inference.A quantitative research method is proposed to generate data from probability models and then estimate parameters.To sum up,this paper proposes a Black-Box Variational Inference method suitable for multivariate normal distribution and an improved Markov Chain Monte Carlo algorithm based on multi-particle system.The algorithm automatically adjusts the operating parameters and acceptance probability,with high stability and accuracy.