Monte Carlo Methods And Their Applications In Some Statistical Model

With the arrival of modern computers, the Monte Carlo methods (i.e. statistical simulation methods) have become the main tools in complicated statistical model and high-dimensional problems. After several years development, the Monte Carlo methods have greatly expanded our scientific horizon. From the development of early computa-tional physics to modern computational biology, Monte Carlo methods allowed people to follow complicated system.A systematic use of the Monte Carlo methods for real scientific problems appeared in1940s, during World War Ⅱ. In order to make a good use of the first programmable "super" computer, scientists (S. Ulam. J. von Neumann, N. Metropolis. E. Fermi, etc.) invented a statistical sampling-based method for complicated problems. and get the numerical solutions of probability problems in atomic bomb designs and eigenvalues of the Schrodinger equation.1950s, statistical physicists (N. Metropolis, A. Rosenbluth, M. Bosenbluth, A. Teller, E. Teller, etc.) introduced the Markov Chain Monte Carlo methods, and use these methods to simulate Ising model and spin glass model etc. The Metropolis algo-rithm [78;70] is perhaps the first sampling algorithm for iterative simulations. Hastings generalized this algorithm by allowing a asymmetric proposal distribution [49]. The Metropolis-Hastings algorithm has an extremely simple form. Suppose the target dis-tribution is π, starting point is X0, this iterating algorithm contain two steps (from Xn to Xn+1): Step1. Propose a " proposal sample " y from any proposal distribution q(Xn,y). Step2. Calculate Metropolis ratio Xn,y)=min{1,π(y)q(y,Xn)/π(Xn)q(Xn,y)}, accept "proposal sample" y with probability α, and set Xn+1=y; otherwise, reject it, and set Xn+1=Xn.1980s, statisticians and computer scientists apply Monte Carlo methods to variety of tasks such as combinatorial optimizations (simulated annealing [57;70]), nonpara-metric statistical inference (jackknife, bootstrap [24;17]), missing data analysis (EM algorithm [19;100], Data Augmentation [101;70]), statistical genetics analysis, Bayesian model and Bayesian computation [100;99].1990s, Monte Carlo methods play an im-portant role in computational biology. Now, Monte Carlo methods is widely used in biology, chemistry, computational mathematics, finance, meteorology, physics, material science, eta. As an special case of Monte Carlo methods, Markov Chain Monte Carlo provides an enormous scope for dealing with very complicated molecules and other physical systems, so it received great attention from statisticians.The heart of Monte Carlo methods is sampling. Given the target distribution (often this target distribution is multimodal or high-dimension), how to sample from the target distribution efficiently is a crucial step for other problems (such as estimation, integral). The random walk Metropolis algorithm is an special case of Metropolis-Hastings algorithm. For its simple case, it is widely used in practice. For the same target distribution π, random walk Metropolis algorithm chose the symmetric proposal distribution, so the Metropolis ratio became Xn,y)=min{1,π(y)/π(Xn)}. But this algorithm is an local update algorithm. it is means that random walk Metropo-lis algorithm will suffer from local-trap problem. Recently, developing a good Markov Chain Monte Carlo sampling algorithm conquering the local-trap problem has been considered as an important topic in Markov Chain Monte Carlo research. During the past two decades, various advanced Markov Chain Monte Carlo sampling algo-rithm have been developed for local-trap problem. These include:simulated tempering [76;35;70]; Metropolis Adjusted Langevin algorithm [91]; parallel tempering [36;70]: adaptive rejection Metropolis sampling [41;40]; multi-try Metropolis algorithm [71;70]. evolutionary Monte Carlo [66;67;64]:dynamic weighting Monte Carlo [108;63;70:64]; Equi-Energy Sampling [58:64], among others.In chapter1, we develop a new Markov Chain Monte Carlo sampling algorithm:B-splines Metropolis-Hastings algorithm, which is an extension of random walk Metropolis algorithm. Since random walk Metropolis algorithm chooses symmetric proposal distri-bution, the efficiency of this algorithm depend critically on the proposal variance σ2. If σ2is t00large,it is easy to lead to a high rejection rate;on the other hand,if σ2is too small the Markov chain will converge slowly. Inspired by importance sampling[70],if we choose the proposal distribution which is reasonably close to the target distribution, we can construct an efficient transition scheme. We use B-splines to approximate the target distribution in certain given intervals: where n is the number of cubic B-splines basis, r is the scale of proposal, cxT (c1,x,…,cn,x)∈[0,+∞]n is the coefficient of B-splines approximation,BxT(y)=(B1,x(y),…,Bn,x(y))is B-splines basis.Next,we list the first theorem in chapter1,which can be used sampling from B-splines approximation. Theorem1.2.1(1)Let B[a,b](x)be one cubic B-splines basis function with equidistant knots on[a,b],and define the continuous random variable(r.v.)η below where,{Ui)i=14are independent identically distributed r.v.with the uniform distribution on[0,1](i.e.U(0,1)).Then77has his unnormalizing probability distribution function B[a,b](x).(2)As the proposal function q(Xn,·),the normalizing constants for bxT(y)=(B1,x(y),…,Bn,x(y))is CXn={1/24,1/2,23/24,1,…,1,23/24,1/2,1/24),and the weights in Algorithm COMIPM is ω={1/24C1,Xn,1/2C2,Xn,23/24C3,XnC4Xn,…,Cm-3,Xn,23/24Cm-2,Xn,1/2Cm-1,Xn1/24Cm,Xn}The second theorem illustrate the efficient of B-splines Metropolis-Hastings algo-rithm.Theorem1.2.2Let the target function be π(x),the scale of the B-splines proposal be r, and suppose the number of B-splines knots is large enough to sufficiently approximate the target distribution on the interval of B-splines proposal.Then the acceptance rate (i.e.,the long-run expected proportion of accepted moves of the Markov chain)ηr can be written below The B-splines Metropolis-Hastings algorithm can be extended to high dimension case:B-splines Hit-and-Run algorithm and B-splines Gibbs sampling. Simulation data and real data examples show that B-splines Metropolis-Hastings algorithm outper-forms other sampling algorithms (random walk Metropolis algorithm, adaptive rejection Metropolis algorithm, parallel tempering).Bayesian methods is widely used in medicine, social sciences, biology, etc. In chapter2, we use Bayesian methods to analyze mixture normal model: where (?)(·;μ,σ) is the probability density function, yi,…, yn is one sample of size n, θ=(a,μ1,σ12,μ2,σ22)T are parameters. Then the posterior distribution of parameters θ is where π0(θ) is noninformative prior distribution. Little et al.[68] and Chen [12] use EM algorithm and data augmentation to infer the parameters of mixture normal model. Finding that Equi-Energy sampling algorithm [58] can conquer "local-trap" problem and efficiently sample from target distribution, we use Equi-Energy sampling algorithm to this model. Simulation results show that the results from Equi-Energy sampling algorithm is more accurate than that from EM algorithm and data augmentation.Since the arrival of modern financial market, there are ever-increasing huge number of multivariate data sets. Data depth for multivariate data has received considerable attention in multivariate nonparametric analysis and robust statistics. However, the computation of data depth, especially the computation of the widely used projection depth [73;119;120;115] has remained as a very challenging problem which hinders the development of the projection depth and its wide use in practice. There are sev-eral algorithms can compute the projection depth [116]. Zuo [116] develop an exact algorithm computing the projection depth in2-dim case. But in high dimension case, they can not work well. In chapter3, we use simulated annealing algorithm [57;70] to compute the projection depth. The key step of projection depth computation is an global optimization where O1(·,·) is an1-dim outlyingness,θ=(θ1,…,θd)∈[0,π]d is high dimension coordinates transformation. We define target function:h(θ)=O1(u’(θ)x,u’(θ)Xn), and bring it into simulated annealing algorithm: Then we can get the computation of outlyingness. simulation results (2-dim,4-dim,7-dim) show that, simulated annealing algorithm is more accurate and efficient than other approximation algorithm.Kendall’s τ and Spearman’s ρ are classical nonparametric measures of the degree of dependence between two real-value random variables (X, Y). It has long been known that, for many joint distributions, they have different values, as they measure different aspects of the dependence structure. Durbin and Stuart [21] gave the first bounds of Kendall’s τ and Spearman’s ρ, but these bounds are not exact and they did not gave the corresponding joint distributions which can attain these bounds. In chapter4, we use simulated annealing method to find a new bounds for τ and ρ and their corresponding joint distribution which can attain those bounds. This is an constrained optimization problem: Then the improved bounds when r∈[-1,1] is where n=2,3,4,…, and τ∈[2/n+1,2/n-1], further, we can get the derivative of r to ρ when τ∈[2/n+1-1,2/n-1]: where τ∈[2/n+1-1,2/n-1], n=2,3,4,5,…, and α∈[0,1].Recently, dimension reduction has become a hot topic in statistics. There are vari-ous methods for this problem, among these sufficient dimension reduction is an efficient one. For sufficient dimension reduction, the important step is the principal compo-nent analysis of symmetric matrix. Li [61] use Sparse principal component analysis to sufficient dimension reduction,but they did not give the sparse eigenvalue.Zhu[113] use LASSO method[102]get the sparse eigenvalue,but they did not give the sparse eigenvector.In chapter5,we extend the results from[113],and get the estimation of sparse eigenvalue and sparse eigenvector: where λ=(λ1,...，λp)T are the eigenvalue of symmetric matrix Λ,ξ=(ξ1,...,ξp)and η=(η1,…,ηp).λi is the ith eigenvalue of Λ,and|λ1|≥|λ2|…≥|λp|≥0,ξ and η are the eigenvector of symmetric matrix Λ corresponding to λi.πjk=|ηjk|-r,ηjk is the consistent estimate of ηjk with order (?). Later we list the asymptotic property of our estimation.Theorem5.3.1(Oracle property) Let ln/(?)�'0.lnn(r-1)/2�'∞,mn/(?)�'0,mnn(r-1)/2�'∞.Then the estimation from sparse principal component analysis has these properties below.(1)Consistency: when n�'∞, A(?)A, for any i,we have, ηi(?)ηi, where A={i|ηi≠0).ηi={j|ηij≠0},and(?)indicate convergence in probability.(2)asymptotie normality: where W0is the limiting distribution of (?)(Λn-Λ).Monte Carlo simulation show the efficacy of sparse principal component analysis in sufficient dimension reduction.