A New Method For Estimating High Dimensional Covariance Matrix

Covariance is a basic concept in probability and statistics,it is used to measure the correlation between two random variables.Covariance matrix is a generaliza-tion of covariance in multi-dimensional which is used to measure the correlation between multiple random variables.The position of covariance matrix is very im-portant in statisties,especially in the theory of multivariate statistieal analysis.However,when dealing with many statistical problems,the covariance matrix of multivariate population is unknown.Therefore,estimating covariance matrix be-comes a basic problem in multivariate statistics,which has been widely used in signal processing,genetics,financial mathematics and other fields.The traditional sample covariance matrix can estimate the population covari-ance matrix well when the sample size is larger than the dimension.However,modern statistics often encounter small sample size which is even less than the data dimension.In this case,the sample covariance matrix is no longer reversible with probability 1,which makes many statistical problems impossible to solve.It should be pointed out that,in 1993,Andersson and Perlman proposed the L-CI(Lattice Conditional Independence)model which is well-known in the field of graph modeling and cross-correlation.The model proves that the precision ma-trix and the covariance matrix of normal population have special structure under conditional independence.Zhang Shun and Zhao Qiang proposed a algorithm for searching strong causal cut in 2016.This algorithm uses measurable sample set to perform strong segmentation search,and provides a method to find conditional independence among many variables.On the basis of LCI model and the algorithm for searching strong causal cut,a new algorithm is presented in this paper for estimating covariance matrix.Firstly,the algorithm for searching strong causal cut is used to search for the conditional independence of a variable set.Then the structure of the population covariance matrix is obtained under eonditional independence by using the LCI model theory.Finally,we estimate each part of the population covaxiance matrix.and obtain an estimation of the population covariance matrix.We also prove theoretically that the new estimator still has strong invertibility in high dimension under some conditions.The conditions mentioned above can be guaranteed by cyclically calling our new algorithm until the preconditions are satisfied.We also prove that the new estimation is a consistent estimation of the population covaxiance matrix.At the end of this paper,we give some applications of the new estimation of covariance matrix obtained by the new algorithm in statistics.