Estimation For High-dimensional Partial Correlation Coefficient

Partial correlation coefficient(PCOR)measures the degree of association between two random variables,with the effect of a set of controlling random variables removed.However,if the dimension of controlling variables is high,its estimation becomes very dif-ficult.Several methods have been proposed for the estimation of pcor in high-dimensional data but mainly for the purpose of testing whether the value is zero instead of estimation efficiency.The first part of the thesis summarizes the existing estimation methods,which can then be directly applied to high-dimensional data through regularization methods.The regularization methods used to estimate the regression model directly affects our estima-tion results.For the same partial correlation coefficient estimation method,the better the dimensionality reduction of the regularization method used,the closer the obtained esti-mate will be to the theoretical value.The second part of the thesis improves the existing estimation methods to make them more applicable to the estimation of high-dimensional partial correlation coefficient.Based on the existing estimation methods,we propose new methods such as Refit(Rf)and second Regression(Reg2),which effectively improve the estimation efficiency.Reg2 can further eliminate the influence of controlling variable.Therefore,combined with the reg-ularization method,Reg2 has the best estimation efficiency in the high-dimensional case.Furthermore,we derive and prove the new definition based on the definition of partial correlation coefficient,and propose a new estimation method.By extensive simulation studies,we find that all the methods "under-estimate" the absolute values of the partial correlation coefficients.Some ad hoc methods based on the existing techniques for high dimensional data cannot make satisfactory improvement of the estimation,suggesting that more efficient methods are still wanted.The third part of the thesis estimates the partial correlation coefficient matrix.Most of the estimation methods do not consider the special structure of the partial correlation coefficient matrix,so their estimation speed and results are poor.In contrast,estimation methods that consider the structure,such as using the inverse of covariance matrix,have good estimation speed and results.At the end of the thesis,we use the stock data in China Shenzhen Stock Exchange to investigate the true relationship between stock returns.Applying the estimation methods of high-dimensional partial correlation coefficient,we can deal with high-dimensional controlling variables,and our study is more able to reveal the true relationship between stock returns compared to previous studies.