Research On Efficient Approximation Methods For Gaussian Process Regression

The Gaussian process?GP?specifies a distribution over functions,and has been widely used in regression,classification,dimensionality reduction and so on.Gaussian process regression?GPR?inherits advantages of both the Bayesian method and the kernel method,but its application is hindered by the expensive computation cost especially on large-scale data.Existing approximation methods attempt to extract compressed information from training samples with the aid of inducing points,but in complex problems,usually a large number of inducing points are needed,which in fact makes it difficult to reduce the time complexity.Inspired by the idea of divide and conquer,the paper proposed a simple and efficient approximation method called Overlapped Local Gaussian Process.The training samples are recursively divided and a ternary tree is constructed in the end.Sibling nodes have intersections where the samples play the role of inducing points to model the dependency between neighboring regions.Each leaf node is associated with a local GPR model.The evidence and predictive distribution in parent nodes are approximated by the combination of the ones from their sons,which reduces the computational cost significantly.A theoretical analysis shows that the time complexity for training and prediction reduces to O?N^t?,for N training samples,where t depends on the proportion of the intersection in each level,and is usually between 1 and 2.A further improvement can be obtained by viewing the inducing points as parameters of the approximate model,and fine tuning the location of the inducing points and the distribution of the corresponding function values through variational inference.The optimized inducing points are usually more effective to model the dependency between neighboring regions.And the marginal likelihood and the predictive distribution can still be decomposed in a hierarchical manner to reduce the time complexity.A theoretical analysis shows that for N training samples,if the number of inducing points scales as the samples to the power of?in each level,the training time complexity can be reduced to O(N^3?).