Multi-task Learning With Matrix Generalized Inverse Gaussian Distribution

The traditional machine learning method is confined by the problem of insufficient sample training in a single task. Multi-task learning is able to mitigate this problem through mining the relationships among the tasks. Based on the work on matrix statistics, we study the application of the Matrix Generalized Inverse Gaussian (MGIG) distribution to model the task relationship matrix in multi-task learning.We first introduce and define the MGIG distribution and explicitly discuss its statistical charac-teristics. Since the statistics of the MGIG distribution have no close form, we propose two sampling strategies for computing the statistics of the MGIG distribution. Based on the proposed sampling techniques. we propose the Gaussian Matrix Generalized Inverse Gaussian (GMGIG) model for low-rank approximation to the task covariance matrix. Through combining the GMGIG model with the residual error structure assumption, we propose the GMGIG regression model for multi-task learning. Experiments show that this model is superior to the peer methods in regression and prediction.Based on the MGIG prior, we propose a novel multi-task learning algorithm, the Bayesian Multi-task Relationship Learning (BMTRL) algorithm. To incorporate the link structure into the framework of BMTRL, we propose link constraints between samples. Through combining the BMTRL algorithm with the link constraints. we propose the Bayesian Multi-task Relationship Learning with Link Constraints (BMTRL-LC) algorithm. To make the computation tractable, we simultaneously use a convex optimization method and sampling techniques. In particular, we adopt two stochastic EM algorithms for BMTRL and BMTRL-LC. respectively. The experimental results on Cora dataset demonstrate the promise of the proposed algorithms.