Low Rank Tensor Regression Methods And Applications

Regression analysis is a crucial subject in machine learning and signal processing,used to study the relationships between variables.It has a wide range of applications,including weather forecasting,human motion trajectory reconstruction,and age estimation based on facial images.However,as sensing and storage technologies advance,the order and dimensionality of data increases,making traditional vector-or matrix-based regression algorithms unable to leverage multidimensional structure information and lead to a huge amount of parameters.The problem is thus intractable especially with restricted samples.Effectively exploring the multidirectional relationship between multiway data without introducing undue complexity to the regression problem-solving is,therefore,a crucial issue.Similarly,with the emergence of multimodal data,joint multi-task learning for regression problems also faces many challenges,such as the multiple indexing of tasks,and accurate modeling of high-order correlations between tasks.The main contributions and innovations of this dissertation are as follows:1.Regarding the difficulty in characterizing high-order correlations among data in high-order data-related regression systems,we propose to use a popular tensor train network structure to characterize the multi-directional correlations between high-order data involved in the regression system.While fully utilizing the internal structural information of high-order data,the required parameter amount is reduced,which alleviates the dimensionality curse caused by using traditional regression techniques to handle highorder data regression tasks.To alleviate the rank imbalance of the tensor train,we further propose a multilinear regression model based on tensor ring structure to improve the modeling accuracy of multi-way relationships among high-order data.In addition,we perform a comparative analysis of different optimization strategies and design a fast tensor network contraction scheme to enhance algorithm efficiency.Compared to traditional CP decomposition-based methods,there is a 9% improvement in the prediction trajectory correlation coefficient on spatiotemporal data and a more than 75% increase in algorithm computational efficiency.2.In order to address the difficulty of model complexity selection for high-order datarelated regression systems,which is both difficult and time-consuming,we propose a new type of group sparsity constraint term to approximate the tensor multilinear rank,and infer the tensor multilinear rank during the training process.The proposed approach achieves a superior balance between data fitting accuracy and model complexity,consequently establishing a more concise and compact tensor regression system.Both simulated and real data experiments have demonstrated that the proposed method,with rank-adaptive adjustment given a rank upper limit,can closely approach the experimental performance upper bound of tensor ring factorization methods.Its performance surpasses more than 95% of different rank combinations.3.Regarding the insufficient utilization of high-Order correlations across tasks in high-order data-related regression systems,a new tensor-based multi-task learning framework is proposed for multi-task learning tasks with multiple index representations.The proposed model preserves the multiple index representations of the involved tasks based on their physical meaning,collects the model parameters of all tasks,and merges them to construct a high-order coefficient tensor.The low-rank CP decomposition is then employed to explore the structural correlations within the coefficient tensor to mine the common subspace among all tasks while preserving task-specific information.The proposed framework can flexibly capture structural dependencies from multiple indices and provides effective solutions for multi-task learning problems with multiple indices.The experimental results on real data show that the proposed tensor multi-task regression model can improve the accuracy by more than 6% compared with existing algorithms.In summary,this dissertation addresses the challenges posed by high-order correlation among data and tasks in high-order data-related regression systems.The primary focus of the research revolves around the optimal representation of tensor regression model coefficients,complexity selection,and the multi-task learning problem with multiple indices.A series of efficient tensor regression analysis models are constructed,aiming to propel machine learning into a new stage to efficiently analyze data with larger scale,more complex structures,or from diverse sources.