Multi-task Learning And Applications Based On Distribution-robust Optimizatio

With the rapid development of big data technology and the increasing complexity of the data processed in practical applications,multi-task learning models that can predict multiple response variables simultaneously have become a research hotspot in recent years and have been widely used in fields such as recommendation systems,image recognition,and text analysis.Previous research on multi-task learning has mostly focused on improving the predictive performance of the model,while risk-averse scenarios such as medical diagnosis,autonomous driving,and financial investment place higher demands on the robustness of multi-task learning.A method based on distributionally robust optimization is proposed to improve the robustness of the multi-task learning model under distributional shifts by minimizing the sum of the maximum expected losses under each task with a constraint of distance between distributions.According to Lagrange method and simplex method,a two-step method is derived to obtain the parameters,and binary search and gradient descent algorithm are used to improve the solution efficiency.In addition,an equivalent form of the objective is obtained based on the Cressie-Read divergence family,which shows that protecting against worst-case distributional shifts is equivalent to optimizing the tail-performance under each task,and further demonstrates the convergence of robust risk under empirical distribution and the lower bound of minimax error for robust risk estimation.The numerical simulation part compares the robustness of the proposed method with classical multi-task learning and uncertainty-weighted multi-task learning by changing the distributional shift angle and further explores the influence of hyperparameter values and task correlations on the model performance.In the empirical analysis part,distributionally robust multi-task learning is applied to multiple diseases prediction and We Chat video recommendation.The former uses a distributionally robust multi-task support vector machine model to illustrate the model’s robustness under data migration in different regions.The latter uses distributionally robust deep multi-task learning to study the model’s robustness on minority subpopulations under imbalanced sample proportions.The experimental results indicate that the distribution-robust multi-task learning exhibits better robustness in both scenarios.