Research On Prediction And Decision-making Methods Based On Multi-source Information Fusion

Multi-source information exists widely in the real world and contains more abundant content.How to make use of multi-source information to improve model performance and robustness is a hot issue in machine learning research.In various machine learning algorithms,two kinds of multi-source information are widely concerned from the perspective of model feature input and environment feedback information:multi-modal representation and multiple auxiliary rewards.In detail,multi-modal representation enriches the input information of the model,and multiple auxiliary rewards enrich reinforcement learning's guidance information.This paper studies how to use two forms of multi-source information to improve the performance of the model,and makes corresponding progress:1.Research on the prediction problems with complex multi-modal objects.Complex objects can always be represented by multi-modal features,e.g.,complex articles contain image and text information.Previous methods assume that the multi-modal data are consistent.However,in real applications,it is often not satisfied,e.g.,the article constitutes the variable number of inconsistent text and image instances.Multi-modal multi-instance multi-label learning performs well in dealing with such complex object prediction problems.However,it is facing two main challenges: 1)how to effectively utilize label correlation;2)how to make use of unlabeled data.To solve these problems,this paper proposes a novel Multimodal Multi-instance Multi-label Deep Network,which trains the deep network end-to-end based on the consistency principle among different modal bag-level predictions.Based on this model,we learn the latent ground label metric with the optimal transport to utilize label correlation.Moreover,we introduce the extrinsic unlabeled data to train the instance-level auto-encoder for single modality and modified bag-level optimal transport in the semi-supervised extension of this model.Thereby it can better predict label and exploit label correlation and unlabeled data.Experiments on benchmark datasets validate the effectiveness of the proposed methods.2.Research on decision-making problems with multiple auxiliary rewards.Environmental feedback information is a crucial component of reinforcement learning,which affects the effectiveness of reinforcement learning to a large extent.Sparse and delayed rewards increase the difficulty of the task,making existing methods are challenging to explore effectively and hindering the application of reinforcement learning.By introducing domain knowledge to construct auxiliary reward,multiple auxiliary rewards reinforcement learning has achieved superior performance,but still faces two problems: 1)how to safely introduce domain knowledge to construct auxiliary reward;2)how to automatically combine reward functions.In order to solve these problems,this paper proposes an Automatic Successive Reinforcement learning method,which introduces domain knowledge into the reinforcement learning environment based on reward shaping and alleviates the problem caused by sparse reward.In order to use the appropriate reward function in different learning stages,the proposed method automatically selects the optimal combination of multiple rewards.The simulation results show that our proposed method achieves superior performance in both classic control environments and video games.In addition,the proposed methods in this paper have been successfully applied in the real world.The prediction method with complex multi-modal objects is applied to the article classification of WKG Game-Hub,which improves the accuracy of complex article classification.The decision-making method with multiple auxiliary rewards is applied to the field of the automated home interior designer,which reduces the economic cost and human labor of home layout.