Recommendation Algorithm Based On Multi-Modal Fusion

With the explosive growth of information,recommendation algorithms can alleviate the problem of information overload.However,in the recommendation system,although the number of users and items is large,the actual interaction data between users and items is very small.In order to effectively alleviate the sparsity of this key data,most of the existing recommendation algorithms use the edge information of users or items to expand the available information.Considering that the information acquired by people in daily life is multi-modal,we can naturally use multi-modal information as another side information to improve the performance of the recommendation system.The existing multi-modal recommendation algorithms are relatively simple and direct in terms of modal fusion.This paper focuses on the selection of fusion strategies and the learning of higher-order features to carry out effective fusion of different modals,and explores the combination of knowledge graph and multi-modal recommendation algorithm.First of all,the self-attention mechanism can learn the relationship between features well.By taking advantage of this advantage,we build an algorithm framework that makes use of both text modes and visual modes for recommendation.In the process of building,we found that most of the existing multi-modal recommendation algorithms used early or late strategies to fuse multi-modal features in the multi-modal feature fusion stage,but these strategies could not simultaneously learn and use the single-mode high-order feature information before fusion and the multi-mode high-order feature information after fusion.In order to solve this problem,we propose a middle fusion strategy that can simultaneously utilize the two kinds of feature information,and implement it on the basis of the built basic algorithm framework.Our middle fusion strategy carries out feature fusion after passing through one feature learning layer,and then sends the fused features into the subsequent feature learning layer.The experimental results show that the multi-modal recommendation algorithm framework we built can indeed bring more information and have a better performance.Secondly,the existing multi-modal recommendation algorithms all use the same feature learning method for both single-modal features and multi-modal features,without additional learning of the relationship between the two modalities.To solve this problem,we try to exchange the feature information of the two modalities before merging them.Since this switching operation is similar to implementing a crossover between two modalities and a mutual characteristic learning,we call this algorithm a Cross-Modal Based Fusion Recommendation Algorithm.By using the inner product method to calculate the cross-modal weight of the two modal features,the direct relationship between the two modal features can be mined and the cross-modal feature representation containing more information can be obtained.Experiments show that our method achieves better results than existing open methods on multiple open datasets.Currently,there are a lot of researches on how to use knowledge graph to propagate user preference information,but only a few focus on the value of multimodal technology in recommendation algorithms based on knowledge graph.Most of the existing works only focus on how to enhance the representation of objects by introducing multimodal information,or just map multimodal features to the same space and then propagate the user's single preference information through the knowledge graph,without really capturing the fine-grained preferences of users for different modal features.To solve the problem of how to use multimodal technology and knowledge graph to enhance user representation,we propose a Multi-Modal Based Knowledge Graph Recommendation Algorithm.We use the self-attention mechanism to calculate the similarity between the target item and the user's related item,and then propagate the user preference information under the two modalities in the knowledge graph respectively.Finally,we use addition,concatenation and cross-modal fusion to obtain the enhanced user representation.Experimental results on open datasets show that our algorithm performs better than existing algorithms.