Research And Application Of Explainable Recommendation System Based On Spark

With the rapid development of big data and artificial intelligence technology,as well as the popularity of intelligent terminals and electronic products,the recommendation system proposed to alleviate the problem of information overload has been widely researched and applied.However,the current recommendation system only gives the result list after calculating the recommendation candidate set from the user behavior data,which would not be attached the corresponding explanation.As a result,users are unable to choose their preferred items clearly,resulting in a digital gap between users and recommendation systems.In order to solve the problems that traditional recommendation algorithms lack the explanation of recommendation process and results,and the accuracy of models is low,this dissertation proposes a semi nonnegative matrix factorization based on Bayesian probability model(BSNMF)and a multi-task potential Dirichlet tensor decomposition explainable recommendation algorithm(MLTER).Firstly,the former is introduced to initialize the original rating matrix to provide latent interpretation semantics.Gaussian-Wishart distribution is used as a priori.Markov Chain Monte Carlo method is used to make Bayesian random approximation inference whose probability interpretation is added.The experimental results show that the proposed recommendation algorithm ensures the intrinsic interpretability and has high prediction accuracy under different number of potential factors and iterations.Although BSNMF is interpretable in the recommendation process,it still lacks the interpretability of the model results.Therefore,the latter first classifies TF-IDF and LSH co-occurrence topic words by LDA text classification method.The sentiment analysis technology is introduced to construct the comment phrases of the classified topic words which the sentiment polarity is attached.Multi-task learning is introduced to deal with the cascade task of tensor decomposition,and practical physical meanings of factor matrix simultaneous are explained.Experimental results show that the proposed explainable recommendation algorithm has the interpretability of recommendation results.Moreover,it has superior performance in model performance.This dissertation constructs an explainable recommendation system based on Spark by applying the two proposed explainable recommendation algorithms.According to the functional requirements,the engineering architecture of the recommendation system is designed in detail.Besides,the data portrait and database models are implemented and designed based on offline and real-time data.Then the implementation of the system is introduced into the perspective of object model and ordered interaction model of the system with UML object modeling method.