Research Of Ensemble Learning Theory And Its Application In Personalized Recommendation

Ensemble learning is a new machine learning paradigm, which can significantly improve the accuracy and generalization ability of learning systems by means of training multiple learners to solve the same problem. Since the beginning of 1990s, the theory and algorithm of ensemble learning have become a hot issue in the field of machine learning. Along with the rapid development of information technology, it has entered the era of information explosion. Information explosion decreases the efficiency and effectiveness of information, the well-known Information Overload. Personalized recommendation is usually considered to be one of the most effective approaches to this problem, which actively collects the users’preferences and provides personalized services for them. So far many personalized recommendation methods were introduced in the literature, but most of them have poor accuracy and generalization.If we can take the advantage of ensemble learning and employ it into recommendation system, the effectiveness and adaptability of personalized recommendation system would be considerably enhanced. In the field of ensemble learning, there are still a number of problems to be solved, such as the relevance and redundancy among weak classifiers in boosting learning. To issue above problems, this dissertation mainly focus on the ensemble learning theory and its application in personalized recommendation, the main contributions are as follows:(1) For the problem of relevance and redundancy among weak learners in boosting learning, we proposed a novel selective boosting algorithm, called SelectedBoost. The new algorithm first calculates the relevance measure between new learner and existed learners, and then, evaluate whether remove the new learner according to the relevance measure and the current strong learner’s accuracy. By doing this, SelectedBoost markedly increases the efficiency of selection compared with conventional selective methods that first generate all weak learners and then select them. Thus, SelectedBoost efficiently decreases the number of weak learner and reduces the relevance among weak learners with higher diversity. Additionally, SelectedBoost improves the convergence rate and has a competitive accuracy. (2) The margin maximized boosting algorithms (e.g. LPBoost, SoftBoost) update sample weight just according to the weak learners. However, compared with weak learners, the strong learner is more representative for current classification hyper plane and has a tighter edge constraint. Hence we propose the algorithm StrongLPBoost which introduces an edge constraint of strong hypothesis to speed up the convergence when maximizing soft margin of the combination of the weak learners. And then, StrongLPBoost updates sample weight not only according to the existed weak learners but also the strong learner. In a benchmark comparison, we show the competitiveness of our approach from the aspect of time consuming, and generalization error.(3) We analyze the reason why boosting framework can be successfully applied to the classification and fails in recommendation system. In order to using boosting learning in the context of recommendation, the original recommendation problem can be converted into a simple classification problem by simplifying and decomposing the user-rating matrix. And then, the boosting algorithms can be applied into recommendation problems. Thus, we propose a new combined algorithm, called RankBoost*, which boosts the multiple KNN algorithms using several different similarity measures and matrix factorization to improve the final predication performance. Experimental results show the effectiveness of boosting framework to improve the single learning algorithm for personalized recommendation.(4) The ultimate purpose of recommendation system is to provide users with personalized ranking or recommendation list and this objective has been argued to be more crucial than the accuracy of rating prediction. For this reason, we focus on ranking or top-N recommendation rather than rating prediction. In this work, the technologies of learning to rank (LTR) are introduced into recommendation because LTR has been successfully applied to information retrieval. Basing on this point, we propose a novel boosting learning framework for LTR and implement a specific boosting algorithm NDCGBoost@K which optimizes NDCG measure in personalized top-N recommendation. In addition, the new boosting framework is equally suitable for other list-wise measure such as MAP (Mean Average Precision).Finally, we summarize this work and present some possible future extensions.