Research And Implementation Of Non-uniform Resource Allocation Recommendation Algorithm Based On Spark

With the rapid development of Internet technology, the network information resources are increasing day by day. The Internet provides users with a large number of optional information to meet the user information needs. However, the problem of information overload is becoming more and more serious, and it becomes more and more difficult to find valuable information in the mass information resources timely and accurately. Based on this, the personalized recommendation technology arises at the historic moment. It can help users to select the most suitable items from massive information according to the user's historical interests and preferences data.Because of high recommend hit rate and no restrictions on items type, uniform resource allocation recommendation algorithm based on the bipartite graph network structure captures widespread attention. But it only relies on the item degree and user degree to allocating resources, and does not distinguish the preference of the user about the item, which greatly reduced recommended quality. After fully understanding the recommendation algorithm, this paper has made the following improvements for its shortcomings.First of all, in order to improve the shortcoming that uniform resource allocation recommendation algorithm too dependents on the role of node's degree when distributing resource, This paper adopts the way of non-uniform resource distribution by Adjusting resource allocation coefficient considering users preference differences on project and project categories. Secondly, the clustering algorithm widely used in data mining and the recommendation algorithm are fused deeply, which can improve the calculation of the similarity between users, and makes recommend more personalized and accurate. Based on the above two points, this paper proposes a non-uniform resource allocation recommendation algorithm based on the bipartite graph network.In order to cope with the huge pressure of massive data processing and improve the timeliness of the recommendation, this paper designs the non-uniform resource allocation recommendation algorithm in parallel by using distributed computing framework based on Spark. The experimental results show that compared with the former algorithm, the improved non-uniform resource allocation recommendation algorithm is significantly improved in the recommendation accuracy and stability,which reflects the research significance and application value.