Research On Top-k Based Service Ranking Optimization For Big Data Environments

As the amount of Internet users increases year after year, the information stored in the network is also growing at an overwhelming rate. How massive information quickly and accurately provides users with the best service to meet their preferences become a new challenge. Top-k technology plays an important role in filtering for users. However, existing top-k algorithms mainly focus on solving top-k ranking service and there is lack of concurrency control in a single thread, causing severe memory retention problems. To address the above problems, it is necessary to pre-process data and improve services during top-k querying process. The main work of this thesis is as follows:(1) According to the working principle of data MapReduce parallel processing, this thesis proposes a Core-C algorithm, using tagged in local child thread to reduce the opening times of MapReduce work process. In addition, based on querying characteristic of top-k, this thesis uses a processing pruning operation for none top-k set of elements with a non-deterministic dataset query method.(2) This thesis proposes a data partitioning and non-dominated top-k ranking algorithm based on distrubuted data model, which calculates the sharing data of server units, with efficient ranking methods server unit obtains k results into memory, and with a central server finally return k results to the user.(3) This thesis proposes a top-k service ranking application based onuser’s preferences.According to the corresponding requirements analysis, preliminary design and detailed design, this thesis completes the realization of each module to verify the correctness and feasibility of proposed method.