Subjective Cognition-Oriented Research Of Data Mining Technology And Platform

Subjective cognition is an important content in cognitive psychology. In the fields of Internet and information technology, based on subjective cognition, we usually carry on research from users’ perspective to guide the optimization of network, application and services taking user satisfaction as a purpose. But in the implementation process, it needs quantize the abstract concept, subjective cognition, which always involves many learning and training methods in data mining. At the same time, along with the arrival of the big data era, combining data mining methods with computing has become one of the main trends of the development of data mining.This dissertation pays attention to two sub projects of subjective cognition--user influence estimation in microblogging platform and QoE assessment for specific services and networks. I have used data mining related methods to analyse and quantize the related problems. And under the development of bigdata fields, I have implemented the used algorithms on hadoop platform. The main contents of this dissertation are as follows:Firstly, this dissertation has studied the information propagation mode in Sina microblogging platform. On the basis of PageRank, MURank algorithm is proposed. It considers concurrently the relationship between user and user, user and tweet, tweet and tweet and improve the information propagation mode, making it in line with the actual situation. Based on the experiment, MURank algorithm has been proved more effectively to solve the problem of zombie fans and rank sinking. The information propagation mode can fit the actual situation much more appropriately. So the result of MURank is more accurate and reliable. Secondly, based on the related international standards,this paper has improved the indicator system and proposed a new hierarchical indicator system consisting of five layers with specifying the meaning and defining methods of each layer. Then I have used the BP neural network algorithm to measure the quality of user experience based on the indicator system. At lasg, taking mobile video service as an example, this dessertation has illustrated the process of establishing the hierarchical indicator system and using BP neural network method to measure QoE.Finally, in order to deal with the challenges in bigdata era, this dissertation has introduced the main functions and architecture of the open source distributed computing platform hadoop.At the same time, the process of how to build a hadoop cluster is given.Then I have decomposed the proposed MURank and BP neural network algorithm as MapReduce processes and implement them on the cluster. Through the experiments, the convergence, correctness and effectiveness of these algorithms have been proved. It has proposed a sollution for the coming challenges of massive data processing in the future.