Research On Anomaly Detection Methods For Online Social Networks

Online social networks represented by the microblog have gradually become the mainstream platform of information dissemination,however,they are also facing a series of problems at the same time,such as spammer fans flooding,rampant account hacking and rumors flood and so on.These problems seriously threatened the benign development of microblog ecosystem.In this paper,we study the anomaly detection methods for online social networks to find the abnormal accounts such as spammers effectively and quickly,block spam messages,malicious marketing and rumors dissemination from the root,and eventually purify the internet environment.Firstly,this thesis summarizes and analyzes the threats that malicious behaviors of the abnormal accounts bring to microblog users,service providers and microblog ecological environment comprehensively.Then,we summarize the detection features and methods involved in the domestic and foreigh existing research,and analyze the deficiency of existing online social network anomaly detection methods.Secondly,this thesis presents a detection model for online social networks anomaly detection.We get the labeled training set through unsupervised learning method,this can get rid of the interference of subjective factors such as a lot of time to spend as well as the labeling process of the supervised learning method,in order to reduce the influence of the sample quantity and quality on the detection results;then,we use cluster analysis and variance analysis to reduce the detection features,this can reduce the dimensionality of the features and extract more effective features at the same time;after that,we use NB,C4.5 and SVM supervised learning algorithm to detect the abnormal accounts and verify the validity of the model.The whole detection model combines unsupervised learning method with supervised learning method effectively,and it does not need to identify the samples in advance,so as to speed up the detection model generation and effectively avoid the limitations of a single learning method.Thirdly,because of the existing methods for anomaly detection of online social networks have some shortcomings,such as limited detection range,not comprehensive detection features and single detection method and so on.This thesis proposes feature system of online social networks anomaly detection,including four aspects:personal attributes,behavior attributes,content attributes and relation attributes,then we do empirical analysis about the differences and correlations of these features,the result show that the features we proposed can be more comprehensive and reflect the status and characteristics of the social network abnormal accounts fully.Finally,based on the validation of detection model,SVM algorithm is the optimal classification algorithm,so we design and implement the parallel SVM algorithm on the Hadoop platform to optimize the detection model.In the training process,we use clustering with random sampling method to partition data to each node,this can effectively guarantee the class distribution of the training set that divided into each node covering the same class distribution as the original data set and avoid the extreme case that random portioning leads to.After that,using genetic algorithm to optimize the kernel function parameter and penalty factor of SVM algorithm,the experimental results show the feasibility and effectiveness of the improved parallel SVM algorithm and its parameter optimization proposed in this paper.