Anomalous User Detection And Behavior Clustering Analysis Based On Telecom Data

Mobile communication services have become an indispensable social infrastructure in today's society.Telecom operators collect large amount of user-related telecom data from their servers every day.Various user behaviors can be revealed from these data,including illegal activities such as telecom fraud and coupon scalping.It is imperative to apply anomaly detection techniques to find anomalous telecom users in order to combat criminals,and apply clustering analysis techniques to construct user profile in order to achieve precision marketing,which will help telecom operators improve service quality and lower operating expenses.Based on machine learning and deep learning techniques,this paper solve the problem of anomaly detection and user behavior analysis based on telecom big data.A multi-faceted telecom user behavior analysis framework is proposed.The Hierarchical Locality Sensitive Hashing-Local Anomaly Detection(LSH-LOF)algorithm for anomalous telecom user detection and Autoencoders with Factorization Machine(FM-AE)algorithm for telecom data dimensionality reduction are proposed.Experimental results prove their effectiveness.K-means is then adopted to accomplish user clustering.The tasks of anomalous telecom user detection and clustering analysis are finished.The major innovations of this paper are as following:(1)The Hierarchical Locality Sensitive Hashing-Local Anomaly Detection(LSH-LOF)algorithm for anomalous telecom user detection is proposed.In the proposed algorithm,the k nearest neighbor searching process is improved,which results in efficiency improvement of outlier detection.Our proposed algorithm adopts the idea of LSH and achieves nearest neighbor searching from coarse to fine by gradually narrowing down candidate scope.Experiments with KD tree-LOF prove that our proposed Hierarchical LSH-LOF algorithm only consumes 1/7 of the time of KD tree-LOF and hardly degrades the detection performance.This paper adopts Hierarchical LSH-LOF for anomalous telecom user detection.Detailed anomalous user type analysis is given.(2)The Autoencoders with Factorization Machine(FM-AE)algorithm for dimensionality reduction is proposed.In the proposed algorithm,the network structure of Autoencoders is improved at the input side,which results in better preservation ability for important information in the original input data.Our proposed algorithm adopts the Factorization Machines to learning second order feature interactions at the input side of Autoencoders,which is then concatenated with the original data and fed into the neural networks.After an end-to-end training,lower dimensional hidden space vectors are obtained.Experiments with PCA,Kernel PCA,LLE,t-SNE and plain Autoencoders prove the information preservation ability of FM-AE.This paper adopts FM-AE for telecom data dimensionality reduction.K-means is then adopted to accomplish telecom user clustering.User profiles are given and precision marketing strategies are proposed.