App Lication Of Spark-based Real-time Efficient Processing Algorithm In Internet User Behavior Analysis Platform

With the development of Internet technology,people's daily life has gradually realized electronic,lightweight and efficiency.Internet technology defines a new way of working and entertainment,and continues to advance with the derivation of user behavior and needs,and strongly promote the development of the current era.Nowadays,customers are able to convey and acquire information anytime and anywhere,regardless of the distance.Information recognized by equipment can be quantifiable saved rather than naked-eye observation.Besides,data created by customers' behaviors is not dispersive but measurable.During the analysis of customer-behavior,it includes anthropology,sociology and psychology.If these data can be fully and efficiently used by ordered and scalarized analysis,it can enable enterprises to understand users' behavior habits in more detail,and more accurately judge problems such as enterprise operation and marketing environment,so that enterprises can make more accurate and efficient decisions on their own policies,and then provide better services for users.Firstly,this thesis introduces the basic concept of Internet user behavior analysis platform.Secondly,some theoretical basis is introduced,including Spark architecture and computing mode,Spark graph computing framework,spectrum clustering algorithm,Hive principle and DubboRPC framework.In this thesis,through the Spark technology and related components,based on the terminal identification,user trajectory data merging,airport scheduling these three actual engineering tasks,we design a corresponding real-time and efficient algorithm.Finally,through DubboRPC framework and parameter transmission,a flexible and universal monitoring system is designed for the Internet user behavior analysis platform to realize real-time status monitoring and fault diagnosis alarm.