Design And Implementation Of User Analysis System Based On Big Data

With the continuous deepening of the degree of informatization,many super apps were born in China,which promoted the popularity of small programs.Software developers are often not limited to the development of APP,but the application of multiple platforms.And now that big data has penetrated into all walks of life,the analysis of user data using big data has become indispensable in the current fierce competition,and Precision marketing is already a common issue.How to help program developers who maybe use the serverless architecture to quickly realize multi-end and multi-platform user data summary and analysis,and help their profit is the most important.This article compares the current status of data processing,the status of big data platforms,and the status of user analysis systems,and extracts the essence to design a user analysis system based on big data ecology and including user tag mining services to solve the above problems at the moment.The system forms a user analysis system mainly including three modules of data processing,data display,and user tag mining by cooperating with a set of Vue-based front-end framework with no trace buried points.The data processing module realizes the connection with the front frame and stores the processed data;the data display module presents the user's analysis data to the software developer from multiple dimensions;the user tag mining module is to form the tag data from the user data mining to provide software development It can be used by other parties to help them achieve accurate marketing;at the same time,access to advertising systems can help them make money.The system uses Kafka and Flume to put the data into HDFS and corresponds to Hive,and uses Spark as the data operation engine to store the data in Mysql.The web end adopts B / S architecture,and uses Spring + Mybaits and Mysql to form a back-end for business data interaction.And cluster tags are generated by K-means,and text mining tags are generated by TF-IDF.The thesis expounds the design and implementation of user analysis system based on big data in combination with the thoughts and steps of software engineering.The transition from business to functional requirements,non-functional requirements,and based on this division of modules,and then shows the system architecture,data warehouse,database outline design.The important functions in the three modules are designed and introduced in detail.Finally,the function and performance of the system are tested.At present,this system has been put into use,and the monthly activity has exceeded 800 million,which provides detailed user data analysis for software developers.