Design And Implementation Of Industry SMS Analysis System Based On Big Data

Industry SMS covers all major industries,including product promotion,event notifications and other forms.This has become one of the key ways companies can do business.SMS service providers provide enterprise customers with SMS sending and receiving services through an interface.All SMS sent via the platform will be stored and recorded,and the data volume can reach millions each month.For SMS service providers,the accumulated short messages will take up a lot of storage space,but the utilization rate of data assets is low.The company lacks fast,effective,and in-depth analysis methods and cannot grasp the business status.In addition,the company lacks effective risk investigation methods for fraudulent,political,and reactionary short messages that have certain risks.In addition,the staff need to frequently extract data from historical accumulated short messages through query,statistics,and number matching methods to make business reports.Finally,the short message data contains valuable information and can be used to build user profile,start new services such as precision marketing and paid information consultation,so that the data can play the value of secondary use.Therefore,for SMS companies,an industry SMS analysis system that can handle large amounts of data is very important.The research background and significance of the industry short message analysis system is first introduced.After that,the application of data mining technology in the field of text analysis and the use of big data technology are introduced.Then,data mining technologies,distributed storage,parallel computing and other technologies used in this system are introduced in detail.Secondly,this thesis has done relevant system requirements analysis according to the background.After that,,the system is designed in outline and in detail based on the results of the demand analysis.Finally,the industry short message analysis system is developed and implemented according to the design plan.The system uses multiple data analysis technologies and big data technology to realize rapid,effective and in-depth analysis of a large number of historically accumulated industry short messages,which meets the frequent and complex data analysis demand.This system has multiple innovation points and advantages:it supports multiple analysis methods,can create user portraits with rich tags,supports multi-industry categories,adopts multi-classifier integration,distributed storage and parallel processing.In addition,through data mining,a multi-label user profile can be established for each SMS user.It can help the company to grasp the business status,it can also cooperate with advertising companies to carry out targeted advertising based on this,so that the SMS data assets Create secondary value.In terms of industry categories,the system supports up to 59 industry categories.Due to the large number of classification tags,and each message may belong to multiple industry categories,in terms of classifiers,the integrated learning XGBoost algorithm that can handle multiple output problems and has great classification performance has significantly improved classification performance.