Design And Development Of High-Concurrency SMS Research System

Didi Chuxing is a huge Internet car company that travels around the world.How to grasp the market changes to expand the scale of the company's operations,it is very important to obtain the first will of users.In the original questionnaires of all business departments,the questionnaires were self-maintained,and the users were separated by themselves.Different departments in the same period issued different research tasks,and there were cases where multiple research information was sent to the same user,causing users to resent.And because the survey data is not interoperable,it is impossible to effectively analyze and optimize the overall situation of the survey.The research system came into being,bringing all user research into one place and providing research services for the entire company.The surge in the number of users surveyed has put the test of the robustness of the system.Therefore,in response to these problems,the system has designed the main modules such as research management,research NPS(Net Promoter Score)questionnaire,data deduplication,and research transmission.The research management module ensures the de-emphasis range of the same research group and the same survey user through the relationship between the research group and the research task,which is the cornerstone of the system.Consider the user's tolerance for the content of the survey questionnaire and the length of the interview,to ensure that all or part of the survey information of the user can be obtained in real time,and the NPS questionnaire guarantee can obtain the user's phased answer.Adding the option of deduplication frequency when creating a survey task,so there are batch users to re-do the operation before the research SMS is sent.De-heavy subsystem module In the case of comparing various de-emphasis methods and referring to their de-weighting characteristics,the de-weighting method based on Bloom filter principle is selected here to achieve fast de-emphasis and low in the case of low false recognition rate.Memory usage.The research and delivery module adopts multi-process multi-tasking design to improve business processing performance.There will be multiple research tasks in the same time period,and the minute level will process hundreds of thousands of transaction data.The system is robust in terms of combining caching,database,task management and process scheduling.In the process of user deduplication,this article mainly compares the ARRAY_DIFF deduplication that comes with PHP,HASH deduplication,BITMAP deduplication and Bloom-Filter deduplication method.By comparing in turn,the BITMAP idea is used to do data storage to reduce the amount of memory space occupied,and the big data user information for the service is quickly deduplicated by the Bloom-Filter.This article combines the two to do big data user information deduplication.In terms of system design and development,the process management module uses the various Redis data constructed by the task management module to schedule and coordinate all processes for task processing,and realize multi-process multi-task parallelism to achieve multi-service data processing speed improvement.The exception handling module is responsible for the processing and automatic repair of tasks and process-related exceptions,and real-time monitoring and alarming for problems that cannot be automatically fixed.After the actual project is completed and run,the whole system runs stably under the condition that the tens of millions of user data is de-duplicated.The research and processing of the short message service involves a transaction volume of 20 W/min,which is in line with the expected effect.