Improvement And Implementation Of Data Processing Schemes For Mobile Internet Audit Platform

With the rise of the fourth generation mobile communication technology and the rapid development of intelligent terminals,the mobile devices count of the access network is increasing faster.In the Mobile Internet Era,data services gradually become the core requirements of users,and the data value will also become the most precious value of operators.Therefore,it is necessary and important to analyze and audit the data of mobile Internet services.However,just because of the popularity of mobile devices and the full coverage of mobile applications,the data is generated anytime and anywhere.The small and miscellaneous data packets,a large number of real-time active connections,a variety of applications have become the main characteristics of the mobile Internet.The requirements of the data processing speed become higher and higher.Only with efficiency real-time data processing can we make the mobile business data value.The original mobile Internet audit platform has a single process and multiple threads.It uses traditional Libpcap tools to capture packets,and uses DPI only for packets analysis,so that many performance issues of the audit platform appear.Frequent interrupt requests,mounts of data copies,time consuming DPI traffic identification and serial single task process severely limit the performance of the mobile Internet audit platform.When the flow rate is large enough,the processing efficiency of the system is worrying.This paper points out the reasons why the original system performance is so poor.In order to meet the real-time,high-speed,fine-grained mobile data audit requirements,improve the ability of data processing of the audit platform,this paper puts forward the design schemes of several stages in the process of data processing.In the data collection phase,we use the PF_RING+NAPI mechanism to capture packets,and make the corresponding improvement based on zero copy.In the data analysis phase,we use the DPI/DFI technology.In addtion,we propose a new method of pretreatment based on the cache hit,in order to reduce the DPI processing pressure.At the same time,the multi process architecture is used to redesign the audit platform,which can make full use of hardware resources and improve the overall performance of the system.Through the stage experiments,the performance of the schemes is tested.Experiments shows that the schemes in this paper complete the task of real-time and stable data processing facing high-speed mobile network traffic.we achieve the anticipated research goal,and help operators to carry out the follow up works,such as the value conversion of the mobile network data and the control of the mobile network etc.