Research And Implementation Of Data Processing And Visual Expression For Social Network

With the popularity of the Internet and rapid development of mobile technology,all enterprises and public institutions,government agencies,research institutions,and even individuals to produce large amounts of data,the data management and analysis of various units are facing huge challenges.For the advantages in relational databases in traditional business logic processing advantages,many units prefer to store the information in relational database.However,relational database can not be compentent for dealing with large numbers of complex,high density,connection and the low structured data,which bring considerable difficulties to the data analysis work.So how to manage the extract massive relational data into a form of figure data management,at the same time,the data in the form of a visual display to obtain important information hiding has caught the researchers' attention.This paper embarks from the actual demand,the first will be stored in a relational database data preprocessing,remove redundant data and null value.Then further data extraction,transformation,storage to non-relational database MongoDB.Finally,the system reads the data in a visual form and shows result.In the process of data preprocessing,this paper improved the existing sorted neighborhood method(SNM),and redesigned for dealing with vacant data algorithm,which is more suitable for the social network data processing.In the process of data extraction transformation,this paper designed a more targeted data structure to meet the requirements of processing a large number of social data.At the same time,for the convenience of users,this paper implements three visual method which are data preview mode,the common clustering model and based on granularity.This article adopt in the process of design and implementation of software engineering,making the system logic is clear,clear structure.In the process of data processing,Python language and Flask framework,NetworkX,D3.js,visualization library are used,to providing a good operation experience in system.The experimental results show that the designed system in fault tolerance and module function are consistent with expected design goal,in fault tolerance enhanced capacity of business related business department,and improved the work efficiency.