Exploratory Correlation Analysis Based On Visualization

With the advent of Web 2.0 and the big data era,the social networking and other types of networks have received a rapid development.Massive multidimensional data have brought great challenges to the relevant analysts and it is more and more difficult to intuitively analyze the data which contains rich relationships.In the traditional data relationship discovery process,data and the results of data analysis are presented in the form of texts or tables generally,so it is difficult for analysts to find hidden relationships from data.This thesis takes Weibo data as the object of study to analyzes the relationship between social network data.Different from the traditional data analysis methods,the method described in this thesis presents the individual user basic information,as well as its fan relations,user concern relationships and other information in a visual way,which uses the strong ability of human eyes to identify the image information to find the hidden relationship between the data.By the visual interaction technology,the data jump is achieved through simple operations to complete the secondary and multiple analyses which have a more explicit orientation.At the same time,the data mining algorithm is performed to analyze the whole data,and the results are displayed in a visual way to assist the analysts to find the data rules and the direction of decision analysis.In this thesis,this analysis method is named exploratory correlation analysis based on visualization.The method can be applied to a variety of industry-based data analyses,so that it is possible to make full use of people the irreplaceable role in the process of exploring data relationships,thereby improving the efficiency of data analysis.In this thesis,we take Weibo data as the research object,and the content covers the following aspects.Firstly,the big data analysis platform framework is designed and implemented.The Weibo data contains user information,user concern relationships,posted microblogs,microblogs forwarding relationships and other massive and complex structured data,so it brings up a higher demand for the relevant data access and analysis.This framework is designed on account of the properties of Weibo's massive and complex data structure and implemented based on the existing big data relevant technology to provide a reliable support for data analysis.Secondly,this thesis proposes an analysis model based on Weibo users.This thesis uses the K-Means algorithm to implement the user clustering based on interest relationship so that we can obtain the relationship circle partitioned by user interest.In addition,the FP-Growth algorithm is performed based on the user concern and microblog forwarding relation,so as to achieve the purpose of associate rules mining.When an analyst analyzes a particular user,this system will recommend other users associated with this user for the analyst to achieve the goal of supporting decision analysis.Finally,this thesis implements the visualization solution of user and microblog information based on Web pages.This thesis displays the basic user information,user concerns relationships,microblog forwarding relationships in a visual way.On the basis of analyzing the hierarchical structure of Weibo data,this system ensures that a large number of data nodes can have a clean and clear layout when presented in a limited space.At the same time,user and microblog relationship forwarding can be obtained by simple interactive operations to improve the analysis efficiency.