Research On The Technology Of Data Analysis And Visualization In Big Data

The data explosion has resulting both opportunities and challenges to the information processing and data analysis.According to IBM Watson,2.5 billion gigabytes of new data is generated every day and the majority of this is unstructured.The numbers are massive and the data is not organized-and this impacts all businesses.A lot of work has been done on how to extract useful information from abundant and noisy data,among which data visualization is a particularly important part.In this thesis,we first reviewed main models and tools in the area of data visualization,and found that there are several aspects that can be improved in the visual analysis process.First,human cognitive characteristics has certain limitations in the data analysis process.While traditional mathematics analysis is insufficient to present what we want efficiently and friendly.Besides,how to select appropriate visualization model in different scenarios remains blocked despite the efforts of many researchers.In addition,some visualization tools pay attention to presentation rather than the affectivity of presenting information.After analyzing the process of data visualization analysis and the application of heuristic method in other fields,we proposed a heuristic visualization method to analyze data more efficiently.The proposed method pays more attention to coordinate the balance between analysts and machines thus helps understands the data.This thesis also introduces the design and implementation of a visual data analysis platform.This platform is based on B/S structure,using a popular visualization framework-D3.js.The core functions of the platform include data attribute analysis,visual model recommendation,dynamic visual model construction and so on.It is a practice of heuristic data analysis strategy.Our platform makes it simple to visualize data.Experiment shows that it improves the data visualization analysis,and gets better visual analysis results.In the future,more specific applications will be created based on this scalable platform.Besides,we will extend the system to visualize more models,optimize recommendation algorithm,and apply the system to other realistic scenarios.