Design And Implementation Of Big Data Rapid Visualization System Based On HDFS

With the development of information technology and its penetration in various fields,many companies or organizations have accumulated a large amount of static historical data.People are already aware of the enormous value hidden behind these data.Therefore,excavating the knowledge it reveals is becoming a hot spot of the times.Visualization technology is one of the important branches in the field of big data analysis and has a wide range of needs.The visualization of massive data is generally used to reduce the amount of unnecessary data in order to quickly generate meaningful visualizations.There are excellent sampling algorithms in all fields,but they vary in their application background,usage methods,applicable visualization chart types,underlying physical implementations,and the principles and characteristics of databases based on them.Therefore,this paper selects three data reduction techniques that are suitable for generating line charts,bar charts,and contour maps that are outstanding in improving visualization performance from different fields or based on different databases,using Apache Phoenix as a data storage method,and designs a corresponding quick query solution based on this storage method.This paper also implements a contour mapping algorithm based on reduced data and provides a browser-based visualization system user interface.HDFS-based Big Data Rapid Visualization System implemented in this article has utility,variety,velocity,and compatibility.The system has a form-guided operator interface and front-end and interactive graphical zoom functions based on native components of the visualization tool and provides six chart types including line charts,bar charts,pie charts,radar charts,general contour maps,filled contour charts and two connectable data sources including HDFS static files,and Phoenix.Through various types of tests that accompany the entire iterative development process,this article ensures that the system is plug-and-play in a suitable environment and can meet the user's visual requirements.