Research And Implementation Of Data Visualization Platform For Log Monitoring

With the rapid development of big data technology in the Internet era,the log data generated by various nodes has grown massively,covering more scenarios and multiple dimensions.The textual log data is displayed in the form of a situation graph,which can visually view the data trend and help the analyst quickly capture the internal laws of the data.Therefore,how to convert multi-source log data into graphic images and display them in real time has become a research hotspot in the direction of network security.The problems with traditional log visualization applications are:(1)Incomplete view types are supported,and the construction process is specialized.(2)The lack of support for the multiple real-time capabilities of the log dashboard and the interactive experience of the dashboard needs to be improved.(3)The lack of a complete product design for the log visualization construction process.In response to the above problems,this topic focuses on analysis and research around real-time dashboards,view construction,and free customization of dashboards.The main research contents are:(1)Propose and implement a lightweight,expansive configuration and self-help analysis of the exploratory graph fusion technology,which supports comparison and visualization of the same index from multiple sources,the same source from multiple index,and the same source and index items from multi-dimensional.Based on the characteristics of the data,three types of views are available for selection,which are timing diagrams,non-time series diagrams,and single value diagrams.(2)Design and implement a variety of real-time visual methods and provide users with rich operations from different dimensions.Real-time visualization supports dashboard-level and view-level real-time,and provides manual mode and configuration mode,respectively,achieving stable real-time of log data in seconds.(3)Designed and implemented the VUE-based data visualization tool DVT,Among them,exploratory graph fusion technology and multiple real-time visual methods provide technical support for the visualization tool DVT.And realized one-stop construction of the dashboard with the goal of visual analysis of logs At the same time,data processing,view generation,dashboard construction,management,interaction and other visual services are implemented.It is easy to use and easy to operate.(4)Design and implement a data visualization platform for log monitoring.The data visualization tool DVT provides visualization capabilities.The whole implements an intelligent log analysis visualization platform that integrates data collection,data processing,data storage,exploratory visualization,and real-time dashboard.It has formed a complete,convenient,one-stop log analysis visualization solution and system implementation capability that integrates data collection,processing,analysis,and visualization.The platform has been fully tested in terms of both function and performance to verify the platform's process integrity and user-friendliness.Based on the research results of the above four key contents,this paper systematically designs and develops a data visualization platform for log monitoring.The platform combines related technical research and application scenarios to implement a data monitoring platform for log monitoring,which is flexible,convenient,rich in interaction,complete in process,and visually beautiful.