Research On The Progressive Visualization Method Of Big Data Of Earthquake Precursor Time Series

With the increasing amount of seismic precursory observation time series data,the existing visualization scheme based on Web,due to the large amount of network transmission data,leads to the system response time becoming longer,which seriously affects the user's interactive experience.Although the big data visualization scheme based on sampling and filtering can greatly reduce the data scale and shorten the data transmission time,it will lose the information behind the high-frequency data.In the face of such a large amount of seismic observation data,how to efficiently store and quickly visualize the data to meet the needs of users has become an urgent problem to be solved.In order to solve the above problems,based on the study of the current mainstream relational database represented by My SQL,distributed file system represented by Open TSDB and No SQL database represented by HBase,this paper proposes a big data storage scheme of earthquake precursor time series using HBase database to solve the problem of low reading and writing efficiency of massive earthquake precursor data.In view of the shortcomings of the existing web-based visualization scheme,and considering that the current seismic precursory observation data are stored in the relational database,a Cloudberry based progressive visualization scheme of seismic big data is proposed.In addition,in the big data environment,the storage and application of big data has become a more and more popular trend.Therefore,a progressive visualization scheme of seismic big data in the big data environment is proposed.The main work of this paper is as follows:(1)This paper proposes a scheme of seismic big data storage based on HBase.The traditional relational database has the problem of low reading and writing efficiency in storing massive earthquake precursor observation data.Considering the characteristics of earthquake precursor observation data,application scenarios,real-time and fast query data,the earthquake precursor observation data is saved in HBase database.The experimental results show that the storage scheme based on HBase has excellent scalability and concurrency,and shows good performance in both read and write operations,which fully proves the effectiveness of the scheme.(2)This paper proposes a Cloudberry based progressive real-time visualization solution for seismic big data for the first time.The scheme greatly shortens the transmission time of each piece of data,improves the user interaction experience,and well meets the needs of long-term and large-scale observation data visualization in seismic business.Aiming at the problem that the correctness of average aggregate function(AVG)is often not guaranteed in current progressive visualization schemes including cloudberry,this paper proposes an innovative AVG conversion rule technology solution based on the cumulability of COUNT and SUM aggregate functions to ensure the correctness of AVG results.The experimental results show that,compared with the non-progressive visualization scheme,the progressive visualization scheme based on cloudberry can see the results immediately without waiting for a long time.Compared with P5 progressive visualization scheme,Cloudberry has shorter response time per batch,and with the increase of data volume,Cloudberry can always keep the response time of each batch within the acceptable range of users.Therefore,Cloudberry progressive visualization scheme shortens the response time of users,avoids long-term waiting,and improves the user interaction experience.(3)This paper proposes a progressive visualization scheme in big data environment to meet the needs of big data scenarios.Because Cloudberry does not support HBase database in big data environment at present,although it can read data from HBase through elasticsearch,this scheme obviously involves large-scale data migration in the network,which is time-consuming and a waste of computing resources.Therefore,this paper imitates Cloudberry to propose a progressive visualization scheme under the big data environment,and applies the research results to the project of "Earthquake Big Data Visualization and Machine Learning Platform".The application results show that the progressive visualization scheme of seismic big data has the characteristics of rapidity and flexibility,and has extensive and practical value for the visualization of massive earthquake precursor observation data.