Research On Scientific Big Data Query Processing Technology Based On Workflow

With the rapid development of scientific technology recent years,the precision of scientific experiment and observation instruments have improved a lot,thus making the scientific data become larger and larger.Meanwhile,with the advance of scientic analysis measures,the scientic analysis becomes more and more complex.So,it’s important to improve the efficiency of field experts’ scientific big data query execution.The paper builds a light cloud platform,based on which build a SciDB cluster to analyze the scientific big data.Based on that the main research content of the paper is listed as below:(1)scientific big data storage and query technology.Focusing on the research status of scientific big data query execution at home and abroad,the paper researches the MapReduce framework and current scientific databases,and employs SciDB cluster as the scientic analysis tool.(2)Data loading technology.The corresponding time of SciDB data loading progress doesn’t perform well with the data size increasing.To solove this problem,the paper gives a novel data loading algorithm.(3)The paper designs a workflow-based scientific query system to simplize the writing of query language.(4)Due to the inefficiency of scidb-based batch workflow schedule,the paper gives a series of optimizeing algorithm,improving the efficiency of batch workflow execution.