Font Size: a A A

Research On Key Technology Of Big Data Management And Query

Posted on:2018-06-09Degree:DoctorType:Dissertation
Country:ChinaCandidate:Y C YangFull Text:PDF
GTID:1318330542491531Subject:Computer application technology
Abstract/Summary:PDF Full Text Request
With the rapid development of information technology,such as cloud computing,mobile internet and smart terminal,data from all sides in economy and society is recorded.The era of big data has come.More and more people realize the great value of big data in commerce,society and science.It also stimulates great interest to development big data application technology in academic and industrial world.The management and analysis in big data based on cloud computing is becoming a major trend.However,the majority of researches in big data just focus on the technical architecture,and exist many problems such as low formalization of big data organization model,lack of heterogeneous data fusion and so on.Therefore,many aspects of research in big data could be expanded such as resource description and organization model,efficient integration of resources with heterogeneous anomalies,application formal modeling theory and so on.This paper mainly researches on the key technology for big data management and query.Based on the national business information service platform of e-goverment modeling and simulation,and application problems existed in this project,this paper mainly studies four issues,including entity information acquisition,effective data organization,multidimensional data retrieval and composite conditional query.The detailed research achievements are as follows:1?Entity information extraction problem:The paper proposes a P3E framework for entity extraction of Internet of things and explores its practical application technology based on the idea of using path expression.The formal definition of entity extraction problem is given by means of path expression and think of relaxation and verification.At the same time,IMP2E algorithm based on efficient automata is proposed.It is proved that the algorithm can effectively and efficiently solve the entity extraction pro'blem of Internet of things.2?Resource organization model in big data:The paper proposes a resource organization model in big data supporting multidimensional conditional query.Big data is reduced to adaptive subject scene traces by means of the complex adaptive system(CAS)theory.The information space is defined,the concepts such as scene,sub scene,entity instance identification,message,time,name and value are introduced into the information space.Trace information space is constructed which is the basic theoretical model of big data resource organization.3?Multimensional data retrieval based on trace information space:Based on the trace information space as the basic model of resource organization and management of large data,the paper researches the big data partition storage model with slicing in scene,entity instances and time,constructs of the two layer index with global index and local index,and designs the multi dimension data retrieval model.Compared with traditional traversal retrieval and hierarchical retrieval,it shows that this method is feasible and effective in support of multidimensional data retrieval.4?Compound condition query method:The paper gives the definition of composite query problem based on trace information space,with the query conditions described by the constraints of space and the query results described by the query region.On this basis,the formal specification of compound condition query method is completed and the system model of compound condition query method is given.Experimental results illustrate the effectiveness of the method.
Keywords/Search Tags:entity extraction, path expression, resource management model in big data, multidimensional data retrieval, compound condition query
PDF Full Text Request
Related items