Research And Implementation Of The Optimized Technology For Continuous Range Query Based On Storm

With the appearence of various location methods,mobile equipments and communication methods,the number of mobile data sources has gained a great increment.LBS,Location based service,which is typical of mobile applications has come into the era of mobile big data.Traditional LBS technologies face lots of challenges in the era of mobile big data,such as the lack of scalability and computing power.As cloud computing draws more and more attention,the industry and academics have emerged a trend of cloud computing recently.Because of its flexible scalability,strong processing capacity and high reliability,cloud computing has become an effective way to solve the problem of big data.This thesis does deep research based on this background.There are two main contributions in this thesis.First,this thesis designs a high throughput,scalable system framework of LBS using the computing resources and technologies of cloud computing.Second,this thesis optimizes the continuous range query which is a common application in LBS.For the system framework of LBS,this thesis firstly analyzes the characteristics of LBS in the era of big data to identify the essential features of LBS query system framework.Then this thesis chooses the system parts according to the characteristics of LBS.For the message queue,we choosed Kafka which is a distributed publish subscribe system.In order to improve the real-time performance,reliability and scalability of the system,this thesis chooses the Twitter Storm as the query processing part.In the aspect of data storage,this thesis uses HBase which is a distributed,column oriented database.Finally,the LBS query framework based on Twitter Storm is proposed.It is an effective solution for the storage,indexing and query of large scale mobile objects.At the same time,in order to keep data consistency of distributed clusters,we make use of the distributed lock service based on ZooKeeper.In addition,this thesis also proposes a general and extensible LBS query topology structure based on the characteristics of Twitter Storm.There are three parts with respect to the optimized topology for continuous range query in this thesis.First,according to the system framework of LBS based on Twitter Storm,this thesis designs and implements a parallel continuous range query algorithm based on Twitter Storm.This algorithm improves the query efficiency by parallizing traditional single thread algorithm.Second,beause of the problem that the overhead to access database is time-consuming,this thesis designs and implements a cache-optimized algorithm based on TimeCacheMap,which improves the the speed and efficiency of the query.Last,for the problem that user response time is too long in some application scenarios of LBS,the thesis does deep research on the the return strategy of the query results and proposes a result return strategy--the direct transmission of the subresults.It effectively reduces the user response time and improves the quality of service.