Research And Application Of Key Technologies Of Mobile Internet Information Integration And Position Retrieval

With the rapid development of the mobile Internet, everyone can get Internet information through the high- speed network in the smart phone, and mobile Internet search is one of the main ways to get the information. Mobile search has its own characteristics, due to the limitations of smart phones such as small screen, weak computing power, so in the face of mass results are often difficult to accurately locate the precise, personalized results.Location service is provided to the user with the support of the GIS platform to provide accurate location services. C urrently, many related industries such as social, life and other applications, are contained in a similar search function, to provide users with accurate services. Specific industry combines with the search function, which is the vertical position search service. Vertical type of location search service is the main integration of industry related resources, through the search to provide users with more personalized information services.In this paper, location service retrieval system was designed and implemented based on open source enterprise full text search engine Solr. Firstly, based on the vertical web crawler architecture, information integration subsystem was designed and implemented. It was divided into the task scheduling, duplicated URL removal, web disjunction and storage four modules respectively to study the related key technologies. Task scheduling module was mainly to study the depth first the URL of the scheduling policy; focused on the analysis and implementation of the Bloom Filter in duplicated URL removal module; In web disjunction module, mainly used the pyquery parsing HTML pages, as well as the research and analysis in the process of web crawler pseudo agent, anti hotlinking prevention techniques,etc; in storage module, mainly for the design and implementation of the My SQL database access interface.Secondly, mobile Internet location retrieval subsystem was designed and imp lemented based on the Solr, which mainly includes spatial indexing, index creation, search, cache and distribution in five aspects. In spatial index module, mainly studied and analyzed the Geo Hash and Cartesian Tiers layered algorithm principle, and specia lly discussed the Geo Hash coding algorithms. In indexing module, mainly implemented the smartcn,IKAnalyzer and Ansj three Chinese word segmentation efficient algorithm. How to build the query was studied in search module. In cache module, this paper emphatically researched and analyzed the Solr filter Q uery Cache, document Cache and query Result Cache three different cache principle and implementd them. about distributed section, mainly studied the Solr distributed indexing and query the princ iple and implemented the three nodes of a distributed cluster, improved the reliability and performance of the system.Finally, the functional tests were made for different functional modules, and the performance of the system was tested and analyzed by Solr Meter and Ganglia.