The Design And Implementation Of Customer Retrieval System Based On Hadoop Platform

There is no doubt that twenty-first century is the age of Internet+,Internet information in the explosive growth in the face of massive data storage and analysis,the traditional centralized search engine is difficult.People had put forward the strategy of distributed search engine.Based on cloud computing,distributed file system can make use of hardware resources reasonably,and it can be used in parallel.The Internet every day the massive data generated is a valuable asset,but there is no search engine,they are just a bunch of messy data,need to spend a lot of manpower to dig.The traditional search engines are based on keyword matching query,cannot fathom people's intentions,which makes it difficult for users to obtain accurate information they want,so the distributed intelligent search engine is the development trend of the future.Branches of hundreds of thousands of employees of enterprise distribution in the world,the need to provide unified for all employees in the enterprise portal search service search includes business data in the enterprise and employee related information.Most companies can't find their own value of the data fully,for example now most enterprise data is unstructured data,which includes the Word document,Excel table,PDF file,scanning pictures,E-mail,voice mail,telephone records,paper documents,photos,web pages,video and other forms of content.Becausemany enterprises lack the technology to understand and make use of these contents effectively,the resources which are very valuable and full of strategic significance are often unable to play its role.Enterprise data and miscellaneous lack of unified management platform,business personnel lack of technical support,the underlying data structure is not familiar with the technical staff only to mention the number,the efficiency is low.Then the natural language based Intelligent Cloud search system for the enterprise value is worth to looking forward.The system is based on big data platform through the new mobile industry terminology thesaurus,dynamic semantic web analytic model,the Lucene/Solr search server,realizes the user can input data by natural language retrieval.Through the dynamic semantic analysis model,the system can automatically collect,analyze,rich semantic entries,continue to improve the natural language and technical language "corresponding thesaurus.The calculation framework of heterogeneous data access using metadata repository and unified fusion documents,traditional databases,XML,MPP and Hadoop,structured data and non-structured data of several types of structural platform,information services provided by a unified platform;using intelligent collaborative tasks,realize the query of distributed processing,rapid response information query service.The system also through the use of Streaming Spark stream processing technology,the use of memory index,the establishment of the background data of the incremental updating mechanism,in a timely manner to provide users with the latest data.In this paper,through the analysis of large data and cloud computing research status at home and abroad,the application of data retrieval engine technology in the practical application of a carrier company.Currently,large data platform technology and distributed file storage system for massive data support has been stable and mature.Information technology companies attach great importance to dataprocessing technology can be applied to the actual operation process.This paper uses all kinds of open-source framework makes some operators according to the company's own characteristics of Application Research on the technology of data retrieval,at the same time through the construction of customer data to improve the level of enterprise information retrieval system,the use of data resources to create more benefits for the enterprise informatization.