Design And Implementation Of Real-Time Search System Based On Elasticsearch

With the rapid spread of the Internet,Internet companies have accumulated a huge amount of data,and the amount of data on a single table often reaches 10 million,and is still growing rapidly.Due to the ease of use and high stability of relational databases,most companies use relational databases as the only way to search for data.However,in the face of such massive amounts of data,directly using relational databases for data search will take several seconds,which is unacceptable for most services that require real-time processing.However,due to the stability of the relational database and the advantages of providing transaction functions,it is indispensable in many business scenarios.In view of the above background,this thesis researches and implements a real-time search system based on Elasticsearch for the purpose of providing efficient search performance when facing large amounts of data.The main work of this thesis is as following:(1)The real-time search system based on Elasticsearch is analyzed in detail based on the actual environment.Based on this,the overall structure,main functions and database structure of the system are designed.(2)The relational database incremental data acquisition method was studied.Combining with the actual environment,the method of obtaining inventory and incremental data separately was proposed.At the same time,for the problem of data acquisition and index construction speed mismatch,the acquisition of Data is cached by the relay processing method.(3)Combining the needs of actual business and the concurrent write function provided by Elasticsearch,the method of building Elasticsearch index is proposed,including data filtering,data assembly and transaction sequence control.The sensitive information is processed through data filtering,and the data assembly function is used to implement customization.Business needs,the use of transaction sequence control to ensure that the order of data written to Elasticsearch consistent with the write relational database.(4)A real-time search system based on Elasticsearch was implemented,and the retrieval performance and data synchronization delay were tested.By comparing the search performance of the system and MySQL,it is proved that the system can provide efficient search performance when facing massive data.