| In recent years,with the rapid development of computer technology and continuous innovation in related fields,the development of politics,economy,science and technology,education,industry and our daily life,work and study depend more and more on big data analysis and intelligent production.Our country provides strong support for the development of science and technology industry.From "the National Science and Technology Innovation Plan of the 13th Five-Year Plan" to "the New and High Technologies Areas Supported by the State",various preferential policies have brought convenience to the development of small and medium-sized micro-science and technology enterprises,and provided preferential policies in finance,taxation,financial subsidies and stipulated restrictions.However,in the specific work,it will be found that it is not easy to find quasi technological innovation enterprises from large enterprises,so in this context,the data analysis technology and intelligent assistant system are particularly important,which can greatly improve the efficiency and accuracy of the work.This system is a data filtering management system based on Elastic Search distributed full-text search engine.At present,the main work is to have completed the screening of the data of small and medium-sized enterprises in Shandong Province,Based on the enterprise names and business scope data provided by Shandong Industry and Commerce Bureau in the list of 2 million small and medium-sized enterprises,according to the instructions of the document for the scientific and technological enterprises in "the National Science and Technology Innovation Plan of the 13th Five-Year Plan","the High-tech Areas Supported by the State",and the Ten Domains of the Transformation of New and Old Kinetic Energy in Shandong Province",the scores of the scientific and technological conformity of nearly 2 million small and medium-sized enterprises in Shandong Province were calculated,and the quasi-scientific and technological enterprises were selected and stored in the quasi-scientific and technological enterprise reserve according to the ranking of the scores.In addition,the system based on Elastic Search distributed full-text search engine achieves a number of functions,including supporting users to dynamically screen,generating filter reports,querying and exporting according to different conditions,managing the database,managing the basic key,Awords,and so on.At present,the data screening management system based on Elastic Search distributed full-text search engine is mainly divided into four functional modules,namely,dynamic screening management-screening source data according to dynamically imported keywords and visually displaying the results of screening;Data-warehouse management-batch management and query management of the data already stored in the database;Base keyword management-provide keyword management and query functions;User Management-role assignment and rights management for users.The system is developed based on B/S architecture,the front and back end are separated.The back end uses the popular S SM framework offline,namely Spring+Spring MVC+Mybatis.The database uses MariaDB,which is more efficient than the traditional MySQL,and the index technology is used when building the database,which makes the index redundancy for the data with large amount of queries and greatly improves the query efficiency.Using the open source distributed full-text search engine Elastic Search and the open source Chinese word tokenizer IK,in order to ensure accuracy,the coarse-grained pattern of IK word segmenter is used to make the matched data as accurate as possible.In order to adapt to the situation that there are many and miscellaneous data filtering and classification standards,we have made an independent base keyword library based on classification standards and custom stop words dictionary that uses the hot update function of IK tokenizer to ensure the real-time updating of the basic keyword library.In data screening operation,multi-threading technology is used,multi-nodes perform screening tasks simultaneously,and synchronize through locking technology,Through the characteristics of tree-like relationship between data nodes,layer-by-layer operations are carried out,and finally merged into the root node to complete the entire screening task,which greatly improves the screening efficiency. |
