Design And Implementation Of Business Information Search Optimization Scheme Based On Big Data Processing

With the development of economy,financial information has become an important reference factor for individuals and enterprises.The development of big data technology has improved the ability of people to process data,enabling people to make more effective use of data.Eikon is the most important financial information service platform for Thomson Reuters.It provides a strong financial market information,analysis,and unique news portfolio for financial professionals who want to benefit globally from trading and investment opportunities.However,because the amount of data needed to be processed is too large and it is expected to continue to increase,Eikon is inefficient in carrying out data update,searching and maintenance.At present,the expansibility of data set is not high,and the cost of expansion and maintenance is huge.The reconstructing of the dataset of Eikon,which can not only improve the performance of data search and maintenance,but also reduce the cost of extending and maintaining the data set,thus improving the ease of use of Eikon.Big data processing technology is used to correlate the data according to certain logic to form the data view.When the user requests the data query,the existing complex SQL query statement will be replaced by a faster and simpler database data acquisition and scan.In this way,it not only helps users to use the data efficiently,but also helps the system to maintain and update the data efficiently.The solution will be implemented in the form of a system project.In the whole project design and development process,I am mainly responsible for the design and development of data management module,data processing module,rule transformation module and task scheduling module.The whole project is based on the Linux platform.It chooses Spark+Hadoop+Hbase open source big data architecture and uses Scala+Python language to ensure that the scheme has strong scalability.In this paper,the key technologies such as the analysis of business information and the construction of tree data structure are studied.At present,the development of the first version of the system has been completed and is in the trial stage.The operation effect accords with the design goal of the scheme.Actively receive feedback from users and data during run time for future updates and improvements.