The Financial Big Data Integration And Processing Technology Based On Semantics

With the arrival of the era of big data, data driven business model has been more and more popular among the enterprise. With the help of data, we can make better business decisions and scientific computing, and do market trend prediction. Data is more worthy than before. However, some new features of the Big Data have brought new challenges to the data processing.At present, for the financial industry, there is a lot of data produced by distributed financial system every day, which need to be collected, integrated and processed for risk management and forecast. On one hand, the complexity of the system structure, especially the diversity of the data structure, bring a challenge for the traditional data warehouse technology with heterogeneous data integration and processing; on the other hand, due to the particularity of the financial industry, is influenced frequently by outside, such as compliance, policies, etc., these changes will directly affect the data analysis method, thus affecting the final results.In view of the above two aspects, this paper bring up with a data processing system based on semantic that can deal with the heterogeneous data in the complicated financial environment. For data integration, the system publishes the heterogeneous data into semantic web ontology through semantics, and realizes the data query and calculation in the semantic level; for data processing, the system realize the distributed computing through Hadoop and MAP-REDUCE technology, and improve data processing performance through the parallel implementation.The structure arrangement in the paper are as follows:Chapter1, combined with the problem of Big Data and the Big Data background in financial field, summarizes the problems that finance industry is faced with, and propose a big data processing system model based on semantic. Chapter2mainly introduces the semantic, data integration, distributed computing and related technologies. Chapter3gives an introduction on the proposed system model. Chapter4, the paper introduces the Big Data integration schemes in detail; Chapter5, do some related research based on Big Data distributed processing technology and do some performance optimization based on the current system; Chapter6, summarize the full text.