| As the fast-growing Internet sets higher and higher requirements for trading systems, low latency andhigh availability have become important indicators of judging a trading system. Trading systems shouldnot only meet the requirements for consistency and integrity of data, but also take the influence brought bydeferred transaction on user experience into consideration. Meanwhile, real-time analysis about datagenerated from the trading system also attract the attention of IT enterprises, because it is a commonchallenge in such a Big Data era that they should try to find the way to give real-time analysis to massivestructural and non-structural data, as well as to guarantee the stability of transaction data.First, this thesis introduces related technologies about trading system latency, that is, networktechnique, online transaction processing system and online analytical processing system. In theintroduction of network technique, a detailed comparison is given between10-gigabit Ethernet andInfiniband. The former is generally adopted in high-level network communication (TCP/IP etc.) techniques,and has a long transmission distance, while the latter has a relatively shorter transmission distance, and isoften used in low-level input or output communication techniques with an aim to improve the input oroutput performance of servers. The trading system is a typical online transaction processing system,real-time system in another way, because of its characteristics of fast-responding to clients’ requests andhigh instantaneity. Thus response time is an important indicator to measure online transaction processingsystem. The processing of data generated from the transaction is a data management technology based ondistributed system, which is also called online analytical processing system. In the society where cloudcomputing and Big Data is developing rapidly, Hadoop has become a mainstream platform of frameworks. The information search technology Dremel, launched by Google in2010, is proved to process data of1PBjust in several seconds.In this thesis, an integrated system framework is designed for real-time trading system and real-timedata processing system, and detailed introductions about each module are elaborated as well. In thereal-time trading system, aiming to improve the low latency and fault-tolerant ability, a memory replicationsolution based on Paxos is proposed. It is feasible to arrange data generated from transaction under theframework of Hadoop in the real-time data processing system, an application framework based ondeploying distributed systems on large cluster equipment, which consists of HDFS and MapReduce. Inaddition, the newest data processing technology Dremel is introduced briefly, which can process the datasaved in HDFS and MapReduce. Key techniques used in this whole system are Paxos algorithm, Hadoopframework technology and Dremel data processing technology. In this thesis, Hadoop frameworktechnology is introduced briefly because of its mature study, the working principles and advantages ofDremel data processing technology are introduced in detail, and the rest key technique, Paxos algorithm, isindeed the core of the article.At present, the main method adopted to guarantee the high availability of online transactionprocessing systems is log replication both in host and backup machines, which emphasizes the importanceof replica technique. The essence of replica technique lies in how to solve the problem of data consistency.Paxos is exactly an effective approach to solve such problem of data. The thesis introduces the basicconceptions of Paxos algorithm and three phases, and also lowers the system latency by residing memoryand persisting asynchronously. In the last part of this thesis, a framework design of trading system basedon Paxos is elaborated in detail, and the feasibility of which is also analyzed by means of studyingexperimental data. |
PDF Full Text Request