Building High Performance Networked Systems On Multicore/Manycore Architectures

In the big data era, the ever-increasing network speed and data volume has ex-ert a great impact on the network devices. Moreover, as the network application is becoming more and more complex, the workloads of network systems have been trans-ferring from simple packet forwarding/filtering to complex takes such as deep packet inspection, network intrusion detection, compute-intensive processing (encryption/de-cryption, compression/decompression), and data caching. However, current network systems mainly rely on specific hardware such as ASIC and network processor, which not only lacks of flexibility but also faces many issues such as long development cycle.With the fast development of general-purpose multicore and manycore processors, utilizing general purpose processors to build high flexibility and high performance net-work systems has drawn lots of attention from both industry and academia. However, building high-performance network systems on general-purpose processors faces lots of challenges, such as 1) general-purpose operating systems are inefficient for high-performance network processing; 2) parallelizing network application is hard as locks involved are harmful for achieving high performance; 3) high-speed network traffic has no locality and therefore cannot utilize the CPU cache; 4) GPU-based acceleration is hard to balance between throughput and latency, etc..This dissertation takes several network systems as the research objectives, and conducts research on the key techniques in building high-performance network systems with general-purpose multicore and manycore processors. Based on these techniques, a set of solutions for the above challenges are proposed, and several system optimization techniques are adopted in building practical high-performance network systems. The main contribution of this dissertation is as follows.1. For the lacking of methodology in building customizable high performance ap-plication layer protocol parser, we propose to build parsers based on open source FLEX software, and build a parallelized parser framework on multicore proces-sors. Based on this methodology, we build a high-performance HTTP parser and a FIX parser.2. For the inefficiency of the TCP/IP stack, we propose a new TCP index data struc-ture. Based on general-purpose multicore processors, the index data structure demonstrating 10Gbps throughput while maintaining one million concurrent TCP sessions.3. For the high and unpredictable latency introduced by GPU processing, we pro-pose a GPU-centric realtime scheduling strategy and a parallel framework. Based on these techniques, we build a GPU-accelerated realtime SRTP reverse proxy, which achieves a high throughput with a predictable latency.4. For the inefficiency of CPUs in processing key-value queries, we propose a GPU-based index data structure and the corresponding algorithms, and utilize GPUs in building an in-memory key-value store system. To the best of our knowledge, this is the fastest in-memory key-value store system currently.This dissertation makes an exploration on utilizing general-purpose processors in building high-performance network systems, providing lots of key techniques and valu-able experiences for building general-purpose processor based network systems. All the projects in the dissertation are open source and ready for practical use.