High Performance Data Center Systems With Programmable Network Interface Cards

Data centers are the infrastructure that hosts Internet services all around the world.Data centers face challenges on hardware and application.On the hardware side,per-formance improvement of general processors is slowing down.On the application side,big data and machine learning impose increasing computational power requirements.Different from Web services that are easy to parallelize,big data and machine learning require more communication among compute nodes,which pushes the performance of data center network to improve rapidly,and also proposes higher requirements for shared data storage performance.However,networking and storage infrastructure ser-vices in data centers still mainly use software processing on general processors,whose performance lags behind the rapidly increasing performance of hardware in network-ing,storage and customized computing.As a result,soft,ware processing becomes a bottleneck in data center systems.In the meantime,in cloud data centers,flexibility is also of great importance.To provide high performance and flexibility at the same time,recent years witnessed large scale deployment of programmable NICs(Network Inter-face Cards)in data centers,which use customized hardware such as FPGAs to accelerate network virtualization services.This thesis aims to explore high performance data center systems with pro-grammable NICs.Besides accelerating network virtualization,programmable NICs can also accelerate network functions,data structures and operating systems.For this pur-pose,this thesis proposes a system that uses FPGA-based programmable NIC for full stack acceleration of compute,network and in-memory storage nodes in cloud data cen-ters.First,this thesis proposes to accelerate virtualized network functions in the cloud with programmable NICs.This thesis proposes ClickNP,the first FPGA acceler-ated network function processing platform on commodity servers with high flexibility and high performance.To simplify FPGA programming,this thesis designs a C-like ClickNP language and a modular programming model,and also develops optimiza-tion techniques to fully exploit the massive parallelism inside FPGA.The ClickNP tool-chain integrates with multiple commercial high-level synthesis tools.Based on ClickNP,this thesis designs and implements more than 200 network elements,and con-structs various network functions using the elements.Compared to CPU-based soft-ware network functions,ClickNP improves throughput by 10 times and reduces latency to 1/10.Second,this thesis proposes to accelerate remote data structure access with pro-grammable NICs.This thesis designs and implements KV-Direct,a high performance in-memory key-value storage system based on ClickNP programming framework.KV-Direct bypasses CPU on the server side and uses programmable NICs to directly access data structures in remote host memory via PCIe.KV-Direct extends memory semantics of one-sided RDMA to key-value semantics and therefore avoid the communication and synchronization overheads in data structure operations.KV-Direct further leverages the reconfigurability of FPGA to enable users to implement more complicated data struc-tures.To tackle with the performance challenge of limited PCIe bandwidth and high latency between NIC and host memory,this thesis design a series of optimizations in-cluding hash table,memory allocator,out-of-order execution engine,load balancing,caching and vector operations.KV-Direct achieves 10 times power efficiency than CPU and microsecond scale latency.KV-Direct is the first general key-value storage system that achieves 1 billion operations per second performance on a single server.Lastly,this thesis proposes to co-design programmable NICs and user-space li-braries to provide kernel-bypass socket communication primitives for applications.This thesis designs and implements SocksDirect,a user-space socket system that is fully compatible with existing applications,achieves throughput and latency that are close to hardware limits,has scalable performance for multi-cores,and preserves high per-formance with many concurrent connections.SocksDirect uses shared memory and RDMA for intra-host and inter-host communication,respectively.To support many concurrent connections,SocksDirect implements an RDMA programmable NIC based.on KV-Direct.SocksDirect further removes overheads such as thread synchronization,buffer management,large payload copying and process wakeup.Compared to Linux,SocksDirect improves throughput by 7 to 20 times,reduces latency to 1/17 to 1/35,and reduces HTTP latency of Web servers to 1/5.5.