Utilizing remote or distributed memory to improve application performance and resource utilization is an idea dated back to the 80s and 90s. Recent datacenter network developments and datacenter applications’ memory needs are making this old idea appealing again. In this talk, I will discuss several networking and systems challenges in building distributed memory systems in today’s datacenter environments and our solutions to them.
Remote Direct Memory Access (RDMA) is a technology that enables fast, direct remote memory access. However, native RDMA is not readily suitable for datacenter applications. It lacks a flexible, high-level abstraction; its performance does not scale; and it does not provide resource sharing or flexible protection. To solve these issues and make RDMA suitable for remote/distributed memory applications, we built LITE, a Local Indirection TiEr in the Linux kernel that virtualizes native RDMA into a flexible, high-level, easy-to-use abstraction. Despite the widely-held belief that kernel bypassing is essential to RDMA’s low-latency performance, we show that using a kernel-level indirection can achieve both flexibility and low-latency, scalable performance at the same time.
With LITE’s flexible abstraction and good network performance, we propose Distributed Shared Persistent Memory (DSPM), a new framework for using persistent memories in distributed, datacenter environments. DSPM allows applications to both perform traditional memory load and store instructions and to name, share, and persist their data. We built Hotpot, the first DSPM system, in the Linux kernel. Hotpot offers low-latency performance, data durability, reliability, and high availability.
1131 Kemper Hall