Memory organization inside a server

  • Memory Controllers and Memory Channels (CHs) are integrated into processor socket.
  • If DIMMs are attached to CH’s slots, call it DIMM CH. If CXL devices are attached, call it CXL CH.

Paper’s thesis

  • The memory capacity and bandwidth should be exposed as separate provisioning resources.
    • The CHs provide bandwidth; The memory devices (DIMMs and CXL devices) provide capacity.
    • From cloud traces, their utilization isn’t necessarily correlated.
  • As the CHs and devices aren’t separable physically, decouple their allocation in SW.

Key techniques

Controlling HW mapping of CHs to VMs

  1. The default DRAM mapping (physical memory-to-CH), which is all-ways interleaving (e.g., 64B/4KB interleaving across the CHs), is turned off in UEFI. Alternately, the physical address space is linearly partitioned into separate contiguous regions, mapped to each DIMM CH.
  2. A CH’s memory region is mapped to VM’s Guest Physical Memory (GPM).
  3. Isolate LLCs across VMs (practically, each VM’s vCPUs are pinned to a chiplet CCX).
  4. Within a VM, each allocated CH is abstracted as channel-NUMA (cNode), and a group of cNodes as server-NUMA (sNode) according to the CH’s position in the physical server’s NUMA structure.
  5. Within a VM, page allocation is interleaved in a balanced manner (across sNodes-cNodes).

Adjusting capacity (C) “or” bandwidth (BW) of a VM

  • While DIMM CHs are statically allocated to VMs (proportional to # vCPUs), CXL CHs are dynamically scaled during runtime.
  • C is adjusted by changing the # of mapped bytes; BW by changing the # of spanned CHs.
    • A CH’s memory region is split into chunks (128 MB) as minimal capacity allocation unit.
    • To expand C while keep BW, keep # allocated CHs the same, while allocate more chunks from memory devices attached on these CHs.
    • To expand BW while keep C, increase # allocated CHs, reduce per-CH (its memory devices’) capacity allocation. Page migrations / redistributions typically follow.
  • Linux meminfo and performance counters are used as proxies to estimate when VM’s capacity or bandwidth utilization exceeds a threshold, triggering adjustment, which takes seconds to resolve.
  • While DIMM CHs are disjointly allocated (also LLCs), providing strong isolation, a CXL CH can be shared by complementary VMs, in which case their BW is contended.
    • Caveat: RamRyder opportunistically does this sharing w/o enforcing an exact BW quota.

Comments

  • RamRyder’s main value is demonstrating that memory capacity and memory bandwidth can be managed as separate cloud resources by controlling how VM pages are distributed across physical DIMM/CXL channels.
  • Its key strength is the channel-centric memory abstraction and software realization: exposing per-channel NUMA nodes, allocating memory in chunks, and dynamically remapping pages so a VM can gain BW w/o gaining capacity, or vice versa. This breaks the common systems assumption that bandwidth is an auxiliary resource implicitly determined by capacity allocation, and aligns w/ their less-correlated trend (from the traces).
  • It has limitations as a precise runtime BW-control mechanism. The adjustment is slow, taking seconds. Shared CXL channels interefere w/o guarantees; finding VM pairs with complementary demands (for capacity and BW), as in the paper, often leads to flexibility and complexity issues.