Core Stateless Fair Queuing (CSFQ; Ion Stoica et al., SIGCOMM’98)

  • Edge routers maintain per-flow state; estimate incoming rate $r_i$ and encode it to packet header.
  • Core routers maintain no per-flow state; try to set a fair share $\alpha$ s.t., $ \sum_{i}^{}min(r_i,\alpha) \approx \text{capacity}$.
    • Drops packets probabilistically, if $r_i \gt \alpha$.

Paper’s thesis

  • Switched CXL memory pooling incurs contention between concurrent load/store streams across entire datapath, specifically at host uncore/adapter, CXL switch, and remote adapter.
  • It should be treated as end-to-end transport problem, and by contextualizing the CSFQ, a sender-driven bandwidth admission can improve fairness and utilization.

Key techniques

mChannel represents end-to-end $\text{Core}_i \leftrightarrow \text{remote-DIMM}_j$ memory stream.

For each 100us scheduling window:

  1. Estimate mChannel’s current rate and unconstrained demand (using per-core CXL request counters).
  2. Obtain mChannel’s fair share: (1) Aggregate traffic at each shared component (e.g., host adapter, CXL switch, and remote adapter) along the path; (2) compute CSFQ fair share for each component; (3) assign mChannel the minimal fair share (limiting its rate by the most congested component).
  3. Enforce through source throttling, specifically by converting into CPU duty cycle (execution/suspension time for the application thread).

Comments

  • MemChannel’s key contributions are: (a) Characterization of various choke points of switched CXL memory system (§2); and (b) Modeling it as an end-to-end transport problem.
  • Its novelty is contextualizing CSFQ into CXL transport, despite it was originally developed for Internet congestion control. For example, given the CXL’s lossless fabric, it replaces the packet drop at core router into a thread duty cycle. Its implementation, where the estimated fair share from the CXL fabric is finally realized in the end host as a simple SW-based thread throttling, is also interesting to see as a future design reference.
  • The proposed transport mechanism may have limitations in directly applying to networks that need sub-us control but limited component-level observability (e.g., lack of counters).