This paper uses Compute-to-Memory (C2M) and Peripheral-to-Memory (P2M; e.g., SSD-DRAM) micro-benchmarks, and finds the two contention regimes within a host network under concurrent execution.

  • Blue regime: Only C2M performance degrades
    • C2M read + P2M read, C2M read + P2M write, C2M write + P2M read
  • Red regime: Both C2M and P2M performance degrades, where C2M degrades earlier (i.e., at lower contention intensity).
    • C2M write + P2M write

Before assessing the two regimes, the paper first identifies and characterizes the intra-host flow-control domains traversed by each transfer. Here, it studies four transfer types by varying its source (C vs. P) and type (read vs. write), and a domain indicates credit consumption/replenishment boundary. Using Intel PMUs, it infers each domain’s characteristics as follows:

  • Latency: Derived using Little’s law from measured average occupancy and request arrival rate.
  • Boundary: Identified via latency inclusiveness: e.g., if LFB latency increases (and matches) CHA→DRAM latency inflation, CHA→DRAM lies within the domain starting at the LFB.
  • Credit count: Inferred from the maximum sustained occupancy. Measured credit counts (in cache lines) are: ~10 for LFB, ~90 for IIO write buffer, and >160 for IIO read buffer.

As a result, for the C2M/P2M datapaths below,

C2M: CPU Core → L1 → LFB → L2 ──┐
                                ├→ CHA (LLC) → MC (RPQ/WPQ) → DRAM
P2M: Peripheral → PCIe → IIO ───┘

the domains are identified, and the bottleneck domain for each transfer are:

C2M-Read:   LFB ───────────────────────────────────→ DRAM
C2M-Write:  LFB ──────────→ CHA

P2M-Read:   IIO ───────────────────────────────────→ DRAM
P2M-Write:  IIO ────────────────────────→ MC

Domain-by-Domain Flow Control. Finally, the paper explains the two regimes by the interplay between their bottleneck domains.

  • Blue regime: Queueing at MC inflates C2M domain latency, directly reducing its throughput as C2M has few credits. P2M remains largely unaffected because: (1) for P2M write, DRAM lies outside its domain, so only admission to the MC’s WPQ matters; and (2) P2M domain has substantially more (unused) credits, hence can hide the latency inflation by pushing more inflight requests. Thus, a shared contention hurts C2M without reducing P2M throughput.
  • Red regime: WPQ saturation backpressures P2M write domain which includes the MC, whereas C2M write is less affected as its domain ends at CHA. At higher load, write backlog fills CHA, and CHA admission backpressure further degrades both C2M and P2M.

Comments

  • The paper’s strongest contribution is its domain-by-domain credit-flow abstraction, explaining why two transfers sharing the same physical resources can experience very different contention outcomes. Careful reverse-engineering results consistently supports the claimed contention regimes. It is much stronger than simply identifying an architectural component as the bottleneck.