Problem: RDMA NICs (RNICs) implement a HW-baked transport layer.

  • Transport mechanisms, such as path-selection and CC, are fixed in HW.
  • This can easily mismatch fast-evolving ML communication needs.
  • E.g., Meta disabled RNIC’s built-CC (DCQCN), instead relying on traffic scheduling at app layer.

Challenge: Decouple the transport control path from the data path, i.e., keep data movement on RNIC and GPU path while moving flexible control into CPU SW, but retain HW-level performance.

  • UCCL sits between CCL and low-level RDMA primitives, remaining transparent to GPU applications while providing an extensible SW-defined transport control path for GPU RDMA networking.

Key techniques

Decoupling by RDMA Unreliable Connection (UC) + Write with IMM

  • RDMA supports various QP types (default: RC; UC/UD) and verbs (read/write/send/recv).
  • UC bypasses RNIC transport control path except packetization and reassembly.
  • Write w/ IMM transfers 32KB data chunk to DMA-mapped GPU while delivering 32bit IMM to CPU.
    • IMM serves as transport control header; A successfully received chunk’s IMM encodes conn/msg IDs, chunk SEQ, end-of-msg, CC bits.
  • UCCL has alternative mechanisms if UC unsupported.
    • E.g., for UD, which transfers in MTU-sized packet per verb and lacks IMM, UCCL uses RDMA scatter-gather to merge CPU control header + GPU payload into one MTU-sized packet and split them back into CPU/GPU memory at the receiver.

Enabling multipath with many QPs

  • Create a shared pool of 256 QPs btw each NIC pair; multiple logical connections share this pool.
    • Many QPs were considered expensive due to QP context swap across NIC SRAM-CPU DRAM.
    • UCCL found it less critical for GPU collectives: (1) efficient hiding with mostly MTU-sized packets; and (2) GPU Direct not colliding with NIC-CPU PCIe path.
  • Multiple QPs translate to different ECMP hash; chunks from the same connection traverse different network paths at RoCE/IB fabric.
  • Packet reordering is handled in SW/GPU memory: Selective retransmission implemented in SW.

Making CPU-based transport efficient despite visibility constraints

  • Reduce increased CPU overhead: e.g., make control decisions per 32KB chunk, not per MTU packet.
  • Despite lacking packet-header congestion signals, use per-path RTT as primary congestion signal.
    • RTT is derived from RNIC TX/RX timestamps.

Three use cases

Multi-path: Hard for RNICs due to expensive per-path states (e.g., RTT)

  • UCCL maintains per-path RTTs and chooses least-RTT path.
  • Then, SW CC (CUBIC/Swift) decides amount and pacing to send out the data.

Receiver-driven CC: No off-the-shelf NIC supports it

  • UCCL implementes EQDS with pacer thread per receiver NIC, issuing credit packets to senders.

Selective retransmission: RNIC Go-back-N drops out-of-order packets due to small NIC SRAM

  • UCCL keeps ROB at GPU memory and track out-of-order in SW, providing more selective control.

Comments

  • UCCL convinces that GPU RDMA networking can move its current transport control at RNIC HW into CPU SW.
  • While vendors are geared toward tigher GPU and RNIC lock-in, such as NVIDIA GPUs and Mellanox NICs through IBGDA – GPU threads directly posting RDMA commands to the NIC – providing the highest performance, UCCL’s niche is on the flexibility, providing workload-specific optimization capabilities.