Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #14423

GPUDirect RDMA

Related event timeline, sources and context from the news index.

EVENT TIMELINE

3

CODING · 1 SOURCE · AWS Machine Learning

vLLM adds push-based NIXL connector to speed KV transfer for disaggregated inference

vLLM introduced a push-based NIXL connector that moves KV-transfer coordination off the decode critical path for prefill/decode disaggregated inference. In benchmarks on EC2 P5en instances with EFA, push mode reportedly reduced time-to-first-token by up to 3× and improved time-per-output-token by up to 30% for Qwen3-32B, with results varying by workload and configuration.

6.0

CODING · 1 SOURCE · AWS Machine Learning

vLLM adds bi-directional KV transfer to reduce TTFT for multi-turn agent workloads

vLLM implemented bi-directional key-value (KV) cache transfer so decode nodes can return cached KV blocks to prefill nodes across turns, avoiding repeated recomputation of earlier responses and context. In 1P1D Amazon EC2 P5en+EFA benchmarks the change reduced time-to-first-token (TTFT) by up to 1.6x for Qwen3-32B and up to 3.3x for zai-org/GLM-5-FP8; results vary by workload and config.

7.0

CODING · 1 SOURCE · NVIDIA Developer

NVIDIA's DOCA GPUNetIO unifies GPU-initiated networking across DOCA and open-source stack

DOCA GPUNetIO provides a common GDA‑KI foundation that lets CUDA kernels directly drive Ethernet, RDMA/Verbs and DMA operations to keep CPUs out of the application critical path. NVIDIA ships GPUNetIO as a full DOCA SDK implementation and a lighter open‑source Verbs‑focused library; major libraries (NCCL GIN since 2.27, NVSHMEM 3.7, UCX/NIXL, NVQLink and Holoscan components) are adopting the shared path, with reported benefits such as improved small‑message scaling and sub‑microsecond-to-microsecond latencies in specialized workflows.

7.0