Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #29708

Ascend910C

Related event timeline, sources and context from the news index.

EVENT TIMELINE

1

RESEARCH · 1 SOURCE · arXiv cs.AI

FlashOffloading: KV-cache offloading architecture for long-context LLM serving (vLLM‑Ascend v0.30.0)

The paper presents FlashOffloading, a host‑DRAM‑centric architecture that fully offloads the KV cache for both prefill and decode phases and adds a rolling HBM cache, hot decode cache, reverse sparse index, and compute–load overlap to minimize offloading latency. In experiments on an Ascend910C PD‑disaggregated platform running GLM‑5.2 with MTP3 speculative decoding, FlashOffloading improved prefill throughput up to 1.38× and decode throughput up to 1.91× under a 30 ms time‑per‑output‑token (TPOT) constraint; the design is integrated into the vLLM‑Ascend open‑source release v0.30.0.

6.0