Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #6567

TensorRT LLM

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

MODELS · 1 SOURCE · NVIDIA Developer

Co-Designing Sparse Attention: indexer and fused Top-k optimizations for long-context inference

A technical primer describes how to co-design model-level sparse attention and GPU kernels to make long-context transformer inference efficient. It analyzes the indexer (a batched matmul with a fused epilogue), Top-k selection, head/config knobs (KH, Hsz, query heads), and kernel-level optimizations (FP16/FP8, fused Top-k) and reports measured gains such as a 2.1x–2.4x speedup from reusing previous-step selections and a fused Top-k implementation.

7.0

CODING · 1 SOURCE · NVIDIA Developer

NVIDIA documents TensorRT LLM adaptations for confidential Blackwell (B200) inference

NVIDIA published guidance and measured results showing how TensorRT LLM adapts to confidential computing on Blackwell B200 hardware, including CC-aware memory selection, async token readback, using the GPU %globaltimer for autotuning, and communication-path detection when NVLS multicast is unavailable. In their tests, CC-enabled runs retained about 96.1–98.2% of token throughput versus CC-off, with per-token latency (TPOT) within roughly 1.2–4.3% of baseline.

7.0