NVIDIA demonstrates TensorRT Edge-LLM workflow to run Nemotron-3.5 Lightning on DGX Spark 64GB
NVIDIA published a tutorial showing how to use TensorRT Edge-LLM (examples tested with Edge-LLM 0.11.1 and TensorRT 10.16.1.11) on a DGX Spark 64GB to prototype, fine-tune and test embedded AI models. The trtedge CLI downloads models and builds optimized TensorRT engines, supports serving via a local OpenAI-compatible API, and can enable DSpark speculative decoding (reported to raise mean generation from 73.4 to 126.9 tokens/s on HumanEval prompts while keeping peak memory at 27.9GB).