GUIDE · CODING · #2367
Profiling TorchNeuron on AWS Trainium — system-level walkthrough (Part 1)
This tutorial shows how to profile PyTorch workloads on AWS Trainium using TorchNeuron (torch-neuronx). It covers configuring torch.profiler with NeuronConfig and NeuronProfiler, exporting matched system/device traces (NEFF/NTFF), inspecting execution with Neuron Explorer, and compares three RMSNorm implementations (naive PyTorch ops, fused torch.rms_norm, and torch.compile) with reproducible examples in the torch-neuronx examples directory.
KEY POINTS
- This tutorial shows how to profile PyTorch workloads on AWS Trainium using TorchNeuron (torch-neuronx).
- It covers configuring torch.profiler with NeuronConfig and NeuronProfiler, exporting matched system/device traces (NEFF/NTFF), inspecting execution with Neuron Explorer, and compares three RMSNorm implementations (naive PyTorch ops, fused torch.rms_norm, and torch.compile) with reproducible examples in the torch-neuronx examples directory.
- Provides practical tooling and a reproducible workflow for diagnosing and optimizing model performance on Trainium, helping engineers locate device vs CPU bottlenecks.
WHY IT MATTERS
Provides practical tooling and a reproducible workflow for diagnosing and optimizing model performance on Trainium, helping engineers locate device vs CPU bottlenecks.