FUNDING · MODELS · #2210
Deepgram ports Nova-3 speech-to-text to AWS Trainium2, gains 11–70% batch cost efficiency
Deepgram worked with AWS to port its Nova-3 speech-to-text model to AWS Trainium2, using TorchNeuron, NKILib kernels and custom Neuron kernels; the final setup runs eight model replicas on a single Amazon EC2 trn2.3xlarge instance. On Deepgram’s offline, throughput-oriented batch workloads (e.g., contact-center analytics, medical transcription) this configuration processed 11–70% more audio per dollar versus the compute options Deepgram previously used, measured using instance pricing at the time of evaluation.
KEY POINTS
- Deepgram worked with AWS to port its Nova-3 speech-to-text model to AWS Trainium2, using TorchNeuron, NKILib kernels and custom Neuron kernels; the final setup runs eight model replicas on a single Amazon EC2 trn2.3xlarge instance.
- On Deepgram’s offline, throughput-oriented batch workloads (e.g., contact-center analytics, medical transcription) this configuration processed 11–70% more audio per dollar versus the compute options Deepgram previously used, measured using instance pricing at the time of evaluation.
- Reducing inference cost per audio lets Deepgram scale voice AI and offer lower prices or easier self-hosting for high-volume batch customers.
WHY IT MATTERS
Reducing inference cost per audio lets Deepgram scale voice AI and offer lower prices or easier self-hosting for high-volume batch customers.