Deepgram and AWS report that Nova-3 processed 11–70 percent more audio per dollar on Trainium2 than Deepgram’s previous compute options.
Deepgram and AWS said on October 19, 2026, that they had adapted Deepgram’s Nova-3 speech-to-text model to AWS Trainium2 for offline batch inference. The companies tested the setup on workloads including contact-center analytics and medical transcription, where throughput and infrastructure cost are central considerations.
In AWS’s evaluation, a single EC2 trn2.3xlarge instance processed 11–70 percent more audio per dollar than the compute options Deepgram had been using. The comparison reflects instance prices at the time of testing and applies to offline, throughput-oriented workloads; AWS did not provide an absolute audio-per-second figure in the post.
The port began with Nova-3’s PyTorch implementation and used AWS’s Neuron software stack. The team compiled the encoder and decoder separately, checked model outputs and word error rate against reference results, then optimized selected operations with existing and custom kernels. AWS reported, for example, that a hand-tuned feed-forward-network kernel reduced latency by 2.0 times versus its reference kernel at batch size 96. A custom convolution kernel also addressed a compilation limitation that had prevented larger batch sizes.
Serving required changes beyond the model itself. Deepgram’s production system is written in Rust, so the team created a runtime that calls compiled Neuron executables without using Python or PyTorch in the serving path. It runs eight Nova-3 replicas on the Trainium2 device. At the throughput-maximizing batch size, AWS measured a 7.5 percent gain over a multiprocessing Python baseline for this runtime.
The results are specific to Deepgram’s tested batch-inference configuration, not a general comparison across all AI workloads. The companies said the work could also benefit customers running Deepgram models in their own virtual private clouds on Amazon SageMaker AI, while noting that setup, reliability, and observability remain areas for improvement.