NVIDIA Nemotron 3 ASR (English)
NVIDIA Nemotron 3 ASR (English)Nemotron 3.5 ASR is a streaming speech recognition model built for high-quality English transcription in both low-latency streaming and batch workloads.
Model details
View repositoryExample usage
This deployment is a Baseten docker_server model: the Baseten WebSockettransport proxies your connection straight through to the NIM's native Riva Realtime WebSocket API. There is no custom Baseten schema, and you speak Riva's OpenAI-Realtime-style JSON event protocol directly.
Two kinds of transcription messages
As audio streams in, the realtime API returns two kinds of transcription events:
Partial results (
...transcription.delta) — live, in-progress hypotheses that update continuously as you speak. Ideal for low-latency captions that refine in real time. Each event is an incremental delta, not the full hypothesis — concatenate the deltas within a segment to build the running caption.Final results (
...transcription.completed) — stable, punctuated transcripts for each completed segment. These won't change and represent the authoritative output.
Prerequisites
pip install websockets
export BASETEN_API_KEY=...Input must be a mono, 16-bit, 16 kHz PCM WAV. The server treats the bytes as raw pcm16 at the declared sample rate, so stereo or off-rate input transcribes to garbage (or nothing) — convert it before streaming (see the note under load_pcm).
Running it
python transcribe_to_json.py audio.wav out.json1import asyncio
2import base64
3import json
4import os
5import sys
6import wave
7
8import websockets
9
10URL = (
11 "wss://model-<id>.api.baseten.co/environments/production"
12 "/websocket?intent=transcription"
13)
14API_KEY = os.getenv("BASETEN_API_KEY")
15MODEL = "cache-aware-parakeet-rnnt-en-US-asr-streaming-sortformer"
16COMPLETED = (
17 "conversation.item."
18 "input_audio_transcription.completed"
19)
20SAMPLE_RATE = 16000
21CHUNK = SAMPLE_RATE * 250 // 1000 * 2 # 250 ms of 16-bit mono PCM
22
23
24async def transcribe(wav_path):
25 with wave.open(wav_path, "rb") as wf:
26 # need mono 16-bit 16 kHz PCM WAV
27 pcm = wf.readframes(wf.getnframes())
28
29 segments = []
30 async with websockets.connect(
31 URL,
32 additional_headers={
33 "Authorization": f"Api-Key {API_KEY}",
34 },
35 ) as ws:
36 await ws.send(
37 json.dumps(
38 {
39 "type": "transcription_session.update",
40 "session": {
41 "input_audio_format": "pcm16",
42 "input_audio_transcription": {
43 "model": MODEL,
44 "language": "en-US",
45 },
46 "input_audio_params": {
47 "sample_rate_hz": SAMPLE_RATE,
48 "num_channels": 1,
49 },
50 "recognition_config": {
51 "enable_automatic_punctuation": True,
52 },
53 },
54 }
55 )
56 )
57
58 for i in range(0, len(pcm), CHUNK):
59 await ws.send(
60 json.dumps(
61 {
62 "type": "input_audio_buffer.append",
63 "audio": base64.b64encode(
64 pcm[i : i + CHUNK]
65 ).decode(),
66 }
67 )
68 )
69 await ws.send(
70 json.dumps({"type": "input_audio_buffer.commit"})
71 )
72 await ws.send(
73 json.dumps({"type": "input_audio_buffer.done"})
74 )
75
76 async for raw in ws:
77 msg = json.loads(raw)
78 if msg.get("type") == COMPLETED:
79 segments.append(msg.get("transcript", ""))
80 if msg.get("is_last_result"):
81 break
82
83 return {
84 "transcript": " ".join(segments),
85 "segments": segments,
86 }
87
88
89if __name__ == "__main__":
90 wav_path, out_path = sys.argv[1], sys.argv[2]
91 result = asyncio.run(transcribe(wav_path))
92 with open(out_path, "w") as f:
93 json.dump(result, f, indent=2)
94 print(f"wrote {out_path}")1{
2 "transcript": "He hoped there would be stew for dinner. Turnips and carrots and bruised potatoes.",
3 "segments": [
4 "He hoped there would be stew for dinner.",
5 "Turnips and carrots and bruised potatoes."
6 ]
7}
