Qwen3 ASR 1.7B Streaming
Qwen3 ASR 1.7B StreamingReal-time streaming speech-to-text from Qwen3-ASR 1.7B over WebSocket, featuring batch-level accuracy, sub-second finals, and 30+ languages.
Model details
Qwen3-ASR 1.7B (Streaming) is a real-time speech-to-text service that runs Alibaba's open-weight Qwen3-ASR model behind Baseten's streaming harness over WebSocket. Because Qwen3-ASR is an offline audio model, the server uses growing-window re-inference: within each speech turn it re-decodes the utterance so far with full acoustic context to produce revisable live partials, then does a full-context decode at the endpoint for a high-accuracy final. Server-side voice-activity detection ends turns automatically, and an explicit commit also finalizes a turn.
This recovers full batch-level accuracy in a streaming setting — roughly 10% macro semantic WER across a 5-locale telephony benchmark (≈4% en-US), while delivering sub-second final latency and low time-to-first-partial, sustaining real-time streaming at high concurrency on a single GPU. It auto-detects language across 30+ languages (52 including Chinese dialects) and can be pinned to a specific language. Responses use the same envelope as Baseten's streaming-Whisper model, so an existing Whisper WebSocket client can switch to it with only a change to the audio-send path (base64 append/commit).
Ideal for voice agents, live captioning, and multilingual customer-support transcription.
Accuracy vs latency — semantic word-error-rate vs median time-to-final-segment (single stream), plotted against closed-source providers.
Latency vs concurrency — how median latency holds up as concurrent streams scale, by GPU.Stream 16 kHz mono PCM16 audio over a WebSocket as base64 input_audio_buffer.append frames and end a turn with input_audio_buffer.commit or automatic voice-activity detection (VAD). The server returns type:"transcription" messages — revisable partials (is_final:false) during the turn and a final (is_final:true) at each endpoint. Responses match the streaming-Whisper format.
We've shown that Qwen3-ASR-1.7B can comfortably keep real-time through up to 64 concurrent streams on both NVIDIA RTX 6000 and H100 GPUs, with P95 finalization latency under 0.50 seconds.
1import asyncio, base64, json, os, wave
2import websockets
3
4URL = "wss://model-<id>.api.baseten.co/environments/production/websocket"
5KEY = os.environ["BASETEN_API_KEY"]
6SR = 16_000
7CHUNK = int(SR * 0.1) * 2 # 100 ms of PCM16
8HEADERS = {"Authorization": f"Api-Key {KEY}"}
9
10
11async def transcribe(path):
12 with wave.open(path, "rb") as w: # 16 kHz mono PCM16 WAV
13 pcm = w.readframes(w.getnframes())
14 async with websockets.connect(URL, additional_headers=HEADERS) as ws:
15 await ws.send(json.dumps({
16 "whisper_params": {"audio_language": "auto"},
17 "streaming_params": {
18 "enable_partial_transcripts": True,
19 "partial_transcript_interval_s": 0.5,
20 },
21 }))
22
23 async def send():
24 for i in range(0, len(pcm), CHUNK):
25 chunk = base64.b64encode(pcm[i:i + CHUNK]).decode()
26 await ws.send(json.dumps({
27 "type": "input_audio_buffer.append",
28 "audio": chunk,
29 }))
30 await asyncio.sleep(0.1) # real-time pace
31 await ws.send(json.dumps(
32 {"type": "input_audio_buffer.commit"}
33 ))
34
35 asyncio.create_task(send())
36 while True:
37 msg = json.loads(await ws.recv())
38 if msg.get("type") != "transcription":
39 continue
40 text = " ".join(s["text"] for s in msg.get("segments", []))
41 print(("final:" if msg["is_final"] else "partial:"), text)
42 if msg.get("is_final") and msg.get("is_end_of_audio_flush"):
43 break
44
45
46asyncio.run(transcribe("example.wav"))1{
2 "type": "transcription",
3 "is_final": false,
4 "transcription_num": 0,
5 "language_code": "English",
6 "segments": [
7 {
8 "start_time": 0,
9 "end_time": 3.2,
10 "text": "<current hypothesis>"
11 }
12 ]
13}
