Qwen3 ASR 1.7B Streaming
Qwen3 ASR 1.7B StreamingReal-time streaming speech-to-text from Qwen3-ASR 1.7B over WebSocket, featuring batch-level accuracy, sub-second finals, and 30+ languages.
Model details
Qwen3-ASR 1.7B (Streaming) is a real-time speech-to-text service that runs Alibaba's open-weight Qwen3-ASR model behind Baseten's streaming harness over WebSocket. Ideal for voice agents, live captioning, and multilingual customer-support transcription.
Feature highlights
Extensive multi-lingual support, including auto-detection across 30+ languages and 20+ Chinese dialects, and easy fine-tuning for even more, achieving batch-level quality at streaming latency.
Configurable update cadence for the partial transcriptions delivered
Consistent real-time latency under high volume of concurrent audio streams
Where accuracy stands
The Pareto plot below shows how this model compares with leading closed-source ASR solutions across accuracy and latency. Results are measured using Pipecat’s open-source STT Benchmark, which evaluates Semantic WER for transcription accuracy and TTFS (time to final segment) for latency.
Accuracy vs latency — semantic word-error-rate vs median time-to-final-segment (single stream), plotted against closed-source providers.Concurrency tradeoffs
The graph below shows how TTFS (time to final segment) and unit economics (cost per audio hour) change as the number of concurrent WebSocket connections per GPU increases. Higher concurrency can significantly reduce cost per audio hour, with a corresponding tradeoff in latency.
Concurrency is configurable and can be tuned to achieve the latency and cost targets that best fit your workload and business requirements. See the concurrency guide for configuration details.
For a balance of latency and throughput, we recommend starting with RTX6000 using concurrency 48, and adjusting based on performance and cost needs based on the chart above if needed.
We've shown that Qwen3-ASR-1.7B can comfortably keep real-time through up to 64 concurrent streams on both NVIDIA RTX 6000 and H100 GPUs, with P95 finalization latency under 0.50 seconds.
Latency vs concurrency — how median latency holds up as concurrent streams scale, by GPU.For workload-specific optimization, you can contact an engineer for further assistance.
Call the model
Stream 16 kHz mono PCM16 audio over a WebSocket as base64 input_audio_buffer.append frames and end a turn with input_audio_buffer.commit or automatic voice-activity detection (VAD). The server returns type:"transcription" messages — revisable partials (is_final:false) during the turn and a final (is_final:true) at each endpoint. Responses match the streaming-Whisper format.
1import asyncio, base64, json, os, wave
2import websockets
3
4URL = "wss://model-<id>.api.baseten.co/environments/production/websocket"
5KEY = os.environ["BASETEN_API_KEY"]
6SR = 16_000
7CHUNK = int(SR * 0.1) * 2 # 100 ms of PCM16
8HEADERS = {"Authorization": f"Api-Key {KEY}"}
9
10
11async def transcribe(path):
12 with wave.open(path, "rb") as w: # 16 kHz mono PCM16 WAV
13 pcm = w.readframes(w.getnframes())
14 async with websockets.connect(URL, additional_headers=HEADERS) as ws:
15 await ws.send(json.dumps({
16 "whisper_params": {"audio_language": "auto"},
17 "streaming_params": {
18 "enable_partial_transcripts": True,
19 "partial_transcript_interval_s": 0.5,
20 },
21 }))
22
23 async def send():
24 for i in range(0, len(pcm), CHUNK):
25 chunk = base64.b64encode(pcm[i:i + CHUNK]).decode()
26 await ws.send(json.dumps({
27 "type": "input_audio_buffer.append",
28 "audio": chunk,
29 }))
30 await asyncio.sleep(0.1) # real-time pace
31 await ws.send(json.dumps(
32 {"type": "input_audio_buffer.commit"}
33 ))
34
35 asyncio.create_task(send())
36 while True:
37 msg = json.loads(await ws.recv())
38 if msg.get("type") != "transcription":
39 continue
40 text = " ".join(s["text"] for s in msg.get("segments", []))
41 print(("final:" if msg["is_final"] else "partial:"), text)
42 if msg.get("is_final") and msg.get("is_end_of_audio_flush"):
43 break
44
45
46asyncio.run(transcribe("example.wav"))1{
2 "type": "transcription",
3 "is_final": false,
4 "transcription_num": 0,
5 "language_code": "English",
6 "segments": [
7 {
8 "start_time": 0,
9 "end_time": 3.2,
10 "text": "<current hypothesis>"
11 }
12 ]
13}
