large language
Qwen3 Coder 480B
Qwen3 Coder 480BMixture-of-experts LLM with advanced coding and reasoning capabilities
Model details
Example usage
Baseten offers Dedicated Deployments and Model APIs for Qwen3 Coder 480B A35B Instruct powered by the Baseten Inference Stack.
Qwen3 has shown strong performance on math and reasoning tasks, but running it in production requires a highly optimized inference stack to avoid excessive latency.
Deployments of Qwen3 are OpenAI-compatible.
Input
1# You can use this model with any of the OpenAI clients in any language!
2# Simply change the API Key to get started
3
4from openai import OpenAI
5
6client = OpenAI(
7 api_key="YOUR_API_KEY",
8 base_url="https://inference.baseten.co/v1"
9)
10
11response = client.chat.completions.create(
12 model="Qwen/Qwen3-Coder-480B-A35B-Instruct",
13 messages=[
14 {
15 "role": "user",
16 "content": "Implement Hello World in Python"
17 }
18 ],
19 stop=[],
20 stream=True,
21 stream_options={
22 "include_usage": True,
23 "continuous_usage_stats": True
24 },
25 top_p=1,
26 max_tokens=1000,
27 temperature=1,
28 presence_penalty=0,
29 frequency_penalty=0
30)
31
32for chunk in response:
33 if chunk.choices and chunk.choices[0].delta.content is not None:
34 print(chunk.choices[0].delta.content, end="", flush=True)JSON output
1{
2 "id": "143",
3 "choices": [
4 {
5 "finish_reason": "stop",
6 "index": 0,
7 "logprobs": null,
8 "message": {
9 "content": "[Model output here]",
10 "role": "assistant",
11 "audio": null,
12 "function_call": null,
13 "tool_calls": null
14 }
15 }
16 ],
17 "created": 1741224586,
18 "model": "",
19 "object": "chat.completion",
20 "service_tier": null,
21 "system_fingerprint": null,
22 "usage": {
23 "completion_tokens": 145,
24 "prompt_tokens": 38,
25 "total_tokens": 183,
26 "completion_tokens_details": null,
27 "prompt_tokens_details": null
28 }
29}
