Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

NVIDIA Nemotron 3 Ultra

nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B

550B hybrid Mamba-Transformer MoE with 55B active params, latent MoE routing, multi-token prediction, and up to 1M token context

Model APIlarge language

OpenAI-compatible chat completion. NVIDIA recommends temperature=1.0 and top_p=0.95; toggle reasoning via chat_template_kwargs.enable_thinking.

Input

python
from openai import OpenAI

client = OpenAI(
    api_key="BASETEN_API_KEY",
    base_url="https://inference.baseten.co/v1"
)

# NVIDIA recommends temperature=1.0 and top_p=0.95 for all tasks.
# Toggle reasoning on/off via chat_template_kwargs.enable_thinking.
response = client.chat.completions.create(
    model="nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B",
    messages=[
        {
            "role": "user",
            "content": "Tell me a fun fact about hummingbirds."
        }
    ],
    stream=True,
    stream_options={
        "include_usage": True,
        "continuous_usage_stats": True
    },
    top_p=0.95,
    max_tokens=256,
    temperature=1,
    presence_penalty=0,
    frequency_penalty=0
)

for chunk in response:
    if chunk.choices and chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)

Output

json
{
  "id": "143",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "message": {
        "content": "[Model output here]",
        "role": "assistant",
        "audio": null,
        "function_call": null,
        "tool_calls": null
      }
    }
  ],
  "created": 1741224586,
  "model": "",
  "object": "chat.completion",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 145,
    "prompt_tokens": 38,
    "total_tokens": 183,
    "completion_tokens_details": null,
    "prompt_tokens_details": null
  }
}