Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

Llama 3.1 Nemotron Ultra 253B

NVIDIA/Llama 3.1 Nemotron Ultra 253B

A high-efficiency distill of Llama 3.2 405B with leading accuracy for reasoning, tool calling, chat, and instruction following.

large language

Input

python
import requests

# Replace the empty string with your model id below
model_id = ""
baseten_api_key = os.environ["BASETEN_API_KEY"]

messages = [
    {"role": "user", "content": "Write a limerick about the wonders of GPU computing.?"},
]
data = {
    "messages": messages,
    "stream": True,
    "max_new_tokens": 512
}

# Call model endpoint
res = requests.post(
    f"https://model-{model_id}.api.baseten.co/production/predict",
    headers={"Authorization": f"Api-Key {baseten_api_key}"},
    json=data,
    stream=True
)

# Print the generated tokens as they get streamed
for content in res.iter_content():
    print(content.decode("utf-8"), end="", flush=True)