Nemotron 3.5 Lightning
Nemotron 3.5 LightningNVIDIA’s Nemotron 3.5 Lightning is a 30B MoE model built for high-throughput AI agents, offering lightning speed, 1M context window, and efficient reasoning.
Model details
NVIDIA's Nemotron 3.5 Lightning is a 30B Mixture-of-Experts (MoE) model engineered specifically for high-throughput, agentic workflows. By distilling the power of the Nemotron 3 Ultra foundation model into a more efficient architecture with 3B active parameters, it achieves nearly four times the throughput of comparable open models. This speed allows agents to perform more reasoning iterations and deeper validation, enabling them to bypass traditional speed-accuracy trade-offs to deliver more reliable performance in complex tasks ranging from personal assistant management to cybersecurity and financial services.
1import os
2from openai import OpenAI
3
4model_id = "" # TODO: replace with this deployment's model ID (from `baseten model push` output or the dashboard URL)
5
6client = OpenAI(
7 base_url=f"https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
8 api_key=os.environ["BASETEN_API_KEY"],
9)
10
11response = client.chat.completions.create(
12 model="nvidia/nemotron-3.5-nano-nvfp4", # must match the served model name in this config
13 messages=[
14 {"role": "user", "content": "Write a short Python function that computes Fibonacci numbers."}
15 ],
16 max_tokens=256,
17 temperature=0.2,
18)
19
20print(response.choices[0].message.content)
21
