Mirendil is a research lab building self-improving AI, founded by former Anthropic researchers Behnam Neyshabur and Harsh Mehta. The lab launched in June with a $200 million seed round and recently released their first model. Its goal is to massively accelerate scientists and engineers and help tackle their most ambitious projects. Getting there means running RL on frontier-scale models with a trainer the lab built and controls.
Their work depends heavily on reinforcement learning (RL) workloads at a massive scale over thousands of GPUs: their trainer continuously publishes new model weights, and a fleet of GPUs must pick up each new policy and keep generating rollouts without stalling the run.
We’re excited to share that Mirendil is one of the AI labs that runs those rollouts on Baseten, and that we're partnering on their RL runs.
Why rollouts are the bottleneck
Rollout GPUs often account for most of the compute needed at the scale that Mirendil operates at. A rollout fleet can use up to 2-5× as many GPUs as the trainer, leaving fewer GPUs for training. Mirendil runs its trainer, data, and environments in-house. It is Baseten’s job to make sure this fleet is running at its peak capacity, and not impaired by the weights swapping out from under it. Baseten operates the sampler fleet and enables Mirendil to scale across our global fleet of clusters.
Figure 1. Fleet throughput during a live RL run, sampled at 30-second intervals. Delta weight syncs cause brief dips to roughly 90% of median throughput, recovering in about a minute.How it works
Mirendil's trainer runs entirely in their own environment and never connects directly to our fleet. It writes checkpoints and compact weight deltas to object storage, then moves on. Baseten's RL rollouts watch that stream, reconstruct each new policy, and apply it to GPUs across independent clusters around the world. A cluster can restart or drop out without blocking the trainer or the other clusters. Mirendil keeps their entire training stack and data in-house, and Baseten keeps the rollout fleet serving the newest policy.
However, one of the hardest parts of this setup is the speed of the weight sync. A weight sync that takes a few minutes leaves thousands of GPUs generating rollouts against a stale policy or worse, sitting idle. On Baseten, a new policy goes from published in object storage to serving live traffic in 40 seconds on average, with roughly six seconds where requests pause which is up to 10x quicker than off-the-shelf implementations of delta weight sync.
Figure 2. Time before a new policy begins serving, as a share of a 15-minute RL step. Baseten’s delta sync takes roughly 4% of the step, compared with 45% for off-the-shelf rebuild and hot reload and 97% for rebuild and vLLM restart.We take care of alignment, ensuring our stack matches Mirendil’s internal numerics exactly. Correctness is another where the system must reconstruct and apply each policy update exactly, without missing changes or corrupting weights, or the trainer learns from bad rollouts.
“We built our trainer and keep our data and environments in-house. For the rollout fleet, we needed a way to get each new policy out to GPUs without turning that infrastructure into another project for our team. Baseten’s RL rollouts offering lets us keep pushing our RL work forward while we focus on the research behind self-improving AI. We’re proud to partner with Baseten and love working with their team.”
Proving it at scale
Mirendil ran the delta weight sync setup end to end with sustained RL runs on Baseten, comparing results against their own baseline. Baseten’s workload-optimized throughput configuration made sampling about 1.9× faster. Mirendil is now scaling up their RL work on Baseten across thousands of GPUs.
Figure 3. Workload-specific optimization increases output from 815 to 1,561 tokens per second per replica—1.9× the initial configuration—measured live on Mirendil’s agentic workload while syncing weights.Alongside RL rollouts, their team uses Baseten Model APIs, the same endpoints available to every Baseten customer.
"Frontier labs can put more compute to work on RL through Baseten’s RL rollout fleet, which dynamically scales across global clusters. Every hour Mirendil spends rebuilding rollout infrastructure is an hour not spent on training. We built RL rollouts so labs like Mirendil can put their time into the research that matters."
If you're running RL and want to scale up your rollout infrastructure, talk to our team.
