Background: Building AI into the data science workflow
The Posit team recently launched Posit AI, a new feature embedding intelligence directly into RStudio through two core features:
Posit Assistant: A conversational agent embedded in RStudio with full context of the user's live R session, capable of writing, refactoring, and executing code, collaborating on exploratory data analysis, debugging, and generating Quarto reports.
Next Edit Suggestions (NES): Context-aware autocompletion that predicts the developer's next edit, similar in spirit to Cursor Tab, but uniquely powered by a suite of small LLMs fine-tuned to incorporate live computational session context (variable names, data frame column types, function signatures) alongside code context. NES is hosted and served through Baseten.
The latency bar
For NES, latency was the critical factor. Because suggestions appear inline as "ghost text" while the user types, and can predict edits elsewhere in the document (such as renaming a column reference downstream), the feature needed to feel instant. Anything above a few hundred milliseconds would break the user experience. This constraint ruled out most LLM providers, who were roughly 400ms too slow at best.

Posit needed infrastructure that could deliver sub-200ms latency and give them the flexibility to rapidly experiment with fine-tuning approaches to push model quality further.


