Engineering

Truss’ key feature for iterative development is live reload. Without live reload, the upload-build-deploy loop for publishing models to production can take anywhere from 3 to 30 minutes. With live reload, it’s practically instantaneous. In this post, we describe implementing live reload.

Pankaj Gupta

February 16, 2023

Horizontal scaling via replicas with load balancing is an important technique for handling high traffic to an ML model. Let’s examine three tips for understanding how to properly replicate your instances to save users time without wasting your money.

Philip Kiely

January 19, 2023

Instance sizing is complicated. In this post, we'll follow a few simple heuristics to select an appropriate instance size that can handle your model while minimizing compute cost.

Philip Kiely

January 18, 2023

Seth and Hayk partnered with Baseten to host and serve Riffusion, which uses a tuned version of Stable Diffusion to generate audio spectrograms and interpret those images as music. Riffusion climbed to the top of Hacker News and handled over 4 million song requests in one day.

Phil Howes

December 21, 2022

Many business processes benefit from integration with existing interfaces. Baseten’s Slack integration abstracts away all of the boilerplate code required to send a message to a Slack channel, and in this tutorial we’ll build a lead classifier with a pre-trained zero-shot classification model.

Philip Kiely

May 19, 2022

While Baseten provides PostgreSQL tables for storing information from model runs, you may want to store your data in your own databases. Plus, for data ill-suited for relational databases, like large files and key-value pairs, you’ll want to use more appropriate technologies.

Philip Kiely

April 22, 2022

Docs as code is a philosophy of writing documentation with the same tools and practices as writing code. Adopting docs as code has lowered friction for engineers to contribute documentation, improving docs quality and saving time.

Philip Kiely

March 9, 2022

Learn how to use scaffolds, the technology that underpins the model serving experience on BaseTen. Use scaffolds to test and iterate on your model servers locally before deploying on BaseTen and so much more.

Alex Gillmor

December 21, 2021