Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

Changelog

Introducing Custom Servers: Deploy production-ready model servers from Docker images

Date

Our new Custom Servers feature lets you deploy production-ready model servers directly from Docker images using just a YAML file.

We built Truss to simplify the process of deploying AI models in production. But sometimes deploying Docker images is the most direct path—such as when using custom in-house images or open-source servers like vLLM.

Custom Servers offer a codeless way to deploy your Dockerized model servers while leveraging all the infrastructure optimizations Baseten provides.

Tianshu Cheng demos how you can deploy a Custom Server on Baseten using just a YAML config file.

Check out our launch blog for more information, or dive into the docs to get started!