This month, we launched Truss, an open-source Python package for model serving and deployment. As a Baseten user, you’ve actually been using Truss for a while now behind the scenes without knowing it, but this open-source launch will unlock new capabilities within Baseten for you.
Check out our launch post in Towards Data Science!
Truss, a new model serving technology
Truss is open source on GitHub under the MIT license. It offers a bunch of features to integrate model deployment into your development loop:
🏎 Turns your Python model into a microservice with a production-ready API endpoint, no need for Flask or Django.
🎚 For most popular frameworks, includes automatic model serialization and deserialization.
🛍 Freezes dependencies via Docker to make your training environment portable.
🕰 Enables rapid iteration with local development that matches your production environment.
🗃 Encourages shipping parsing and even business logic alongside your model with integrated pre- and post-processing functions.
🤖 Supports running predictions on GPUs. (Currently limited to certain hardware, more coming soon)
🙉 Bundles secret management to securely give your model access to API keys.
For Baseten users, Truss is even more powerful. Platform features like GPU support, secret management, and model pulling and sharing all run through Truss, giving you more control over your deployed models. And our roadmap includes ideas for where we want to take Truss, but this is a collaborative project; we’ll build what you need!
Truss is a foundational technology for Baseten that will power the next sets of model deployment features and grow with your use cases. We’re excited to see what models you build and share with Truss!
Model deployment improvements with Truss
Thanks to Truss, you can now deploy models from more frameworks with the out-of-the-box baseten.deploy() command. This doesn’t require using Truss explicitly, it’s a free win from Baseten switching to Truss under the hood. New model frameworks supported include:
- Hugging Face Transformers (built with the pipeline command), a flexible
- XGBoost, an optimized distributed gradient boosting library
- LightGBM, a gradient boosting framework that uses tree based learning algorithms
Here’s how simple creating, invoking, and deploying a Hugging Face model with Truss is: just eight lines of code, counting imports.
This month, we joined forces with Brittany, a product designer, and Nish, a software engineer. We might be looking for you, too! We’re hiring for a lead infrastructure engineer and a lead ML engineer.
We’ll venture back toward your inbox next month with another update!
The team at Baseten
Data + Curiosity: How a Data Scientist turned Software Engineer Rich Iannone leverages graphic design to make beautiful documentation
Rich Iannone started his career as a data scientist with an ad tech firm before becoming a software engineer responsible for the creation of data quality packages such as pointblank. In this interview we dive into his career and how it’s been influenced by his interests in graphic design.