Changelog

Baseten’s model deployment stack runs on Truss, our open-source library for model serving. Any model framework supported by Truss is also supported in the Baseten CLI. As such, the Baseten Python client now supports three new frameworks:

• Hugging Face Transformers
• LightGBM
• XGBoost

And of course you can deploy any model to Baseten as a custom model.

To use these newly supported frameworks, upgrade to the latest version of the Baseten Python client with:

pip install --upgrade baseten

Example deployment

Let’s take a look at what deploying a model built from Hugging Face Transformers looks like:

import baseten
from transformers import pipeline

model = pipeline('fill-mask', model='bert-base-uncased')

baseten.login("*** INSERT API KEY ***")  # https://docs.baseten.co/settings/api-keys

baseten_model = baseten.deploy(
    model,
    model_name='bert base uncased',
)

As we continue developing Truss, more frameworks will be supported out-of-the-box on Baseten. Let us know what framework you’d like to see next by opening an issue.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• You can view and edit your workspace name in settings
• Code files show when they were last edited
• The file upload component checks uploaded files to ensure they are below the maximum size before saving
• Bug fixes and performance improvements

Baseten’s data explorer in the view builder helps you take your ML-powered project from “web page” to “web app.” But the concepts introduced in the data explorer are clearly centered in front-end development, not data science. While the view builder provides helpful abstractions for concepts like application state, it also exposes a great deal of control to let developers choose to build more powerful interfaces.

To assist developers in venturing into front-end concepts, we’ve added tooltips to every field in the data explorer providing immediate context and a direct link to relevant documentation. We encourage you to fearlessly explore building featureful UIs by peeking under the hood of front-end development.

While we’ve been releasing a steady cadence of quality-of-life features and user experience improvements, we have some big things coming in the second half of July that we’re excited to share with you. Tune in next week for Baseten’s first-ever open source project!

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• In the console, use the new “Shift-Enter” hotkey to execute the current cell and create a new one, just like in a Jupyter notebook
• View your account email and workspace role (owner/admin/creator/operator) in your account profile
• Bug fixes and performance improvements

We found from user testing that selecting the page canvas was unintuitive. So we added a component-style label above the canvas to make it easy to select and access a menu of powerful properties.

Just like the components on a page, the canvas has configurable properties. You can adjust canvas width depending on the user interface you want to build and add event handlers that fire on page load. These event handlers can initialize state, read query parameters, or kick off worklets and database queries, making the page dynamic from first paint.

At least it’s not Clippy

Within the application builder, we want to give you essential information and reminders while staying out of your way as you work. One of the most important concepts in the view builder is a binding, the double-bracket reference to on-page state. A new dismissible inline reference, shipping soon, tells you all about bindings.

Also, don’t miss tooltips over many component properties explaining their input and function.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• A recent effort removed tens of thousands of lines of leftover code from Baseten’s previous iterations, improving build speed and innovation velocity
• A slightly altered sign-up flow helps us get to know users a little better
• Bug fixes and performance improvements

Copy and paste are any developer’s foremost time-saving tools. Stack Overflow even offered a keyboard with only the copy and paste keys — first as a joke, then as a real product.

Now, Baseten’s view builder offers this essential feature for components. You can copy and paste components within a view, preserving all properties, event handlers, bindings, and more between the copies. 

Because copy and paste copies all of the component’s properties, you cannot copy and paste components between views or applications. Instead, try cloning the entire view or application to build variants.

Combined with the delete hotkey and, of course, undo and redo, copy and paste makes using the view builder feel like flying a nimble starfighter, but with fewer explosions.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• A badge on the logs tab within the application builder indicates new changes as you work
• Autosave is now permanently enabled in files with a system in place to prevent code file save clobbers
• The dot grid in the view builder now only displays when you are adding, resizing, or moving a component
• Bug fixes and performance improvements

To do, or not to do, that is the question. Whatever your answer, you can now change your mind! In the view builder, you can undo and redo recent actions with cmd-z and shift-cmd-z, respectively. These keyboard shortcuts match undo and redo in our code editor (for Windows, ctrl-z and shift-ctrl-z). 

The undo command stacks, so you can undo and redo multiple changes. Add, alter, and delete components with freedom; you can revert any mistakes instantly.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• The list of pre-trained models shows a badge next to models for which you already have an active deployment
• Updated onboarding cards walk new users through the entire application builder
• Bug fixes and performance improvements

In the application builder, the logs tab now shows events from the view in addition to worklet runs. This gives you more information when building and debugging complex user interactions.

If a single user action triggers multiple events, each event will have its own log entry. Each log entry includes the component name and event type, along with any relevant information like worklet name and parameters.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Clone and delete applications from the dropdown menu within the application builder
• When you deploy your own model to Baseten, its type now shows as “Imported”
• Additional security tooling on production clusters
• Bug fixes and performance improvements

Baseten’s library of pre-trained models now includes a new addition that we’ve been having a lot of fun with: Dall·E mini. 

You can deploy this model to your Baseten account, and it comes with a basic starter app so that you can start generating images right away. 

‍

Look out for more cool pictures and projects with Dall·E mini soon!

Deploy CodeGen models

We also added two pre-trained models that generate code from a prompt. CodeGen mono and multi are each two-billion-parameter models trained on datasets of code and natural language. The twin models come with identical starter apps for model evaluation and to inspire your own projects.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Image components support base64 encoded images (see the Dall·E mini starter app for an example)
• Transactional emails now include plain text links for expanded compatibility with various email clients and firewalls
• Bug fixes and performance improvements

‍

As you develop an application on Baseten, you might want to edit your worklets. Now, you can add blocks in between existing blocks in your worklet graph.

‍

‍

You can also delete blocks from the middle of worklets without worry, the graph will snap into place as if the deleted block was never there. 

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Each component in the view builder now has its own minimum width and height
• Removed unnecessary scrollbars from application builder UI
• Bug fixes and performance improvements

‍

We added charts to visualize the health and performance of your models on BaseTen. Even better, we used pretty colors and fun shapes. From the health tab, you can monitor a model's traffic, response time, and compute resource usage over time.

Just like model logs, the health tab is available for all custom models deployed on your BaseTen account, but not for pre-trained models from our model zoo.

You’ll now have full visibility 👀 into the following:

Prediction volume

The number of requests per minute made to your deployed model:

Response time

Your model's 50th, 90th, 95th, and 99th percentile response time to incoming requests:

CPU and memory usage

The total CPU and memory usage across all replicas of your model–or, focus in on a single replica:

🔮 Back in time

For every model metric, you can use the time picker to select a time range from the last five minutes to the last six hours. Along with model logs, this access will make you feel like Sherlock Holmes as you crack the case of any model health concerns.

‍

You’re now able to access queries from a code block in a Worklet! These queries can be in the form of raw SQL (as a string) or a name of a query you’ve defined earlier.

For example:

new_query_output = context.run_query_str("select * from test_table") # to run raw SQL queries

existing_query_output = context.run_query_name("testquery") # to run a specific query you’ve already defined

These examples show queries against Baseten Postgres data tables. If you’d like to query external data sources you have connected to your Baseten account, check out our documentation.

To access the results of your queries, either return or print their output (new_query_output, existing_query_output) from your code block!

‍

We've recently pushed out a change that allows you to associate pieces of information with models that are deployed in the BaseTen infrastructure. You can now track configurations, hyperparameters, and specific files with your models. Learn more about tracking model metadata in our documentation!

https://docs.baseten.co/ml-models/tracking-model-metadata

‍

You can now access queries across all your applications in one place! You’ll now find all your queries in the Queries tab of your Data page. Query your own BaseTen Postgres tables (any tables defined in the Data page under BaseTen Postgres) and all connected external data sources (in the Connections portion of the Data page). Happy querying!

‍

Instead of unnecessarily updating Docker images every single time a model is deployed, we're now periodically updating base images for every model framework we support. We're seeing that model deployments can be 2-5 times faster as a result.

If you've been testing your application but want to start from a clean slate, you can easily delete all the rows in your BaseTen-provided Postgres table(s). Click the red "Delete all data" button. Before your data is deleted, a confirmation pop-up will appear with details on how many rows will be deleted and any referenced tables that will also be affected. If the table you're deleting data from is a foreign key that is referenced in other tables, then the columns that reference the table you're deleting data from will be set to null.

If you need to delete individual rows instead of all the rows in your table, you can still do that with code. Just follow the instructions in this doc on deleting objects.

‍

You can now specify system-level packages that you want to install in your Python environment on BaseTen.

BaseTen runs your python code on a container using a standard Python image for the Debian Operating System. Occasionally, you may come across Python packages that require additional system packages to function. For example, librosa, a Python package for audio analysis requires the ffmpeg system package for understanding a number of common audio formats such as MP3. Similarly, OpenCV, the popular image-processing package, requires the libjpeg system package to read JPEG files.

Currently, only a standard set of package repositories are looked up but we intend to expand the number of available repositories. Please let us know if you're unable to install a package you need.

‍