Changelog

Deactivate and delete model versions

Baseten supports deploying multiple versions of the same model, so you can iterate, test, and experiment to your heart’s content. Now, you can either deactivate or delete model versions when they are no longer useful.

A deactivated model version cannot be invoked or used in applications, but can be re-activated. A deleted model version is permanently gone. Either way, neither deactivated nor deleted model versions count against your deployed model limit.

Record audio with new microphone component

When building UI models like Whisper and wav2vec that process audio, you’ve been able to let users upload audio clips with the file upload component. With the new microphone component, you can instead let users capture audio directly in the app.

🎃 The pumpkin patch

This week’s small-but-mighty changes to bring more magic to your models!

Share state between views: If you’re building a complex application on Baseten with multiple views, you might want to share state between those views. This is useful for building interactions like clicking on a row in a table and going to a detail page pre-populated with that row’s information.

New account profile and API key pages: Go to Settings in the main sidebar and you’ll find Account settings broken out from Workspace settings for easier access

Set object fit in image components: Select from five options to set the object fit that works best in the context of your application.

As a data scientist, you can do a lot with Python: train models, build servers, write scripts. Now, in the Baseten app builder, you can do something special with Python: build dynamic front-end web interactions that run in the browser.

JavaScript’s claim to fame is that it is the language of the web browser. But building a dynamic front-end experience shouldn’t require learning a new programming language. So we now support Python as the default way to write front-end functions in the view builder. JavaScript is still supported as well.

Before, with Javascript:

let nums = [1, 2, 3, 4, 5]
let evens = nums.filter(function (num) {
  return (num % 2 == 0);
});
return evens

Now, with Python:

nums = [1, 2, 3, 4,5]
return [n for n in nums if n % 2 == 0]

Python in the view builder isn’t just for basic operations. You can import many packages, including all Python built-in packages and popular data science libraries like numpy. Unfortunately, due to limitations in the technology powering Python in the web browser, you can’t install your own packages like you can in the back end.

To help you get started with this exciting new capability, we prepared a tutorial on building a stateful web app with Python in the view builder. Follow along to build an auto loan payoff calculator in fifteen minutes using drag and drop components, state, and a Python function. Read the tutorial here.

Deploy models with any interface

By default, when you deploy a model to Baseten, like in this XGBoost classifier example, the deployed model expects to receive a dictionary with the key inputs and a list of input values, and will return a dictionary with the key predictions and a list of model results.

model_input = {"inputs": [[0, 0, 0, 0, 0, 0]]}
model_output = {'predictions': [0.21339938044548035]}

Until now, that default behavior has not been changeable. All models deployed to Baseten had to follow that spec. However, this interface was too inflexible, so as of the most recent version of the Baseten Python package (0.2.7), you can set your own interface for your models.

Setting your model interface

Baseten uses Truss under the hood for model deployment. You can customize your model’s interface by editing the predict function in models/model.py in Truss. The auto-generated predict function for the aforementioned XGBoost example looks like this:

def predict(self, request: Dict) -> Dict[str, List]:
    response = {}
    inputs = request["inputs"]
    dmatrix_inputs = xgb.DMatrix(inputs)
    result = self._model.predict(dmatrix_inputs)
    response["predictions"] = result
    return response

Modify this function to parse whatever request and response you want, and remember that Truss also supports pre- and post-processing functions that can further modify input and output when more complicated parsing is needed.

Backwards compatibility

This change does not modify the behavior of existing deployed models, nor the default behavior of future models. However, it does change how deployed models are invoked through the Baseten Python client.

Previously, the predict() function wrapped its argument in a dictionary with the inputs key. Now that said key is not required, the predict() function passes its inputs as-is, which means you have to enter the entire model input yourself.

Before (1.0 spec):

baseten.predict([[0,0,0,0,0,0]])

Now (2.0 spec):

baseten.predict({"inputs": [[0,0,0,0,0,0]]})

The syntax for invoking a model via an API call has not changed.

However, to prevent breaking existing scripts using baseten.predict, a new spec_version flag is now included in Truss. This parameter is set to 2.0 by default for all new models, so they will use the new input spec, but all existing models will continue to function exactly as they have been on the 1.0 spec. You can upgrade your model to the latest interface spec by changing the flag in the config.yaml file in Truss.

Enjoy this unrestricted interface by installing the newest versions of Truss and the Baseten client.

pip install --upgrade baseten truss

Pass a Truss to baseten.deploy()

We also cleaned up the deployment experience in the Baseten Python client. You no longer have to use different functions to deploy an in-memory model versus a model packaged as a Truss. Whatever you have, toss it into baseten.deploy() and we’ll take care of it.

Plus, when you deploy an in-memory model, the deploy function now gives you insight into what is happening to package that model, including the path to where the auto-generated Truss folder lives. This is useful if you want to change something about your deployed model’s behavior (like updating the interface) after deploying the model, just edit the truss and pass it into baseten.deploy() to ship a new version!

INFO Autogenerating Truss for your model,
find more about Truss at https://truss.baseten.co/
INFO You can find your auto generated Truss at
/root/.truss/models/lightgbm-MMVJD0

🎃 The pumpkin patch

This week’s small-but-mighty changes to bring more magic to your models!

Tab complete bindings: In the view builder, you can create a binding tag by typing “{{“ and you can now close the binding by pressing “Tab” after selecting the data you want to use in the binding.

Model building banners: When you deploy a new version of a model to your Baseten workspace, you’ll see a banner on other versions of the model letting you know that the new version is building.

‍

Search your resources

Where did I leave that cutting-edge machine learning model? Never lose track of your application resources again with full workspace search. Just type Command-k (Ctrl-k on Windows) or click the search bar and you’ll be able to find the resource you're looking for: view, data table, code file, anything!

Search results are segmented by scope: the application you're in your Baseten workspace as a whole. Click on any resource and you’ll be taken directly to it!

🎃 The pumpkin patch

This week’s small-but-mighty changes to bring more magic to your models!

Copy data explorer values: Use values from the data explorer with a convenient copy button, just like worklet outputs. The copy button appears per value on hover and copies the information as JSON.

‍

‍

Back up your apps to GitHub

This week, a feature for workspaces that use our starter or business plans: back up your Baseten apps to GitHub. Paired with the draft environment, this gives you a safety net while developing your production applications. Get started by following the GitHub sync docs.

Once GitHub sync is turned on in your workspace, you can enable it app-by-app to back up applications to GitHub. Applications will be backed up immediately, and a new version will be committed to the repository each time you publish.

🎃 The pumpkin patch

This week’s small-but-mighty changes to bring more magic to your models!

Set multiline secret values: In your workspace settings, you can now store multiline secret values. This is useful for storing things like PGP and SSH keys.

Reorder event handlers: In the view builder, you can add multiple event handlers to each component. Event handlers are executed sequentially. If you want to change the order that event handlers run in, you can now do so by dragging and dropping them into place.

Model deployment emails

Watching code build is the digital equivalent of watching paint dry. Model deployment on Baseten generally runs in less than five minutes, but that's enough time to:

• Make yourself a cup of coffee
• Listen to your favorite song
• Run a mile (if you're Usain Bolt)
  

So take a microbreak — we'll send you an email once the model deployment is finished!

We also send a notification email if a deployment fails or if an active model goes down. And you can click the link in any model status email to be taken directly to the deployed model’s page.

🎃 The pumpkin patch

This week’s small-but-mighty changes to bring more magic to your models!

Set state with math: When using the “set state” action, the value field will automatically evaluate mathematical expressions.

‍

App & model tags: Starter applications and pre-trained models are now tagged in their respective tables. These resources do not count against your workspace limits.

‍

Include secrets in deployed models

The word “secret” is derived from Latin, secreto, which literally translates to “don’t store your API keys and AWS access tokens in your source code.” Amazing what insights linguistics offers!

For models that use secrets to access file stores, databases, external APIs, or other resources, you need to store secrets to access these resources. Secrets also pop up often in pre- and post-processing code to store model inputs and outputs. When you use Truss to deploy your model on Baseten, you can add secrets to your model’s config as follows:

secrets:
    my_api_key: null

‍

Then, you can access the secrets in the Truss' model file by passing them into the init function:

def __init__(self, secrets) -> None:
    self._api_key = secrets["my_api_key"]

‍

To set secrets in production, use Baseten’s secret management feature. Get started with secrets by following the documentation to add and reference secrets in your model.

Connect to external Postgres databases with SSL

Baseten supports data connections to external sources. Now, data connections to PostgreSQL tables come with an optional additional layer of security: connecting with SSL. Simply check the SSL option and paste in a client key to connect via SSL. You also have the option to provide a self-signed certificate.

Let us know if you need SSL for another data connection like MySQL and we will expand the functionality based on user requests!

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Workspace settings are now organized as full-page tabs rather than a modal
• The app builder tour is dismissible for those who prefer to explore on their own

Deploy your ML model, step-by-step

The model deployment modal is gone, replaced with a full page of instructions to help you check off the prerequisite steps then deploy your model to Baseten with confidence. Dynamic examples across modeling frameworks give you paste-able deployment scripts to use in your code, while the “Import from SageMaker” tab helps you establish a data connection then import your SageMaker models to Baseten.

Along with this on-page guidance, we now have seven new end-to-end model deployment and invocation examples as Google Colab notebooks that you can run from your browser or download. Try deploying models built with HuggingFace, PyTorch, scikit-learn, and more!

Fix errors faster in code files

Scientists estimate that there are 10 quintillion bugs alive on the planet at any given time. Make sure none of them are lurking in your code with new error messages when you try to save invalid syntax in code files.

‍

Explore pre-trained models

The new pre-trained model exploration page showcases more than twenty pre-trained models that can be deployed and immediately used in your Baseten applications or called via API requests.

‍

Find a model that works for your use case by filtering by tags and libraries. And if there is an open-source model that you’d like to see us add to our pre-trained model library, let us know at support@baseten.co. 

Select model version in worklets

When you add a model block to a worklet, you can now select which version of the model is invoked. By default, the primary version of the model will be selected.

‍

This is especially useful for keeping production behavior consistent as you deploy new model versions until you have a chance to update the applications that depend on these models.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Added documentation around specific versions of model frameworks that are officially supported by the Baseten Python client
• Fixed issue with AWS SageMaker input format selector when deploying a model from SageMaker
• Performance fixes and bug improvements

Deploy Hugging Face Transformers, LightGBM, and XGBoost models

Baseten’s model deployment stack runs on Truss, our open-source library for model serving. Any model framework supported by Truss is also supported in the Baseten CLI. As such, the Baseten Python client now supports three new frameworks:

• Hugging Face Transformers
• LightGBM
• XGBoost

And of course you can deploy any model to Baseten as a custom model.

To use these newly supported frameworks, upgrade to the latest version of the Baseten Python client with:

pip install --upgrade baseten

Example deployment

Let’s take a look at what deploying a model built from Hugging Face Transformers looks like:

import baseten
from transformers import pipeline

model = pipeline('fill-mask', model='bert-base-uncased')

baseten.login("*** INSERT API KEY ***")  # https://docs.baseten.co/settings/api-keys

baseten_model = baseten.deploy(
    model,
    model_name='bert base uncased',
)

As we continue developing Truss, more frameworks will be supported out-of-the-box on Baseten. Let us know what framework you’d like to see next by opening an issue.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• You can view and edit your workspace name in settings
• Code files show when they were last edited
• The file upload component checks uploaded files to ensure they are below the maximum size before saving
• Bug fixes and performance improvements

Explore the data explorer

Baseten’s data explorer in the view builder helps you take your ML-powered project from “web page” to “web app.” But the concepts introduced in the data explorer are clearly centered in front-end development, not data science. While the view builder provides helpful abstractions for concepts like application state, it also exposes a great deal of control to let developers choose to build more powerful interfaces.

To assist developers in venturing into front-end concepts, we’ve added tooltips to every field in the data explorer providing immediate context and a direct link to relevant documentation. We encourage you to fearlessly explore building featureful UIs by peeking under the hood of front-end development.

While we’ve been releasing a steady cadence of quality-of-life features and user experience improvements, we have some big things coming in the second half of July that we’re excited to share with you. Tune in next week for Baseten’s first-ever open source project!

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• In the console, use the new “Shift-Enter” hotkey to execute the current cell and create a new one, just like in a Jupyter notebook
• View your account email and workspace role (owner/admin/creator/operator) in your account profile
• Bug fixes and performance improvements

Select the view canvas

We found from user testing that selecting the page canvas was unintuitive. So we added a component-style label above the canvas to make it easy to select and access a menu of powerful properties.

Just like the components on a page, the canvas has configurable properties. You can adjust canvas width depending on the user interface you want to build and add event handlers that fire on page load. These event handlers can initialize state, read query parameters, or kick off worklets and database queries, making the page dynamic from first paint.

At least it’s not Clippy

Within the application builder, we want to give you essential information and reminders while staying out of your way as you work. One of the most important concepts in the view builder is a binding, the double-bracket reference to on-page state. A new dismissible inline reference, shipping soon, tells you all about bindings.

Also, don’t miss tooltips over many component properties explaining their input and function.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• A recent effort removed tens of thousands of lines of leftover code from Baseten’s previous iterations, improving build speed and innovation velocity
• A slightly altered sign-up flow helps us get to know users a little better
• Bug fixes and performance improvements

Copy and paste components in the view builder

Copy and paste are any developer’s foremost time-saving tools. Stack Overflow even offered a keyboard with only the copy and paste keys — first as a joke, then as a real product.

Now, Baseten’s view builder offers this essential feature for components. You can copy and paste components within a view, preserving all properties, event handlers, bindings, and more between the copies. 

Because copy and paste copies all of the component’s properties, you cannot copy and paste components between views or applications. Instead, try cloning the entire view or application to build variants.

Combined with the delete hotkey and, of course, undo and redo, copy and paste makes using the view builder feel like flying a nimble starfighter, but with fewer explosions.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• A badge on the logs tab within the application builder indicates new changes as you work
• Autosave is now permanently enabled in files with a system in place to prevent code file save clobbers
• The dot grid in the view builder now only displays when you are adding, resizing, or moving a component
• Bug fixes and performance improvements

Undo in the view builder

To do, or not to do, that is the question. Whatever your answer, you can now change your mind! In the view builder, you can undo and redo recent actions with cmd-z and shift-cmd-z, respectively. These keyboard shortcuts match undo and redo in our code editor (for Windows, ctrl-z and shift-ctrl-z). 

The undo command stacks, so you can undo and redo multiple changes. Add, alter, and delete components with freedom; you can revert any mistakes instantly.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• The list of pre-trained models shows a badge next to models for which you already have an active deployment
• Updated onboarding cards walk new users through the entire application builder
• Bug fixes and performance improvements

View event logs

In the application builder, the logs tab now shows events from the view in addition to worklet runs. This gives you more information when building and debugging complex user interactions.

If a single user action triggers multiple events, each event will have its own log entry. Each log entry includes the component name and event type, along with any relevant information like worklet name and parameters.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Clone and delete applications from the dropdown menu within the application builder
• When you deploy your own model to Baseten, its type now shows as “Imported”
• Additional security tooling on production clusters
• Bug fixes and performance improvements