Changelog

Baseten’s library of pre-trained models now includes a new addition that we’ve been having a lot of fun with: Dall·E mini. 

You can deploy this model to your Baseten account, and it comes with a basic starter app so that you can start generating images right away. 

‍

Look out for more cool pictures and projects with Dall·E mini soon!

Deploy CodeGen models

We also added two pre-trained models that generate code from a prompt. CodeGen mono and multi are each two-billion-parameter models trained on datasets of code and natural language. The twin models come with identical starter apps for model evaluation and to inspire your own projects.

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Image components support base64 encoded images (see the Dall·E mini starter app for an example)
• Transactional emails now include plain text links for expanded compatibility with various email clients and firewalls
• Bug fixes and performance improvements

‍

As you develop an application on Baseten, you might want to edit your worklets. Now, you can add blocks in between existing blocks in your worklet graph.

‍

‍

You can also delete blocks from the middle of worklets without worry, the graph will snap into place as if the deleted block was never there. 

🎃 The pumpkin patch

A handful of small-but-mighty changes to make Baseten more joyful to use:

• Each component in the view builder now has its own minimum width and height
• Removed unnecessary scrollbars from application builder UI
• Bug fixes and performance improvements

‍

We added charts to visualize the health and performance of your models on BaseTen. Even better, we used pretty colors and fun shapes. From the health tab, you can monitor a model's traffic, response time, and compute resource usage over time.

Just like model logs, the health tab is available for all custom models deployed on your BaseTen account, but not for pre-trained models from our model zoo.

You’ll now have full visibility 👀 into the following:

Prediction volume

The number of requests per minute made to your deployed model:

Response time

Your model's 50th, 90th, 95th, and 99th percentile response time to incoming requests:

CPU and memory usage

The total CPU and memory usage across all replicas of your model–or, focus in on a single replica:

🔮 Back in time

For every model metric, you can use the time picker to select a time range from the last five minutes to the last six hours. Along with model logs, this access will make you feel like Sherlock Holmes as you crack the case of any model health concerns.

‍

You’re now able to access queries from a code block in a Worklet! These queries can be in the form of raw SQL (as a string) or a name of a query you’ve defined earlier.

For example:

new_query_output = context.run_query_str("select * from test_table") # to run raw SQL queries

existing_query_output = context.run_query_name("testquery") # to run a specific query you’ve already defined

These examples show queries against Baseten Postgres data tables. If you’d like to query external data sources you have connected to your Baseten account, check out our documentation.

To access the results of your queries, either return or print their output (new_query_output, existing_query_output) from your code block!

‍

We've recently pushed out a change that allows you to associate pieces of information with models that are deployed in the BaseTen infrastructure. You can now track configurations, hyperparameters, and specific files with your models. Learn more about tracking model metadata in our documentation!

https://docs.baseten.co/ml-models/tracking-model-metadata

‍

You can now access queries across all your applications in one place! You’ll now find all your queries in the Queries tab of your Data page. Query your own BaseTen Postgres tables (any tables defined in the Data page under BaseTen Postgres) and all connected external data sources (in the Connections portion of the Data page). Happy querying!

‍

Instead of unnecessarily updating Docker images every single time a model is deployed, we're now periodically updating base images for every model framework we support. We're seeing that model deployments can be 2-5 times faster as a result.

If you've been testing your application but want to start from a clean slate, you can easily delete all the rows in your BaseTen-provided Postgres table(s). Click the red "Delete all data" button. Before your data is deleted, a confirmation pop-up will appear with details on how many rows will be deleted and any referenced tables that will also be affected. If the table you're deleting data from is a foreign key that is referenced in other tables, then the columns that reference the table you're deleting data from will be set to null.

If you need to delete individual rows instead of all the rows in your table, you can still do that with code. Just follow the instructions in this doc on deleting objects.

‍

You can now specify system-level packages that you want to install in your Python environment on BaseTen.

BaseTen runs your python code on a container using a standard Python image for the Debian Operating System. Occasionally, you may come across Python packages that require additional system packages to function. For example, librosa, a Python package for audio analysis requires the ffmpeg system package for understanding a number of common audio formats such as MP3. Similarly, OpenCV, the popular image-processing package, requires the libjpeg system package to read JPEG files.

Currently, only a standard set of package repositories are looked up but we intend to expand the number of available repositories. Please let us know if you're unable to install a package you need.

‍

You can now see your BaseTen-provided Postgres tables and external connections in one place. Click into the Data overview page to manage all your tables and connections.

Coming very soon, you'll also be able to manage all your queries from this Data page.

‍

The Applications and Models overview pages are now table-based instead of card-based. With these tables, we’ve streamlined the information on each page and removed the reliance on icons to designate metadata. The tables are also better suited for larger numbers of models and applications and set us up to support filtering and sorting in the future.

‍

We've released the first of many small navigation changes, all intended to make it easier for you to get around the product and differentiate between exploration mode (when you need to understand how all your models, apps, and data relate to each other) and focus mode (when you’re actively building in BaseTen).

Your main navigation now includes words, because the icons in the old icon-only navigation didn’t speak for themselves. The main navigation also minimizes when you’re editing worklets and views now, to give you more space to build.

We're always trying to make it easier for you to work on BaseTen, both on your own and with others. Here’s a little improvement that should help your editing experience when you have multiple windows of BaseTen open or are collaborating on BaseTen with someone else.

If you accidentally end up editing a worklet, view, or query in multiple windows, we kindly prevent you from saving changes that’ll overwrite your work. We do the same thing when you and a colleague are collaborating on an application together, so no more accidentally editing over each other's work.

‍

ML tasks can be fairly time-consuming. Processing audio, video, and images can take minutes, sometimes even hours. While BaseTen already supported processing of up to 5 minutes per python node invocation, even that can be limiting.

We're happy to announce that long-running tasks are now a first-class citizen in BaseTen. Each python node invocation can take up to 4 hours, as long as you invoke the worklet in async mode.

If your worklet has long-running tasks, i.e. tasks that take more than a few minutes, we highly recommend marking the worklet for background execution. Then, the worklet will always execute in async mode.

Alternatively, you can pass the async flag in the API request to execute any worklet in async mode:

curl -X POST https://app.baseten.co/applications/{YOUR_APP_ID}/worklets/{YOUR_WORKLET_ID}/invoke -H 'Authorization: Api-Key YOUR_API_KEY' -d '{ "worklet_input": WORKLET_INPUT , "async": true}'

When writing any sort of software, it's inevitable to run into errors that Googling, scrolling through myriad Stack Overflow answers, and random fixes ultimately cannot resolve. Here at BaseTen, we’ve made an effort to rid you of some of that misery by adding more descriptive error messages to worklet and node outputs.

Here are a few of the errors we’ve made more clear:

• New requirements still being applied to your python environment: If you’re trying to use new requirements right away, you’ll see a message letting you know they’re still being applied.

• (ModuleNotFoundError): No module named 'cv2'; this might be due to the new requirements that are still being applied.

• Timing out in a worklet or node: When you run a worklet or node with some code that times out because it runs for longer than 5 minutes, you’ll see a message encouraging you to optimize your code or run it as a background task.

• (ReadTimeout): The node timed out. Nodes must be executed in 300s or less; you can make this error go away by optimizing your code, or running it as a background task.

• Server errors: When you run a worklet or node with some code that causes a server error, you’ll see a message letting you know that you should try again or contact us.

• (HTTPError): An error has occurred. Please try running your code again. If the error persists, reach out to BaseTen for more information.

Here’s to hoping you never run into unexplained errors in BaseTen 🤞 But if you do, we’re here to help. Happy coding!

‍