Please rotate your device

To continue exploring this website, please rotate your device to a vertical orientation

Changelog

Get 500 response code on model invocation error

Date

Models deployed on Baseten using Truss 0.7.1 or later can now send the 500 response code when there is an error during model invocation. This change only affects newly deployed models.

Any exception raised will result in a 500 response code. For example, this Truss code:

python
class Model:
    def predict(...):
         raise Exception("hello")

Will yield a response with the following content:

json
{
    "error": "Internal Server Error"
}

You can see details in the model logs for each exception raised, and errors are aggregated on the model metrics page.

Inference volume showing 200, 400, and 500 response codes
Inference volume showing 200, 400, and 500 response codes