WizardLMWizardLM

A seven billion parameter large language model fine tuned from Llama for general chat tasks.

Deploy WizardLM behind an API endpoint in seconds.

Deploy model

Example usage

This code example shows how to invoke the model using the requests library in Python. The model has a couple of key inputs:

  1. prompt: The input text sent to the model.

  2. max_new_tokens: Allows you to control the length of the output sequence.

The output of the model is a JSON object which has a key called output that contains the generated text.

Input
1import requests
2import os
3
4# Replace the empty string with your model id below
5model_id = ""
6baseten_api_key = os.environ["BASETEN_API_KEY"]
7
8data = {
9    "prompt": "What is the most powerful wizard in the world?",
10    "max_new_tokens": 512
11}
12
13# Call model endpoint
14res = requests.post(
15    f"https://model-{model_id}.api.baseten.co/production/predict",
16    headers={"Authorization": f"Api-Key {baseten_api_key}"},
17    json=data
18)
19
20# Print the output of the model
21print(res.json())
JSON output
1{
2    "output": "What is the most powerful wizard in the world?\n\n    ### Response:\n    \n    Merlin, from the King Arthur legend, is often considered the most powerful wizard in the world of fiction. He is known for his wisdom, magical powers, and his ability to see into the future. Merlin is said to have advised King Arthur and helped him become the legendary ruler of Camelot. His powers include the ability to manipulate time, control the elements, and even shape-shift into various animals. Merlin's story has been retold in many different versions and adaptations, making him one of the most well-known and beloved wizards in literature and popular culture."
3}

Deploy any model in just a few commands

Avoid getting tangled in complex deployment processes. Deploy best-in-class open-source models and take advantage of optimized serving for your own models.

$

truss init -- example stable-diffusion-2-1-base ./my-sd-truss

$

cd ./my-sd-truss

$

export BASETEN_API_KEY=MdNmOCXc.YBtEZD0WFOYKso2A6NEQkRqTe

$

truss push

INFO

Serializing Stable Diffusion 2.1 truss.

INFO

Making contact with Baseten πŸ‘‹ πŸ‘½

INFO

πŸš€ Uploading model to Baseten πŸš€

Upload progress: 0% | | 0.00G/2.39G