Falcon-40B is an impressive LLM that made it to the top of the OpenLLM leaderboard. It can be used commercially but requires serious hardware to run. You can deploy Falcon-40B on Baseten using this Truss model package. Or, start with the smaller, less expensive Falcon-7B which you can deploy in a couple clicks from our model library.
Falcon-40B is an LLM released by Technology Innovation Institute (TII) in Dubai that’s been causing waves in the AI community for making it to the top of the OpenLLM leaderboard. The benchmarks that have come out put Falcon-40B in the ranks of GPT-3.5, and from an anecdotal point of view, are very impressive.
We deployed Falcon-40B on two A100s on Baseten this week and were amazed. Here are a few example prompts and responses that impressed us:
Write a recipe for almond tea cake
1 1/2 cups unsalted butter
1 1/2 cups granulated sugar
1/2 cup packed light brown sugar
3 large eggs
2 1/4 cups almond flour
1/2 cup all-purpose flour
1/4 cup cornstarch
2 teaspoons vanilla extract
1/2 teaspoon almond extract
Preheat oven to 350°F. Line a 9-inch springform pan with parchment paper.
Beat butter, granulated sugar, brown sugar, and eggs in a large bowl with an electric mixer on medium speed until light and fluffy, about 5 minutes. Add almond flour, all-purpose flour, cornstarch, vanilla extract, and almond extract, and beat until just combined, about 1 minute.
Pour batter into prepared pan and smooth the top with a spatula.
Bake for 50 minutes to 1 hour, until a toothpick inserted in the center of the cake comes out clean. Let cool for 10 minutes, then remove the sides of the springform pan and cool cake on wire rack for 2 to 3 hours.
Serve and enjoy!
Come up with a movie plot about Falcons scheming to take over the world
In the movie, Falcons are portrayed as highly intelligent and cunning creatures who have a hidden agenda of taking over the world. They use their natural abilities to plan and execute nefarious activities that are seemingly impossible to detect. From breaking into top-secret facilities and stealing sensitive military information to influencing politicians to change laws in their favor, they will stop at nothing to achieve their ultimate goal of world domination.
Write a Flask Endpoint for receiving a stripe webhook and printing it
To receive Stripe webhooks in Flask, you can use the Flask library to handle POST requests and print the data for logging or debugging purposes. Here's an example endpoint that listens to a certain endpoint URL and prints the received data:
from flask import Flask, request # Initialize app app = Flask(__name__) # Define url that webhook should be sent to WEBHOOK_ENDPOINT_URL = '/stripe/webhook' @app.route(WEBHOOK_ENDPOINT_URL, methods=['POST']) def webhook_handler(): # Parse the received data payload = request.json.decode('utf-8') # Print the payload for debugging or logging print(payload) # Return a success response return 'OK' # Run the app if __name__ == '__main__': app.run()
Now when a Stripe webhook request is sent to your endpoint URL, it will be received and handled by your Flask endpoint, which will parse the payload and print it to the console. You can modify the endpoint to perform any additional processing or storage of the received data as per your needs.
Unlike some of the other top open source LLMs, Falcon-40B can be used for commercial purposes. That’s easier said than done though. Say you want to use Falcon-40B in an application that you’re building. It’s tough! It’s a big model, so getting reasonable performance requires serious hardware (the elusive A100s). And using Falcon-40B in a real app means you need monitoring, easy scaling, logs, and other features that you’d generally need to run something in prod.
To make that easier, we packaged the model with Truss, the open source package we use for deploying models, so you can now quickly deploy Falcon-40B on Baseten.
Once you have a Baseten account, clone this Github repo and run the following to deploy Falcon-40B:
pip install --upgrade truss git clone https://github.com/basetenlabs/truss-examples cd falcon/falcon-40b-truss truss push
Once you deploy Falcon-40B, you’ll get a REST API endpoint that you can use to get responses from the model. You’ll also get the full power of Baseten’s model serving infrastructure, including logging, monitoring, and autoscaling, powering your model.
Falcon-40B needs to run on two A100s GPU, which can get expensive quickly. You can also choose to start with the smaller Falcon-7B, which can run on one A10 GPU. You can deploy Falcon-7B in a couple clicks from our model library – and new users get $30 of model resource credit to start deploying models on Baseten for free.
Read more about Falcon-40B: