Blaxel is joining Baseten. Read more

Blog

Inference engineering

All Model performance AI engineering Infrastructure News Community AI models Product Foundations

Infrastructure

Testing Llama 3.3 70B inference performance on NVIDIA GH200 in Lambda Cloud

Pankaj Gupta

Philip Kiely

Pankaj Gupta

1 other

Testing GH200 GPUs

News

Baseten Chains is now GA for production compound AI systems

Marius Killinger

Tyron Jung

Marius Killinger

2 others

Baseten Chains

AI engineering

Private, secure DeepSeek-R1 in production in US & EU data centers

Amir Haghighat

Philip Kiely

Yineng Zhang

2 others

DeepSeek R1

Model performance

Driving model performance optimization: 2024 highlights

Pankaj Gupta

Pankaj Gupta

MP 2024 highlights

News

New observability features: activity logging, LLM metrics, and metrics dashboard customization

Aaron Relph

Marius Killinger

Sid Shanker

Suren Atoyan

4 others

Observability

Model performance

How we built production-ready speculative decoding with TensorRT-LLM

Pankaj Gupta

Philip Kiely

Pankaj Gupta

2 others

Speculative Decoding with TensorRT-LLM

Model performance

A quick introduction to speculative decoding

Pankaj Gupta

Philip Kiely

Pankaj Gupta

2 others

Intro to Speculative Decoding

News

Introducing our Speculative Decoding Engine Builder integration for ultra-low-latency LLM inference

Abu Qader

Bryce Dubayah

Justin Yi

3 others

Speculative Decoding in Engine Builder

Model performance

Generally Available: The fastest, most accurate and cost-efficient Whisper transcription

William Gao

Derrick Yang

William Gao

3 others

The fastest Whisper

1...16 17 18...27