Blaxel is joining Baseten. Read more
Blog

Inference engineering

AllModel performanceAI engineeringInfrastructureNewsCommunityAI modelsProductFoundations
AI engineering

Streaming real-time text to speech with XTTS V2

Het Trivedi
1 other
Model performance

Continuous vs dynamic batching for AI inference

Matt Howard
1 other
Infrastructure

Using fractional H100 GPUs for efficient model serving

Matt Howard
3 others
Model performance

Benchmarking fast Mistral 7B inference

Abu Qader
3 others
Model performance

33% faster LLM inference with FP8 quantization

Pankaj Gupta
1 other
Model performance

High performance ML inference with NVIDIA TensorRT

Justin Yi
1 other
Model performance

FP8: Efficient model inference with 8-bit floating point numbers

Pankaj Gupta
1 other
News

Announcing our Series B

Tuhin Srivastava
Infrastructure

The benefits of globally distributed infrastructure for model serving

Phil Howes
1 other
1...202122...27