Baseten acquires Parsed: Own your intelligence by unifying training and inference. READ
Blog

Blog

Expert guides and engineering deep dives to help you ship faster, scale easier, and learn along the way.

AllModel performanceAI engineeringInfrastructureNewsCommunityFoundations
Model performance

Comparing tokens per second across LLMs

Philip Kiely
Comparing TPS across LLMs
AI engineering

CI/CD for AI model deployments

Vlad Shulman
3 others
CI/CD for AI models
AI engineering

Streaming real-time text to speech with XTTS V2

Het Trivedi
1 other
Streaming TTS
Model performance

Continuous vs dynamic batching for AI inference

Matt Howard
1 other
Continuous vs Dynamic batching
Infrastructure

Using fractional H100 GPUs for efficient model serving

Matt Howard
3 others
H100 MIGs
Model performance

Benchmarking fast Mistral 7B inference

Abu Qader
3 others
Mistral 7B
Model performance

33% faster LLM inference with FP8 quantization

Pankaj Gupta
1 other
Faster inference with FP8
Model performance

High performance ML inference with NVIDIA TensorRT

Justin Yi
1 other
NVIDIA TensorRT
Model performance

FP8: Efficient model inference with 8-bit floating point numbers

Pankaj Gupta
1 other
8-bit floating point numbers
1...91011...16