# Appendix B: Recommended Reading

_Inference Engineering_ by Philip Kiely. © 2026 Baseten Labs, Inc. All rights reserved. [Full book index](https://www.baseten.co/inference-engineering/llms.txt)

This book is an introduction to the field of inference engineering. There is endless depth to explore in every one of the technologies and techniques behind performant inference at scale.

If you’re in the market for another book to continue learning, I have three recommendations:

- _AI Engineering: Building Applications with Foundation Models_ by Chip Huyen (O’Reilly Media, 2025): This incredibly popular book introduces the full breadth of AI engineering topics.
- _Build a Large Language Model (From Scratch)_ by Sebastian Raschka (Manning, 2024): This hands-on book provides a detailed look at LLM architecture.
- _AI Systems Performance Engineering: Optimizing Model Training and Inference Workloads with GPUs, CUDA, and PyTorch_ by Chris Fregly (O’Reilly Media, 2025): This brand-new book focuses on building for performance.

The AI industry moves fast, and new models, research, and implementations are constantly being released. My colleagues and I publish our latest work on the Baseten blog, which you can access at [https://www.baseten.com/blog](https://www.baseten.com/blog).

This appendix provides a list of papers, documentation, books, and blogs to further support your next steps as an inference engineer. Resources are organized by topic and alphabetized by title within each section.

## Architecture

- “Attention is All You Need,” by Ashish Vaswani et al. (_Neural Information Processing Systems_, 2017), [https://arxiv.org/abs/1706.03762](https://arxiv.org/abs/1706.03762)
- “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” by Jacob Devlin et al. (_North American Chapter of the Association for Computational Linguistics, 2019),_ [https://arxiv.org/abs/1810.04805](https://arxiv.org/abs/1810.04805)
- “BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models,” by Junnan Li et al. (International Conference on Machine Learning, 2023), [https://arxiv.org/abs/2301.12597](https://arxiv.org/abs/2301.12597)
- _Deep Learning_, by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (The MIT Press, 2016), [https://www.deeplearningbook.org/](https://www.deeplearningbook.org/)
- _Deep Learning with Python (2nd Edition)_, by François Chollet (Manning, 2021), [https://www.manning.com/books/deep-learning-with-python-second-edition](https://www.manning.com/books/deep-learning-with-python-second-edition)
- “Denoising Diffusion Probabilistic Models,” by Jonathan Ho et al. (_ArXiv_ abs/2006.11239, 2020), [https://arxiv.org/abs/2006.11239](https://arxiv.org/abs/2006.11239)
- “DiT: Scalable Diffusion Models with Transformers,” by William Peebles and Saining Xie _(International Conference on Computer Vision (ICCV)_, 2022), [https://arxiv.org/abs/2212.09748](https://arxiv.org/abs/2212.09748)
- “FlashAttention: Fast and Memory-Efficient Exact Attention with IO Awareness,” by Tri Dao et al. (_ArXiv_ abs/2205.14135, 2022), [https://arxiv.org/abs/2205.14135](https://arxiv.org/abs/2205.14135)
- “FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning,” by Tri Dao (_ArXiv_ abs/2307.08691, 2023), [https://arxiv.org/abs/2307.08691](https://arxiv.org/abs/2307.08691)
- “FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision,” by Jay Shah et al. (_ArXiv_ abs/2407.08608, 2024), [https://arxiv.org/abs/2407.08608](https://arxiv.org/abs/2407.08608)
- _Flash-Attention-4_, by Tri Dao (Dao AI Research Lab, 2025), [https://github.com/Dao-AILab/flash-attention](https://github.com/Dao-AILab/flash-attention)
- “Imagen Video: High Definition Video Generation with Diffusion Models,” by Jonathan Ho et al. (_ArXiv_ abs/2210.02303, 2022), [https://arxiv.org/abs/2210.02303](https://arxiv.org/abs/2210.02303)
- “Language Models Are Few-Shot Learners,” by Tom Brown et al. (_ArXiv_ abs/2005.14165), [https://arxiv.org/abs/2005.14165](https://arxiv.org/abs/2005.14165)
- “Learning Transferable Visual Models from Natural Language Supervision,” by Alec Radford et al. (_International Conference on Machine Learning_, 2021), [https://arxiv.org/abs/2103.00020](https://arxiv.org/abs/2103.00020)
- “Longformer: The Long-Document Transformer,” by Iz Beltagy et al. (_ArXiv_ abs/2004.05150, 2020), [https://arxiv.org/abs/2004.05150](https://arxiv.org/abs/2004.05150)
- “Mamba: Linear-Time Sequence Modeling with Selective State Spaces,” by Albert Gu and Tri Dao (_ArXiv_ abs/2312.00752, 2023), [https://arxiv.org/abs/2312.00752](https://arxiv.org/abs/2312.00752)
- “Matryoshka Representation Learning,” by Aditya Kusupati et al. (_Neural Information Processing Systems_, 2022), [https://arxiv.org/abs/2205.13147](https://arxiv.org/abs/2205.13147)
- “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,” by Noam Shazeer et al. (_ArXiv_ abs/1701.06538, 2017), [https://arxiv.org/abs/1701.06538](https://arxiv.org/abs/1701.06538)
- “Reformer: The Efficient Transformer,” by Nikita Kitaev et al. (_ArXiv_ abs/2001.04451, 2020), [https://arxiv.org/abs/2001.04451](https://arxiv.org/abs/2001.04451)
- “Robust Speech Recognition via Large-Scale Weak Supervision,” by Alec Radford et al. (_International Conference on Machine Learning_, 2022), [https://arxiv.org/abs/2212.04356](https://arxiv.org/abs/2212.04356)
- “RoFormer: Enhanced Transformer with Rotary Position Embedding,” by Jianlin Su et al. (_ArXiv_ abs/2104.09864, 2021), [https://arxiv.org/abs/2104.09864](https://arxiv.org/abs/2104.09864)
- “SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis,” by Dustin Podell et al. (_ArXiv_ abs/2307.01952, 2023), [https://arxiv.org/abs/2307.01952](https://arxiv.org/abs/2307.01952)
- “Segment Anything,” by Alexander Kirillov et al. (_2023 IEEE/CVF International Conference on Computer Vision (ICCV),_ 2023), [https://arxiv.org/abs/2304.02643](https://arxiv.org/abs/2304.02643)
- “Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks,” by Nils Reimers and Iryna Gurevych (_ArXiv_ abs/1908.10084, 2019), [https://arxiv.org/abs/1908.10084](https://arxiv.org/abs/1908.10084)
- “The Llama 3 Herd of Models,” by Aaron Grattafiori et al. (_ArXiv_ 2407.21783, 2024), [https://arxiv.org/abs/2407.21783](https://arxiv.org/abs/2407.21783)
- “Video Diffusion Models,” by Jonathan Ho et al. (_ArXiv_ abs/2204.03458, 2022), [https://arxiv.org/abs/2204.03458](https://arxiv.org/abs/2204.03458)
- “Visual Instruction Tuning,” by Haotian Liu et al. (_ArXiv_ abs/2304.08485, 2023), [https://arxiv.org/abs/2304.08485](https://arxiv.org/abs/2304.08485)

## Developer Tools

- BitsAndBytes by bitsandbytes-foundation, [https://github.com/TimDettmers/bitsandbytes](https://github.com/TimDettmers/bitsandbytes)
- ComfyUI by comfyanonymous, [https://github.com/comfyanonymous/ComfyUI](https://github.com/comfyanonymous/ComfyUI)
- “CUDA by Example: An Introduction to General Purpose GPU Programming,” by Jason Sanders and Edward Kandrot (NVIDIA developer, 2025), [https://developer.nvidia.com/cuda-example](https://developer.nvidia.com/cuda-example)
- “CUDA C++ Programming Guide Release 13.0,” by NVIDIA (2025), [https://docs.nvidia.com/cuda/cuda-c-programming-guide/](https://docs.nvidia.com/cuda/cuda-c-programming-guide/)
- “CUDA cuBLAS Release 13.0,” by NVIDIA (2025),[https://docs.nvidia.com/cuda/cublas/](https://docs.nvidia.com/cuda/cublas/)
- CUTLASS by NVIDIA, [https://github.com/NVIDIA/cutlass](https://github.com/NVIDIA/cutlass)
- DeepGEMM by DeepSeek-ai, [https://github.com/deepseek-ai/DeepGEMM](https://github.com/deepseek-ai/DeepGEMM)
- Hugging Face Diffusers by Hugging Face, [https://huggingface.co/docs/diffusers/index](https://huggingface.co/docs/diffusers/index)
- LMCache by LMCache Project, [https://github.com/LMCache/LMCache](https://github.com/LMCache/LMCache)
- “NVIDIA Dynamo Documentation,” by NVIDIA (2025), [https://docs.nvidia.com/dynamo/latest/index.html](https://docs.nvidia.com/dynamo/latest/index.html)
- NVIDIA Nsight Systems by NVIDIA, [https://developer.nvidia.com/nsight-systems](https://developer.nvidia.com/nsight-systems)
- NVIDIA Triton Inference Server by NVIDIA, [https://github.com/triton-inference-server/server](https://github.com/triton-inference-server/server)
- ONNX Runtime by Microsoft, [https://onnxruntime.ai/](https://onnxruntime.ai/)
- “PyTorch Performance Tuning Guide,” by Szymon Migacz (PyTorch Foundation, 2020), [https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html](https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html)
- “PyTorch Profiler,” by Shivam Raikundalia (PyTorch Foundation, 2021), [https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html)
- SGLang Project by LMSYS Org, [https://github.com/sgl-project/sglang](https://github.com/sgl-project/sglang)
- “TensorRT Documentation,” by NVIDIA (2025), [https://docs.nvidia.com/deeplearning/tensorrt/](https://docs.nvidia.com/deeplearning/tensorrt/)
- TensorRT-LLM by NVIDIA, [https://github.com/NVIDIA/TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM)
- Transformers by Hugging Face, [https://huggingface.co/docs/transformers/index](https://huggingface.co/docs/transformers/index)
- vLLM Project by The Linux Foundation, [https://github.com/vllm-project/vllm](https://github.com/vllm-project/vllm)

###

## Frontier Open Models

- _DeepSeek,_ by DeepSeek AI, [https://huggingface.co/deepseek-ai](https://huggingface.co/deepseek-ai)
- _FLUX,_ by Black Forest Labs**,** [https://huggingface.co/black-forest-labs](https://huggingface.co/black-forest-labs)
- _Gemma,_ by Google, [https://huggingface.co/google](https://huggingface.co/google)
- _GLM,_ by Z.ai, [https://huggingface.co/zai-org](https://huggingface.co/zai-org)
- _GPT OSS,_ by OpenAI, [https://huggingface.co/openai](https://huggingface.co/openai)
- _Kimi,_ by Moonshot AI, [https://huggingface.co/moonshotai](https://huggingface.co/moonshotai)
- _Llama,_ by Meta Llama, [https://huggingface.co/meta-llama](https://huggingface.co/meta-llama)
- _MiniMax,_ by MiniMax AI, [https://huggingface.co/MiniMaxAI](https://huggingface.co/MiniMaxAI)
- _Mistral,_ by Mistral AI, [https://huggingface.co/mistralai](https://huggingface.co/mistralai)
- _Nemotron,_ by NVIDIA, [https://huggingface.co/nvidia](https://huggingface.co/nvidia)
- _Orpheus,_ by Canopy Labs, [https://huggingface.co/canopylabs](https://huggingface.co/canopylabs)
- _Qwen,_ by Alibaba Qwen, [https://huggingface.co/Qwen](https://huggingface.co/Qwen)
- _Wan,_ by Wan-AI, [https://huggingface.co/Wan-AI](https://huggingface.co/Wan-AI)
- _Whisper,_ by OpenAI, [https://huggingface.co/openai](https://huggingface.co/openai)

## GPU Infrastructure

- _Designing Data-Intensive Applications,_ by Martin Kleppmann (O’Reilly Media, 2017) [https://dataintensive.net/](https://dataintensive.net/)
- Grace Hopper / Grace Blackwell Systems by NVIDIA, [https://www.nvidia.com/en-us/data-center/grace-cpu/](https://www.nvidia.com/en-us/data-center/grace-cpu/)
- _GPU Glossary,_ by Frye et al. (Modal, 2025), [https://modal.com/gpu-glossary](https://modal.com/gpu-glossary)
- _InfiniBand,_ by NVIDIA, [https://www.nvidia.com/en-us/networking/products/infiniband/](https://www.nvidia.com/en-us/networking/products/infiniband/)
- _Kubernetes Documentation,_ by The Kubernetes Authors (The Linux Foundation, 2025), [https://kubernetes.io/docs/home/](https://kubernetes.io/docs/home/)
- _NVIDIA Blackwell Architecture Technical Brief: Built for the Age of AI Reasoning,_ (NVIDIA, 2025), [https://resources.nvidia.com/en-us-blackwell-architecture?ncid=no-ncid](https://resources.nvidia.com/en-us-blackwell-architecture?ncid=no-ncid)
- _NVIDIA H100 Tensor Core GPU Architecture: Exceptional Performance, Scalability and Security for the Data Center,_ (NVIDIA, 2023), [https://resources.nvidia.com/en-us-hopper-architecture/nvidia-h100-tensor-c](https://resources.nvidia.com/en-us-hopper-architecture/nvidia-h100-tensor-c)
- "NVIDIA Tesla: A Unified Graphics and Computing Architecture," by E. Lindholm et al. (_IEEE Micro_, March–April 2008), [https://ieeexplore.ieee.org/document/4523358](https://ieeexplore.ieee.org/document/4523358)
- _NVLink / NVSwitch,_ by NVIDIA, [https://www.nvidia.com/en-us/data-center/nvlink/](https://www.nvidia.com/en-us/data-center/nvlink/)
- _Programming Massively Parallel Processors: A Hands-on Approach,_ by Wen-mei Hwu, David Kirk, Izzat El Hajj (Morgan Kaufmann, 2022), [https://www.goodreads.com/work/editions/10244675-programming-massively-parallel-processors-a-hands-on-approach](https://www.goodreads.com/work/editions/10244675-programming-massively-parallel-processors-a-hands-on-approach)
- _SemiAnalysis,_ by Dylan Patel (SemiAnalysis, 2025), [https://semianalysis.com/](https://semianalysis.com/)
- _Site Reliability Engineering: How Google Runs Production Services_, edited by Betsy Beyer et al. (O’Reilly Media, 2017), [https://sre.google/books/](https://sre.google/books/)

## Inference Optimization Research

- “Adversarial Diffusion Distillation,” by Axel Sauer et al. (_European Conference on Computer Vision_, 2023), [https://arxiv.org/abs/2311.17042](https://arxiv.org/abs/2311.17042)
- “Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation,” by Ofir Press, Noah Smith, and Mike Lewis (_ArXiv_ abs/2108.12409, 2021), [https://arxiv.org/abs/2108.12409](https://arxiv.org/abs/2108.12409)
- “AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration,” by Song Han (MIT, 2024), [https://github.com/mit-han-lab/llm-awq](https://github.com/mit-han-lab/llm-awq)
- Cache-DIT by Vipshop, [https://github.com/vipshop/cache-dit](https://github.com/vipshop/cache-dit)
- “CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion,” by Jiayi Yao et al. (_Proceedings of the Twentieth European Conference on Computer Systems_, 2024), [https://arxiv.org/abs/2405.16444](https://arxiv.org/abs/2405.16444)
- “Adding Conditional Control to Text-to-Image Diffusion Models,” by Lymin Zhang et al. (_International Conference on Computer Vision,_ 2023), [https://arxiv.org/abs/2302.05543](https://arxiv.org/abs/2302.05543)
- “Beyond the Buzz: A Pragmatic Take on Inference Disaggregation,” by Tiyasa Mitra et al. (_ArXiv_ abs/2506.05508, 2025), [https://arxiv.org/abs/2506.05508](https://arxiv.org/abs/2506.05508)
- “Break the Sequential Dependency of LLM Inference Using Lookahead Decoding,” by Yichao Fu et al. (_ArXiv_ abs/2402.02057, 2024), [https://arxiv.org/abs/2402.02057](https://arxiv.org/abs/2402.02057)
- “EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty,” by Yuhui Li et al. (_ArXiv_ abs/2401.15077, 2024), [https://arxiv.org/abs/2401.15077](https://arxiv.org/abs/2401.15077)
- “EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees,” by Yuhui Li et al. (_Conference on Empirical Methods in Natural Language Processing_, 2024), [https://arxiv.org/abs/2406.16858](https://arxiv.org/abs/2406.16858)
- “EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test,” by Yuhui Li et al. (_ArXiv_ abs/2503.01840, 2025), [https://arxiv.org/abs/2503.01840](https://arxiv.org/abs/2503.01840)
- “FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving,” by Ye et al. (_ArXiv_ abs/2501.01005, 2025), [https://arxiv.org/abs/2501.01005](https://arxiv.org/abs/2501.01005)
- “GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers,” by Elias Frantar (_ArXiv_ abs/2210.17323, 2022), [https://arxiv.org/abs/2210.17323](https://arxiv.org/abs/2210.17323)
- “High-Resolution Image Synthesis with Latent Diffusion Models,” by Robin Rombach et al. (_Conference on Computer Vision and Pattern Recognition (CVPR)_, 2021), [https://arxiv.org/abs/2112.10752](https://arxiv.org/abs/2112.10752)
- “Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference,” by Simian Luo (_ArXiv_ abs/2310.04378, 2023), [https://arxiv.org/abs/2310.04378](https://arxiv.org/abs/2310.04378)
- “LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale,” by Tim Dettmers et al. (_ArXiv_ abs/2208.07339, 2022), [https://arxiv.org/abs/2208.07339](https://arxiv.org/abs/2208.07339)
- “Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads,” by Tianle Cai et al. (_ArXiv_ abs/2401.10774, 2024), [https://arxiv.org/abs/2401.10774](https://arxiv.org/abs/2401.10774)
- “Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism,” by Mohammad Shoeybi et al. (_ArXiv_ abs/1909.08053, 2019), [https://arxiv.org/abs/1909.08053](https://arxiv.org/abs/1909.08053)
- “Efficient Memory Management for Large Language Model Serving with PagedAttention,” by Woosuk Kwon et al. (_Proceedings of the 29th Symposium on Operating Systems Principles_, 2023), [https://arxiv.org/abs/2309.06180](https://arxiv.org/abs/2309.06180)
- “Fast Inference from Transformers via Speculative Decoding,” by Yaniv Leviathan et al. (_International Conference on Machine Learning_, 2022), [https://arxiv.org/abs/2211.17192](https://arxiv.org/abs/2211.17192)
- “Ring Attention with Blockwise Transformers for Near-Infinite Context,” by Hao Lin et al. (_ArXiv_ abs/2310.01889, 2023), [https://arxiv.org/abs/2310.01889](https://arxiv.org/abs/2310.01889)
- “SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration,” by Jintao Zhang et al. (_ArXiv_ abs/2410.02367, 2024), [https://arxiv.org/abs/2410.02367](https://arxiv.org/abs/2410.02367)
- _Sequence/Context Parallelism,_ by Megatron-LM for NVIDIA [https://github.com/NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM)
- SmoothQuant by Song Han (MIT), [https://github.com/mit-han-lab/smoothquant](https://github.com/mit-han-lab/smoothquant)
- “SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot,” by Elias Frantar and Dan Alistarh, (_ArXiv_ abs/2301.00774, 2023), [https://arxiv.org/abs/2301.00774](https://arxiv.org/abs/2301.00774)
- “SpecVLM: Fast Speculative Decoding in Vision-Language Models,” Haiduo Huang et al. (_ArXiv_ abs/2509.11815, 2025), [https://arxiv.org/abs/2509.11815](https://arxiv.org/abs/2509.11815)
- “TeaCache: Efficient KV Cache Compression via Tensor Decomposition,” by Feng Lu et al. (Alibaba TongYi Vision Intelligence Lab, _ArXiv_ abs/2411.19108, 2025), [https://github.com/ali-vilab/TeaCache](https://github.com/ali-vilab/TeaCache)

## Intelligence Evaluation

- ARC AGI Prize by Greg Kamradt (2025), [https://arcprize.org/](https://arcprize.org/)
- _Evals for AI Engineers: Systematically Measuring and Improving AI Applications_, by [Shreya Shankar](https://www.oreilly.com/search/?query=author:%22Shreya%20Shankar%22&sort=relevance&highlight=true) and Hamel Husain (O’Reilly Media, forthcoming 2026) [https://www.oreilly.com/library/view/evals-for-ai/9798341660717/](https://www.oreilly.com/library/view/evals-for-ai/9798341660717/)
- _Grade School Math: Training Verifiers to Solve Math Word Problems,_ by Karl Cobbe and Vineet Kosaraju (_ArXiv_ abs/2110.14168, 2021), [https://github.com/openai/grade-school-math](https://github.com/openai/grade-school-math)
- “How to Fine-Tune Qwen3 to GPT-4o Level Performance,” by Greg Schoeninger (_Fine-Tune Fridays_, Oxen AI, 2025), [https://ghost.oxen.ai/how-to-fine-tune-qwen3-to-gpt-4o-level-performance/](https://ghost.oxen.ai/how-to-fine-tune-qwen3-to-gpt-4o-level-performance/)
- “Humanity's Last Exam,” by Long Phan et al. (_Center for AI Safety and Scale AI, ArXiv_ abs/2501.14249, 2025), [https://www.lastexam.ai/](https://www.lastexam.ai/)
- “HumanEval: Evaluating Large Language Models Trained on Code,” by Michelle Pokrass, Qiming Yuan, and Yichen Xu (OpenAI, 2021), [https://github.com/openai/human-eval](https://github.com/openai/human-eval)
- “MMLU: Measuring Massive Multitask Language Understanding,” by Dan Hendrycks et al. (_Proceedings of the International Conference on Learning Representations (ICLR)_, 2021), [https://github.com/hendrycks/test](https://github.com/hendrycks/test)
- “MTEB: Massive Text Embedding Benchmark,” by Niklas Muenninghoff et al. (_Conference of the European Chapter of the Association for Computational Linguistics_, 2022), [https://arxiv.org/abs/2210.07316](https://arxiv.org/abs/2210.07316)
- “SWE-Bench: Can Language Models Resolve Real-World Github Issues?” by Carlos Jimenez et al. (_Proceedings of the International Conference on Learning Representations (ICLR)_, 2024), [https://www.swebench.com/](https://www.swebench.com/)