vLLM's famous 24x claim, fact-checked.
The catch? Read the baseline
vLLM launched with "up to 24x" the throughput of HuggingFace Transformers. The chart is real, but the baseline is a library, and the peer-reviewed paper claims 2-4x.
- Claim
- 24x faster than HuggingFace?
- Catch
- 24X, BEST CASE
- Verdict
- Quote the baseline
"Up to 24x higher throughput than HuggingFace Transformers." That line launched vLLM on June 20, 2023, and the 24x has followed the project ever since. vLLM deserves the attention, and the numbers below show why it caught on. But the 24x has a specific baseline and a specific setup behind it, and both matter if you're deciding what to run.
The project
vLLM was originally developed in UC Berkeley's Sky Computing Lab and is now a community project. On October 26, 2026, vllm-project/vllm had 95,595 GitHub stars, 1,445 of them that week on GitHub Trending, an Apache-2.0 license, and v0.32.0 (released October 18) as its latest release.
The idea
The launch post's diagnosis is about memory. While a model generates text, it keeps the attention keys and values of every token in GPU memory: the KV cache. The post says the cache can take up to 1.7 GB for a single LLaMA-13B sequence, and its size depends on how long the sequence gets, which nobody knows in advance. By the team's measurement, existing systems wasted 60% to 80% of that memory to fragmentation and over-reservation.
PagedAttention borrows the operating system's trick for RAM: split the cache into fixed-size blocks that don't have to sit next to each other in memory. Waste then only happens in a sequence's last block, under 4% in practice, according to the post. Less waste means more sequences fit in one batch, and bigger batches mean more throughput.
Where 24x comes from
The post benchmarks two setups, with request lengths sampled from the ShareGPT dataset and one output per request:
| Requests per minute | HF Transformers | HF TGI | vLLM |
|---|---|---|---|
| LLaMA-13B, A100-40GB | 6.4 | 61.8 | 154.2 |
| LLaMA-7B, A10G | 8.3 | 50.4 | 112.2 |
154.2 divided by 6.4 is 24.1. That's the headline. In the second setup the same ratio is about 13.5x (the post rounds the range to 14x-24x). Against TGI, which the post itself calls the previous state of the art, the gain was 2.2x to 2.5x, and up to 3.5x when each request asked for three parallel completions.
The catch
The 24x is measured against HuggingFace Transformers, which the post describes as "the most popular LLM library," not against a serving system. The serving system in the same chart, TGI, comes in at 2.2x to 2.5x. The peer-reviewed paper behind vLLM (Kwon et al., SOSP 2023) makes the more careful claim: 2-4x the throughput of the state-of-the-art systems of the time, FasterTransformer and Orca, at the same level of latency.
And today's README doesn't repeat any multiplier. Under "vLLM is fast with" it lists "state-of-the-art serving throughput" and the techniques behind it. That's the honest framing: real throughput depends on your model, your request lengths, your batch sizes, quantization and hardware, and a 2023 benchmark can't tell you how those play out on your setup.
Verdict
A memory trick, not a magic 24x. PagedAttention is genuinely great engineering, and vLLM earned its place. If you're choosing an inference engine, compare it with the serving systems you'd actually consider, on your own model and traffic, and quote the baseline, not the ceiling.
SOURCES
- Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica. "Efficient Memory Management for Large Language Model Serving with PagedAttention." SOSP 2023. arXiv:2309.06180. DOI: 10.48550/arXiv.2309.06180
- vLLM blog, "vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention," June 20, 2023 (vllm.ai/blog/2023-06-20-vllm) — the 24x claim, the 60-80% waste figure, the under-4% figure, the 1.7 GB KV cache figure, and the benchmark charts (HF 6.4, TGI 61.8, vLLM 154.2 req/min on LLaMA-13B, A100-40GB; HF 8.3, TGI 50.4, vLLM 112.2 on LLaMA-7B, A10G).
- vllm-project/vllm README and About page, github.com/vllm-project/vllm, accessed October 26, 2026 ("state-of-the-art serving throughput", no numeric claim).
- GitHub API, October 26, 2026 (95,595 stars, Apache-2.0 license, v0.32.0 released October 18, 2026).
- GitHub Trending, Python, weekly view, October 26, 2026 (1,445 stars that week).
Not affiliated with the vLLM project, UC Berkeley, or Hugging Face. Educational content, not engineering or buying advice.
