vLLM

vLLM Project

The open-source inference engine behind much of modern LLM serving: PagedAttention delivers state-of-the-art throughput with an OpenAI-compatible API and broad model support on your own GPUs.

Product preview

Overview

vLLM is the quiet default of open LLM serving. Its PagedAttention technique — treating KV-cache memory the way an operating system treats RAM — delivered step-change throughput gains when it appeared and remains the benchmark other engines chase. Spin up an OpenAI-compatible server on your own hardware, point your existing OpenAI SDK at it, and the same application code now serves Llama, Qwen, Mistral or any of hundreds of supported architectures.

It is infrastructure, not a product: expect to bring your own GPUs, capacity planning and ops. Continuous batching, tensor parallelism, quantization support and an ecosystem that includes most cloud providers shipping “vLLM-compatible” endpoints have nonetheless made it the lowest-regret choice for self-hosting — and the reason token prices across the industry keep falling.

Pros / Cons

Pros

Best-in-class throughput and memory efficiency
Drop-in OpenAI compatibility for existing code
Huge model and hardware coverage
Free, open source and community-vetted

Cons

You own the GPUs, scaling and ops burden
No GUI — API-first infrastructure
Peak performance needs tuning per model

Who it's for

Self-hosting open models for privacy or cost

High-volume batch inference on owned GPUs

Backing internal LLM gateways

Research serving with reproducible stacks

Bottom line

If you are serving open models at any real scale, the question is not whether to use vLLM — it is how to tune it.

Key Features

PagedAttention throughput

OpenAI-compatible server

Continuous batching

Broad model coverage

Multi-GPU & quantization

Compare vLLM with alternatives

vLLM vs Hugging Face vLLM vs Replicate vLLM vs Ollama 4 products · AI Platforms

Frequently asked questions

What is vLLM?

vLLM is a AI Platforms tool developed by vLLM Project. The company was founded in 2023 and is headquartered in Open source (community). The open-source inference engine behind much of modern LLM serving: PagedAttention delivers state-of-the-art throughput with an OpenAI-compatible API and broad model support on your own GPUs.