United States
vLLM
High-throughput GPU inference server for production and multi-user serving. GPU-first; benefits from large VRAM.
GPU acceleration
Yes
CPU-only
No
RAM offload
No
VRAM overhead
2.5 GB
Popular models to run in vLLM
Qwen2.5 32B
Qwen · 32.8B
Qwen2.5-Coder 32B
Qwen · 32.8B
Llama 3.1 8B
Llama · 8B
DeepSeek-R1 Distill Qwen 32B
DeepSeek · 32.8B
Qwen2.5 7B
Qwen · 7.6B
Qwen2.5-Coder 7B
Qwen · 7.6B
Llama 3.3 70B
Llama · 70.6B
Qwen2.5 14B
Qwen · 14.8B
DeepSeek-R1 Distill Qwen 14B
DeepSeek · 14.8B
DeepSeek-R1 Distill Qwen 7B
DeepSeek · 7.6B
DeepSeek-R1 Distill Llama 70B
DeepSeek · 70.6B
Qwen2.5 72B
Qwen · 72.7B