United States
llama.cpp
The inference engine under most local LLM tools. Maximum control, GGUF quantization, CPU/GPU split offload.
GPU acceleration
Yes
CPU-only
Possible
RAM offload
Yes
VRAM overhead
0.8 GB
Popular models to run in llama.cpp
Qwen2.5 32B
Qwen · 32.8B
Qwen2.5-Coder 32B
Qwen · 32.8B
Llama 3.1 8B
Llama · 8B
DeepSeek-R1 Distill Qwen 32B
DeepSeek · 32.8B
Qwen2.5 7B
Qwen · 7.6B
Qwen2.5-Coder 7B
Qwen · 7.6B
Llama 3.3 70B
Llama · 70.6B
Qwen2.5 14B
Qwen · 14.8B
DeepSeek-R1 Distill Qwen 14B
DeepSeek · 14.8B
DeepSeek-R1 Distill Qwen 7B
DeepSeek · 7.6B
DeepSeek-R1 Distill Llama 70B
DeepSeek · 70.6B
Qwen2.5 72B
Qwen · 72.7B