Local AI runtimes

The tools that actually run models on your machine. Each one has different hardware implications — pick one to see what it needs.

Local LLM runtimes

The most popular way to run local LLMs. One-command model pulls, GPU offload, and CPU/GPU split for models larger than your VRAM.

Desktop app for discovering, downloading and running GGUF models with a friendly UI and OpenAI-compatible server.

The inference engine under most local LLM tools. Maximum control, GGUF quantization, CPU/GPU split offload.

Self-hosted web interface that pairs with Ollama or any OpenAI-compatible backend. Adds RAG, chat history and multi-user.

High-throughput GPU inference server for production and multi-user serving. GPU-first; benefits from large VRAM.

Node-based workflow tool for image and video diffusion models (Stable Diffusion, FLUX). GPU-first with model offload options.

The classic AUTOMATIC1111 web UI for Stable Diffusion image generation.