All articles
Self-hosted LLMs
Running language models on your own hardware
Self-hosted LLMs · · 13 min readServing LLMs in production with vLLM: a practical guideDeploy vLLM as an OpenAI-compatible server: Docker setup, the flags that control memory and concurrency, metrics to watch and mistakes to avoid.Self-hosted LLMs · · 13 min readSecuring a self-hosted LLM API: auth, TLS and rate limitsProtect a self-hosted LLM endpoint in layers: private binding, an nginx reverse proxy with TLS, per-client API keys, route allowlists and rate limits.Self-hosted LLMs · · 11 min readHow to run an LLM locally with Ollama: a complete setup guideInstall Ollama on Linux, pull a model, call its API, tune context length and memory, run it as a hardened service and fix the usual problems.Self-hosted LLMs · · 11 min readSelf-host a private ChatGPT with Open WebUI and OllamaDeploy Open WebUI with Ollama using Docker Compose, set up accounts, add documents, put it behind HTTPS and keep it backed up and updated.Self-hosted LLMs · · 12 min readOllama vs llama.cpp vs vLLM: which should you self-host with?A practical comparison of Ollama, llama.cpp and vLLM: how each loads models, uses memory and handles concurrency, with commands to try all three.Self-hosted LLMs · · 12 min readOne API for every model: setting up a LiteLLM gatewaySet up a LiteLLM proxy in front of vLLM, Ollama and hosted models: config file, virtual keys, per-key limits, load balancing and fallbacks.Self-hosted LLMs · · 12 min readHardware for local LLMs: GPU, VRAM and RAM explainedHow to size hardware for local LLMs: estimate memory from parameter count and quantization, and decide between GPU, CPU and Apple Silicon.Self-hosted LLMs · · 12 min readGGUF and quantization explained: choosing Q4, Q5 or Q8What GGUF files are, how to read names like Q4_K_M and IQ3_M, how much memory each level needs, and how to pick or build the right quantization.