Self-Hosting Models

You don't have to use a provider like OpenCode Console. You can be your own provider by self-hosting an LLM. But running an LLM on your machine is a bit harder than self-hosting most "normal" software. The upside is privacy, control, and no per-token billing. The downside is more setup, weaker models (usually), and hardware constraints.

The biggest practical bottleneck is usually VRAM, your GPU's memory. Model weights have to fit somewhere, and bigger models with large context windows increase memory pressure fast.

You'll hear models described as "X billion parameters." That's a (very) rough indicator of model capacity, but it's not a true quality score. For example, here are two open-weight models:

At 16 bits per parameter, 1 billion total parameters requires about 2 GB of memory... so Kimi K2.6's 1 trillion parameters would require about 2 TB just to store the weights, before any additional inference overhead.

So you can see why most self-hosters run smaller models, often quantized to use less memory.

{
  "$schema": "https://opencode.ai/config.json",
  "providers": {
    "lmstudio": {
      "settings": {
        "baseURL": "http://<gpu-host>:1234/v1",
      },
    },
  },
}