Triage where your mail never leaves the machine.

2026-08-26 · guide · self-hosting with local models

The strongest privacy argument for AI email sorting isn't a policy page — it's a network diagram with no outbound arrow. Klorn self-hosted with a local model gives you that: classification, drafts and summaries computed on hardware you own, with message text never crossing your network boundary. Here's the setup, and — because this page follows the house rule — what you actually trade away.

The setup

Klorn talks to any OpenAI-compatible endpoint, which is the interface Ollama, LM Studio and vLLM all speak. On a self-hosted install (docker compose up -d with the prebuilt images), two variables point the judge at your local model:

  1. Run a model locally — with Ollama: ollama pull qwen3:8b.
  2. In your env file:
    OPENAI_COMPAT_BASE_URL=http://host.docker.internal:11434/v1
    OPENAI_COMPAT_MODEL=qwen3:8b

LM Studio and vLLM are the same shape — point the base URL at their server. Full walkthrough: the self-hosting guide.

What you give up: consistency, not safety

Smaller local models score the four features less consistently than frontier ones — expect more mail landing one lane away from where you'd put it. What you do not give up is the safety shape, because it never depended on the model: the deterministic rule still decides, uncertainty still degrades to the visible queue, and the CI-enforced invariant — urgent mail is never silently hidden — holds regardless of what's scoring. Even with no model at all, a keyword floor still classifies; on the committed eval set that path scores 82.1% overall with urgent recall down to 46.2% — degraded mode costs you the interrupt, never the message.

Don't trust your model. Measure it.

The eval set ships in the repo, and the harness treats a local endpoint like any other provider — so instead of guessing whether your model is good enough:

OPENAI_COMPAT_BASE_URL=... OPENAI_COMPAT_MODEL=qwen3:8b pnpm eval:judge

Fifty-six labelled emails, gate floors printed at the end. It's the same harness behind the six-model bake-off — run it against your own hardware and you have a number instead of a vibe.

The part nobody mentions: cost

Local inference makes the AI spend line exactly zero, which also makes Klorn's daily cost caps moot — though they stay armed, because the cap is a runtime guard, not a billing feature. If you later mix in a hosted model for harder mail, the cap is already standing.

Try Klorn free