LLMHarbor

Self-hosted, local-first, OpenAI compatible

Route LLM traffic through one local control plane.

LLMHarbor exposes one OpenAI-compatible endpoint, then handles provider credentials, browser accounts, limits, fallback, and per-key access policy.

Install in one command Read the docs
Install curl -fsSL https://raw.githubusercontent.com/PLASMA-FR/LLMHarbor/main/install.sh | bash

A small local control plane routes requests to healthy provider keys and OAuth-backed accounts, applies per-client access policy, and falls back without changing your client code.

OpenAI-compatible/v1/chat/completions
Local by default127.0.0.1 control plane
Provider-awarehealth, fallback, limits
Guardrailedper-key route, provider, model policy

Why it exists

Model access is messy. Your apps should not care.

Most AI tools want one base URL and one API key. Real life is different: API providers use different formats, browser subscriptions expose different model inventories, Cloudflare needs account-scoped keys, local servers move around, and free tiers get rate-limited.

LLMHarbor gives you one stable local endpoint while it handles encrypted credentials, OAuth-backed accounts, model registration, per-client access policy, fallback order, health checks, and request analytics behind the scenes.

01

Scoped client keys

Create revocable llmharbor-* keys for apps and agents, each with its own route, provider, and model allow/block policy.

02

Provider key vault

Add Google, Groq, Mistral, OpenRouter, Cloudflare, NVIDIA, HuggingFace, and custom OpenAI-compatible endpoints.

03

OAuth-backed accounts

Connect ChatGPT/OpenAI and Antigravity through loopback OAuth, or Freebuff through its browser device flow, then discover routeable models.

04

Fallback routing

Order models by preference. The router skips unhealthy, disabled, rate-limited, reconnect-required, and incompatible routes.

05

Bulk imports

Drop a text file with one key per line and import an entire batch into the provider target you choose.

06

Readable analytics

See model usage, latency, error reasons, token use, and traffic patterns from a local dashboard.

Account access, without raw token paste

Browser OAuth and local key policy.

Connect supported browser accounts from OAuth. LLMHarbor handles PKCE loopback callbacks or device verification as required, encrypted storage, refresh, provider-reported limits, and inventory cleanup so stale models do not leak into /v1/models. A remote Tailscale/VPN dashboard can securely submit the provider-returned localhost callback URL when that fixed redirect lands on the browser device instead of the server.

Create one local API key per app, device, or agent. Keep the same /v1 base URL, then restrict that key to specific local routes, providers, or models before an upstream call is attempted.

Client key

POST /api/client-keys
{
  "label": "editor agent"
}

OAuth accounts

POST /api/oauth/connect/openai/start
POST /api/oauth/connect/antigravity/start
POST /api/oauth/connect/:provider/callback
GET  /api/oauth/accounts/:id/models

Access policy

GET   /api/client-keys/:id/access-policy
PATCH /api/client-keys/:id/access-policy
{
  "platforms": [{ "platform": "groq", "enabled": false }],
  "models": [{ "modelDbId": 42, "enabled": false }]
}

Provider operations

Bulk API key importer

Open Providers, pick a target from the current provider list, upload a .txt file, and LLMHarbor creates one stored credential per non-empty, non-comment line. The dashboard submits the stable provider platform returned by GET /api/provider-keys/providers; legacy numeric positions are accepted only for older clients.

keys.txt

AIzaSy...one
AIzaSy...two
# comments are ignored
AIzaSy...three

API shape

POST /api/provider-keys/import
{
  "platform": "google",
  "contents": "key-one\nkey-two",
  "labelPrefix": "Google batch"
}

Full docs

Run, configure, and operate LLMHarbor

Quickstart

  1. Install with the curl command above.
  2. Start the app with llmharbor start or run from the repo with npm run dev.
  3. Open the local dashboard and create a client key under Providers. Use Settings to scope that key by route, provider, and model if it is for an agent or shared tool.
  4. Add at least one provider key or connect a supported OAuth account, then run Check all or refresh account inventory.
  5. Point any OpenAI-compatible client at http://127.0.0.1:3001/v1.
curl http://127.0.0.1:3001/v1/chat/completions \
  -H "Authorization: Bearer YOUR_LLMHARBOR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"hello"}]}'

Core concepts

Client API keys
Keys used by your apps to call LLMHarbor. They are separate from provider credentials, can be revoked independently, and can carry route, provider, and model access policies.
Provider keys
Encrypted upstream credentials for Google, Groq, Mistral, OpenRouter, custom endpoints, and other providers.
OAuth accounts
Browser-account connections for OpenAI/ChatGPT, Antigravity, and Freebuff, with provider-appropriate login, automatic model discovery, and reconnect handling.
Models
Provider model rows with limits, context windows, display names, and routing metadata.
Fallback chain
The ordered eligible route list used for model: auto, omitted model fields, and retryable upstream failures.

Providers and custom endpoints

Built-in providers appear first in Providers and Models. Custom OpenAI-compatible endpoints can be added under Providers and then populated from Models. Resolve the stable bulk-import platform from the current UI or GET /api/provider-keys/providers.

  • OpenAI
  • Google
  • OpenAI / ChatGPT OAuth
  • Antigravity OAuth
  • Freebuff
  • Groq
  • Cerebras
  • SambaNova
  • NVIDIA
  • Mistral
  • OpenRouter
  • GitHub Models
  • Cohere
  • Cloudflare
  • Zhipu
  • Ollama
  • Kilo Gateway
  • Pollinations
  • LLM7
  • HuggingFace
  • Custom

OAuth accounts and client policy

The OAuth page starts PKCE loopback login for OpenAI/ChatGPT and Antigravity, or browser device verification for Freebuff, and stores account tokens encrypted. On a remote Tailscale/VPN dashboard, paste the complete returned localhost callback URL into the short-lived fallback field; the server verifies the exact loopback route and consumes the original state and PKCE verifier once. Discovery refreshes account models and provider-reported usage windows; accounts that need reconnect are hidden from routing until fixed. Antigravity includes its native desktop client configuration; the server environment values are overrides.

Local API keys keep a single OpenAI-compatible /v1 base URL while Settings scopes each key independently. Block /v1/models, an upstream provider family, or an individual model row for a selected key without changing other apps.

Routing and fallback

LLMHarbor routes requests through enabled models with eligible credentials. Routing shows priority, health, quota/cooldown state, and why a row is skipped. Retryable failures skip that credential for the current request; only rate-limit and quota failures create a persistent cooldown. A stream can fall back before response data starts, but never after partial output has reached the client. Analytics summaries count final client outcomes, while the recent-error feed retains failed fallback attempts for diagnosis.

API reference

GET /api/pingPublic liveness check.
GET /api/provider-keysList masked provider credentials.
POST /api/provider-keysAdd one provider credential.
GET /api/provider-keys/providersList stable provider targets for bulk import.
POST /api/provider-keys/importImport provider credentials from text lines.
GET /api/client-keysList local client keys.
POST /api/client-keysCreate local client keys for apps and agents.
GET /api/client-keys/:id/access-policyInspect routes, providers, and models scoped to one local key.
PATCH /api/client-keys/:id/access-policyAllow or block local routes, provider endpoints, and model rows.
GET /api/oauth/accountsList connected OAuth-backed accounts.
GET /api/providersList built-in and custom endpoints.
GET /v1/modelsList the models eligible for the authenticated client key.
POST /v1/chat/completionsOpenAI-compatible chat completions proxy.

Verification

GitHub Actions installs from the lockfile, validates Bash, PowerShell, and static-site JavaScript syntax, runs ShellCheck and dashboard lint, executes the server tests, builds the server and dashboard, and audits production dependencies. Release candidates should also be exercised through the real CLI lifecycle and API-backed browser workflows on their target operating systems.

Security model

The dashboard/control-plane API is loopback-only by default. Trusted-network mode permits every client that can reach its private listener; it does not add dashboard login. Provider keys and OAuth tokens are encrypted with AES-256-GCM and a fresh nonce per write, while local llmharbor-* client keys are stored as one-way hashes and shown only when created or regenerated. Listing endpoints return only masked credential metadata. Custom endpoints allow local/private HTTP(S) targets but block credential-bearing URLs, metadata/link-local destinations, unsafe DNS resolution, and redirects. Per-client access policies and CORS are guardrails, not substitutes for a private network boundary. Do not expose the control plane directly to the public internet; keep it on loopback or a private VPN address.

Operations

  • Run on a supported Node.js LTS line: ^22.12.0 or ^24.0.0.
  • Use HOST=127.0.0.1 for local-only operation.
  • Use PORT=3001 or another valid port between 1 and 65535.
  • Run npm run check and npm audit before deploying changes.
  • Back up the entire SQLite data directory and .env. If no environment key is configured, also preserve the database .key sidecar; streamed database backups intentionally exclude it. Imports are validated and staged, require a restart to activate, and must use the matching credential-encryption key.