Scott.Idem/Cortex-Inara
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: model registry V2 — provider-aware schema with multi-account support

Browse Source 
Adds a providers section to the per-user model registry for Anthropic and
Google as first-class providers alongside local hosts. Google accounts
(API keys) are now stored as a list so multiple Google accounts can coexist.

Changes:
- model_registry.py: V2 schema, auto migration V1→V2 (pulls gemini_api_key
  from auth.json into providers.google.accounts), _resolve_model() merges
  account API key for gemini_api type models
- routers/orchestrator.py: uses model-resolved api_key when orchestrator
  role resolves to a gemini_api model with account_id
- ANTHROPIC_CATALOG and GOOGLE_CATALOG constants for model picker (Phase 2)
- New functions: get_google_api_key(), save/remove_google_account(), get_catalog()
- Documentation: ARCH__BACKENDS.md updated to V2 schema, DESIGN doc added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Scott Idem
2026-04-27 20:21:04 -04:00
parent 27ca7c7efd
commit 45c95d20ba
6 changed files with 715 additions and 134 deletions
Download Patch File Download Diff File Expand all files Collapse all files

361
cortex/model_registry.py
View File

@@ -1,57 +1,72 @@
"""
Per-user unified model registry.
Per-user unified model registry — V2.
Stored in: home/{user}/model_registry.json
Schema:
V2 Schema:
{
"version": 1,
"hosts": [{"id", "label", "api_url", "api_key",
"host_type": "openwebui" | "openai"}, ...],
#
# host_type controls the API path layout:
# "openwebui" (default) — Open WebUI / Ollama:
# chat: POST {url}/api/chat/completions
# models: GET {url}/api/models
# "openai" — OpenRouter, LiteLLM, Anthropic-compatible, etc.:
# chat: POST {url}/chat/completions
# models: GET {url}/models
# Set api_url to the base path that ends just before /chat/completions,
# e.g. https://openrouter.ai/api/v1 for OpenRouter.
"version": 2,
# Per-provider accounts / credentials (user-configured)
"providers": {
"anthropic": {
"credentials": [
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
]
},
"google": {
"accounts": [
{"id": "<hex>", "label": "My Google account", "api_key": "AIza..."}
]
}
},
# Local OpenAI-compatible hosts (unchanged from V1)
"hosts": [{"id", "label", "api_url", "api_key", "host_type"}, ...],
# User-registered model entries (all providers)
"models": [
{
"id": str, # unique within this registry
"type": str, # "local_openai" | "claude_cli" | "gemini_cli" | "gemini_api"
"label": str, # human-readable display name
"model_name": str, # model identifier sent to the API
"host_id": str | null, # only for local_openai — references hosts[].id
"context_k": int, # context window in thousands of tokens (informational)
"tags": [str], # user-defined capability tags
"id": str, # unique within this registry
"type": str, # see TYPES below
"label": str, # human-readable
"model_name": str, # identifier sent to the API / CLI
"provider": str | null, # "anthropic" | "google" | "local" | null
"host_id": str | null, # local_openai only — references hosts[].id
"credential_id":str | null, # claude_cli only — references providers.anthropic.credentials
"account_id": str | null, # gemini_api only — references providers.google.accounts
"context_k": int, # context window in k tokens (informational)
"tags": [str], # user-defined capability tags
},
],
# Role assignments — any model (any provider) can fill any role
"roles": {
"<role>": {
"primary": "<model_id>" | null,
"backup_1": "<model_id>" | null,
"backup_2": "<model_id>" | null,
"backup_3": "<model_id>" | null,
...
"backup_4": "<model_id>" | null,
},
},
}
Built-in model IDs (always resolvable, no registry entry required):
"claude_cli" — Claude CLI subprocess (~/.claude/.credentials.json)
"gemini_cli" — Gemini CLI subprocess
"gemini_api" — Gemini API (google-genai SDK; used by orchestrator engine, not llm_client)
Types:
"claude_cli" — Claude CLI subprocess (~/.claude/.credentials.json)
"gemini_cli" — Gemini CLI subprocess
"gemini_api" — Gemini API (google-genai SDK); account_id → api_key from providers.google
"local_openai" — OpenAI-compatible endpoint; host_id → api_url/api_key from hosts[]
Standard roles are defined by settings.defined_roles (default: chat,orchestrator,distill,coder,research).
Additional custom roles can be added freely to roles{}.
Built-in model IDs (always resolvable without a registry entry):
"claude_cli" — resolves to the default Claude CLI model
"gemini_cli" — resolves to Gemini CLI
"gemini_api" — resolves to Gemini API using GEMINI_API_KEY from .env
Resolution for get_model_for_role(username, role):
1. User registry: roles[role].primary → backup_1 → backup_2 → backup_3 → backup_4
2. .env default: ROLE_<ROLE>=<builtin_id> (e.g. ROLE_CHAT=claude_cli)
Role resolution for get_model_for_role(username, role):
1. User registry: roles[role].primary → backup_1 → ... → backup_4
2. .env default: ROLE_<ROLE>=<builtin_id>
3. Hardcoded last-resort defaults per role
4. claude_cli (absolute fallback)
"""
import json
@@ -63,11 +78,28 @@ from config import settings
logger = logging.getLogger(__name__)
# ── Provider model catalogs ───────────────────────────────────────────────────
# Server-side defaults. Update here when providers release new models.
# Users can add entries via the settings UI (Phase 2).
ANTHROPIC_CATALOG: list[dict] = [
{"id": "claude-opus-4-7", "label": "Claude Opus 4.7", "context_k": 200},
{"id": "claude-sonnet-4-6", "label": "Claude Sonnet 4.6", "context_k": 200},
{"id": "claude-haiku-4-5-20251001", "label": "Claude Haiku 4.5", "context_k": 200},
]
GOOGLE_CATALOG: list[dict] = [
{"id": "gemini-2.5-pro", "label": "Gemini 2.5 Pro", "context_k": 1000},
{"id": "gemini-2.5-flash", "label": "Gemini 2.5 Flash", "context_k": 1000},
{"id": "gemini-2.0-flash", "label": "Gemini 2.0 Flash", "context_k": 1000},
{"id": "gemini-1.5-pro", "label": "Gemini 1.5 Pro", "context_k": 2000},
]
# ── Built-in model definitions ────────────────────────────────────────────────
# These IDs are always resolvable without a registry entry.
def _builtins() -> dict[str, dict]:
"""Return built-in model definitions (lazy so settings are resolved at call time)."""
return {
"claude_cli": {
"id": "claude_cli",
@@ -96,7 +128,6 @@ def _builtins() -> dict[str, dict]:
}
# Hardcoded last-resort defaults per role (used only if .env is also unset)
_ROLE_LAST_RESORT: dict[str, str] = {
"chat": "claude_cli",
"orchestrator": "gemini_api",
@@ -118,14 +149,40 @@ def _local_llm_path(username: str) -> Path:
return settings.home_root() / username / "local_llm.json"
def _auth_path(username: str) -> Path:
return settings.home_root() / username / "auth.json"
def _empty() -> dict:
return {"version": 1, "hosts": [], "models": [], "roles": {}}
return {
"version": 2,
"providers": _default_providers(),
"hosts": [],
"models": [],
"roles": {},
}
def _default_providers() -> dict:
return {
"anthropic": {
"credentials": [
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
]
},
"google": {
"accounts": []
},
}
def _normalize(data: dict) -> dict:
"""Back-fill any missing fields introduced by schema additions."""
"""Back-fill missing fields introduced by schema additions."""
for h in data.get("hosts", []):
h.setdefault("host_type", "openwebui")
data.setdefault("providers", _default_providers())
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
data["providers"].setdefault("google", {"accounts": []})
return data
@@ -135,12 +192,15 @@ def _load(username: str) -> dict:
try:
data = json.loads(path.read_text())
if isinstance(data, dict) and "version" in data:
if data["version"] == 1:
data = _migrate_v1_to_v2(username, data)
_save(username, data)
return _normalize(data)
except (json.JSONDecodeError, OSError):
logger.warning("model_registry.json for %s is unreadable — starting fresh", username)
return _empty()
# No registry yet — try migrating from local_llm.json
# No registry — try migrating from local_llm.json
legacy = _local_llm_path(username)
if legacy.exists():
data = _migrate_from_local_llm(username, legacy)
@@ -157,8 +217,45 @@ def _save(username: str, data: dict) -> None:
# ── Migration ─────────────────────────────────────────────────────────────────
def _migrate_v1_to_v2(username: str, data: dict) -> dict:
"""
Upgrade a V1 registry to V2.
Changes:
- Adds providers section with default structure
- Migrates gemini_api_key from auth.json → first Google account entry
- Sets version to 2
"""
logger.info("Migrating model_registry.json V1 → V2 for %s", username)
data["version"] = 2
if "providers" not in data:
data["providers"] = _default_providers()
else:
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
data["providers"].setdefault("google", {"accounts": []})
# Pull existing Gemini key from auth.json (stored there in V1) → first account entry
accounts = data["providers"]["google"]["accounts"]
if not accounts:
try:
auth = json.loads(_auth_path(username).read_text())
existing_key = auth.get("gemini_api_key")
if existing_key:
accounts.append({
"id": secrets.token_hex(4),
"label": "Gemini API Key",
"api_key": existing_key,
})
logger.info("Migrated gemini_api_key from auth.json → providers.google.accounts for %s", username)
except (OSError, json.JSONDecodeError):
pass
return data
def _migrate_from_local_llm(username: str, path: Path) -> dict:
"""Convert local_llm.json (hosts/models/active_model_id) → model_registry format."""
"""Convert local_llm.json → V2 model_registry format."""
try:
old = json.loads(path.read_text())
except Exception:
@@ -190,30 +287,25 @@ def _migrate_from_local_llm(username: str, path: Path) -> dict:
"type": "local_openai",
"label": m.get("label") or m.get("model_name", ""),
"model_name": m.get("model_name", ""),
"provider": "local",
"host_id": m.get("host_id"),
"context_k": 0,
"tags": [],
})
# Build initial role assignments
active_id = old.get("active_model_id")
distill_type = settings.distill_backend_mid or None
roles: dict[str, dict] = {}
if active_id and any(m["id"] == active_id for m in data["models"]):
roles["chat"] = {"primary": active_id}
data["roles"]["chat"] = {"primary": active_id}
if distill_type == "local" and active_id:
roles["distill"] = {"primary": active_id}
data["roles"] = roles
# Migrate Gemini key from auth.json
data = _migrate_v1_to_v2(username, {"version": 1, **data})
return data
# ── Model resolution ──────────────────────────────────────────────────────────
def _resolve_model(registry: dict, model_id: str) -> dict | None:
"""Resolve a model_id to its full config dict, or None if not found."""
"""Resolve a model_id to its full config dict (credentials merged in), or None."""
builtins = _builtins()
# Built-in IDs take priority over user-defined entries with the same ID
@@ -224,7 +316,9 @@ def _resolve_model(registry: dict, model_id: str) -> dict | None:
if not model:
return None
if model.get("type") == "local_openai":
model_type = model.get("type")
if model_type == "local_openai":
host_id = model.get("host_id")
host = next((h for h in registry.get("hosts", []) if h["id"] == host_id), None)
if not host:
@@ -237,6 +331,19 @@ def _resolve_model(registry: dict, model_id: str) -> dict | None:
"host_type": host.get("host_type", "openwebui"),
}
if model_type == "gemini_api":
account_id = model.get("account_id")
if account_id:
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
account = next((a for a in accounts if a["id"] == account_id), None)
if account:
return {**model, "api_key": account.get("api_key", "")}
logger.warning("model %s references missing account_id %s", model_id, account_id)
return dict(model)
if model_type == "claude_cli":
return dict(model)
return dict(model)
@@ -277,7 +384,6 @@ def get_best_local_model(username: str, role: str = "chat") -> dict | None:
"""
Return the best available local_openai model for the given role.
Used when the user explicitly selects "local" backend in the UI.
Tries the role's priority chain first, then any configured local model.
"""
registry = _load(username)
role_cfg = registry.get("roles", {}).get(role, {})
@@ -290,7 +396,6 @@ def get_best_local_model(username: str, role: str = "chat") -> dict | None:
if resolved and resolved.get("type") == "local_openai":
return resolved
# Fall back to first configured local model
for model in registry.get("models", []):
if model.get("type") == "local_openai":
resolved = _resolve_model(registry, model["id"])
@@ -300,15 +405,38 @@ def get_best_local_model(username: str, role: str = "chat") -> dict | None:
return None
# ── Read API (for UI and callers) ─────────────────────────────────────────────
def get_google_api_key(username: str, account_id: str | None = None) -> str | None:
"""
Return the best available Gemini API key for the user.
If account_id is specified, returns that account's key (or None if not found).
Otherwise returns the first configured account key, falling back to the
server-level GEMINI_API_KEY from .env.
"""
registry = _load(username)
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
if account_id:
account = next((a for a in accounts if a["id"] == account_id), None)
return account.get("api_key") if account else None
# First configured account
if accounts:
return accounts[0].get("api_key") or None
# Fall back to .env server key
return settings.gemini_api_key or None
# ── Read API ──────────────────────────────────────────────────────────────────
def get_registry(username: str) -> dict:
"""Return the full registry (with built-in models injected for display)."""
"""Return the full registry (providers + hosts + models + roles)."""
return _load(username)
def get_all_models(username: str) -> list[dict]:
"""Return all user-defined models (resolved — hosts merged in)."""
"""Return all user-defined models (resolved — credentials/hosts merged in)."""
registry = _load(username)
out = []
for m in registry.get("models", []):
@@ -319,24 +447,94 @@ def get_all_models(username: str) -> list[dict]:
def get_defined_roles(username: str) -> dict[str, dict]:
"""Return the roles section of the registry, filling gaps with empty dicts."""
"""Return the roles section, filling gaps with empty dicts."""
registry = _load(username)
roles = registry.get("roles", {})
result = {}
for role in settings.get_defined_roles():
result[role] = roles.get(role, {})
return result
return {role: roles.get(role, {}) for role in settings.get_defined_roles()}
# ── Write API (CRUD) ──────────────────────────────────────────────────────────
def get_google_accounts(username: str) -> list[dict]:
"""Return Google account entries (api_key masked for display)."""
registry = _load(username)
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
return [
{
"id": a["id"],
"label": a.get("label", ""),
"hint": (a.get("api_key") or "")[:8] + "…" if a.get("api_key") else "",
}
for a in accounts
]
def get_catalog(provider: str, username: str | None = None) -> list[dict]:
"""
Return the model catalog for a provider.
For now returns server defaults. Phase 2 will merge in per-user additions.
"""
if provider == "anthropic":
return list(ANTHROPIC_CATALOG)
if provider == "google":
return list(GOOGLE_CATALOG)
return []
# ── Write API — Google accounts ───────────────────────────────────────────────
def save_google_account(username: str, account_id: str | None,
label: str, api_key: str) -> str:
"""Create or update a Google account entry. Returns the account ID."""
data = _load(username)
accounts = data["providers"]["google"]["accounts"]
if account_id:
for a in accounts:
if a["id"] == account_id:
a["label"] = label.strip()
if api_key.strip():
a["api_key"] = api_key.strip()
_save(username, data)
return account_id
account_id = secrets.token_hex(4)
accounts.append({
"id": account_id,
"label": label.strip(),
"api_key": api_key.strip(),
})
_save(username, data)
return account_id
def remove_google_account(username: str, account_id: str) -> bool:
"""Remove a Google account. Clears any model entries that reference it."""
data = _load(username)
accounts = data["providers"]["google"]["accounts"]
before = len(accounts)
data["providers"]["google"]["accounts"] = [a for a in accounts if a["id"] != account_id]
# Clear role assignments for models that referenced this account
removed_model_ids = {
m["id"] for m in data.get("models", [])
if m.get("account_id") == account_id
}
data["models"] = [m for m in data.get("models", []) if m["id"] not in removed_model_ids]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["providers"]["google"]["accounts"]) < before
# ── Write API — Hosts ─────────────────────────────────────────────────────────
def save_host(username: str, host_id: str | None,
label: str, api_url: str, api_key: str,
host_type: str = "openwebui") -> str:
"""Create or update a host. Returns the host ID.
host_type: "openwebui" (default) or "openai" (OpenRouter, LiteLLM, etc.)
"""
"""Create or update a host. Returns the host ID."""
data = _load(username)
host_type = host_type if host_type in ("openwebui", "openai") else "openwebui"
@@ -350,7 +548,7 @@ def save_host(username: str, host_id: str | None,
h["api_key"] = api_key.strip()
_save(username, data)
return host_id
host_id = None # not found — create new
host_id = None
host_id = secrets.token_hex(4)
data["hosts"].append({
@@ -365,25 +563,26 @@ def save_host(username: str, host_id: str | None,
def remove_host(username: str, host_id: str) -> bool:
"""Remove a host and all models that reference it. Returns True if found."""
"""Remove a host and all models that reference it."""
data = _load(username)
before = len(data["hosts"])
data["hosts"] = [h for h in data["hosts"] if h["id"] != host_id]
data["models"] = [m for m in data["models"] if m.get("host_id") != host_id]
# Clear any role assignments that pointed to removed models
removed_ids = {m["id"] for m in data["models"] if m.get("host_id") == host_id}
removed_model_ids = {m["id"] for m in data["models"] if m.get("host_id") == host_id}
data["hosts"] = [h for h in data["hosts"] if h["id"] != host_id]
data["models"] = [m for m in data["models"] if m.get("host_id") != host_id]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_ids:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["hosts"]) < before
# ── Write API — Models ────────────────────────────────────────────────────────
def save_model(username: str, model_id: str | None, host_id: str,
label: str, model_name: str, context_k: int = 0,
tags: list[str] | None = None) -> str:
"""Create or update a model entry. Returns the model ID."""
"""Create or update a local_openai model entry. Returns the model ID."""
data = _load(username)
tags = tags or []
@@ -405,6 +604,7 @@ def save_model(username: str, model_id: str | None, host_id: str,
"type": "local_openai",
"label": label.strip() or model_name.strip(),
"model_name": model_name.strip(),
"provider": "local",
"host_id": host_id,
"context_k": context_k,
"tags": tags,
@@ -418,12 +618,10 @@ def remove_model(username: str, model_id: str) -> bool:
data = _load(username)
before = len(data["models"])
data["models"] = [m for m in data["models"] if m["id"] != model_id]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) == model_id:
role_cfg[key] = None
_save(username, data)
return len(data["models"]) < before
@@ -434,8 +632,7 @@ def set_role(username: str, role: str, priority: str, model_id: str | None) -> b
priority must be one of: primary, backup_1, backup_2, backup_3, backup_4
model_id None clears the slot.
model_id "claude_cli" / "gemini_cli" / "gemini_api" are valid built-in IDs.
Returns False if model_id is set but not found.
Built-in IDs (claude_cli, gemini_cli, gemini_api) are always valid.
"""
if priority not in PRIORITY_KEYS:
return False
@@ -455,10 +652,14 @@ def set_role(username: str, role: str, priority: str, model_id: str | None) -> b
return True
def fetch_models_from_host(api_url: str, api_key: str) -> list[str]:
# ── Utility ───────────────────────────────────────────────────────────────────
def fetch_models_from_host(api_url: str, api_key: str,
host_type: str = "openwebui") -> list[str]:
"""Synchronously fetch the model list from an OpenAI-compatible host."""
import httpx
url = api_url.rstrip("/") + "/api/models"
path = "/api/models" if host_type == "openwebui" else "/models"
url = api_url.rstrip("/") + path
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
resp = httpx.get(url, headers=headers, timeout=10)
resp.raise_for_status()

8
cortex/routers/orchestrator.py
View File

@@ -171,12 +171,18 @@ async def _run_job(job_id: str, req: OrchestrateRequest, user: str) -> None:
respond_with_final=req.respond_with_claude,
)
else:
# Use the API key embedded in the resolved model config (V2 registry with
# account_id), then fall back to the per-user key from auth.json, then .env.
gemini_key = (
(orch_model.get("api_key") if orch_model else None)
or get_user_gemini_key(user)
)
result = await orchestrator_engine.run(
task=req.task,
system_prompt=system_prompt,
session_messages=session_messages,
respond_with_claude=req.respond_with_claude,
gemini_api_key=get_user_gemini_key(user),
gemini_api_key=gemini_key,
)
# Save the turn to the session store so it survives a page refresh