feat: SSH dev routing, model registry UX, chat input toolbar, doc sync

Backend / infrastructure: - cortex/tools/_projects.py (new): shared project alias registry with ssh_host for workstation projects (aether_api, aether_frontend, aether_container) - cortex/tools/git.py: all git tools route to workstation via SSH when ssh_host set - cortex/tools/aider.py: aider_run SSH-routes to workstation using bash -l -c - cortex/routers/local_llm.py: POST /api/models/{id}/edit AJAX endpoint — save model edits without page reload or tab reset; returns JSON {ok, label, model_name} - cortex/llm_client.py: remove Gemini CLI and Claude CLI backends; clean up fallback chain and process group tracking (continuation of Gemini CLI removal) - cortex/routers/auth.py: strip Claude/Gemini CLI auth status checks (CLI removed) - cortex/routers/chat.py: remove legacy claude/gemini backend fields - cortex/config.py: clean up CLI-related settings - cortex/main.py: remove CLI lifecycle hooks UI: - cortex/static/local_llm.html: model edit forms now save via fetch() + toast; stay on Models tab; update row header label in place on success - cortex/static/index.html: restructure input area to column layout — textarea above, compact toolbar below (Chat/Tools/Attach + Send); fixes dead space at M/L/XL sizes; context panel "Role" → "Model" section label - cortex/static/style.css: column input-area layout; #input-toolbar; flex:1 → width:100% on textarea (fixes scrollHeight in column flex context); compact send/stop button padding - cortex/static/app.js: add XL (720px) to height cycle; default M (240px) Docs: - cortex/static/HELP.md: S/M/L → S/M/L/XL; add Rebuild to distill table; fix "Role selector" references (no such UI); fix "your active role" → Chat role; fix ⚡ toggle description; Model Registry section cleanup - documentation/ARCH__BACKENDS.md: reflect CLI removal, current backend state Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 22:14:07 -04:00
parent 85223326b0
commit b144d8385f
15 changed files with 378 additions and 586 deletions
--- a/documentation/ARCH__BACKENDS.md
+++ b/documentation/ARCH__BACKENDS.md
@@ -1,20 +1,21 @@
 # Architecture: LLM Backends

 > How Cortex selects and talks to AI models.
-> Last updated: 2026-05-06
+> Last updated: 2026-06-18

 ---

 ## Providers

-Cortex supports four model types, each dispatched differently:
+Cortex supports two model types, each dispatched differently:

 | Type | Auth | Use |
 |---|---|---|
-| `claude_cli` | OAuth token from `~/.claude/.credentials.json` | Chat, persona responses |
-| `gemini_cli` | Gemini CLI credentials | Chat fallback / explicit selection |
-| `gemini_api` | API key from registry account or `.env` | Orchestrator tool loop |
-| `local_openai` | API key per host in model registry | Open WebUI, Ollama, OpenRouter, LiteLLM, etc. |
+| `local_openai` | API key per host in model registry | Open WebUI, Ollama, OpenRouter, LiteLLM, any OpenAI-compatible endpoint |
+| `anthropic_api` | API key in model registry (Anthropic cloud provider) | Claude models via Anthropic SDK |
+
+The Gemini API (`gemini_api`) is a third type used exclusively by the orchestrator engine —
+it is not dispatched through `llm_client.py` and is not available for chat/distill roles.

 ---

@@ -22,40 +23,36 @@ Cortex supports four model types, each dispatched differently:

 ### Default: Role-Based Routing (Auto)

-When no explicit backend is selected, Cortex routes to the model configured for the
-request's **role** in the user's model registry. Roles: `chat`, `orchestrator`, `distill`,
-`coder`, `research` (extensible via `DEFINED_ROLES` in `.env`).
+All routing goes through the user's model registry. When a request arrives, `complete()` in
+`llm_client.py` resolves the model for the given role:

-Resolution order for a role:
-1. User registry: `roles[role].primary → backup_1 → backup_2 → backup_3 → backup_4`
-2. `.env` role default: `ROLE_CHAT=claude_cli`, `ROLE_DISTILL=claude_cli`, etc.
-3. Hardcoded last-resort: `chat/distill/coder → claude_cli`, `orchestrator/research → gemini_api`
-
-### Explicit Override
-
-The **Role** toggle in the Context & Memory panel cycles through configured role slots for the `chat` role: **Primary → Backup 1 → Backup 2 → auto**.
-
- Each slot shows the configured model label
- `auto` uses the Primary without forcing a specific backend type
- The ⚡ Tools toggle is independent — it routes to the `orchestrator` role regardless of the chat role selection
-
-**Fallback chain** (automatic, only when no explicit registry entry exists):
 ```
-claude  → gemini
-gemini  → claude
-local   → claude
+slot specified  →  resolve that exact slot (primary / backup_1 / backup_2)
+no slot         →  get_model_for_role(username, role)
+no registry entry  →  RuntimeError: "No model configured for role '...'"
 ```
-When a model is explicitly configured in the registry, errors surface immediately — no silent fallback.

-Each response shows a model tag (bottom-right of the message bubble) with the model label and host.
+Roles: `chat`, `orchestrator`, `distill`, `janitor`, `coder`, `research` (extensible via
+`DEFINED_ROLES` in `.env`).
+
+There is no implicit fallback to a built-in model. If no model is configured for a role,
+the request fails with a clear error directing the user to `/settings/models`.
+
+### Explicit Slot Selection
+
+The **Role** toggle in the Context & Memory panel cycles through configured role slots:
+**Primary → Backup 1 → auto**. Each slot resolves the configured model for that position.
+
+When a model is explicitly configured (via slot or registry entry), errors surface
+immediately — no silent fallback to another backend.

 ---

-## Model Registry — V2 Schema
+## Model Registry Schema

 Per-user configuration stored in `home/{user}/model_registry.json`.

-Managed at **Settings → Models** (`/settings/models`). Full provider UI coming in Phase 2.
+Managed at **Settings → Models** (`/settings/models`).

 ```json
 {
@@ -64,7 +61,7 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
  "providers": {
    "anthropic": {
      "credentials": [
-        {"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
+        {"id": "key1", "label": "My Anthropic Key", "type": "api_key", "api_key": "sk-ant-..."}
      ]
    },
    "google": {
@@ -77,6 +74,13 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
  "hosts": [
    {
      "id": "abc123",
+      "label": "OpenRouter",
+      "api_url": "https://openrouter.ai/api/v1",
+      "api_key": "sk-or-...",
+      "host_type": "openai"
+    },
+    {
+      "id": "def456",
      "label": "Gaming Laptop",
      "api_url": "http://192.168.x.x:3000",
      "api_key": "",
@@ -87,23 +91,22 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
  "models": [
    {
      "id": "m1",
-      "type": "claude_cli",
-      "label": "Sonnet 4.6 (CLI)",
-      "model_name": "claude-sonnet-4-6",
-      "provider": "anthropic",
-      "credential_id": "cli",
+      "type": "local_openai",
+      "label": "Claude Sonnet 4.6 (OpenRouter)",
+      "model_name": "anthropic/claude-sonnet-4-6",
+      "host_id": "abc123",
      "context_k": 200,
      "tags": ["chat", "persona"]
    },
    {
      "id": "m2",
-      "type": "gemini_api",
-      "label": "Gemini 2.5 Flash (OSIT)",
-      "model_name": "gemini-2.5-flash",
-      "provider": "google",
-      "account_id": "a1b2",
-      "context_k": 1000,
-      "tags": ["orchestrator", "research"]
+      "type": "anthropic_api",
+      "label": "Claude Sonnet 4.6 (Direct)",
+      "model_name": "claude-sonnet-4-6",
+      "provider": "anthropic",
+      "credential_id": "key1",
+      "context_k": 200,
+      "tags": ["chat"]
    },
    {
      "id": "m3",
@@ -111,7 +114,7 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
      "label": "Gemma 4 E4B",
      "model_name": "gemma4:e4b",
      "provider": "local",
-      "host_id": "abc123",
+      "host_id": "def456",
      "context_k": 72,
      "max_rounds": 5,
      "tools": true,
@@ -120,8 +123,8 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
  ],

  "roles": {
-    "chat":         {"primary": "m1", "backup_1": "m2", "backup_2": "m3"},
-    "orchestrator": {"primary": "m2", "backup_1": "m3"},
+    "chat":         {"primary": "m1", "backup_1": "m2"},
+    "orchestrator": {"primary": "m2"},
    "distill":      {"primary": "m1"}
  }
 }
@@ -145,52 +148,9 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
 Set `api_url` to the base path before `/chat/completions`:
 - OpenRouter: `https://openrouter.ai/api/v1`

-### Built-in model IDs
-
-Always resolvable without a user-created registry entry. Used as role defaults.
-
-| ID | Type | Notes |
-|---|---|---|
-| `claude_cli` | `claude_cli` | Model from `DEFAULT_MODEL` in `.env` |
-| `gemini_cli` | `gemini_cli` | Gemini CLI subprocess |
-| `gemini_api` | `gemini_api` | Model from `ORCHESTRATOR_MODEL` in `.env`; key from `GEMINI_API_KEY` |
-
-### V1 → V2 migration
-
-Automatic on first load. Changes:
- Adds `providers` section (Anthropic CLI credential + empty Google accounts)
- Migrates `gemini_api_key` from `auth.json` → `providers.google.accounts[0]`
- All existing hosts, models, and role assignments are preserved
-
 ---

-## Claude Backend (`_claude()`)
-
-Runs `claude --print --no-session-persistence --output-format text` as a subprocess.
-
- System prompt passed via `--system-prompt`
- Conversation history formatted as `<conversation>` block
- Token read live from `~/.claude/.credentials.json` on every call — never uses the
-  env var, which goes stale after `claude auth login`
- Model override via `--model` flag when `model_name` is set in the registry entry
-
-Timeout: `TIMEOUT_CLAUDE=60` seconds (`.env`)
-
---
-
-## Gemini CLI Backend (`_gemini()`)
-
-Runs `gemini --output-format text --extensions "" -p <prompt>` as a subprocess.
-
- `--extensions ""` disables all MCP extensions — prevents child processes keeping pipes open
- `start_new_session=True` puts the process in its own group for clean `os.killpg` on timeout
- Output is cleaned to strip CLI noise (loading messages, retry notices, quota warnings)
-
-Timeout: `TIMEOUT_GEMINI=120` seconds (`.env`)
-
---
-
-## Local Backend (`_local()`)
+## Local/OpenAI-Compatible Backend (`_local()`)

 HTTP POST to an OpenAI-compatible endpoint. Model config is resolved via the model registry.

@@ -199,13 +159,36 @@ HTTP POST to an OpenAI-compatible endpoint. Model config is resolved via the mod
 # host_type "openai":     POST {api_url}/chat/completions
 ```

+System prompt is sent as the first `{"role": "system", "content": "..."}` message.
+Image attachments are injected into the last user message as `image_url` content blocks.
+Token usage is recorded when returned by the endpoint.
+
+Streaming variant: `_local_streaming()` — SSE line-by-line, yields tokens via `token_sink`.
+
 Timeout: `TIMEOUT_LOCAL=300` seconds (`.env`) — local models may need to load from disk.

 ---

-## Gemini API (Orchestrator)
+## Anthropic API Backend (`_anthropic_api()`)

-Used by `orchestrator_engine.py` for the ReAct tool loop. Not used for general chat.
+Direct call to the Anthropic Messages API via the `anthropic` Python SDK.
+
+System prompt passed as top-level `system` field. Messages stripped to `role`/`content` only.
+Token usage is always recorded from `resp.usage`.
+
+Streaming variant: `_anthropic_api_streaming()` — uses `client.messages.stream()`, yields
+tokens via `token_sink`.
+
+API key comes from the model registry: `providers.anthropic.credentials[n].api_key`.
+
+Timeout: governed by httpx defaults and the Anthropic SDK's own connection handling.
+
+---
+
+## Gemini API (Orchestrator only)
+
+Used by `orchestrator_engine.py` for the ReAct tool loop. Not dispatched through
+`llm_client.py` and not available for chat, distill, or other roles.

 API key resolution order:
 1. `api_key` embedded in the resolved orchestrator model config (V2 registry with `account_id`)
@@ -217,9 +200,7 @@ API key resolution order:
 ## Distillation

 Memory distillation uses `role="distill"`. Configure via Model Registry → Role Assignments.
-
-`.env` override: `ROLE_DISTILL=claude_cli` (default).
-
+Any `local_openai` or `anthropic_api` model can be assigned to the distill role.

 ---

@@ -232,4 +213,4 @@ Memory distillation uses `role="distill"`. Configure via Model Registry → Role
 | `cortex/routers/local_llm.py` | Settings UI routes + `/api/models/role` AJAX |
 | `cortex/routers/chat.py` | `_backend_label()`, `fallback_used` flag |
 | `cortex/routers/orchestrator.py` | Engine selection, Gemini API key resolution |
-| `cortex/config.py` | `ROLE_*` env defaults, `DEFINED_ROLES`, `PRIMARY_BACKEND` |
+| `cortex/config.py` | `ROLE_*` env defaults, `DEFINED_ROLES`, `TIMEOUT_LOCAL` |