Scott.Idem/Cortex-Inara
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: audit log, usage tracking UI, OpenAI orchestrator compaction, onboarding + docs

Browse Source 
Tool audit log:
- Every orchestrator tool call logged to home/{user}/tool_audit/YYYY-MM-DD.jsonl
- Files panel sidebar: audit log group (collapsed), date-linked read-only table
- Admin endpoints: /api/audit/files, /api/audit/day, /api/audit/recent, /api/audit/stats
- Engine and model name recorded per entry

OpenAI orchestrator improvements:
- Context budget enforcement: 75% of model context_k (min 16k)
- Message compaction: truncates old tool results when approaching budget
- max_rounds respected per model config (intersected with server cap)

OpenRouter onboarding (setup.html, onboarding.py, app.js, settings.html):
- Step 3 of 3: /setup/model with curated model picker
- Chat banner for users on server-default model (informational, not alarmist)
- Settings quick-link card; /setup/model works standalone for existing users

Model registry + session store:
- set_role_config / get_role_config for per-role tool lists and system_append
- session_store: session rename, session name backfill endpoint

UI updates (app.js, index.html, style.css, local_llm.html):
- Role toggle in context panel
- Off-the-record mode
- Agent notes read-only viewer
- OPERATIONS.md loaded at T2+ in context

Documentation:
- HELP.md: full tool table, per-role tool sets, Agent Notes, usage tracking
- TOOLS.md: Agent Notes section, count corrected to 44
- ARCH__SYSTEM.md, ARCH__BACKENDS.md, MASTER.md updated to match reality
- CLAUDE.md: onboarding flow, documentation philosophy sections
- README.md: stack in practice, DeepSeek TUI mention, architecture diagram updated
- TODO__Agents.md: onboarding task completed with deviation notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Scott Idem
2026-05-08 21:26:43 -04:00
parent c02d2462b0
commit f8f7cd75da
25 changed files with 1088 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
documentation/ARCH__BACKENDS.md
View File

@@ -1,7 +1,7 @@
# Architecture: LLM Backends
> How Cortex selects and talks to AI models.
> Last updated: 2026-04-27 (V2 schema)
> Last updated: 2026-05-06
---
@@ -33,11 +33,11 @@ Resolution order for a role:
### Explicit Override
The UI backend toggle cycles: **auto → claude → gemini → local → auto**
The **Role** toggle in the Context & Memory panel cycles through configured role slots for the `chat` role: **Primary → Backup 1 → Backup 2 → auto**.
- **auto** (default): role-based routing as above
- **claude / gemini / local**: bypasses role routing; forces that backend type
- The toggle will be redesigned in Phase 3 to cycle through chat role slots (Primary / Backup 1 / Backup 2)
- Each slot shows the configured model label
- `auto` uses the Primary without forcing a specific backend type
- The ⚡ Tools toggle is independent — it routes to the `orchestrator` role regardless of the chat role selection
**Fallback chain** (automatic, only when no explicit registry entry exists):
```
@@ -113,6 +113,8 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
"provider": "local",
"host_id": "abc123",
"context_k": 72,
"max_rounds": 5,
"tools": true,
"tags": ["fast", "local"]
}
],
@@ -125,6 +127,14 @@ Managed at **Settings → Models** (`/settings/models`). Full provider UI coming
}
```
### Optional model fields
| Field | Type | Default | Meaning |
|---|---|---|---|
| `context_k` | int | 32 | Context window in thousands of tokens. Used for compaction budget (75% of window). |
| `max_rounds` | int \| null | null | Per-model tool loop cap. `null` = use global `orchestrator_max_rounds`. Effective limit = `min(per_model, global)`. |
| `tools` | bool | true | Whether this model supports tool calling. `false` = skip tool loop entirely; model gets a plain chat request. |
### host_type (local hosts)
| `host_type` | Chat endpoint | Models endpoint | Use for |
@@ -210,13 +220,6 @@ Memory distillation uses `role="distill"`. Configure via Model Registry → Role
`.env` override: `ROLE_DISTILL=claude_cli` (default).
---
## Future: Phase 3 — Backend Toggle Redesign
The `claude → gemini → local` toggle will be replaced with a slot toggle that cycles
through the chat role's configured models (Primary → Backup 1 → Backup 2), showing
the actual model label. See `DESIGN__Model_Registry_V2.md`.
---

51
documentation/ARCH__SYSTEM.md
View File

@@ -1,7 +1,7 @@
# Architecture: System Overview
> How the pieces fit together.
> Last updated: 2026-04-03
> Last updated: 2026-05-06
---
@@ -56,7 +56,9 @@ Details: [`ARCH__BACKENDS.md`](ARCH__BACKENDS.md) | [`ARCH__PERSONA.md`](ARCH__P
| `context_loader.py` | Builds system prompt from persona files (tiers 1–4) |
| `llm_client.py` | All LLM backends — Claude, Gemini CLI, Local |
| `orchestrator_engine.py` | Gemini API ReAct tool loop → Claude handoff |
| `session_store.py` | In-memory + file session persistence |
| `openai_orchestrator.py` | OpenAI-compatible ReAct tool loop (local models via Open WebUI/OpenRouter) |
| `model_registry.py` | Per-user model registry V2 — providers, hosts, models, role assignments |
| `session_store.py` | In-memory + file session persistence (`session_data/{id}.json`) |
| `session_logger.py` | Writes session turns to `sessions/YYYY-MM-DD.md` |
| `memory_distiller.py` | Short/mid/long distill jobs |
| `scheduler.py` | APScheduler — distill jobs + user crons |
@@ -64,20 +66,23 @@ Details: [`ARCH__BACKENDS.md`](ARCH__BACKENDS.md) | [`ARCH__PERSONA.md`](ARCH__P
| `notification.py` | Outbound channel messages (distill alerts, cron proactive) |
| `auth_utils.py` | bcrypt passwords, JWT, invite tokens, channel config |
| `auth_middleware.py` | JWT cookie validation on all routes |
| `user_settings.py` | Per-user local LLM config (hosts, models, active model) |
| `tool_audit.py` | JSONL audit log for every orchestrator tool invocation |
| `usage_tracker.py` | Per-user token usage tracking (daily buckets → `usage.json`) |
| `event_bus.py` | Internal SSE pub/sub (NC Talk → browser mirror) |
| `email_utils.py` | SMTP invite emails |
| `persona_template.py` | Bootstrap a new persona directory from templates |
| `routers/` | One file per endpoint group (chat, orchestrator, auth, files, channels, ui, settings…) |
| `tools/` | Orchestrator tool implementations (web, ae_knowledge, tasks, scratch, reminders, cron, system) |
| `static/` | Web UI — `index.html`, `app.js`, `style.css`, `login.html`, `setup.html`, `HELP.md` |
| `tests/` | pytest suite (80 tests) |
| `routers/` | One file per endpoint group — `chat`, `orchestrator`, `auth`, `files`, `ui`, `settings`, `local_llm`, `distill`, `audit`, `usage`, `push`, `help`, `onboarding`, `auth_google`, `nextcloud_talk`, `google_chat` |
| `tools/` | Orchestrator tool implementations — `web`, `tasks`, `scratch`, `reminders`, `cron`, `system`, `notify`, `ae_journals`, `ae_tasks`, `agent_notes` |
| `static/` | Web UI — `index.html`, `app.js`, `style.css`, `login.html`, `setup.html`, `HELP.md`, `local_llm.html`, `settings.html` |
| `tests/` | pytest suite |
---
## Key Design Decisions
**Two-brain pattern** — Gemini API handles tool use (function calling, planning, web search). Claude CLI handles all user-facing responses. Direct chat bypasses the orchestrator entirely.
**Two-brain pattern (Gemini orchestrator)** — Gemini API handles tool use (function calling, planning, web search). Claude CLI handles all user-facing responses. Direct chat bypasses the orchestrator entirely.
**Single-model pattern (local orchestrator)** — When the `orchestrator` role resolves to a `local_openai` model, `openai_orchestrator.py` runs the full ReAct loop and produces the final response itself. No Claude handoff — the local model does both reasoning and response.
**Subprocess backends** — Claude and Gemini run as CLI subprocesses (`claude --print`, `gemini -p`). This keeps auth transparent (Claude Code manages tokens) and avoids API costs on the Pro subscription path.
@@ -88,3 +93,33 @@ Details: [`ARCH__BACKENDS.md`](ARCH__BACKENDS.md) | [`ARCH__PERSONA.md`](ARCH__P
**Per-user filesystem layout** — `home/{user}/persona/{name}/` mirrors Linux home directories. Each persona is a directory of markdown files and JSON. No database. Easy to inspect, edit, and back up.
**No single point of coupling** — tools live in `cortex/tools/`, separate from `ae_*` MCP tools. Channels live in `cortex/routers/`, each self-contained. Adding a channel or tool doesn't touch other subsystems.
**Agent private notes** — `AGENT_NOTES.md` per persona, writable only by the orchestrator via `agent_notes_*` tools. Never loaded into user-facing context. Three rolling backups (`bak1`–`bak3`) are visible read-only in the Files panel. Declared in `tools/agent_notes.py`; usage guidance in `PROTOCOLS.md`.
**No black boxes** — Every component, flow, and design decision is documented. Documentation is updated before implementation of significant changes and verified after. HELP.md is the user-facing contract; ARCH__*.md files are the developer contract; PROTOCOLS.md is the agent contract. If any of these drift from reality, that is a bug.
---
## Onboarding Flow
New users are invited via a one-time token and complete a three-step setup before reaching the chat:
```
1. /setup/{token} → Set password (POST creates session cookie, consumes token)
2. /setup/persona → Create persona (slug, display name, emoji, description)
3. /setup/model → Connect a model — OpenRouter recommended
(skip link goes straight to /{user}/{persona})
```
Step 3 is the planned addition (see `TODO__Agents.md § Guided onboarding`). Before it exists,
users land in the chat with no model configured and must navigate Settings → Model Registry
manually — which is confusing for non-technical users.
**After Step 3:**
- `save_host()` adds OpenRouter (`https://openrouter.ai/api/v1`, type `openai`)
- `save_model()` creates a model entry for the chosen model
- `set_role(chat, primary, model_id)` assigns it as the chat role primary
- Redirect to `/{user}/{persona}`
**Existing users with no model configured** — a dismissable banner is shown in the chat on
load, linking to `/setup/model` (the Step 3 form works standalone, without step labels).

24
documentation/MASTER.md
View File

@@ -1,7 +1,10 @@
# Cortex / Inara — Master Index
> Start here. This document is a map, not a manual.
> Last updated: 2026-04-28
> Last updated: 2026-05-06
>
> **Documentation philosophy:** Cortex is a no-black-box system. Docs must match reality.
> Update docs before implementing significant changes. Verify they still match after.
---
@@ -17,20 +20,27 @@ Cortex is a self-hosted personal AI platform. It routes messages from any input
| Component | Status | Notes |
|---|---|---|
| Web UI | ✅ Live | SPA, dark theme, mobile-responsive, session auth |
| Web UI | ✅ Live | SPA, dark theme, mobile-responsive, PWA-installable |
| Nextcloud Talk bot | ✅ Live | HMAC-signed, per-user routing |
| Google Chat Add-on | ✅ Live | JWT-verified, per-user routing |
| Claude backend | ✅ Live | Primary — via Claude Code CLI |
| Gemini backend | ✅ Live | Fallback — via Gemini CLI |
| Local backend | ✅ Live | Third option — Open WebUI/Ollama on scott_gaming |
| Gemini orchestrator | ✅ Live | Tool loop → Claude response, ⚡ Tools toggle in UI (27 tools) |
| Model registry V2 | ✅ Live | Providers (Anthropic/Google/Local), multi-account Gemini |
| Local backend | ✅ Live | Open WebUI/Ollama on scott_gaming; per-user multi-model config |
| Gemini orchestrator | ✅ Live | Tool loop → Claude response, ⚡ toggle in UI (40 tools) |
| Local orchestrator | ✅ Live | OpenAI-compatible ReAct loop; used when orchestrator role → local model |
| Model registry V2 | ✅ Live | Providers (Anthropic/Google/Local), multi-account Gemini, role assignments |
| Memory distillation | ✅ Live | Short (daily) / Mid (weekly) / Long (monthly) |
| Multi-user | ✅ Live | Scott, Holly, Brian — each with own personas |
| Session search | ✅ Live | Full-text search across past session logs |
| Proactive cron | ✅ Live | `message` and `brief` job types → NC Talk |
| Proactive cron | ✅ Live | `message` and `brief` job types → NC Talk / web push |
| Tool audit log | ✅ Live | Every orchestrator tool call logged to `home/{user}/tool_audit/` |
| Token usage tracking | ✅ Live | Per-user daily buckets in `home/{user}/usage.json`; visible in Settings |
| Web push notifications | ✅ Live | VAPID push; `web_push` orchestrator tool; subscribe via ☰ menu |
| Agent private notes | ✅ Live | `AGENT_NOTES.md` — orchestrator-only notepad; 3 rolling backups; user-visible as read-only |
| Distill safety | ✅ Live | Per-persona asyncio lock, per-endpoint cooldowns, Rebuild option |
| Guided onboarding | ✅ Live | Setup Step 3 for OpenRouter; existing-user banner; settings quick-link |
**Active users / personas:** scott/inara, scott/developer, holly/tina, brian/wintermute
**Active users / personas:** scott/inara, holly/tina, brian/wintermute
---

1
documentation/ROADMAP.md
View File

@@ -54,7 +54,6 @@
## Phase 5 — Routing Intelligence & Scale
- [ ] Intelligent model routing (by task type, privacy, context length)
- [ ] Agent-to-agent task delegation across fleet
- [ ] Permanent hosting on home server (currently on `scott_lpt`)
## Phase 6 — Infrastructure
- [ ] Server DMZ finalized

46
documentation/TODO__Agents.md
View File

@@ -7,16 +7,41 @@
## 🔴 High Priority
### [UX] User onboarding — guided model setup
New users complete password + persona setup and land directly in the chat with no working
AI model configured. This closes that gap with a guided Step 3 and a fallback for existing
users who skipped it or were onboarded before this existed.
Design spec: `documentation/ARCH__SYSTEM.md` § Onboarding Flow
- [x] **Setup Step 3 page** — new `/setup/model` GET/POST in `onboarding.py` — 2026-05-06
- Recommends OpenRouter: "one API key, access to Claude, Gemini, and dozens of other models"
- API key field + curated model dropdown (claude-3-5-haiku, claude-3-7-sonnet, gemini-2.0-flash, llama-3.3-70b)
- On submit: `save_host()` (OpenRouter) + `save_model()` + `set_role(chat, primary, model_id)` in `model_registry.py`
- Skip: `POST /setup/model/skip` reads `cx_setup_persona` cookie, redirects to chat; JS fetch on skip-link click
- Step labels updated: setup.html "1 of 3" / "2 of 3" / "3 of 3" (was "1 of 2" / "2 of 2")
- Standalone: `/setup/model` works without step labels (no `cx_setup_persona` cookie → no label)
- Persona creation now redirects to `/setup/model` instead of directly to chat
- [x] **Existing user banner** — displayed in chat if no role has a model assigned — 2026-05-06
- Checks `GET /backend` on load (uses `available_roles` — already does role-resolution)
- Dismissable amber callout strip above chat: "No AI model configured — Set up OpenRouter →"
- Dismissed via `localStorage` key `cx_no_model_banner_dismissed`; auto-removed when a model is added
- [x] **Settings quick-link** — amber card in settings Model Registry section — 2026-05-06
- Checks `GET /backend` on page load; shown if `available_roles` is empty
- Links to `/setup/model`
- [x] Update `cortex/static/HELP.md` — Getting Started section + model registry quick-connect note — 2026-05-06
- [x] Update `CLAUDE.md` — documented `/setup/model` endpoint, setup flow description, docs philosophy — 2026-05-06
### [Local] Local orchestrator — reach full parity with Gemini orchestrator
`openai_orchestrator.py` is partially built and wired into `POST /orchestrate`.
When the `orchestrator` role resolves to a `local_openai` model it routes there
automatically. Remaining work is quality/reliability parity, not ground-up design.
- [ ] Audit tool schema conversion — Gemini `FunctionDeclaration` → OpenAI `tools` format
(minor field rename, already partially done)
- [ ] Context budget enforcement per iteration (40–50k for E4B, 35–40k for 26B A4B)
- [ ] Context compaction — trim stale tool results mid-run when approaching limit
- [ ] Error handling parity with Gemini orchestrator (retry logic, malformed tool calls)
- [x] Tool schema conversion — Gemini FunctionDeclaration → OpenAI tools format
- [x] Context budget: `_context_budget()` uses `context_k * 1000 * 0.75`, min 16k — 2026-05-06
- [x] Context compaction: `_compact_messages()` trims old tool results before each round and before the confirmation-gate call — 2026-05-06
- [x] Error handling: malformed tool args caught + logged; tool execution errors returned as strings
- [ ] Retry logic on transient API errors (connection timeout, 429, 503)
- [ ] Test end-to-end with Gemma 4 E4B and 26B A4B on scott_gaming
- [ ] Review `ARCH__FUTURE.md` agent architecture ideas before finalising design
- Reference: `docs/OPEN_WEBUI_API.md`, `documentation/ARCH__FUTURE.md` §1
@@ -117,7 +142,7 @@ Multi-user setup with real Gemini/Claude API costs. Track per-user token consump
so Scott can see who's spending what.
- [x] Count input + output tokens — local backend (OpenAI `usage` field) + Gemini API (`usage_metadata`) — 2026-05-05
- [x] Append to `home/{user}/usage.json` — daily buckets, per-model breakdown — 2026-05-05
- [ ] Expose via `/api/usage` endpoint; add a summary row to the Settings page
- [x] Expose via `/api/usage` + `/api/usage/summary` + `/api/usage/all` (admin); usage table in Settings — 2026-05-06
- [ ] Optional: soft spending limit with a warning toast when exceeded
### [Security] Tool call audit log — 2026-05-05
@@ -166,15 +191,6 @@ the foundation. What remains is removing the need to toggle manually.
- Fast/cheap queries → local E4B (25 t/s, no API cost)
- [ ] Routing logic in `llm_client.py` or new `router.py`; expose override in UI
### [Ops] Permanent fleet hosting — home server deployment
Currently running on `scott-lt-i7-rtx` (gaming laptop). Long-term target is the
home server for always-on reliability. `docker-compose.yml` already exists.
- [ ] Copy project to home server
- [ ] Configure Nginx reverse proxy (already Docker-hosted on that machine)
- [ ] Point `cortex.dgrzone.com` → home server internal IP (pfSense alias update)
- [ ] WireGuard required for all access — not internet-exposed
- [ ] Update `FLEET_MANIFEST.md` to reflect new hosting location
### [Future] Cortex Mesh — multi-instance fleet coordination
Each fleet device runs its own Cortex instance. Instances delegate tasks to each
other based on resources and specialisation. No central coordinator required.