Cortex-Inara/CLAUDE.md

CLAUDE.md — Cortex / Inara Project

This file is loaded automatically by Claude Code when working in this directory. Read it before touching any files.

---

Identity & Context

• Project: Cortex (dispatcher) + Inara (resident agent)
• Owner: Scott Idem (One Sky IT / Danger Zone)
• Machine context: See ~/CLAUDE.md for fleet identity (scott_lpt = General Manager)
• Named after: The 'verse-wide communications network (Firefly)

---

Directory Map

Cortex_and_Inara_dev/
  cortex/                ← FastAPI service (the dispatcher)
    main.py              ← App entry point, router registration
    config.py            ← All settings (pydantic-settings, reads .env)
    persona.py           ← Two-level identity: user + persona, path resolution, ContextVars
    llm_client.py        ← Claude CLI + Gemini CLI subprocess backends
    orchestrator_engine.py ← Gemini API ReAct tool loop → Claude handoff
    context_loader.py    ← Builds system prompt from persona files (tier 1–4)
    session_store.py     ← In-memory + file session persistence
    session_logger.py    ← Writes session turns to home/{user}/persona/{name}/sessions/
    memory_distiller.py  ← Short/mid/long distill jobs (APScheduler)
    cron_runner.py       ← Cron job storage, schedule parsing, job execution
    scheduler.py         ← APScheduler setup (distill + user crons)
    event_bus.py         ← Internal SSE pub/sub (NC Talk → browser)
    auth_utils.py        ← bcrypt passwords, JWT create/decode, invite token system
    auth_middleware.py   ← SessionAuthMiddleware — JWT cookie validation on all routes
    persona_template.py  ← Bootstrap a new persona directory from string templates
    email_utils.py       ← SMTP_SSL email helpers (invite emails, future notifications)
    routers/
      chat.py            ← POST /chat (streaming SSE)
      orchestrator.py    ← POST /orchestrate, GET /orchestrate/{job_id}
      auth.py            ← GET /auth/status (Claude + Gemini CLI token checks)
      distill.py         ← POST /distill/*, GET /distill/status
      files.py           ← GET /files (persona file browser)
      nextcloud_talk.py  ← POST /webhook/nextcloud (NC Talk bot)
      google_chat.py     ← POST /webhook/google (Google Chat Add-on)
      ui.py              ← Login/logout, /{user}/{persona} UI route, /api/personas
      onboarding.py      ← /setup/{token} password step + /setup/persona creation
    tools/
      __init__.py        ← Tool registry (Gemini FunctionDeclarations + dispatcher)
      web.py             ← DuckDuckGo web_search tool
      scratch.py         ← Scratchpad tools (scratch_read/write/append/clear)
      tasks.py           ← Personal task management (task_create/list/update/complete)
      cron.py            ← Scheduled job tools (cron_list/add/remove/toggle)
      system.py          ← Local machine tools (claude_allow_dir)
    tests/               ← pytest test suite (80 tests)
    static/              ← Single-page web UI (index.html, style.css, app.js)
                           login.html — login form (dark theme, POST /login)
                           setup.html — onboarding form (password + persona creation)
    data/sessions/       ← Persisted session JSON files

  home/                  ← User and persona data (Linux home layout)
    scott/
      persona/
        inara/           ← Inara identity, memory, context, sessions
          IDENTITY.md    ← Who Inara is
          SOUL.md        ← Values, personality, voice
          PROTOCOLS.md   ← Behavioral rules
          CONTEXT_TIERS.md ← What each tier (1–4) includes in the system prompt
          USER.md        ← Scott's profile (loaded into context)
          HELP.md        ← In-app help content (rendered in UI)
          MEMORY_LONG.md ← Long-term memory (auto-distilled monthly)
          MEMORY_MID.md  ← Mid-term memory (auto-distilled weekly)
          MEMORY_SHORT.md ← Short-term memory (auto-distilled daily)
          REMINDERS.md   ← Pending reminders (auto-surfaced in context at tier 2+)
          SCRATCH.md     ← Ephemeral scratchpad
          TASKS.json     ← Personal task list
          CRONS.json     ← Scheduled jobs
          sessions/      ← Session turn logs (YYYY-MM-DD.md)
    holly/
      persona/
        tina/            ← Tina (Holly's persona) — same structure as inara/

  docs/                  ← Integration reference docs
    NEXTCLOUD_TALK_BOT.md
    OPEN_WEBUI_API.md    ← Open WebUI API: tool calling, RAG, model management

  documentation/         ← Architecture decisions and agent task list
    TODO__Agents.md      ← READ THIS FIRST — active task list
    ARCH__Intelligence_Layer.md ← Orchestrator, dev agent, knowledge architecture

  docker-compose.yml     ← Docker deployment
  .env.default           ← Reference config (copy to .env, fill in secrets)
  README.md              ← Project orientation

---

Run Commands

# First-time setup or update on any machine
python3 install.py

# Restart service (after any Python change)
systemctl --user restart cortex

# Syntax check a file before restarting
python3 -m py_compile cortex/<file>.py

# Syntax check all routers
for f in cortex/routers/*.py cortex/tools/*.py cortex/orchestrator_engine.py; do
    python3 -m py_compile "$f" && echo "OK: $f"
done

# Install/update dependencies
cd cortex && .venv/bin/pip install -r requirements.txt

# Logs
journalctl --user -u cortex -f

# Web UI (local)
http://localhost:8000

# Swagger docs
http://localhost:8000/docs

---

Key Design Decisions

Two-Brain Architecture (Orchestrator / Responder)

• Gemini API (orchestrator_engine.py) — runs the ReAct tool loop; handles tool calling, planning, research
• Claude CLI (llm_client.py) — produces all user-facing responses; receives enriched context from Gemini
• Direct chat bypasses the orchestrator entirely — POST /chat goes straight to Claude (faster)
• Orchestrated tasks go to POST /orchestrate — returns a job_id, result is polled

LLM Backends

• llm_client.py manages Claude CLI (claude --print) and Gemini CLI (gemini -p) subprocesses
• orchestrator_engine.py uses the Gemini API (google-genai SDK) — completely separate from the Gemini CLI
• Claude OAuth token is read live from ~/.claude/.credentials.json (never rely on stale env var)

Tool Strategy

• Orchestrator tools live in cortex/tools/ — separate from the ae_* MCP tools
• Do not modify the ae_* MCP server to support orchestrator needs; add new tools to cortex/tools/ instead
• Tools are registered in cortex/tools/__init__.py as both Gemini FunctionDeclarations and Python callables

Context / Memory

• context_loader.py assembles Inara's system prompt from inara/ files based on tier (1–4)
• Tier 1 = minimal (identity only); Tier 2 = standard (+ memory + user profile); Tier 3 = + last 2 sessions; Tier 4 = + last 7 sessions
• Memory files are written by the distiller or manually — do not delete them

Security / Safety

• Never rm — move files to ~/tmp/gemini_trash
• Never commit secrets — .env is gitignored; use .env.default as the reference
• NEXTCLOUD_TALK_BOT_SECRET and GEMINI_API_KEY live in .env only
• /channels/* and /health are publicly exposed (webhook auth is handled at app layer — JWT/HMAC)
• /login, /logout, /setup/*, /static/* are public — all other routes require a valid JWT session cookie
• SessionAuthMiddleware (auth_middleware.py) validates the cookie on every request; browsers are redirected to /login, API calls get 401
• Passwords are bcrypt-hashed and stored in home/{username}/auth.json — never in .env or the DB
• Invite tokens are one-time-use, 72-hour expiry, stored in home/{username}/invite.json

Onboarding Flow

New users follow a three-step setup before reaching the chat:

1. GET /setup/{token} → password form → POST /setup/{token} sets password + session cookie
2. GET /setup/persona → persona creation form → POST /setup/persona bootstraps persona directory
3. GET /setup/model → OpenRouter quick-connect → POST /setup/model saves host + model + role assignment

Step 3 is optional (skip link goes straight to /{user}/{persona}). /setup/model also works standalone (accessible from Settings) for existing users who haven't configured a model.

All in cortex/routers/onboarding.py. Model writes use model_registry.py: save_host(), save_model(), set_role(username, "chat", "primary", model_id).

Documentation Philosophy

Cortex is a no-black-box system. Docs must match reality — at all times.

• Docs first: When planning significant changes, update TODO__Agents.md and the relevant ARCH__*.md to describe the intended design before implementing. This creates a spec to implement against.
• Verify after: Once implementation is complete, re-read the pre-written docs and confirm they match what was actually built. Update anything that drifted.
• HELP.md is a user contract: It describes what users can do. Never let it describe features that don't exist or omit features that do.
• CLAUDE.md + ARCH__*.md are the developer contract: Update them as the architecture evolves.
• Stale docs are bugs. If you notice drift, fix it before moving on.

Doc update checklist (run after any significant change)

Doc	Update when
CLAUDE.md	New tool, channel, router, major design change, tool count
cortex/static/HELP.md	Any user-visible feature — tools, settings, UI, API endpoints
documentation/TODO__Agents.md	Mark completed items; add new planned work
documentation/MASTER.md	New capability goes live; tool count changes
documentation/ROADMAP.md	Phase items completed or added
documentation/ARCH__CHANNELS.md	New channel, notification trigger, or scheduler job
documentation/ARCH__SYSTEM.md	New module, router, or tools/ file
README.md	Architecture diagram, channels table, or setup steps change

---

Adding a New Tool

1. Implement the tool function in cortex/tools/<domain>.py
   • Must be async def; use asyncio.to_thread for blocking calls
   • Return a plain string result
2. Add a FunctionDeclaration and register it in cortex/tools/__init__.py:
   • Import the callable
   • Add to TOOL_CATEGORIES (pick an existing category or create one)
   • Add to _CALLABLES
   • Add a TOOL_RISK rating (low/medium/high)
   • Add to TOOL_ROLES if admin-only; add to CONFIRM_REQUIRED if destructive
   • Add module to _ALL_DECLARATIONS
3. Syntax check: python3 -m py_compile cortex/tools/<domain>.py
4. Restart Cortex

Managing Claude Code Directory Permissions

Claude Code prompts (or silently hangs) when it needs to read or write a directory outside its current working directory. The claude-allow-dir script patches ~/.claude/settings.json to add auto-allow rules so Claude no longer blocks on those paths.

Script: ~/.local/bin/claude-allow-dir

# Allow read + write (default)
claude-allow-dir ~/OSIT_dev/aether_api_fastapi

# Read-only
claude-allow-dir ~/agents_sync r

# Write-only
claude-allow-dir /tmp w

Adds Read(path/*) and/or Edit(path/*) + Write(path/*) entries to the permissions.allow array in ~/.claude/settings.json. Idempotent — safe to run twice on the same path. Changes take effect in the next Claude Code session (or after opening /hooks in the UI).

Orchestrator tool: claude_allow_dir

Cortex exposes this as a Gemini tool (cortex/tools/system.py) so the orchestrator can add allow rules on Inara's behalf without human intervention.

Security note: This tool modifies Claude Code's own permission settings. The Gemini orchestrator calling it can grant Claude access to any directory on the machine. Keep this in mind when evaluating orchestrator behavior — it should only be invoked when Scott has clearly asked for a directory to be unblocked.

Adding a New Router

1. Create cortex/routers/<name>.py with router = APIRouter()
2. Import and register in cortex/main.py
3. Syntax check, restart

---

Current State (2026-05-12)

Cortex is running and stable. All channels are live:

Channel	Status	Notes
Web UI	✅ Live	https://cortex.dgrzone.com — PWA-installable
Nextcloud Talk	✅ Live	HMAC-signed webhook, async reply
Google Chat	✅ Live	Workspace Add-on, hostAppDataAction response format
Local backend	✅ Live	Open WebUI/Ollama on scott_gaming, per-user multi-model config
Gemini orchestrator	✅ Live	Gemini API tool loop → Claude response; ⚡ toggle in UI
Local orchestrator	✅ Live	OpenAI-compatible ReAct loop; fires when orchestrator role → local model
Tool audit log	✅ Live	Every tool call logged to home/{user}/tool_audit/YYYY-MM-DD.jsonl
Token usage tracking	✅ Live	Per-user home/{user}/usage.json; summary in Settings
Web push	✅ Live	VAPID push notifications; web_push tool; subscribe via ☰ menu
Proactive notifications	✅ Live	Daily reminder check (09:00); distill/cron completions; GET /settings/notifications dedicated page

Active users: scott (inara), holly (tina), brian (wintermute)

62 orchestrator tools across 16 domain modules: web_search/http_fetch/web_read/http_post, project_file_read/list + file_stat/grep/diff/syntax_check (project-scoped), file_read/list/write/session_read/session_search (system-scoped, admin), git_status/git_log/git_diff (read-only git inspection, project-scoped), shell_exec/claude_allow_dir, cortex_restart/logs/status/update, task_list/create/update/complete, cron_list/add/remove/toggle, reminders_add/list/remove/clear, scratch_read/write/append/clear, web_push/email_send/nc_talk_send/nc_talk_history, ae_journal_list/search/entries_list/entry_read/create/update/disable/append/prepend, ae_task_list, agent_notes_read/write/append/clear, spawn_agent, ha_get_state/ha_get_states/ha_call_service.

Each tool has a TOOL_RISK rating (low/medium/high). Configure access at /settings/tools (max_risk threshold + per-tool whitelist/blacklist). Risk policy stored in home/{user}/tool_policy.json.

See documentation/TODO__Agents.md for the active task list. See documentation/ROADMAP.md for phases and what's next.

---

Related Docs

File	Purpose
documentation/MASTER.md	Start here — index, current state, all doc links
documentation/TODO__Agents.md	Active task list — read before starting work
documentation/ROADMAP.md	Phases — what's done, what's next
documentation/ARCH__SYSTEM.md	System architecture and component map
documentation/ARCH__BACKENDS.md	LLM backends, routing, per-user config
documentation/ARCH__PERSONA.md	Persona system, context tiers, memory distillation
documentation/ARCH__CHANNELS.md	Input channels — web, NC Talk, Google Chat, cron
documentation/ARCH__FUTURE.md	Planned: local orchestrator, dev agents, knowledge layer
~/agents_sync/projects/CORTEX.md	Project vision and philosophy
~/agents_sync/CLAUDE.md	Fleet coordination rules
~/CLAUDE.md	Machine identity (scott_lpt)