Cortex-Inara/CLAUDE.md

# CLAUDE.md — Cortex / Inara Project

This file is loaded automatically by Claude Code when working in this directory.
Read it before touching any files.

---

## Identity & Context

- **Project:** Cortex (dispatcher) + Inara (resident agent)
- **Owner:** Scott Idem (One Sky IT / Danger Zone)
- **Machine context:** See `~/CLAUDE.md` for fleet identity (`scott_lpt` = General Manager)
- **Named after:** The 'verse-wide communications network (Firefly)

---

## Directory Map

```
Cortex_and_Inara_dev/
  cortex/                ← FastAPI service (the dispatcher)
    main.py              ← App entry point, router registration
    config.py            ← All settings (pydantic-settings, reads .env)
    persona.py           ← Two-level identity: user + persona, path resolution, ContextVars
    llm_client.py        ← Claude CLI + Gemini CLI subprocess backends
    orchestrator_engine.py ← Gemini API ReAct tool loop → Claude handoff
    context_loader.py    ← Builds system prompt from persona files (tier 1–4)
    session_store.py     ← In-memory + file session persistence
    session_logger.py    ← Writes session turns to home/{user}/persona/{name}/sessions/
    memory_distiller.py  ← Short/mid/long distill jobs (APScheduler)
    cron_runner.py       ← Cron job storage, schedule parsing, job execution
    scheduler.py         ← APScheduler setup (distill + user crons)
    event_bus.py         ← Internal SSE pub/sub (NC Talk → browser)
    auth_utils.py        ← bcrypt passwords, JWT create/decode, invite token system
    auth_middleware.py   ← SessionAuthMiddleware — JWT cookie validation on all routes
    persona_template.py  ← Bootstrap a new persona directory from string templates
    email_utils.py       ← SMTP_SSL email helpers (invite emails, future notifications)
    routers/
      chat.py            ← POST /chat (streaming SSE)
      orchestrator.py    ← POST /orchestrate, GET /orchestrate/{job_id}
      auth.py            ← GET /auth/status (Claude + Gemini CLI token checks)
      distill.py         ← POST /distill/*, GET /distill/status
      files.py           ← GET /files (persona file browser)
      nextcloud_talk.py  ← POST /webhook/nextcloud (NC Talk bot)
      google_chat.py     ← POST /webhook/google (Google Chat Add-on)
      ui.py              ← Login/logout, /{user}/{persona} UI route, /api/personas
      onboarding.py      ← /setup/{token} password step + /setup/persona creation
    tools/
      __init__.py        ← Tool registry (Gemini FunctionDeclarations + dispatcher)
      web.py             ← DuckDuckGo web_search tool
      scratch.py         ← Scratchpad tools (scratch_read/write/append/clear)
      tasks.py           ← Personal task management (task_create/list/update/complete)
      cron.py            ← Scheduled job tools (cron_list/add/remove/toggle)
      system.py          ← Local machine tools (claude_allow_dir)
    tests/               ← pytest test suite (80 tests)
    static/              ← Single-page web UI (index.html, style.css, app.js)
                           login.html — login form (dark theme, POST /login)
                           setup.html — onboarding form (password + persona creation)
    data/sessions/       ← Persisted session JSON files

  home/                  ← User and persona data (Linux home layout)
    scott/
      persona/
        inara/           ← Inara identity, memory, context, sessions
          IDENTITY.md    ← Who Inara is
          SOUL.md        ← Values, personality, voice
          PROTOCOLS.md   ← Behavioral rules
          CONTEXT_TIERS.md ← What each tier (1–4) includes in the system prompt
          USER.md        ← Scott's profile (loaded into context)
          HELP.md        ← In-app help content (rendered in UI)
          MEMORY_LONG.md ← Long-term memory (auto-distilled monthly)
          MEMORY_MID.md  ← Mid-term memory (auto-distilled weekly)
          MEMORY_SHORT.md ← Short-term memory (auto-distilled daily)
          REMINDERS.md   ← Pending reminders (auto-surfaced in context at tier 2+)
          SCRATCH.md     ← Ephemeral scratchpad
          TASKS.json     ← Personal task list
          CRONS.json     ← Scheduled jobs
          sessions/      ← Session turn logs (YYYY-MM-DD.md)
    holly/
      persona/
        tina/            ← Tina (Holly's persona) — same structure as inara/

  docs/                  ← Integration reference docs
    NEXTCLOUD_TALK_BOT.md

  documentation/         ← Architecture decisions and agent task list
    TODO__Agents.md      ← READ THIS FIRST — active task list
    ARCH__Intelligence_Layer.md ← Orchestrator, dev agent, knowledge architecture

  docker-compose.yml     ← Docker deployment
  .env.default           ← Reference config (copy to .env, fill in secrets)
  README.md              ← Project orientation
```

---

## Run Commands

```bash
# Start (Docker)
docker compose up -d

# Restart service (after any Python change)
systemctl --user restart cortex

# Syntax check a file before restarting
python3 -m py_compile cortex/<file>.py

# Syntax check all routers
for f in cortex/routers/*.py cortex/tools/*.py cortex/orchestrator_engine.py; do
    python3 -m py_compile "$f" && echo "OK: $f"
done

# Install/update dependencies
cd cortex && .venv/bin/pip install -r requirements.txt

# Logs
journalctl --user -u cortex -f

# Web UI (local)
http://localhost:8000

# Swagger docs
http://localhost:8000/docs
```

---

## Key Design Decisions

### Two-Brain Architecture (Orchestrator / Responder)
- **Gemini API** (`orchestrator_engine.py`) — runs the ReAct tool loop; handles tool calling, planning, research
- **Claude CLI** (`llm_client.py`) — produces all user-facing responses; receives enriched context from Gemini
- **Direct chat** bypasses the orchestrator entirely — `POST /chat` goes straight to Claude (faster)
- **Orchestrated tasks** go to `POST /orchestrate` — returns a job_id, result is polled

### LLM Backends
- `llm_client.py` manages Claude CLI (`claude --print`) and Gemini CLI (`gemini -p`) subprocesses
- `orchestrator_engine.py` uses the Gemini **API** (google-genai SDK) — completely separate from the Gemini CLI
- Claude OAuth token is read live from `~/.claude/.credentials.json` (never rely on stale env var)

### Tool Strategy
- Orchestrator tools live in `cortex/tools/` — separate from the `ae_*` MCP tools
- **Do not modify** the `ae_*` MCP server to support orchestrator needs; add new tools to `cortex/tools/` instead
- Tools are registered in `cortex/tools/__init__.py` as both Gemini FunctionDeclarations and Python callables

### Context / Memory
- `context_loader.py` assembles Inara's system prompt from `inara/` files based on tier (1–3)
- Tier 1 = minimal (identity only); Tier 2 = standard (+ memory + user profile); Tier 3 = full
- Memory files are written by the distiller or manually — do not delete them

### Security / Safety
- **Never `rm`** — move files to `~/tmp/gemini_trash`
- **Never commit secrets** — `.env` is gitignored; use `.env.default` as the reference
- `NEXTCLOUD_TALK_BOT_SECRET` and `GEMINI_API_KEY` live in `.env` only
- `/channels/*` and `/health` are publicly exposed (webhook auth is handled at app layer — JWT/HMAC)
- `/login`, `/logout`, `/setup/*`, `/static/*` are public — all other routes require a valid JWT session cookie
- `SessionAuthMiddleware` (`auth_middleware.py`) validates the cookie on every request; browsers are redirected to `/login`, API calls get 401
- Passwords are bcrypt-hashed and stored in `home/{username}/auth.json` — never in `.env` or the DB
- Invite tokens are one-time-use, 72-hour expiry, stored in `home/{username}/invite.json`

---

## Adding a New Tool

1. Implement the tool function in `cortex/tools/<domain>.py`
   - Must be `async def`; use `asyncio.to_thread` for blocking calls
   - Return a plain string result
2. Add a `FunctionDeclaration` and register it in `cortex/tools/__init__.py`
3. Syntax check: `python3 -m py_compile cortex/tools/<domain>.py`
4. Restart Cortex

## Managing Claude Code Directory Permissions

Claude Code prompts (or silently hangs) when it needs to read or write a directory outside
its current working directory. The `claude-allow-dir` script patches `~/.claude/settings.json`
to add auto-allow rules so Claude no longer blocks on those paths.

### Script: `~/.local/bin/claude-allow-dir`

```bash
# Allow read + write (default)
claude-allow-dir ~/OSIT_dev/aether_api_fastapi

# Read-only
claude-allow-dir ~/agents_sync r

# Write-only
claude-allow-dir /tmp w
```

Adds `Read(path/*)` and/or `Edit(path/*)` + `Write(path/*)` entries to the `permissions.allow`
array in `~/.claude/settings.json`. Idempotent — safe to run twice on the same path.
Changes take effect in the next Claude Code session (or after opening `/hooks` in the UI).

### Orchestrator tool: `claude_allow_dir`

Cortex exposes this as a Gemini tool (`cortex/tools/system.py`) so the orchestrator can add
allow rules on Inara's behalf without human intervention.

**Security note:** This tool modifies Claude Code's own permission settings. The Gemini
orchestrator calling it can grant Claude access to any directory on the machine. Keep this
in mind when evaluating orchestrator behavior — it should only be invoked when Scott has
clearly asked for a directory to be unblocked.

## Adding a New Router

1. Create `cortex/routers/<name>.py` with `router = APIRouter()`
2. Import and register in `cortex/main.py`
3. Syntax check, restart

---

## Current State (2026-03-27)

Cortex is running and stable. All three primary channels are live:

| Channel | Status | Notes |
|---|---|---|
| Web UI | ✅ Live | `https://cortex.dgrzone.com` |
| Nextcloud Talk | ✅ Live | HMAC-signed webhook, async reply |
| Google Chat | ✅ Live | Workspace Add-on, `hostAppDataAction` response format |

### Active Tasks

See `documentation/TODO__Agents.md` for the full list. Current priorities:

- **[High]** Ollama backend — local LLM via `scott_gaming` over WireGuard
- **[Medium]** NC Talk — complete bot registration docs (`docs/NEXTCLOUD_TALK_BOT.md`)
- **[Medium]** Knowledge consolidation — markdown → AE Journals

### Recently Completed

- ✅ Help & Reference shared base — `cortex/static/HELP.md` served to all users; persona-specific additions appended if present — 2026-03-27
- ✅ Google OAuth sign-in — `/auth/google` flow, pre-register via `manage_passwords.py google-add` — 2026-03-27
- ✅ Per-user Gemini API key — stored in `auth.json`, used by orchestrator, manageable in `/settings` — 2026-03-27
- ✅ Connected accounts + Gemini key in settings UI — `/settings` shows Google account, key hint, remove link — 2026-03-27
- ✅ `/{username}` persona picker page — card grid instead of 404 — 2026-03-26
- ✅ Session persistence across navigation — localStorage TTL 30 min, auto-restore on page load — 2026-03-26
- ✅ Lucide icons throughout UI — mode selector, send/stop/action buttons, edit/del/copy/save/cancel — 2026-03-25
- ✅ Persona-specific favicon — emoji SVG set from persona config — 2026-03-25
- ✅ Session auth — bcrypt passwords, JWT cookies, login/logout, `SessionAuthMiddleware` — 2026-03-20
- ✅ Persona onboarding — invite tokens, self-service password setup, persona creation form — 2026-03-20
- ✅ Multi-user/multi-persona support (`home/{username}/persona/{name}/` two-level layout) — 2026-03-20
- ✅ SMTP invite email — `noreply@oneskyit.com`, HTML + plain text, `manage_passwords.py invite` — 2026-03-20
- ✅ Scratchpad, task management, and cron/scheduled job tools — 2026-03-20
- ✅ Test suite (80 tests) covering API, persona routing, tools, security — 2026-03-20
- ✅ Google Chat bot (Workspace Add-on, JWT auth, `hostAppDataAction` format) — 2026-03-20
- ✅ Orchestrator Agent mode UI + session persistence — 2026-03-18
- ✅ Memory distiller (APScheduler, short/mid/long) — 2026-03

---

## Related Docs

| File | Purpose |
|---|---|
| `documentation/TODO__Agents.md` | Active task list — read before starting work |
| `documentation/ARCH__Intelligence_Layer.md` | Full architecture design |
| `~/agents_sync/projects/CORTEX.md` | High-level project vision and phases |
| `~/agents_sync/CLAUDE.md` | Fleet coordination rules |
| `~/CLAUDE.md` | Machine identity (`scott_lpt`) |