Compare commits

..

201 Commits

Author SHA1 Message Date
Scott Idem
e8819773ee docs: update TODO — mark completed items from 2026-06-03 session
- aider_run async/notify: done
- L2→L3 boundary enforcement: done (default _agent_level=2)
- aider_run multi-provider credentials: done
- Added remaining item: pass _agent_level=1 from main orchestrators

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 23:06:47 -04:00
Scott Idem
0c1cf3989a feat: aider multi-provider credentials + test suite green (182/182)
aider_run multi-provider credentials (tools/aider.py):
- _resolve_credentials() — general credential resolver; replaces the previous
  OpenRouter-only injection; resolution priority: Anthropic model hint → explicit
  host_label → model prefix (openrouter/*, groq/*, deepseek/*, …) → OpenRouter
  default → Anthropic API key → any keyed cloud host → local/generic host
- _host_flags() — generates --api-key slug=key for known cloud providers (OpenRouter,
  OpenAI, Groq, Together, Fireworks, X.ai, DeepSeek, Mistral); generates
  --openai-api-base + --openai-api-key for generic/local hosts (Open WebUI, Ollama);
  appends /api suffix for openwebui host_type; auto-prefixes model with 'openai/'
  for generic endpoints when model has no / prefix
- Anthropic API keys from providers.anthropic.credentials (not a host entry)
- host_label param added to aider_run and FunctionDeclaration — pick a configured
  host by partial label match (e.g. 'OpenRouter', 'Local', 'scott-lt-i7-rtx')
- 16 unit tests for _resolve_credentials covering all resolution paths

main.py: move @app.get("/health") before app.include_router(ui.router) — the
/{username} catch-all in ui.router was swallowing the /health path

Test suite: 37 pre-existing failures → 182/182 passing
- test_tools.py: _task_list() missing priority arg (6 callsites); cron ID regex
  c_\w+ → c_[\w-]+ (token_urlsafe includes '-', causing intermittent truncation)
- test_webhooks.py: rewritten for per-user channel config architecture —
  patch routers.nextcloud_talk/google_chat.get_user_channels instead of removed
  settings fields; corrected endpoints /webhook/nextcloud/scott and
  /channels/google-chat/scott; non-empty cfg dicts so falsy-guard passes
- test_health.py: test_unknown_route_404 now uses 3-segment path (/{u}/{p}/x)
  since single-segment paths hit the /{username} UI catch-all
- test_api_files.py: removed '../config.py' from not-in-allowed test (ASGI
  normalizes it to /config.py which hits /{username} catch-all, not files router)
- test_security.py: same webhook patch target fix; per-user endpoint URLs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 23:00:45 -04:00
Scott Idem
658c508925 feat: multi-level agent management — background agents, lifecycle tools, 3-level hierarchy
agent_manager.py (new):
- AgentRecord dataclass: agent_id, level (1/2/3), role, task, status, started,
  parent_id (lineage), finished, result, notify, _task_ref
- register() / finish() / cancel_agent() / list_agents() / get() / set_task_ref()
- Calls notification.notify() on completion when notify=True (same channel as
  reminders and cron completions)
- 24-hour pruning of completed records on each new registration

spawn_agent (tools/agents.py):
- background=True: fires asyncio.create_task(), registers in agent_manager, returns
  agent_id string immediately — sync path unchanged (no regression)
- notify=True: push/Talk notification when the background task completes
- Level enforcement: _agent_level param tracks hierarchy depth; when spawning from
  Level 2, child automatically gets spawn_agent + aider_run denied so Level 3 agents
  cannot delegate further

New lifecycle tools (tools/agents.py + __init__.py):
- agent_status(agent_id) — status, role, level, elapsed, task, result preview; user-level
- agent_list(status, limit) — all agents for current user, newest first; user-level
- agent_cancel(agent_id) — kills background task; admin-only, confirm-required

tests/test_agent_manager.py (new, 41 tests):
- agent_manager CRUD, pruning, notification hook
- spawn_agent background: returns immediately, completes async, timeout, failure
- Level enforcement: L1→L2 permits spawn, L2→L3 auto-denies; explicit tool_list path
- agent_status / agent_list / agent_cancel output formatting
- aider_run background: returns agent_id, completes async, sync path unchanged
- All tests run without browser or Cortex service (~2.5s total)
  Run: cd cortex && .venv/bin/python -m pytest tests/test_agent_manager.py -v

Docs: ARCH__FUTURE.md §13 (full design), ROADMAP.md, TODO__Agents.md, MASTER.md,
HELP.md (orchestrator description corrected, tool schema line updated to reflect
keyword routing), CLAUDE.md tool count 66→69.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 22:40:20 -04:00
Scott Idem
29d8aa4aae feat: tool schema optimization, keyword routing, aider_run coding agent
Tool schema optimization (PLAN__Tool_Schema_Optimization.md Phases 1-3):
- model_registry.py: ROLE_DEFAULT_TOOLS — distill gets [], research/coder get
  narrow tool lists by default; applied in get_role_config() when user hasn't
  configured a custom list
- openai_orchestrator.py: keyword routing via narrow_tools_by_keywords() — scans
  user message + last assistant turn; narrows active schemas to matched categories
  only (e.g. "weather" → 3 web tools instead of 69); zero tools sent for pure chat
- openai_orchestrator.py: _get_cached_tools() — module-level schema cache keyed by
  (role, sorted_tool_list, risk_params); eliminates redundant schema rebuilds
- openai_orchestrator.py: _TOOL_SCHEMA_OVERHEAD 3000 → 500 tokens (schemas now
  excluded from the per-call fixed estimate since they're cached separately)
- tools/__init__.py: CATEGORY_TOOL_MAP + _KEYWORD_CATEGORY_MAP + classify_tool_categories()
  + narrow_tools_by_keywords() — the classifier logic lives here so both orchestrators
  can share it

aider_run tool (cortex/tools/aider.py):
- Invokes Aider as a subprocess with --message --yes-always --no-pretty --no-stream
- Project aliases: cortex / aether_api / aether_frontend / aether_container
- Auto-injects OpenRouter API key from Cortex model registry (no ~/.env needed)
- background=True fires async + registers in agent_manager; notify=True sends push
  notification on completion
- admin-only, confirm-required, TOOL_RISK=high
- .gitignore: added .aider.chat.history.md / .aider.input.history / .aider.llm.history

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 22:39:44 -04:00
Scott Idem
29940c299b docs: fix Chat and Tools section accuracy in HELP.md
- Copy button appears on all messages (user + assistant), not just assistant
- Delete requires a confirm step — clarified in description
- Model tag position is below-left for assistant messages, not bottom-right
- Tool calls render as expandable per-call cards, not a single  N summary note
- help.html: default-open sections updated to Getting Started / Chat / Sessions / Model Registry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:36:24 -04:00
Scott Idem
105ff8507f fix: restore proper tab button CSS in help.html (browser-reset issue with Tailwind preflight: false)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:19:08 -04:00
Scott Idem
c2a12a895a docs: rename Backends → Switching Models in HELP.md; fix stale Backup 2 reference
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:14:55 -04:00
Scott Idem
df1f358912 docs: fix role slots description in HELP.md (Primary + Backup 1, not Backup 2)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:11:03 -04:00
Scott Idem
7a27190ffe feat: custom roles, Tailwind settings pages, pg.css fixes, doc cleanup
Model Registry:
- Add per-user custom roles (add/remove via UI); required roles chat/orchestrator/distill
  are always present and cannot be removed
- Auto-migrate legacy .env-defined roles to custom_roles on first access
- Role config panel (gear): Remove role button moved inside panel; required badge below name
- Role select: Primary + Backup slots only (was three)

Settings pages — Tailwind CSS migration (CDN, preflight: false):
- local_llm.html, settings.html, help.html, notifications.html, tools_settings.html,
  crons.html, integrations.html all migrated; pg-* color tokens; dark/light via data-theme

pg.css fixes:
- input[type=checkbox/radio]: width: auto — prevents pg.css width:100% from stretching checkboxes
- btn-submit: responsive sizing via Tailwind w-full md:w-96 on each button (no longer full-width on desktop)

Documentation:
- MASTER.md, TODO__Agents.md: remove "/ Inara" from titles; description updated to "per-user AI personas"
- HELP.md: persona-agnostic language throughout (NC Talk, Google Chat, push, schedules, HA sections);
  roles section restructured to show required vs. custom roles with examples
- notifications.html: subtitle and HA description use "your persona" not "Inara"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:03:11 -04:00
Scott Idem
070f1ce156 fix: align settings/models nav and heading with all other settings pages
local_llm.html had a stale nav (hardcoded / and /help, missing Notifications /
Tools / Schedules / Integrations links) and used a different heading markup
(.page-header) than the rest of the settings pages (.page-title/.page-subtitle).

- Add pg.css link; strip duplicate base CSS (vars, reset, body, nav) from inline
  <style> — page-specific styles (provider blocks, model rows, roles, badges) kept
- Nav: use {{ back_href }} / {{ help_href }} / {{ integrations_nav }} placeholders;
  add Notifications, Tools, Schedules, Integrations (admin) links
- Heading: .page-header → <h1 class="page-title"> + <p class="page-subtitle">
- Body structure: move <nav> outside the .page wrapper; rename .page → .page-wrap
- local_llm.py: _render() accepts request, injects back_href / help_href /
  integrations_nav; adds _preferred_persona(), _integrations_nav() helpers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 21:51:06 -04:00
Scott Idem
a92fd90f0d feat: Anthropic SDK backend — API key alternative to Claude CLI OAuth
Adds `anthropic_api` model type so users can authenticate with a direct
Anthropic API key instead of (or alongside) the CLI OAuth session.

- model_registry.py: `anthropic_api` type; `save/get/remove_anthropic_api_key()`
  mirroring the Google account pattern; `save_cloud_model()` now picks type
  based on credential type (cli → claude_cli, api_key → anthropic_api);
  `_resolve_model()` merges api_key from the credential entry
- llm_client.py: `_anthropic_api()` backend (AsyncAnthropic SDK); dispatch
  and fallback wiring; usage tracking
- routers/local_llm.py: Anthropic API key management routes
  (POST /settings/local/anthropic-key, /anthropic-key/{id}/remove);
  `anthropic_api` badge and edit-form credential selector
- static/local_llm.html: Anthropic Cloud Provider block now shows API key
  management (add/remove); Add Model → Anthropic tab has credential selector
  (CLI vs API key)
- requirements.txt: enable anthropic>=0.40.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 21:30:56 -04:00
Scott Idem
70665fadff feat: schedules UI, task cron type, monthly/yearly schedules, AE DB tools, integrations page
- Schedules web UI (/settings/crons): list, add, edit, pause/resume, delete jobs
- cron task type: full orchestrator tool loop on a schedule, result → notification channel
- parse_schedule: monthly/yearly formats (monthly:DD:HH:MM, yearly:MM:DD:HH:MM)
- HA inbound webhook tools toggle: orchestrator loop vs. direct LLM, configurable in UI
- ae_db_query/describe/show_view: SELECT-only Aether MariaDB access (admin, per-user creds)
- /settings/integrations: admin-only page for Aether DB credentials
- Schedules nav link added to all settings pages
- pymysql added to requirements
- Docs updated: HELP.md, MASTER.md, CLAUDE.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 21:06:43 -04:00
Scott Idem
96b3c796c5 feat: file attachment support in chat (images + text/code files)
Text files (.md, .py, .js, .json, etc.): read client-side and injected
into the message body as a fenced code block — works with all backends
with zero model capability requirements.

Images (PNG/JPG/WebP/GIF, max 5 MB): encoded as base64 data URL on the
client and sent as a separate attachment field. Backend formats them as
OpenAI multimodal content (text + image_url) for local_openai backends.
Claude CLI and Gemini CLI see the text message with a "📎 filename.png"
note; image data is never written to session history.

- index.html: 📎 button + hidden file input in mode-select row;
  attachment-row preview area with thumbnail (images) or filename chip
- app.js: _resolveAttachment(), file reader, clearAttachment();
  sendMessage/sendOrchestrate updated to allow no-text sends when a
  file is pending; attachment spread into chat payload for images
- chat.py: Attachment model; attachment field on ChatRequest;
  llm_attachment extracted in _stream_chat and passed to complete()
- llm_client.py: attachment param through complete()/_dispatch()/_local();
  _local() builds multimodal content array for vision calls
- style.css: #attach-btn, #attachment-row, #attachment-preview, thumb

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:46:50 -04:00
Scott Idem
50c1997e91 docs: mark Phase 2/3 done; add file_diff, git tools, spawn_agent restrictions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:34:26 -04:00
Scott Idem
3716e5974f feat: Phase 3 model toggle — cycle chat-role slot models in UI
Replaces the role-cycle toggle with a slot model toggle in the Context &
Memory panel. The active model label is shown on the button; clicking cycles
through Primary → Backup 1 → Backup 2 slots configured for the Chat role.

- app.js: remove activeRole()/availableRoles role-cycling; add
  activeChatModel()/chatModels slot cycling; update send/orchestrate
  payloads to send slot + chat_role:"chat"; fix updateSendBtnTitle and
  startRunTimer to use activeChatModel()
- chat.py: add slot field to ChatRequest; pass slot= to complete();
  resolve backend_label from slot config; add _chat_slot_models() helper;
  include chat_models in GET /backend response
- HELP.md: update Model toggle description, tool count (62/16),
  Backends section, API chat payload example

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:32:43 -04:00
Scott Idem
85e13314a2 feat: add confirm step to message-level delete
Clicking del now shows 'confirm delete / cancel' inline in the action
bar. Cancel rebuilds the original buttons; confirm proceeds as before.
Matches the session delete pattern added in the prior commit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:20:13 -04:00
Scott Idem
20f3fe4f71 docs: mark completed items in TODO__Agents.md
task_list priority filter, session delete confirm, spawn_agent tool
restrictions, HA API tools/config/confirm-required, file_diff,
git_status/log/diff — all completed 2026-05-12.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:17:42 -04:00
Scott Idem
f336ae9687 feat: task_list priority filter, session delete confirm, spawn_agent tool restrictions
- task_list: add priority param ('low'/'normal'/'high') alongside existing status filter
- Session delete: inline confirm row (Delete / Cancel) instead of immediate delete
- spawn_agent: allow_tools and deny_tools per-call params; role config remains ceiling;
  deny_tools falls back to confirm_deny gate when no explicit tool_list is set

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:09:50 -04:00
Scott Idem
76fef827c5 docs: update tool count to 62 and current state to 2026-05-12
Reflects file_diff and git_status/log/diff additions, pg.css refactor,
and reasoning level controls added this session.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 00:14:12 -04:00
Scott Idem
b7144d5903 feat: add git_status, git_log, git_diff orchestrator tools
Read-only wrappers around git commands, project-scoped. Covers working
tree status, commit history browsing (with optional path filter), and
diffs between refs or the working tree — cleaner than shell_exec for
code review and change verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 00:12:00 -04:00
Scott Idem
3c9b8f5909 feat: add file_diff orchestrator tool
Runs diff -u on two project-scoped files. Low risk, no admin required.
Covers code review, config comparison, and before/after verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 00:08:36 -04:00
Scott Idem
54eef73b74 docs: add spawn_agent per-invocation tool restriction design (ARCH__FUTURE §12)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:23:17 -04:00
Scott Idem
4c3d9a7a65 refactor: extract shared settings stylesheet (pg.css) and clean up inline styles
- Create cortex/static/pg.css with shared variables, nav, containers, form
  elements, button classes, text utilities, and feedback messages
- All four settings pages (settings, notifications, tools, help) now link to
  pg.css and have page-specific-only <style> blocks
- Style block line counts reduced: settings 244→70, notifications 189→42,
  tools 126→88, help 122→75
- Inline style= attributes reduced: settings 45→4, notifications 7→3,
  tools 12→4, help 0
- Introduced shared CSS classes: .btn-submit, .btn-save, .btn-cancel,
  .btn-secondary, .action-link, .hint, .section-note, .page-title,
  .page-subtitle, .page-wrap, .usage-table, .tool-cat-row
- Fix tools_settings.py: replace server-generated inline styles on category
  header rows with .tool-cat-row CSS class
- Fix settings.py: add btn-save/btn-cancel/persona-rename-cancel classes to
  server-rendered persona rename form buttons

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:22:59 -04:00
Scott Idem
8ab1942514 refactor: migrate Tool Permissions from Settings to /settings/tools
- Remove Tool Permissions form from settings.html; replace with a
  "Tool Settings →" link that redirects to /settings/tools
- Add Confirmation Gate section to tools_settings.html (allow/deny
  textareas) inside the same form as risk policy — one save covers all
- tools_settings.py save handler now writes allow/deny alongside
  max_risk/whitelist/blacklist into tool_policy.json
- Remove /settings/tool-policy POST route from settings.py (no longer needed)
- Remove get_tool_policy, save_tool_policy, CONFIRM_REQUIRED imports
  from settings.py (now owned by tools_settings.py)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:50:48 -04:00
Scott Idem
69ec2f667d feat: tool risk policy UI + wiring through all orchestrators
- New /settings/tools page: max_risk selector (low/medium/high) + per-tool
  override dropdowns (Default / Force include / Force exclude) for all 58 tools
  grouped by category with color-coded risk badges; JS updates Auto status live
- get_tools_for_role() + get_openai_tools_for_role() now accept max_risk,
  whitelist, blacklist; _apply_risk_policy() handles the filtering logic
- get_risk_policy() helper in auth_utils reads from tool_policy.json
- Risk policy wired through orchestrator.py, openai_orchestrator.py,
  orchestrator_engine.py, nextcloud_talk.py, homeassistant.py
- Tools nav link added to settings.html and notifications.html
- CLAUDE.md and ARCH__SYSTEM.md updated: tool count 50→58, risk system docs,
  tool access control three-layer model documented

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:45:04 -04:00
Scott Idem
c9c1ca7de6 feat: TOOL_RISK ratings for all 58 orchestrator tools
Add informational security risk ratings (low/medium/high) to every tool.
Groundwork for future auto-allow tiers (max_risk + whitelist + blacklist).
Breakdown: 36 low, 12 medium, 10 high.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:32:22 -04:00
Scott Idem
ac06b3bc7b feat: project-scoped file tools — grep, stat, syntax_check, offset reads
Add five project-scoped tools (user-level, no admin required):
  project_file_read — read with 1-based offset for paging large files
  project_file_list — list with sizes + timestamps
  file_stat         — size, modified time, line count / entry count
  file_grep         — regex search with context lines, up to 50 matches
  file_syntax_check — py_compile (.py) or json.loads (.json)

Also add offset support to existing file_read (system scope).
Rename "Files" tool category to "System Files"; add "Project Files" category.
Project scope restricted to Cortex_and_Inara_dev/ project root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:23:50 -04:00
Scott Idem
fc6600c33e feat: Home Assistant API tools (ha_get_state, ha_get_states, ha_call_service)
Register three HA orchestrator tools so Inara can read device states and
control devices via the HA REST API. ha_call_service requires admin role
and user confirmation. Also includes accumulated UI fixes (setProcessing
helper, wasNewSession flag cleanup).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:39:35 -04:00
Scott Idem
ba91de37c5 feat: Home Assistant settings UI + fix channels.json
notifications.html: add Home Assistant section with two collapsible
blocks — Connection (HA URL + Long-Lived Access Token) and Inbound
webhook (webhook ID with endpoint URL hint showing the username).
Token field uses keep-existing pattern (blank = no change).

settings.py: wire ha_url, ha_token, ha_webhook_id through
_notifications_page() template substitution and save_notifications()
POST handler. Preserves existing HA config fields (persona, tier,
role, tools) on save.

TODO__Agents.md: add Home Assistant integration planning section
(event design, richer payload template, HA API tools).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:18:45 -04:00
Scott Idem
1d361fe809 feat: NCT orchestrator support + Home Assistant webhook
nextcloud_talk.py:
- Fix missing import hmac / import hashlib (NameError bug in _verify_signature)
- Add orchestrator routing when channels.json "tools": true — sends
  " Working on it…" immediately, then runs the full tool loop and
  replies with the result; checkpoint case gets a web UI confirmation note
- Read tier and role from channel config (defaults: default_tier / "chat")
- Pass cfg through to _process_message

homeassistant.py (new):
- POST /webhook/ha/{username}/{webhook_id}
- Auth: webhook_id path segment matched against channels.json
- Accepts JSON or form-encoded body from HA automations
- Builds natural-language task from payload (uses "message" key if present,
  otherwise serialises full body as context)
- Same orchestrator/direct dispatch as NCT
- Delivers response via notify() — NC Talk, web push, or configured channel
- Session key: ha_{username} for continuity across HA events
- Registered in main.py; /webhook/ prefix already public in auth_middleware

channels.json schema addition:
  "homeassistant": {
    "webhook_id": "your-secret-id",
    "persona": "inara",
    "tier": 2,
    "role": "chat",
    "tools": false
  }

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 19:45:59 -04:00
Scott Idem
19d6f004ed feat: reasoning level select (Off/Light/Moderate/High/Max)
Replace free-form reasoning_budget_tokens number input with a 5-level
select in both the edit form (local_llm.py) and add-model form
(local_llm.html). Values: 0 / 1024 / 4096 / 8192 / 32768 tokens.
Edit form pre-selects the stored value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:44:20 -04:00
Scott Idem
a66c5a7f84 feat: reasoning token budget + session name in header
- reasoning_budget_tokens: optional int field on local_openai models;
  when set, injects {"reasoning": {"budget_tokens": N}} via extra_body
  into every OpenRouter API call (both tool-loop and confirmation-gate
  rounds). Field exposed in the model edit form in Settings.

- session name moved from standalone full-row div between #messages
  and #input-area into the persona-switcher block in the header, as a
  third dim line under "Cortex · Local". Collapses when empty via
  :empty CSS. No JS changes required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:35:23 -04:00
Scott Idem
85792a7bcf feat: per-role inject_mode, OTR fixes, hover metadata, send/stop tooltip
- inject_mode: per-role toggle (parallel to inject_datetime) gates the
  "Current mode: Off The Record" line in the system prompt; wired through
  model_registry, context_loader, chat router, orchestrator router, and
  local_llm settings UI

- OTR orchestrator fix: OrchestrateRequest now carries off_record;
  _finalize_job stores it per message and gates log_turn on it; JS
  orchestrate payload sends off_record correctly

- Per-message hover metadata: removed always-visible .model-tag; replaced
  with .msg-meta strip in the action bar (hover-only); shows model label,
  host, fallback indicator, and OTR badge; stored in session JSON

- Send/stop button tooltip: shows role + model and (when tools on)
  separate orchestrator model + engine label; live elapsed timer on stop
  button via startRunTimer/stopRunTimer

- OrchestratorResult.backend_label: new field; openai_orchestrator fills
  it; finalize_job propagates it to job dict and session messages

- GET /backend: exposes orchestrator_model label so the frontend tooltip
  can show both models separately

- TODO: session delete confirmation added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 16:12:03 -04:00
Scott Idem
0afa135ce9 docs: document System block and OTR mode in ARCH__PERSONA.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:40:21 -04:00
Scott Idem
128d8a7c1e feat: inject session mode into persona system prompt
context_loader.load_context() now accepts a mode param ("chat"|"otr").
In OTR mode, the --- System --- block gains a second line:

  Current mode: Off The Record — this conversation is private
  and will not be logged or included in memory distillation

routers/chat.py passes mode="otr" when req.off_record is True.
Normal chat and all orchestrator calls stay at mode="chat" (no change
to the System block). The System block consolidates date/time and mode
in one place, matching the existing timestamp pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:35:09 -04:00
Scott Idem
3a4f518300 docs: add SELF_UPDATE.md — agent self-maintenance bootstrap
Short reference covering: git repo location, Syncthing fleet sync,
ignore files (.gitignore / .stignore), helper scripts (install.py,
dev-restart.sh, backup.sh), standard change workflow, doc update
checklist, pip dependency process, and key paths on the service host.
Linked from MASTER.md document map.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:00:28 -04:00
Scott Idem
348ca120c1 feat: full channels.json UI + http_allowlist settings
Notifications page:
- NC Talk section expanded: url, bot_secret, notification_room,
  nc_username, nc_app_password — all fields from channels.json now editable
- Per-channel sections use <details>/<summary> collapsibles; auto-open
  when values are present
- Secrets use type=password with "leave blank to keep" semantics
- Google Chat outbound webhook in its own collapsible section

Account settings:
- HTTP POST Allowlist section added (same textarea pattern as email allowlist)
- POST /settings/http-allowlist route saves home/{user}/http_allowlist.json
- Example placeholder shows ha.dgrzone.com and n8n patterns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:57:18 -04:00
Scott Idem
7b443b40a4 feat: http_post tool, nc_talk_history tool, local orchestrator retry
- http_post: POST to external URLs with per-user URL prefix allowlist
  (home/{user}/http_allowlist.json); admin-only, confirm-required
- nc_talk_history: read recent NC Talk messages via Basic Auth (requires
  nc_username + nc_app_password in channels.json under nextcloud)
- openai_orchestrator: _chat_with_retry() wraps both API calls with
  exponential backoff (3 attempts, 1s/2s) on connection errors and
  transient status codes (429, 500, 502, 503, 504)
- Docs updated: CLAUDE.md, HELP.md, TODO, MASTER, ROADMAP (50 tools)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:38:38 -04:00
Scott Idem
b9a78819ac docs: add LLM wiki concept (Karpathy pattern) to ARCH__FUTURE.md
Inara's exploration of a living-wiki knowledge compilation architecture
as an alternative to RAG — three-layer model, ingest/query/lint ops,
and a mapping to existing Cortex concepts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:22:55 -04:00
Scott Idem
3672fa1506 docs: comprehensive doc audit — sync all docs to current state
- MASTER.md: tool count 40→47, add proactive notifications + spawn_agent rows, date bump
- ROADMAP.md: mark local orchestrator/web push/proactive notifs/spawn_agent/web_read/session_read as done, date bump
- ARCH__CHANNELS.md: rewrite notification channel config section — all 4 channels, all triggers, on-demand endpoints
- ARCH__SYSTEM.md: update tools/ module list to include files, agents
- README.md: update LLM backends in architecture diagram, add browser push to channels table
- CLAUDE.md: add doc update checklist to Documentation Philosophy section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:13:45 -04:00
Scott Idem
52c19afbcc fix: raise web_read and http_fetch max_chars cap to 128K
Both tools now accept max_chars up to 131072 to accommodate long
documentation pages and large API responses.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:08:17 -04:00
Scott Idem
17e8869d12 docs: update tool count (45→47), HELP.md, and TODO for new web/file tools
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:05:04 -04:00
Scott Idem
7c3291960a feat: web_read (trafilatura), session_read, http_fetch max_chars
web_read(url, max_chars=16000) — fetches a URL and extracts clean article
text via trafilatura, stripping ads/nav/boilerplate. Returns markdown.

session_read(date) — reads a full session log by YYYY-MM-DD date; lists
available dates if the requested one is not found.

http_fetch gains a max_chars param (default 8192, max 32768) so the cap
is configurable instead of hardcoded.

Tool count: 45 → 47.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:04:24 -04:00
Scott Idem
a99ebb8c30 feat: retry button for orchestrator errors + explicit client timeout
Extract orchestrator inner loop into _doOrchestrate() so the retry button
can re-run without re-adding the user message to DOM or history — same
pattern as the existing chat retry.

Also set AsyncOpenAI(timeout=settings.timeout_local) so slow remote models
(OpenRouter/DeepSeek) get the same 300s budget as local chat calls instead
of the SDK default which varies by connection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 12:39:34 -04:00
Scott Idem
ff154b1ec0 docs: update CLAUDE.md, HELP.md, and TODO for notifications page + push fix
- CLAUDE.md: date → 2026-05-08, add Proactive notifications row to channel table
- HELP.md: update Notifications settings entry, expand Push Notifications section
  with channel config link, add test API endpoints to reference table
- TODO__Agents.md: mark notifications dedicated page and pywebpush fix as done

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:58:47 -04:00
Scott Idem
c21f9a23ec fix: use Vapid.from_pem() instead of passing PEM string to webpush()
pywebpush 2.x routes string keys through Vapid.from_string() which only
handles raw/DER base64 — not PEM. Pre-build the Vapid object so the key
deserializes correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:56:17 -04:00
Scott Idem
19475610be feat: move Notifications to its own settings sub-page
Adds GET /settings/notifications (dedicated page with channel form + two
test buttons) and updates POST /settings/notifications to render that page.
Settings page now shows a compact link card instead of the full form.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:43:52 -04:00
Scott Idem
3c7ecf4e4f feat: notification test endpoints — POST /api/push/test and /api/push/reminders/check
- POST /api/push/test: sends "Test notification from Cortex" via the
  user's configured notification channel (web_push / NCT / email / etc.)
- POST /api/push/reminders/check: runs the daily reminder check immediately
  for the current user, returns reminders_found count

Both require an active session cookie. Useful for verifying channel setup
without waiting for the 09:00 scheduler job.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:34:58 -04:00
Scott Idem
64020ad982 feat: proactive notifications — web_push channel + daily reminder check
Routes web_push through notification.py alongside NCT/email/Google Chat,
and fires daily reminder summaries via the scheduler.

- notification.py: _notify_web_push() + "web_push" case in notify();
  all four channels (web_push/email/nextcloud/google_chat) now routable
- scheduler.py: _run_reminder_check() daily at 09:00 — reads due reminders
  per persona via set_context(), formats up to 3 entries, calls notify()
- routers/settings.py: "web_push" added to valid notification_channel values
- static/settings.html: "Browser Push Notification" option in channel selector
- TODO__Agents.md: proactive notifications section marked complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:28:49 -04:00
Scott Idem
47d23a7b2f feat: per-model max_rounds for Gemini orchestrator engine
Mirrors the pattern already in openai_orchestrator.py. The Gemini engine
was still hardcoded to the global orchestrator_max_rounds setting.

- orchestrator_engine.py: max_rounds param on run() and _run_from_contents();
  effective_limit = min(per_model_limit, global_limit); stored in checkpoint
  so resume() respects it across confirmation gates
- routers/orchestrator.py: passes orch_model.get("max_rounds") to run()
- tools/agents.py: passes model_cfg.get("max_rounds") for gemini_api spawns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:54:37 -04:00
Scott Idem
09d775b47b feat: spawn_agent tool + host max_concurrent + docs
Adds a synchronous sub-agent spawning tool that lets the orchestrator
delegate tasks to a specific role's model and tool set.

- cortex/tools/agents.py: spawn_agent(task, role, tier, timeout, max_rounds)
  - Supports local_openai and gemini_api model types
  - Per-host asyncio semaphore (keyed by host_id or model type)
  - asyncio.wait_for() enforces timeout; admin-only tool
- cortex/model_registry.py: max_concurrent field in host schema (default 3,
  clamped 1-20); backfilled on _normalize() for existing hosts
- cortex/routers/local_llm.py + local_llm.html: "Max parallel" number input
  in host add/edit forms
- cortex/tools/__init__.py: spawn_agent registered in TOOL_CATEGORIES["Agents"],
  _CALLABLES, TOOL_ROLES (admin), and _ALL_DECLARATIONS
- Docs: TOOLS.md count 44→45, spawn_agent section; HELP.md tool table updated;
  ARCH__FUTURE.md Round 2 completed items; TODO__Agents.md spawn_agent checked;
  CLAUDE.md tool count and list updated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:48:21 -04:00
Scott Idem
6ad7597db8 feat: per-role inject_datetime toggle for system prompt
Each role can now disable the current date/time header injected into the
system prompt. Default is true (all existing roles unchanged). Useful for
pure processing roles (summarizer, classifier, translator) where temporal
context is irrelevant or could cause unexpected model behavior.

Changes:
- model_registry: set_role_config/get_role_config gain inject_datetime field
- context_loader: load_context gains inject_datetime param (default True)
- orchestrator router: passes inject_datetime from role_cfg to load_context
- local_llm router: reads inject_datetime from POST body, passes to registry;
  role_config_data_js includes the field
- local_llm.html: checkbox in role config panel; populate on open, save on submit

Session logs still timestamp every turn (HH:MM header in YYYY-MM-DD.md files)
regardless of this setting — the toggle only affects the system prompt header.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:53:35 -04:00
Scott Idem
8e512d4e11 feat: reminders due-date support + context filtering
reminders_add now accepts optional due: YYYY-MM-DD parameter.
Due date stored as first line of section body in REMINDERS.md.

context_loader.py calls load_due_reminders() instead of loading REMINDERS.md
wholesale — future-dated reminders are suppressed in the system prompt until
their date arrives. Undated reminders always surface (backward compatible).

reminders_list shows due status per entry: [OVERDUE by N days], [due TODAY],
or [due: YYYY-MM-DD] for future items. All reminders visible via the tool
regardless of date; only context surfacing is filtered.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:46:45 -04:00
Scott Idem
750cde489d feat: session_search tool + tool expansion docs update
session_search (tools/files.py):
- Full-text search across past session logs, exposed to the orchestrator
- Params: query (required), limit (default 5, max 20)
- Returns dated excerpts, newest first; own sessions only via ContextVars
- User-level — no TOOL_ROLES gating needed
- Registered in __init__.py callables + TOOL_CATEGORIES["Files"]

ARCH__FUTURE.md §2: updated tool count to 44, marked prior tools complete,
added Round 2 planned tools table (session_search now done, reminders due dates,
http_post, nc_talk_history, task_list priority filter, http_fetch max_chars),
noted datetime_now is not needed (already in system prompt via context_loader)

TODO__Agents.md: session_search checked off, Round 2 task list added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:41:26 -04:00
Scott Idem
f8f7cd75da feat: audit log, usage tracking UI, OpenAI orchestrator compaction, onboarding + docs
Tool audit log:
- Every orchestrator tool call logged to home/{user}/tool_audit/YYYY-MM-DD.jsonl
- Files panel sidebar: audit log group (collapsed), date-linked read-only table
- Admin endpoints: /api/audit/files, /api/audit/day, /api/audit/recent, /api/audit/stats
- Engine and model name recorded per entry

OpenAI orchestrator improvements:
- Context budget enforcement: 75% of model context_k (min 16k)
- Message compaction: truncates old tool results when approaching budget
- max_rounds respected per model config (intersected with server cap)

OpenRouter onboarding (setup.html, onboarding.py, app.js, settings.html):
- Step 3 of 3: /setup/model with curated model picker
- Chat banner for users on server-default model (informational, not alarmist)
- Settings quick-link card; /setup/model works standalone for existing users

Model registry + session store:
- set_role_config / get_role_config for per-role tool lists and system_append
- session_store: session rename, session name backfill endpoint

UI updates (app.js, index.html, style.css, local_llm.html):
- Role toggle in context panel
- Off-the-record mode
- Agent notes read-only viewer
- OPERATIONS.md loaded at T2+ in context

Documentation:
- HELP.md: full tool table, per-role tool sets, Agent Notes, usage tracking
- TOOLS.md: Agent Notes section, count corrected to 44
- ARCH__SYSTEM.md, ARCH__BACKENDS.md, MASTER.md updated to match reality
- CLAUDE.md: onboarding flow, documentation philosophy sections
- README.md: stack in practice, DeepSeek TUI mention, architecture diagram updated
- TODO__Agents.md: onboarding task completed with deviation notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:26:43 -04:00
Scott Idem
c02d2462b0 feat: agent notes, OpenRouter onboarding, usage tracking, per-role tools docs
Agent notes tool (cortex/tools/agent_notes.py):
- Private durable notepad for the orchestrator — not user-visible
- agent_notes_read/write/append/clear with 3 rolling backups
- Per-persona isolation via ContextVars; no TOOL_ROLES gating needed
- PROTOCOLS.md updated to make this a core proactive tool

OpenRouter guided onboarding:
- Setup Step 3 (/setup/model) — OpenRouter quick-connect with curated model list
- Amber banner in chat for users on server-default model
- Settings quick-link card (/settings/models OpenRouter section)
- POST /setup/model/skip for users who want to bypass Step 3
- Holly pre-configured: DeepSeek V4 Flash (OpenRouter) → Gemma Medium (local) → claude_cli

Usage tracking:
- cortex/routers/usage.py — GET /api/usage, /api/usage/summary, /api/usage/all (admin)

Documentation:
- HELP.md: Tools section rewritten — full tool table by category, per-role tool sets explained
- TOOLS.md: Agent Notes section added; count corrected to 44
- ARCH__SYSTEM.md, ARCH__BACKENDS.md, MASTER.md, CLAUDE.md, README.md updated
- TODO__Agents.md: onboarding task checked off with deviation notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:25:31 -04:00
Scott Idem
5d4f5ee598 feat: OPERATIONS.md bootstrap doc + load at T2+; patch stale persona files
- New home/scott/persona/inara/OPERATIONS.md: self-maintenance workflow
  (cortex_update → review → cortex_restart), access control table, key
  paths, memory file map, distillation cadence, channel/architecture notes
- context_loader.py: load OPERATIONS.md at Tier 2+ after PROTOCOLS.md
- TOOLS.md: count 39→40, add web_push to Notifications section
- PROTOCOLS.md: replace stale 10-tool list with reference to TOOLS.md
- CONTEXT_TIERS.md: fix memory file names (MEMORY.md → LONG/MID/SHORT),
  update Tier 2 load list, fix Hard Rules credentials note

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 22:21:03 -04:00
Scott Idem
a75546485b feat: context budget enforcement + compaction in OpenAI orchestrator
Protects all models in the Primary/Backup chain regardless of context window:
- _context_budget(): 75% of model_cfg["context_k"] * 1000 (default 32k if unset)
- _estimate_tokens(): char count / 4 + 3k overhead for tool schemas
- _compact_messages(): truncates old tool results to 400 chars, keeps last 6
  intact (~2 recent rounds), logs chars saved per compaction pass
- Compaction runs before every API call; log line now shows estimated token count
- Malformed tool call args logged with model/args detail instead of silent {}
- finish_reason check accepts "stop" and None alongside "tool_calls" (some
  models return wrong reason even when tool_calls are present)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 22:01:54 -04:00
Scott Idem
7d221863dc feat: engine/model in audit log + docs update
- tool_audit: ContextVars (engine, model) set at orchestrator run start; fields added to every entry
- orchestrator_engine: tool_audit.set_context("gemini", model_name) at run() start
- openai_orchestrator: tool_audit.set_context("openai", model label) at run() start
- audit table: Model column between Status and Args
- HELP.md: push notifications section, audit log in Files section, tool count 30→40, new API endpoints
- TODO__Agents.md: web_push and audit log marked complete with full detail

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:42:32 -04:00
Scott Idem
02accefe8f feat: audit log in Files panel sidebar
Adds an "Audit Log" section (collapsed by default) at the bottom of the Files
panel showing tool_audit/YYYY-MM-DD.jsonl files for the current user.

- GET /api/audit/files  — lists available dates (newest first, any auth user)
- GET /api/audit/day    — returns entries for one date as JSON (any auth user)
- tool_audit.read_day() — reads a single day's JSONL file chronologically
- Clicking a date renders a read-only table: time / tool / status / args / result
- Status cells are colour-coded (green ok, red error, amber denied)
- Edit/Raw/Preview/Save buttons are hidden in audit view, restored on file switch
- Audit group starts collapsed; expands on click like other file groups

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:36:08 -04:00
Scott Idem
584ae679a6 feat: tool call audit log
Every orchestrator tool invocation is recorded to home/{user}/tool_audit/YYYY-MM-DD.jsonl.
Each entry captures: timestamp, user, tool, args (truncated), status (ok/error/denied),
result length, and a 300-char result snippet.

- tool_audit.py: JSONL writer with per-file asyncio locks; read_recent / read_recent_all_users helpers
- tools/__init__.py: hook in call_tool() — fire-and-forget record on every dispatch
- routers/audit.py: GET /api/audit/recent and /api/audit/stats (admin-only)
- tools/files.py: add home_root() to file_read allowed roots so agents can read audit JSONL

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:55:59 -04:00
Scott Idem
ddf44a2aee feat: web push notifications (VAPID)
- push_utils.py: subscription storage + send helper (auto-prunes 410 endpoints)
- routers/push.py: GET /api/push/vapid-key (public), POST/DELETE /api/push/subscribe
- sw.js: push event listener shows notification; notificationclick focuses/opens tab
- app.js: subscribe/unsubscribe flow + "Enable notifications" toggle in settings dropdown
- tools/notify.py: web_push orchestrator tool (user-level, no admin required)
- VAPID keys in .env; pywebpush added to requirements.txt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:38:58 -04:00
Scott Idem
0b96772fa6 fix: show session friendly name in resume message and status bar
/history/{session_id} now returns a 'name' field alongside messages.
resumeSession() uses data.name first, then the sessionNames map, then
raw ID as fallback — so named sessions display correctly even on page
load before the sessions panel has been opened.

'Resumed session X' message also now shows the friendly name.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:14:59 -04:00
Scott Idem
5d23d04e7e fix: session panel wider + two-line layout for session names
Root cause: 300px panel minus edit btn (28px) + meta (~130px) + delete
btn (28px) + gaps/padding left only ~70px (~7 chars) for the session name.

- Panel: 300px → 420px desktop, 300px → 380px mobile drawer
- Max-height: 340px → 400px
- Session item: name and meta now in a .session-body flex column, so the
  name gets full body width (panel minus two buttons) — meta lives below
- Edit mode: hides .session-body + delete, input takes the full body slot

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:07:33 -04:00
Scott Idem
7a0fbdb659 feat: session rename UX overhaul
- Edit button (✎) moved to left of row, separated from delete (×)
- Clicking ✎ hides name/meta/delete and expands input to full row width
- Button changes to ✓ (accent color) while editing
- Enter or ✓ click = save; Escape = cancel without saving
- Removed accidental-save-on-blur behavior
- Edit button: 30% opacity at rest, 75% on row hover, 100% on direct hover
- Touch devices: edit button always at 60% opacity (no hover to reveal it)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:00:39 -04:00
Scott Idem
508fb638ad feat: distill safeguards — rolling backups + sanity checks
Before any memory file is overwritten, _rotate_backup() keeps 2 rolling
backups: MEMORY_*.bak1.md (most recent) and MEMORY_*.bak2.md (older).

_sanity_check() now also guards against size anomalies: the new content
must be between 40% and 250% of the old file size — anything outside that
range looks like truncation or runaway output and aborts the write.
Existing checks (min length, refusal phrases) still apply.

Backup files exposed in the Files panel (ALLOWED set) so they can be
reviewed and manually restored if needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:54:27 -04:00
Scott Idem
0ffcd57c95 fix: multi-user distillation + datetime in context + session log labels
Distillation was silently operating on scott/inara for all users due to
ContextVar defaults. All three distill endpoints now require ?user=&persona=
query params and validate them via persona.validate(). Memory distiller
signatures changed from Optional to required positional args — no more
global settings fallback. Scheduler now iterates all users/personas instead
of hardcoding the primary user.

- context_loader: inject current date/time as first system prompt section
- session_logger: use get_user()/get_persona() from context instead of
  settings globals so Holly/Brian sessions show correct speaker labels
- memory_distiller: system prompts now reference u.title()/p.title()
  instead of settings.user_name/settings.agent_name
- distill router: Query(...) enforces params; _resolve() validates persona
- scheduler: _all_personas() helper iterates every user/persona for distill
- app.js: runDistill() now appends ?user=&persona= via _fileParams

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:44:51 -04:00
Scott Idem
8d4aa4094c feat: usage tracking + knowledge import script
- usage_tracker.py: daily token/call buckets per user (home/{user}/usage.json)
- Hook into local backend (OpenAI usage field) and Gemini API (usage_metadata)
- Claude/Gemini CLI backends produce no structured token data and are not tracked
- Fix CLAUDE.md stale tool count (27 → 39) and refresh tool list
- scripts/import_knowledge.py: walk markdown dirs, chunk by H2, call local LLM
  for summaries, create AE journal entries with path-derived tags; resumable via
  state file; --dry-run and --limit flags for safe testing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 20:38:31 -04:00
Scott Idem
eab92d876d refactor: split tool declarations into domain files + role config UI
tools/__init__.py shrinks from 1,137 → 250 lines. Each domain file now
owns both its callables and its FunctionDeclarations (DECLARATIONS list),
so adding a new tool only touches one file.

New TOOL_CATEGORIES dict exported from __init__ — used by the UI for
grouped tool checkboxes.

Role config UI (Settings → Model Registry → Role Assignments):
- ⚙ button per role expands an inline configure panel
- Textarea for system_append (injected into system prompt for this role)
- Grouped checkboxes for tool allow-list (all checked = no restriction)
- POST /api/models/role-config saves both fields; updates ROLE_CONFIG_DATA
  in-page so re-open reflects current state without a page reload

Backend:
- model_registry.set_role_config() writes system_append + tools to registry
- TOOL_CATEGORIES exported from tools/__init__ for UI rendering
- TOOLS.md header updated: 30 → 39 tools (ae_journal_* and cortex_* additions)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 20:40:50 -04:00
Scott Idem
49123cdd5c feat: per-role tool lists and system prompt overlays
Each role in model_registry.json can now carry two optional keys:
  system_append — injected into the system prompt at position 7 (after
                  memory, closest to the turn) for the active chat_role
  tools         — explicit tool allow-list; intersected with the user's
                  access-level filter so it can only restrict, never elevate

No changes needed for existing users — missing keys fall back to current
behavior. Add keys to a role to give it a specialty focus:

  "coder": {
    "primary": "claude_cli",
    "system_append": "You are in code-specialist mode...",
    "tools": ["web_search", "file_read", "shell_exec", "scratch_write"]
  }

Changes:
- model_registry.py: get_role_config() returns system_append + tools
- context_loader.py: role_append param appended as "--- Role Context ---"
- tools/__init__.py: get_tools_for_role/get_openai_tools_for_role accept
  optional tool_list and intersect with access-level filter
- orchestrator_engine.py: tool_list threaded through run/resume/checkpoint
- openai_orchestrator.py: tool_list threaded through run/resume/checkpoint;
  _build_client now calls get_openai_tools_for_role instead of returning
  unfiltered OPENAI_TOOL_SCHEMAS
- routers/orchestrator.py: pulls role_cfg for chat_role, passes both
  role_append and tool_list to context loader and engine

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 20:00:38 -04:00
Scott Idem
5ad2e50d69 feat: split help into tabbed UI Guide / Tools / Persona pages
- cortex/static/TOOLS.md — tool reference extracted from HELP.md; uses ##
  headers so each category is collapsible. All 30 tools with descriptions.
- cortex/static/HELP.md — UI guide only; tools section replaced with a
  one-line pointer to the Tools tab.
- help.html — three tabs (UI Guide / Tools / Persona); tab choice persists
  in localStorage. Tools tab defaults all sections open. Persona tab shows
  home/{user}/persona/{name}/HELP.md with an empty-state message if unset.
- context_loader.py — loads cortex/static/TOOLS.md into context at tier 2+
  (replaces the previously empty persona HELP.md load). Persona HELP.md
  still loaded if non-empty, as persona-specific additions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:25:47 -04:00
Scott Idem
552fd56abb docs: expand tools section in HELP.md with per-tool descriptions
Replaces single flat table with category sections, each with a description
column. Footnotes moved to the top of the section for clarity. Covers all
30 tools including the new cortex_status, cortex_update, reminders_remove,
ae_journal_entry_read, ae_journal_entries_list, and email_send.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:11:22 -04:00
Scott Idem
77997bc4ae feat: add cortex_status and cortex_update tools
cortex_status: git branch/commit/ahead-behind + systemctl state — read-only
cortex_update: git pull + syntax check all .py files + report; does NOT auto-restart.
  If syntax errors are found after pull, warns and blocks restart suggestion.
  Call cortex_restart separately to apply a clean update.

Both are admin-only. cortex_update is confirm-required (modifies files on disk).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:01:42 -04:00
Scott Idem
1ffa846edd docs: sync HELP.md tools table and files list with current implementation
- Add reminders_remove (targeted single-reminder removal, no confirm needed)
- Add ae_journal_entry_read, ae_journal_entries_list to AE Journals row
- Add email_send (admin-only) to Notifications row
- Remove TASKS.json from Files table (not in the Files panel)
- Add email_allowlist.json to Files table (Settings group in Files panel)
- Update last-updated date

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:54:50 -04:00
Scott Idem
98546abe21 docs: update ARCH__AE_INTEGRATION with verified API behavior
- query_string required for and/or filters to apply; use "%" as wildcard
- Total count is in meta.data_list_count, not top-level
- id_random is None in responses; Vision ID convention uses {obj_type}_id
- tags comes back as string on read, not list — normalize before joining
- Replace stale "Planned: Search Improvements" with current signature + notes
- Clarify date_to boundary (lte midnight, use next day to include full day)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:17:19 -04:00
Scott Idem
1fa5151d8a fix: correct V3 search filter key and response field names in ae_knowledge
- Filter key is "and" not "and_filters" (V3 API format)
- Entry IDs use journal_entry_id/id, not id_random (id_random is None)
- Dates use updated_on/created_on, not updated_at/created_at
- Total count lives in meta.data_list_count, not top-level total/count
- Inject query_string="%" when and filters present but no query, since
  the V3 search engine requires query_string for filters to apply
- Normalize tags from string to list in both entry_read and entries_list
- Fix order_by to use updated_on (not updated_at) in entries_list
- Correct ARCH__AE_INTEGRATION.md: and_filters → and, or_filters → or

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:12:44 -04:00
Scott Idem
71e472bebe feat: improved ae_journal_search + AE integration docs
Search improvements:
- Switched from LIKE on default_qry_str to query_string path (fulltext
  MATCH/AGAINST IN BOOLEAN MODE — uses the index, supports +/- boolean ops)
- Added tag filter (icontains on tags field)
- Added date_from / date_to filters (created_on gte/lte)
- Added type_code / topic_code exact-match filters
- Added sort_by / sort_order control (updated, created, name, priority)
- Added status / priority filters
- Added page parameter for pagination
- Richer output: updated date, tags, pagination hint
- Updated Gemini tool declaration with all new params

Docs:
- documentation/ARCH__AE_INTEGRATION.md — journal_entry full schema,
  search operator reference, current tool inventory, planned phases
  (broader AE integration: tasks, people, calendar, knowledge import)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 20:10:04 -04:00
Scott Idem
77327d97ad feat: improve AE Journal read toolset
- ae_journal_entry_read: expose full entry content by id_random (title,
  journal, tags, summary, full content with configurable truncation)
- ae_journal_entries_list: browse all entries in a journal newest-first,
  numbered with id/title/tags/summary/date and pagination support
- ae_journal_search: richer output — tags, updated date, 400-char preview
  (was 200), show summary OR preview (not both when summary exists)

_get_entry() was already implemented; read tool just exposes it properly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:47:59 -04:00
Scott Idem
36fdda6728 feat: add reminders_remove tool for single-reminder removal
- reminders_remove(index) removes one reminder by 1-based index
- reminders_list now returns numbered output (1. heading / body)
  so any model can easily identify which index to pass
- _parse_sections() / _sections_to_text() helpers for clean round-trip
- Not in CONFIRM_REQUIRED — targeted removal is safe without a gate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:27:53 -04:00
Scott Idem
6405dd338d feat: proper confirmation-resume flow + per-user tool policy
Fixes the broken confirmation gate where users had no way to approve
or deny a blocked tool call in the web UI.

Changes:
- orchestrator_engine.py: add OrchestrateCheckpoint dataclass, extract
  loop into _run_from_contents(), add resume() function
- openai_orchestrator.py: same treatment — _run_from_messages(), resume()
- routers/orchestrator.py: POST /{job_id}/confirm and /deny endpoints,
  separate _checkpoints store, _resume_job() + _finalize_job() helpers,
  "awaiting_confirmation" job status with pending_confirmation payload
- auth_utils.py: get_tool_policy() and save_tool_policy() helpers reading
  home/{user}/tool_policy.json (allow/deny lists)
- routers/orchestrator.py: loads tool_policy per user and passes
  confirm_allow/confirm_deny to both engines
- app.js: poll loop handles awaiting_confirmation — shows Confirm/Deny
  buttons inline, resumes polling after user action
- settings.html + settings.py: Tool Permissions section with allow/deny
  textareas, POST /settings/tool-policy route
- style.css: .confirm-gate, .confirm-btn, .deny-btn styles

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:14:53 -04:00
Scott Idem
bce7de647c feat: proactive notifications — email, NC Talk, Google Chat per user
notification.py now handles all three outbound channels. Email defaults
to the user's login address (google_email from auth.json); an optional
override can be set in channels.json. Google Chat uses an incoming
webhook URL. NC Talk was already wired, just needs notification_room set.

Settings page gains a Notifications section: channel dropdown, optional
email override, NC room token, and Google Chat webhook URL. All stored
in per-user channels.json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 22:32:22 -04:00
Scott Idem
165cf3552d docs: update TODO and ROADMAP for 2026-04-29 session
Mark completed: email_send + allowlist UI, model inline edit, cross-session
search. Remove completed items from active lists (distill review, model edit,
session search). Update ROADMAP Phase 3 tool suite line and last-updated date.
Remove resolved Unsloth entry from Deferred.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 22:03:50 -04:00
Scott Idem
db3dd465b2 feat: email allowlist management in Settings + Files panel
Settings page gets an editable textarea (POST /settings/email-allowlist)
so users can view and update their per-user regex allowlist without
touching the raw JSON file.

Files panel gains a "Settings" group containing email_allowlist.json as
a raw JSON editor backup — served from home/{user}/ via files.py USER_FILES.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:56:45 -04:00
Scott Idem
e0e3170de3 feat: regex support in email allowlist
Each entry in email_allowlist.json is treated as a re.fullmatch pattern
(case-insensitive). Allows domain wildcards, plus-addressing, and any
variation expressible as a regex. Invalid patterns are logged and skipped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:43:38 -04:00
Scott Idem
b8bc4ea21f feat: email_send allowlist — block sends to non-whitelisted addresses
Reads home/{username}/email_allowlist.json (JSON array of addresses).
Fails safe: if file is missing or address not listed, send is blocked with
an informative message. home/ is gitignored; create the file manually per user.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:40:10 -04:00
Scott Idem
fd0fb76c08 feat: add email_send orchestrator tool
Wraps the existing email_utils.send_email helper as an admin-only tool.
Accepts to, subject, body (plain text); newlines converted to <br> for HTML part.
Registered in _CALLABLES, _ALL_DECLARATIONS, and TOOL_ROLES (admin).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:35:29 -04:00
Scott Idem
a5658eb3c4 feat: edit existing model entries in the Model Registry
- Inline edit form per model row (label, model name/ID, host/account, context, tags)
- Fetch models button in edit form for local models — same live-picker UX as Add Model
- POST /settings/local/models/{id}/edit route in local_llm.py
- Admin role badge (ADMIN/USER pill) in Account Settings page
- HELP.md updated: new tools table with admin/confirm markers, PWA install section
- TODO updated: tool expansions marked done, distill review and Unsloth resolved,
  role-based access and admin badge added to completed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:08:09 -04:00
Scott Idem
334e7f0dea feat: role-based tool access, confirmation gates, and new orchestrator tools
- auth_utils: get_user_role() reads role from auth.json (admin|user, default user)
- manage_passwords: new `role` command to promote/demote users (admin-only by convention)
- tools/__init__: TOOL_ROLES map, CONFIRM_REQUIRED set, get_tools_for_role(),
  get_openai_tools_for_role() — both orchestrators now filter tools by caller's role
- tools/system: cortex_restart (detached subprocess, 5s delay), cortex_logs (admin-only)
- tools/web: http_fetch — direct URL fetch, distinct from web_search
- tools/files: file_list (directory listing), file_write (restricted paths, admin-only)
- tools/notify: nc_talk_send — proactive outbound via notification.py
- orchestrator_engine + openai_orchestrator: user_role param; CONFIRM_REQUIRED tools
  return a confirmation-request result instead of executing — loop breaks after Claude
  asks user to confirm in a follow-up message
- home/scott/auth.json: role set to admin

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:23:53 -04:00
Scott Idem
1603ad5124 docs: sync TODO and ARCH__FUTURE — local orchestrator status, new tools, fleet/mesh plans
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:05:11 -04:00
Scott Idem
25182a1765 feat: PWA support — manifest, service worker, icons, public auth exemption
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:46:33 -04:00
Scott Idem
f726d78979 docs: add five new feature items to TODO (PWA, proactive notifications, attachments, search, usage)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:23:59 -04:00
Scott Idem
217c7c3d6a feat: CodeMirror markdown editor for identity/memory file editor
Replace plain textarea with CodeMirror 5 + markdown mode loaded from
jsDelivr CDN. Editor fills the modal body via flex layout, theme-aware
via CSS vars (cursor, selection, headings, bold/em/links/code all mapped
to Cortex dark/light palette). Lazy init on first file open; history
cleared per-file so undo doesn't bleed across files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:05:57 -04:00
Scott Idem
66cb197de0 feat: last-used persona cookie, emoji dropdown, theme support, auth status move
- cx_last_persona cookie set on serve_ui; root/login/help/settings
  redirects use preferred persona from cookie instead of alphabetically first
- /api/personas returns [{name, emoji}] objects; persona switcher dropdown
  renders emoji + name with flex layout and .pd-emoji span
- Help, Settings, Model Registry pages apply localStorage theme on load
  (no flash); CSS variables for dark/light replacing all hardcoded hex values
- Claude CLI auth status moved from prominent chat banner to Anthropic
  provider block in Model Registry — live dot indicator (ok/warn/err)
- Auth banner removed from main chat UI (index.html, app.js, style.css)
- Add Model collapsed into Models section as <details> to shorten page
- Light-mode overrides for provider icons, model badges, ctx-badge, tags
  (Anthropic/Google/local colors now readable in both themes)
- Help page gains table, pre/code, hr styles for HELP.md rendered content

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 22:52:34 -04:00
Scott Idem
1222f806ce docs: sync all documentation to current state (2026-04-28)
HELP.md:
- Replace stale "Agent Mode" section with accurate "Tools ()" section
  including full tool table (27 tools across 9 categories)
- Fix header controls table: distinguish Context & Memory panel (sliders)
  from Settings dropdown (☰)
- Update Settings panel section: "Backend" → "Role", add S/M/L and ⌃↵
  to Display controls list
- Update Backends section to match Role toggle terminology

TODO__Agents.md:
- Mark 5 new journal tools as complete (ae_journal_list, entry_update,
  entry_disable, entry_append, entry_prepend)
- Add completed sections for: shell_exec tool, Tools toggle decoupling,
  UI input area polish (2026-04-28)

MASTER.md:
- Date: 2026-04-27 → 2026-04-28
- Orchestrator row: "Agent mode in UI" → " Tools toggle in UI (27 tools)"

CLAUDE.md:
- Date: 2026-04-03 → 2026-04-28
- Add orchestrator row to Current State table
- Add full 27-tool list for quick reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 22:07:48 -04:00
Scott Idem
ed191cf0b4 feat: add journal entry update, disable, append, prepend tools
Four new tools for full journal entry lifecycle management:

- ae_journal_entry_update  — PATCH any combination of fields (title,
  content, summary, tags, enable); only provided fields are changed
- ae_journal_entry_disable — soft-delete via enable=false
- ae_journal_entry_append  — fetch entry, append timestamped section
  to the bottom (ideal for running logs / data logs)
- ae_journal_entry_prepend — fetch entry, prepend timestamped section
  to the top (most-recent-first pattern)

Shared _get_entry / _patch_entry helpers keep the read-modify-write
logic DRY. Also fixed journal_entry_create to prefer the canonical
journal_entry_id field over the legacy id_random alias.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 22:02:22 -04:00
Scott Idem
44f215c764 feat: add ae_journal_list tool
Lists all Aether Journals for the configured account via
POST /v3/crud/journal/search with no filters (account scoped by header).
Returns name + id_random for each journal so the agent can discover
available journals before searching or writing entries.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:50:02 -04:00
Scott Idem
d61e39d614 feat: S/M/L height drives mode-select row vs column layout
When height is set to S, mode-select collapses to a row (mode button +
compact tools toggle side by side). M and L keep the vertical column
layout where each control gets its own full-width row. Driven by
data-size attribute set in JS so the switch is instant on click, not
reliant on a viewport media query. Removed the redundant max-height
landscape query.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:20:28 -04:00
Scott Idem
93a692f3f0 feat: vertical mode-select column on desktop
Stack Chat/Note/OTR button and tools toggle vertically (flex-direction:
column, align-items: stretch) on desktop so they share a tidy left column.
Mobile (≤520px) restores row layout; landscape phone (≤400px height) also
reverts to row to avoid crowding a short viewport.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:14:16 -04:00
Scott Idem
af4d78136a fix: textarea height setting now visibly changes empty-state size
The previous approach used a 600ms preview animation + syncHeight() which
collapsed the textarea back to 1 line (empty scrollHeight). Now syncHeight
enforces minHeight = maxHeight/3, so each setting (S/M/L) has a visibly
distinct resting height even when the input is empty.

  S (120px): min ~40px  ≈ 1-2 lines at rest
  M (240px): min ~80px  ≈ 3 lines at rest
  L (480px): min ~160px ≈ 5-6 lines at rest

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:08:46 -04:00
Scott Idem
af7d8b40e2 feat: single cycling height button, panel mutual exclusion, consistent shadows
- Replace 3 S/M/L height buttons with one cycling button (like font size)
- Fix closeAllPanels() to include ctx-panel so Context and Settings menus
  cannot be open simultaneously
- Fix ctxOpenBtn handler to use the same toggle-via-closeAllPanels pattern
  as the settings button
- Align .hdr-dropdown shadow to var(--shadow) instead of hardcoded rgba
- Align #ctx-panel z-index to 200 (match .hdr-dropdown)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:03:56 -04:00
Scott Idem
4159f470d6 fix: context panel polish — height buttons, amber theme vars, label cleanup
- Replace height <select> with S/M/L buttons (data-height); active class shows
  current setting; clicking an empty textarea briefly expands it as a preview
  so the effect is immediately visible, then auto-shrinks back
- Add --amber/--amber-border/--amber-glow CSS vars to all 4 theme blocks:
  dark=#f59e0b (bright), light=#b45309 (deep, 4:1 contrast on light bg)
  Fixes local-on/tools-toggle/backend-hint being nearly invisible in light mode
- Rename "Backend" ctx-section to "Role" (matches the role-cycle toggle)
- Update backend-toggle title from stale "primary backend" to "Active role"
- Capitalize distill buttons (Short/Mid/Long/All) to match Memory layer style
- Improve all ctx-panel button titles for clarity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:53:16 -04:00
Scott Idem
e2a61bb78d fix: mode-select row layout so tools toggle doesn't push textarea down
- #mode-select changed from flex column to flex row (desktop + mobile unified)
- Chat/ buttons now sit side-by-side at the same height as the textarea
- Removed stale mode-agent CSS rules (mode removed in prior commit)
- Mobile: simplified override — flex:1 only, direction/align already desktop default

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:44:46 -04:00
Scott Idem
80702a21e2 fix: add distinct off/on styles for tools toggle button
OFF: very dim (nearly invisible) — makes it clear tools are inactive
ON: amber with glow — matches local-on pattern, clearly active

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:38:38 -04:00
Scott Idem
2b9dd53566 feat: replace Agent mode with independent Tools toggle
- Remove 'agent' from mode dropdown; Chat/Note/OTR remain
- Add  tools toggle button in input bar (persisted in localStorage)
  When on: routes to POST /orchestrate (Gemini tool loop); send btn → "Run"
  When off: routes to POST /chat (direct to active role); no change
- Role selector and tools toggle are now fully independent:
  active chat_role sent in orchestrate payload → used for final response
- orchestrator_engine.run() accepts response_role param; passes it to
  complete(role=...) instead of hardcoded model="claude"
- OrchestrateRequest gains chat_role field (default "chat")
- Migrate stored 'agent' mode/MRU entries to 'chat' on load

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:36:15 -04:00
Scott Idem
1cc7988953 feat: add shell_exec tool and fix orchestrator model name resolution
- Add shell_exec to orchestrator tool suite (system.py + __init__.py)
  Runs arbitrary shell commands on the Cortex host with timeout (1–120s),
  combined stdout/stderr output, optional working_dir, and exit code reporting.
  Enables system diagnostics (df, ls, ps, journalctl, etc.) from Agent mode.

- Fix orchestrator_engine.run() to use model_name from resolved registry entry
  Previously used settings.orchestrator_model (.env hardcode) regardless of
  what model was assigned to the orchestrator role. Now accepts model_name param
  and falls back to settings value only when registry has no model_name.

- Update ARCH__FUTURE.md: date, running host, local orchestrator status,
  model registry V2 progress, added Cortex Mesh concept (section 9)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:29:46 -04:00
Scott Idem
8baab874f1 feat: replace backend/slot toggle with role selector
The backend toggle now cycles through configured roles (chat, coder,
research, distill, etc.) instead of backup model slots within the chat
role. Each role uses its own primary→backup chain from the registry.

- ChatRequest.slot replaced by chat_role (default "chat")
- GET /backend returns available_roles instead of chat_models
- _available_roles_for_toggle() builds list from defined_roles, excluding
  orchestrator (which has its own Agent mode)
- Model label on responses now reflects the actual role's assigned model
- Toggle is inert when only one role is configured (avoids useless cycling)
- Add "Clear browser cache" button to Account Settings (Connected Accounts)
- Add _role_model_label() helper for cleaner response tag labeling

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 19:23:18 -04:00
Scott Idem
962d58d2e2 feat: model registry Phase 3 — slot-based backend toggle
Backend toggle now cycles through chat role models by label instead of
cycling service type strings (auto/claude/gemini/local).

- model_registry: get_model_for_slot() — resolves a specific priority
  slot without walking the fallback chain
- llm_client: complete() gains slot param; explicit slot selection
  dispatches directly to that model with no silent fallback
- routers/chat.py: ChatRequest.slot; GET /backend returns chat_models
  [{slot, label, type}] for the UI; _stream_chat uses resolved model
  label for the response tag when a slot is pinned
- app.js: toggle loads chat_models from /backend, cycles by label,
  sends slot in chat payload; legacy model field removed from payload
- app.js: fix Gap B — agent mode placeholder no longer says "Gemini
  tool loop"; now says "orchestrator"
- DESIGN doc: updated to reflect phases 1+2 complete, catalog-as-code
  decision, Gap A/B documented, Phase 3 implementation details

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:43:08 -04:00
Scott Idem
3bc6b45f9f fix: update Anthropic model catalog — add previous versions, fix context sizes
- Add Claude Opus 4.6 and Sonnet 4.5 (previous versions, still available)
- Fix context_k for Opus 4.7 and Sonnet 4.6: both have 1M context (was 200)
- Haiku 4.5 context_k 200 is correct

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:16:22 -04:00
Scott Idem
ef07596955 fix: update Google model catalog to current models (April 2026)
Remove outdated gemini-2.0-flash and gemini-1.5-pro.
Add gemini-2.5-flash-lite (GA) and the three Gemini 3.x preview
models (gemini-3.1-pro-preview, gemini-3-flash-preview,
gemini-3.1-flash-lite-preview). All have 1M token context windows.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:14:23 -04:00
Scott Idem
6e56024815 fix: settings page and help docs updated for model registry V2
settings.html:
- Remove Gemini API Key section (keys now managed in Model Registry)
- Rename "Local Models" → "Model Registry" with updated description
  covering all providers (Anthropic, Google, local hosts)
- Update button text: "Manage local models" → "Manage models"

settings.py: remove dead gemini_key template variable lookups

HELP.md:
- Fix navigation path: ☰ → Account → Model Registry → Manage models
- Restructure Model Registry section as ordered steps (1: providers/hosts,
  2: add models, 3: assign roles) so dependency order is clear
- Add explicit note that accounts/hosts must exist before adding models

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:07:05 -04:00
Scott Idem
9f6b162fbd docs: update end-user HELP.md for model registry V2
- Backends section: add local as third backend option, explain model
  tag on responses, clarify auto vs explicit toggle behavior
- Agent Mode: remove hard-coded "Gemini" reference — orchestrator model
  is now configurable via role assignments
- New Model Registry section: step-by-step for adding Google accounts,
  local hosts, cloud/local model entries, and role assignments
- API reference: add local to model field, add /settings/models endpoint
- Remove outdated In Progress section (local backend + multi-user shipped)
- Header controls table: update Backend description

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 20:57:05 -04:00
Scott Idem
f08b033d6c feat: model registry Phase 2 — cloud provider UI (Anthropic + Google)
Adds cloud provider management to /settings/models:
- Google Accounts section: add/remove Gemini API keys with labels
- Add Model form: provider tabs (Local / Google / Anthropic) with
  catalog dropdowns that auto-fill label and context_k
- Provider badges on model rows (Anthropic / Google / Local)
- /settings/local now redirects to /settings/models (canonical URL)
- save_cloud_model() in model_registry for Anthropic/Google entries
- Distill role migration restored in _migrate_from_local_llm
- Test fixes: version assertions updated to V2

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 20:41:06 -04:00
Scott Idem
45c95d20ba feat: model registry V2 — provider-aware schema with multi-account support
Adds a providers section to the per-user model registry for Anthropic and
Google as first-class providers alongside local hosts. Google accounts
(API keys) are now stored as a list so multiple Google accounts can coexist.

Changes:
- model_registry.py: V2 schema, auto migration V1→V2 (pulls gemini_api_key
  from auth.json into providers.google.accounts), _resolve_model() merges
  account API key for gemini_api type models
- routers/orchestrator.py: uses model-resolved api_key when orchestrator
  role resolves to a gemini_api model with account_id
- ANTHROPIC_CATALOG and GOOGLE_CATALOG constants for model picker (Phase 2)
- New functions: get_google_api_key(), save/remove_google_account(), get_catalog()
- Documentation: ARCH__BACKENDS.md updated to V2 schema, DESIGN doc added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 20:21:04 -04:00
Scott Idem
27ca7c7efd fix: apply host_type path correction in OpenAI orchestrator
The AsyncOpenAI client always appends /chat/completions to base_url.
Open WebUI's endpoint is at /api/chat/completions, so for openwebui
host_type the base_url must include the /api prefix — same logic as
_local() in llm_client.py.

Also strip non-standard metadata fields (backend, host, etc.) from
session_messages before passing them to the API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 23:16:33 -04:00
Scott Idem
869a596f4b fix: don't fall back when a model is explicitly configured in registry
Previously any backend error would silently fall back to Claude.
Now if the user has a model configured via the model registry, errors
propagate to the UI so the actual problem is visible rather than hidden
behind a transparent backend switch.

Fallback still applies when using the default/auto backend with no
registry config.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:30:55 -04:00
Scott Idem
3b3456600a feat: store and display backend + host metadata on chat messages
Each assistant message in the session JSON now carries:
  backend, backend_label, host (platform.node())

These fields are shown as model tags in the UI — on live responses and
when loading session history. Session log entries (sessions/YYYY-MM-DD.md)
include the backend label and host in the turn header.

The local (OpenAI-compat) backend strips non-standard fields before
sending messages to the API so extra fields don't leak upstream.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:16:48 -04:00
Scott Idem
6c84d6ae72 docs: update README and CLAUDE.md for install script and backup
- README: add Setup/Install section (install.py), Development Workflow
  (dev-restart.sh), and Backup (restic/backup.sh) sections; fix
  .env.example reference → .env.default
- CLAUDE.md: replace stale Docker start command with install.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 21:36:48 -04:00
Scott Idem
2629983452 fix: create cortex config dir and restic password file during install
Previously the ~/.config/cortex/ directory and restic-password were only
created on the first backup run. Now install.py creates them eagerly so
the user can verify setup and back up the password immediately.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 21:24:08 -04:00
Scott Idem
f668826f79 fix: bootstrap pip via ensurepip if missing from venv
On Arch Linux, venvs can be created without pip seeded in.
Detect the missing module before attempting pip install and
recover with `python -m ensurepip --upgrade`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 21:17:52 -04:00
Scott Idem
576f22216a feat: restic backup of home/ with systemd daily timer
- backup.sh — backs up home/ (persona data, memory, tasks, crons) to a
  local restic repo; auto-generates password on first run, prunes to
  7d/4w/6m retention; excludes sessions/ and session_data/
- install.py — setup_backup_timer() installs cortex-backup.service +
  cortex-backup.timer (daily 03:00, Persistent=true); skips gracefully
  if restic is not installed

Password lives at ~/.config/cortex/restic-password (chmod 600, not in git).
Repo defaults to ~/backups/cortex-home-restic; override via RESTIC_REPOSITORY.

Per-user encrypted backup is a noted future feature.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:36:22 -04:00
Scott Idem
bf800acca8 chore: remove home/ from git, update gitignore
All user persona data (memory, tasks, crons, identity files, sessions)
was previously committed. This removes it from tracking and adds home/
to .gitignore — files remain on disk and will continue to sync via
Syncthing. Backup should be handled via encrypted per-user means.

Also added: cortex/.env*.bak and cortex/=* (pip artifact) to gitignore.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:19:23 -04:00
Scott Idem
d9a322164a feat: OpenAI-compatible orchestrator + backend auto-routing
- openai_orchestrator.py — new ReAct tool loop engine for any
  OpenAI-compatible endpoint (OpenRouter, Open WebUI, Ollama, LiteLLM);
  model handles both tool loop and final response, no Claude handoff needed
- tools/__init__.py — auto-derive OpenAI JSON Schema from existing Gemini
  FunctionDeclarations so tool definitions have a single source of truth
- routers/orchestrator.py — route to openai_orchestrator when model registry
  "orchestrator" role resolves to a local_openai type host
- routers/chat.py — pass role to _backend_label(); fix fallback_used logic
  (only meaningful for explicit backend overrides, not auto-routing)
- static/app.js — add null/"auto" to backend cycle; fetch local model hint
  without overriding the auto default on page load
- model_registry.py — _normalize() back-fills host_type on old registry files
- requirements.txt — add openai>=1.0.0
- ARCH__BACKENDS.md — document OpenAI-compat backend and routing logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:18:18 -04:00
Scott Idem
8ba5247ef5 tooling: install script, workspace file, and dev-restart helper
- install.py — idempotent setup script (venv, systemd service, linger,
  auth checks); supports --check for read-only status inspection
- .stignore — exclude .venv and runtime dirs from Syncthing so each
  host maintains its own machine-local venv
- Cortex_and_Inara.code-workspace — VS Code workspace (service, personas,
  docs folders; launch config for uvicorn --reload)
- dev-restart.sh — SSH wrapper to restart Cortex on the gaming laptop
  and tail logs; supports restart / logs / status subcommands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:11:27 -04:00
Scott Idem
a6e404c143 feat: host_type field for OpenRouter / OpenAI-compatible API support
Adds host_type ("openwebui" | "openai") to the host schema so Cortex can
talk to both Open WebUI/Ollama and OpenRouter/standard-OpenAI endpoints.

Path differences per type:
  openwebui (default): /api/chat/completions, /api/models
  openai:              /chat/completions,     /models

model_registry.py:
  - host_type added to host schema (default "openwebui", backward compat)
  - save_host() accepts host_type parameter
  - _resolve_model() passes host_type through with the merged host fields

llm_client._local():
  - Reads host_type from resolved model_cfg
  - Selects correct chat completions path accordingly

routers/local_llm.py:
  - save_host route accepts host_type form field
  - fetch-models uses /models for openai type, /api/models for openwebui
  - Existing host rows show type selector pre-filled from stored value

local_llm.html:
  - "Add host" form includes type selector

To use OpenRouter:
  - Add host: URL = https://openrouter.ai/api/v1, Type = OpenAI-compatible
  - API key from openrouter.ai (store in .env or model_registry.json only)
  - Fetch models or add manually (e.g. anthropic/claude-sonnet-4-5-20251022)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 21:11:22 -04:00
Scott Idem
2dd94696d5 fix: model-tag color was #334155 (invisible on dark theme) → #475569 2026-04-05 22:25:09 -04:00
Scott Idem
8570e8d852 fix: backend toggle not sent to server; add per-message model tag
Fixes:
  - app.js was tracking primaryBackend locally but never included
    model: primaryBackend in the /chat POST body, so the server always
    used settings.primary_backend regardless of what the user clicked.
    Now model: primaryBackend is sent on every chat request.

  - Responses were only annotated when fallback occurred. Now every
    assistant message shows a small model tag at the bottom right.

chat.py:
  - _backend_label() resolves human-readable name:
      claude → "Claude", gemini → "Gemini",
      local → registry label (e.g. "Gemma 4 E4B") or model_name
  - SSE payload now includes backend_label field

app.js:
  - model: primaryBackend added to /chat fetch body
  - After every response, appends .model-tag div with backend_label
  - Fallback shows " fallback → <label>" in amber; normal is muted
  - Removed separate system message for fallback (tag covers it)

style.css:
  - .model-tag: small muted text, right-aligned, separated by thin line
  - .model-tag.fallback: amber (#f59e0b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 22:10:40 -04:00
Scott Idem
9299ce5ba6 test: model registry unit test suite (45 tests)
Covers model_registry.py without requiring a running service or LLM:

  Empty/fresh state: no files, missing user dir
  Save/load: round-trip, corrupt file fallback
  Migration: v1 hosts/models, v1 no active, v0 flat, v0 empty url,
             distill_backend_mid=local → distill role, saves file after migrate
  Built-in resolution: claude_cli, gemini_api, gemini_cli, unknown → None
  User model resolution: local_openai merges host, missing host → None
  get_model_for_role: registry primary, built-in from registry, skips missing,
                      walks full backup chain, .env fallback, hardcoded fallback,
                      custom roles
  get_best_local_model: prefers role chain, falls back to first local, None if no local
  Host CRUD: create, update, unknown ID creates new, remove + cascades to models
  Model CRUD: create, update, remove + clears role refs
  set_role: assign model, assign built-in, clear with None, invalid slot,
            unknown model ID, creates new role key
  get_defined_roles: returns all settings roles, fills gaps with {}
  Multi-user isolation: registries don't bleed across users

All tests use tmp_path + patch.object(config.settings, ...) — no real files touched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:48:00 -04:00
Scott Idem
608e1de246 feat: model registry UI — hosts, models, role assignments
Replaces the single-host local model settings page with a full model
registry interface at /settings/local.

Hosts section:
- List existing hosts with inline edit + save + remove
- Collapsible "Add host" form
- Per-host "Fetch models" button

Models section:
- List all models with label, model name, host, context_k badge, tags
- Remove button

Add Model section:
- Host dropdown, label, model name, context_k, tags (comma-separated)
- "Fetch models from host" with auto-fill picker

Role Assignments section:
- One row per defined role (chat, orchestrator, distill, coder, research)
- Primary + backup_1 + backup_2 dropdowns per role
- Dropdowns pre-filled from registry on load
- AJAX save on change (POST /api/models/role) with toast confirmation
- Built-in models (claude_cli, gemini_cli, gemini_api) always available in dropdowns

Backend:
- All user_settings references replaced with model_registry
- host/{id}/remove route added
- fetch-models now accepts host_id query param
- POST /api/models/role for AJAX role assignment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:31:32 -04:00
Scott Idem
6a1a1c2686 feat: unified model registry with role-based routing
Introduces model_registry.py as the single source of truth for all LLM
backend configuration. Replaces scattered backend settings across user_settings,
config distill_backend_*, and the UI toggle.

model_registry.py:
- Per-user home/{user}/model_registry.json with version, hosts, models, roles
- Models have: type (local_openai|claude_cli|gemini_cli|gemini_api), label,
  model_name, host_id, context_k (tokens), tags (capability labels)
- Roles map to priority chains: primary, backup_1..backup_4
- Built-in IDs (claude_cli, gemini_cli, gemini_api) always resolvable
- Auto-migrates existing local_llm.json on first access
- CRUD: save_host, remove_host, save_model, remove_model, set_role
- get_model_for_role(): registry → .env default → hardcoded fallback

config.py:
- role_chat/orchestrator/distill/coder/research .env defaults
- defined_roles: comma-separated standard role list (extensible)
- get_defined_roles() and get_role_default() helper methods

llm_client.complete():
- New role= parameter (default "chat") for registry-based routing
- model= still accepted for explicit UI toggle override
- _claude() and _local() accept model_cfg dict instead of raw string
- _local() uses pre-resolved config from registry

memory_distiller.py:
- distill_mid/long now use role="distill" (no more distill_backend_* .env vars needed)

cron_runner.py:
- brief jobs use role="chat"

routers/chat.py + auth.py:
- Use model_registry instead of user_settings for local model info

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:25:18 -04:00
Scott Idem
a4daebdc9b feat: local LLM multi-model, session search, cron proactive types, notifications, docs overhaul
Local LLM:
- user_settings.py: per-user hosts/models config (local_llm.json)
- routers/local_llm.py + static/local_llm.html: dedicated settings page
- llm_client.py: local OpenAI-compatible backend via httpx
- config.py: LOCAL_API_URL/KEY/MODEL + per-backend timeouts
- Active model shown near backend toggle (amber hint text)

Memory distillation:
- memory_distiller.py: DISTILL_BACKEND_MID/LONG .env overrides
- scheduler.py + notification.py: notify NC Talk after mid/long distill
- notification.py: outbound channel abstraction (NC Talk, extensible)

Session search:
- routers/files.py: GET /sessions/search?q= with excerpts grouped by date
- static/index.html + app.js: search UI in file sidebar with highlight
- _esc() helper to prevent XSS in search results

Proactive cron:
- cron_runner.py: new job types — message (send directly) and brief (LLM + send)
- Both support optional per-job channel override

Channels:
- routers/nextcloud_talk.py: consolidated using notification._send_nct_message()
- routers/auth.py: local backend status in /auth/status
- routers/chat.py: /backend returns {primary, fallback, local_model} object

UI / UX:
- Copy button for user messages (matching assistant)
- Autocomplete disabled on sensitive form fields
- settings.html: local model section replaced with link to /settings/local

Docs overhaul:
- MASTER.md hub + ARCH__SYSTEM/BACKENDS/PERSONA/CHANNELS/FUTURE.md
- ARCH__Intelligence_Layer.md replaced with redirect table
- CORTEX.md trimmed to vision only; README updated
- OPEN_WEBUI_API.md added to docs/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 20:53:06 -04:00
Scott Idem
bd6532e93a feat: shared nav bar on Help and Settings pages
Replaces the lone "← Back to Cortex" link with a consistent page-nav
on both pages: ← Chat | Help | Settings | Sign out

Active page is highlighted purple; others are muted gray.
Settings page gets a {{ help_href }} template var from settings.py.
Help page builds nav links from the existing cfg JS object.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 22:09:08 -04:00
Scott Idem
a94fdc869d docs: fix Gitea SSH URL to use git.dgrzone.com
cortex subdomain works incidentally but git.dgrzone.com is correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:53:54 -04:00
Scott Idem
1fefd42e19 docs: Gitea SSH port 2222 verified working
WAN port forward confirmed end-to-end. Clone URL:
ssh://git@cortex.dgrzone.com:2222/<user>/<repo>.git

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:52:36 -04:00
Scott Idem
0c17b4b1ab docs: overhaul TODO__Agents.md to reflect current state
Moved to completed: token expiry restart, Holly onboarding, per-user
channel config, Google OAuth, per-user Gemini key, session persistence,
persona picker, Lucide icons, favicon, Help shared base, reminders tools,
Brian onboarding.

Updated in-progress: knowledge consolidation tools (ae_journal_* done,
import script still pending). NC Talk and Google Chat notes updated for
per-user routing. Removed stale "default user only" notes.

High priority now: Ollama backend, Gitea SSH verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:44:56 -04:00
Scott Idem
cec6d3e23a docs: update README for current state
- .env location: cortex/.env + cortex/.env.example (not project root)
- Webhook endpoints: per-user /webhook/nextcloud/{username} and /channels/google-chat/{username}
- Personas table: added brian/wintermute and scott/developer
- Docs table: added GOOGLE_CHAT_BOT.md, cortex/static/HELP.md
- Channels section: per-user webhook note + links to setup docs
- User management: added google-add command and channels.json note
- Removed stale Inara/Tina-only framing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:42:11 -04:00
Scott Idem
2d3a380d6b docs: add Tools & Modes protocol to all personas and template
Every persona now knows: direct chat has no tools, Agent mode () has
the full tool suite. If asked to write a reminder/task/etc in chat mode,
tell the user to switch modes rather than silently failing.

Updated: inara, tina, donut, wintermute, developer, cleo PROTOCOLS.md
Updated: persona_template.py so all future personas get this by default

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:35:54 -04:00
Scott Idem
662924c6a1 fix: pass user to _run_job so get_user_gemini_key resolves correctly
NameError: name 'user' is not defined in orchestrator._run_job —
user was resolved in the endpoint but not forwarded to the background task.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:21:14 -04:00
Scott Idem
e5b6d58889 feat: reminders_add and reminders_list tools
- New cortex/tools/reminders.py with reminders_add, reminders_list, reminders_clear
- reminders_clear moved here from cron.py (cron still imports from same file)
- __init__.py: wired up new callables and Gemini declarations
- Inara can now add/read reminders in Agent mode via the orchestrator

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:14:22 -04:00
Scott Idem
6b725afc3e docs: update NC Talk doc with real container name, commands, and logs section
- Use dgr_zone_nextcloud-app-1 throughout (actual container name)
- talk:bot:uninstall (not remove — wrong command in previous version)
- Added Logs section: occ log:tail + journalctl
- Bruteforce reset command now includes full docker exec form

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 13:47:43 -04:00
Scott Idem
ddf5dd6338 docs: add Google Chat setup guide, update NC Talk for per-user routing
- docs/GOOGLE_CHAT_BOT.md: new step-by-step guide covering channels.json,
  Google Cloud Console config, JWT audience, and troubleshooting
- docs/NEXTCLOUD_TALK_BOT.md: updated for per-user endpoints
  (/webhook/nextcloud/{username}), channels.json config, removed old
  server-level .env references, updated Multi-User note

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 13:24:36 -04:00
Scott Idem
93f7f44e51 feat: per-user channel config for Google Chat and Nextcloud Talk
- New endpoints: POST /channels/google-chat/{username} and /webhook/nextcloud/{username}
- Channel secrets/config live in home/{username}/channels.json (gitignored)
- auth_utils: get_user_channels() helper reads channels.json
- Both routers load persona, audience/secret, backend, timeout per user;
  set_context() wires the correct persona before building the system prompt
- Removed server-level channel settings from config.py and .env —
  no user gets a channel until they create their own channels.json
- .gitignore: home/**/channels.json added

To migrate: update Google Chat Add-on webhook URL to /channels/google-chat/{username}
and re-register NC Talk bot at /webhook/nextcloud/{username}

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 13:02:45 -04:00
Scott Idem
496da58f58 chore: consolidate .env files — one .env in cortex/, one .env.example
- Removed orphaned root .env and .env.default (values already in cortex/.env,
  which is what the systemd service actually loads)
- Replaced outdated cortex/.env.example with the comprehensive .env.default content
- Also tracks: tested/persona/cleo/ (new test persona), Inara memory updates

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 12:22:49 -04:00
Scott Idem
8e20bfbea8 feat: shared Help base, Google OAuth live, new personas, cleanup
- cortex/static/HELP.md: shared Help & Reference base served to all users
- help.html: loads shared base + appends persona-specific HELP.md if present
- inara/HELP.md: cleared (content moved to shared base)
- Google OAuth: registered scott.idem@oneskyit.com; flow now working end-to-end
- .gitignore: exclude home/**/sessions/ (runtime logs)
- New personas tracked: home/holly/persona/donut/, home/scott/persona/developer/
- Removed orphans: holly/, personas/, cortex-holly.service
- CLAUDE.md: updated current state and recently completed list to 2026-03-27

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 22:55:45 -04:00
Scott Idem
3a94df1eaf fix: prevent password managers autofilling Gemini API key field
Change type="password" to type="text" — the main signal password
managers use. Also add autocomplete="off", data-lpignore, data-1p-ignore
for broader coverage across Bitwarden, 1Password, LastPass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:57:14 -04:00
Scott Idem
ce806e52ed fix: google-add now also sets profile.json email
The invite command reads email from profile.json, not auth.json.
google-add was only writing to auth.json so invite had no address
to send to. Now calls set_email() as well.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:32:48 -04:00
Scott Idem
7438031797 feat: connected accounts + Gemini API key in account settings UI
Settings page gains two new sections:
- Connected Accounts: shows linked Google email (read-only)
- Gemini API Key: paste personal key from aistudio.google.com,
  shows masked hint of saved key, remove link to revert to server key

POST /settings/gemini-key saves/clears gemini_api_key in auth.json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:16:37 -04:00
Scott Idem
8aec6aafcc feat: Google OAuth sign-in + per-user Gemini API key
Users with Google accounts can now sign in without a password.

Auth flow:
- GET /auth/google → Google consent page (CSRF state cookie)
- GET /auth/google/callback → exchange code, lookup user, set JWT
- auth.json gains google_sub + google_email fields
- set_password() no longer overwrites unrelated auth.json fields

Admin setup:
  python manage_passwords.py google-add <username> <email>
  # add GOOGLE_CLIENT_ID + GOOGLE_CLIENT_SECRET to .env

Per-user Gemini key:
- get_user_gemini_key() reads gemini_api_key from auth.json
- orchestrator_engine.run() accepts gemini_api_key param
- orchestrator router passes user's key, falls back to server key

login.html: "Sign in with Google" button above the password form.
manage_passwords.py list: now shows auth method columns (pw / google).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:01:52 -04:00
Scott Idem
62fde62653 feat: persona-specific favicon + fix favicon.ico 404
app.js updates the <link rel="icon"> to the active persona's emoji on
load (CORTEX_EMOJI is already injected server-side). /favicon.ico route
added as a fallback for login/settings/help pages that don't have
persona context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:45:36 -04:00
Scott Idem
f8d89bc272 fix: close SSE connection cleanly on page navigation
beforeunload closes the EventSource explicitly so the browser doesn't
log "connection interrupted while page was loading". onerror handler
suppresses auto-reconnect noise if the connection temporarily drops.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:44:12 -04:00
Scott Idem
92350f7a7b feat: persist active session across page navigation with inactivity TTL
Session ID is stored in localStorage keyed to user+persona. On page load
it's silently restored if within 30 min of last activity. Timestamp
updates on every sent message. New session / delete session clears the
stored ID so the TTL logic stays consistent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:38:04 -04:00
Scott Idem
4f09823afe feat: Lucide icons on edit/del/copy and inline edit save/cancel buttons
pencil → edit, trash-2 → del, copy → copy, check → copied feedback,
check → Save, x → Cancel. All small action buttons get inline-flex
alignment for consistent icon+label layout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:32:19 -04:00
Scott Idem
826bd6cfe3 feat: /{username} persona picker landing page
Visiting /scott (or any user root) now shows a clean card page listing
all their personas with emoji + name, each linking to /{user}/{persona}.
Previously the route was unhandled (404 or wildcard match).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:19:04 -04:00
Scott Idem
c3507f8e11 fix: help page back link preserves active persona
Pass ?persona= query param on the help link so the server knows which
persona to return to. Previously always defaulted to personas[0], causing
navigation back to the wrong persona.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:13:52 -04:00
Scott Idem
65548ebf36 feat: Lucide SVG icons throughout main UI
Replace all emoji/unicode icons with Lucide SVG icons:
- Mode select dropdown: message-circle / pencil / lock / bot
- Send button: arrow-up (chat/OTR), pencil (note), zap (agent)
- Stop button: square icon
- Header nav already had Lucide SVGs; render_icons() now called at init

Add icon_html() + render_icons() helpers; update update_mode_ui() and
open_mode_dropdown() to use innerHTML + lucide.createIcons(). CSS: .btn-icon
alignment, inline-flex on .hdr-btn / .hdr-dd-item / #send / #stop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:06:01 -04:00
Scott Idem
24c9f52b49 fix: agent mode icon — 🥸 in dropdown, on Run button
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:51:51 -04:00
Scott Idem
3bada5f311 fix: font sizes 21/25/17px, icon labels on send btn, no-wrap fix
- Sync preload script font sizes to match app.js (21/25/17px)
- Send button variants now show icons: ↑ Send, 📝 Note,  Run
- Remove fixed width on send-col; add white-space:nowrap + padding
  so "📝 Note" never wraps regardless of font size

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:43:34 -04:00
Scott Idem
cf277f822e feat: Inter font, weight 450, bumped base sizes across all pages
- Load Inter variable font from Google Fonts on all 5 HTML pages
- font-weight: 450 on body (between regular and medium — fixes thin feel)
- -webkit-font-smoothing: antialiased for cleaner screen rendering
- Base font size: normal 16→17px, large 18→19px, small 14→15px
- Applies consistently to main UI, login, setup, settings, and help pages

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:20:10 -04:00
Scott Idem
6cf10d2755 feat: UI redesign — compact mode select, consolidated header nav
Header:
- Sessions, ⚙ context panel, ≡ settings dropdown (Files, Account,
  Sign Out), and  help — down from 6+ individual buttons
- Responsive: flex-row on desktop, wraps on mobile with labels hidden

Footer (input area):
- 4-way mode select replaces the button row — shows only the active
  mode as [icon] [label] ▲; click opens an upward dropdown
- Options sorted by MRU: most recently used floats to the bottom
  (closest to the trigger button) for quick re-selection
- Current mode marked with ✓
- Note mode shows a small prv/pub sub-toggle below the select button
- Mobile: textarea on top (full width), mode select + send on one row

Mode state consolidated from 3 booleans into a single current_mode
variable with localStorage persistence and MRU tracking.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:08:33 -04:00
Scott Idem
fa04b5e6b0 feat: off the record mode (OTR)
Adds a third input mode toggle alongside Note and Agent. When active:
- Textarea gets a subtle purple tint with dashed border
- OTR button highlights purple
- Placeholder reads "Off the record — not logged or distilled…"
- off_record=True is sent to /chat; session_logger is skipped
- In-memory session context is preserved within the session

Switching to Note or Agent mode deactivates OTR, and vice versa.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 21:07:21 -04:00
Scott Idem
b3f40cf437 fix: message input placeholder uses active persona name
Hardcoded 'Inara' replaced with CORTEX_PERSONA in all placeholder
strings (chat mode and agent task mode).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 20:59:37 -04:00
Scott Idem
8487645224 fix: claude auth expiry warning — correct field name and smarter threshold
- Fix 'undefined' in auth banner: read access_token_hours_remaining (not hours_remaining)
- Fix false-positive warning on fresh tokens: when refresh token present, only warn
  within 1 hour of expiry (not 24h) since the CLI should auto-rotate but sometimes misses
- Emit claude_auth_expired SSE event on 401 so UI shows inline red banner immediately
- app.js: handle claude_auth_expired SSE event with persistent top banner + dismiss button

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:22:18 -04:00
Scott Idem
0cf0d65e9e feat: session naming, username/persona rename, help page, contrast fixes
- Session name field: PATCH /sessions/{id} endpoint, inline rename button in UI
- Persona rename: inline ✏ toggle form in settings, POST /settings/persona/rename
- Username rename: inline form in settings, POST /settings/username (renames home dir, forces re-login)
- Help page: dedicated /help route replacing modal, collapsible sections
- Per-persona isolation: files.py and session_store.py now scope to correct user/persona
- Contrast/visibility: muted text bumped to slate-400+, session rename btn at 0.4 opacity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:10:12 -04:00
Scott Idem
1b425a539f feat: account settings page + dedicated help page
- Add /settings page with password change form and personas list
- Add /help dedicated page (replaces help modal); renders HELP.md with
  collapsible sections, dark theme, back link to active persona
- Add 👤 account button and convert ? button to link in header
- Remove help modal HTML and ~55 lines of modal JS from main app
- Register settings and help routers in main.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 21:41:18 -04:00
Scott Idem
c01ef663f5 fix: per-persona session/file isolation + onboarding route order
- session_store: store sessions under home/{user}/persona/{name}/session_data/
  instead of the shared cortex/data/sessions/ bucket
- chat endpoints: add user/persona query params to /sessions, /history/*,
  /sessions/*, /note so they resolve the correct persona context
- files router: add user/persona query params to /files and /files/{name}
  so the file browser loads the right persona's files
- app.js: pass user/persona on all session, history, and file fetches;
  move _fileParams to top-level scope so it is available everywhere
- onboarding: fix FastAPI route ordering — register /persona before /{token}
  so the literal path wins and does not get captured as a token value
- ui.py: read Emoji field from IDENTITY.md and inject into CORTEX_CONFIG
  so the header icon reflects each persona's chosen emoji
- .gitignore: exclude home/**/session_data/ (runtime state)
- migrate scott/inara sessions from cortex/data/sessions/ to session_data/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-22 00:01:07 -04:00
Scott Idem
99f8961bec feat(tina): add dinosaurs as a core Holly interest
Lifelong passion — Jurassic Park seen countless times. Tina should
engage with this genuinely, not just acknowledge it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 00:18:38 -04:00
Scott Idem
c2825194d4 docs: update project docs, NC Talk guide, Tina persona, and gitignore
- CLAUDE.md: add new auth/onboarding files to directory map, update
  security section (JWT/bcrypt/invite details), expand recently completed
- README.md: fix Web UI auth description, add User Management section
- TODO__Agents.md: mark NC Talk docs and auth/onboarding complete,
  update Holly onboarding plan to reflect single-instance multi-user approach
- docs/NEXTCLOUD_TALK_BOT.md: complete guide — occ commands, nginx config,
  clarify incoming vs outgoing HMAC difference, multi-user note, full
  troubleshooting table
- home/holly/persona/tina/: flesh out all four persona files with real
  content (DCC name origin, metal music, reading, foster cats, Holly's profile)
- .gitignore: exclude home/**/auth.json, invite.json, profile.json

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 00:13:35 -04:00
Scott Idem
8c61c28b7d fix: mount /static before ui.router to prevent wildcard route catching static files
The ui.router's /{username}/{persona} wildcard was matching /static/style.css
(username="static", persona="style.css") because app.mount("/static") was
registered after app.include_router(ui.router). FastAPI processes routes in
registration order, so /static must be mounted first.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 23:28:13 -04:00
Scott Idem
69f38ca7dc feat: SMTP email support for invite links + profile.json for user email storage
- email_utils.py: send_email() via smtplib.SMTP_SSL (port 465, same server
  as AE API); send_invite_email() renders plain-text + HTML invite template
- config.py: smtp_server, smtp_port, smtp_username, smtp_password,
  smtp_from_email, smtp_from_name, cortex_base_url settings
- manage_passwords.py:
  - profile.json helpers (get/set email stored in home/{username}/profile.json)
  - invite command now accepts optional email arg, sends invite automatically;
    falls back to stored email; prints link either way
  - new 'email' command to store/update a user's email address
  - 'list' command now shows email alongside password status
- .env.default: SMTP_* and CORTEX_BASE_URL documented

Usage after adding SMTP_PASSWORD to .env:
  python manage_passwords.py invite holly holly@example.com
  → generates token, stores email, sends invite, prints link as fallback

All 80 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 23:19:09 -04:00
Scott Idem
46b65d087c feat: persona onboarding — invite tokens, self-service setup, persona creation, switcher
New user flow:
  1. Admin: python manage_passwords.py invite <username>  → generates URL
  2. User visits /setup/<token> → sets own password → logged in
  3. User redirected to /setup/persona → fills name/emoji/description
  4. persona_template.py generates all starter files → lands at /{user}/{persona}

Multiple personas:
  - Header persona name is now a clickable dropdown listing all personas
  - "New persona" link at bottom → /setup/persona (available to logged-in users)
  - /api/personas endpoint returns persona list for current session user

New files:
  - persona_template.py: generates IDENTITY/SOUL/PROTOCOLS/USER/HELP.md + data files
  - routers/onboarding.py: /setup/{token}, /setup/persona GET+POST
  - static/setup.html: two-step form (password → persona), emoji picker, mobile-friendly

Updated:
  - auth_utils.py: create_invite(), validate_invite(), consume_invite()
  - manage_passwords.py: invite command with URL output
  - auth_middleware.py: /setup/* prefix is public (invite tokens need no auth)
  - routers/ui.py: /api/personas endpoint; post-login redirect if no personas
  - static/app.js: persona switcher dropdown with navigation + Add persona link
  - static/style.css: .persona-switcher, .persona-dropdown, mobile adjustments

Mobile: login/setup pages are card-centered with responsive padding;
dropdown avoids edge-clipping on narrow screens; logout button stays visible.

All 80 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 23:10:32 -04:00
Scott Idem
a9bbb668b5 feat: session auth + per-user/persona UI at /{user}/{persona}
Replaces nginx basic auth with a proper per-user session system:

- auth_utils.py: bcrypt password hashing, JWT cookie creation/decode
- auth_middleware.py: validates JWT cookie on all routes except /login,
  /health, /static/, and webhook endpoints (/channels/, /webhook/)
- routers/ui.py: GET /login, POST /login, POST /logout,
  GET /{username}/{persona} — serves index.html with CORTEX_CONFIG injected
- static/login.html: minimal login form (dark theme, matches UI)
- main.py: registers SessionAuthMiddleware + ui.router
- config.py: jwt_secret, jwt_expire_days settings
- manage_passwords.py: CLI tool to set/check/list user passwords
- app.js: reads window.CORTEX_CONFIG (user + persona), sends both on
  every /chat and /orchestrate request; persona name shown in header;
  logout button (⏏) added to header
- requirements.txt: bcrypt, PyJWT, python-multipart
- .env.default: JWT_SECRET, JWT_EXPIRE_DAYS documented
- tests: client fixture injects JWT cookie; security test assertions
  updated for URL-normalized path traversal paths (still secure, codes differ)

All 80 tests pass.

Setup for a new user:
  python manage_passwords.py set scott
  python manage_passwords.py set holly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 22:54:12 -04:00
Scott Idem
77e770cdb2 feat: multi-user/multi-persona support with two-level home directory layout
Restructures persona storage from a flat personas/{name}/ layout to
home/{username}/persona/{name}/, mirroring Linux home directories.

Changes:
- persona.py: two ContextVars (user + persona), Linux-style name validation,
  set_context(), get_user(), get_persona(), validate(), list_users(),
  list_user_personas(); persona_path() takes (username, name)
- config.py: replaces personas_dir with home_dir + home_root()
- git mv personas/inara → home/scott/persona/inara (history preserved)
- home/holly/persona/tina/: Holly's persona stub added
- cron_runner.py: all storage functions take (username, persona) params
- tools/cron.py: stamps user + persona on jobs; APScheduler IDs are
  {user}:{persona}:{job_id} to prevent collisions across users
- memory_distiller.py: distill_short/mid/long take (username, persona);
  added missing Path + settings imports
- scheduler.py: _load_user_crons() iterates home/*/persona/* (two-level)
- routers/chat.py, orchestrator.py: user field added; set_context() called
- tests/conftest.py: home_root fixture with two-level structure;
  patches home_dir instead of personas_dir
- tests/test_persona.py: fully rewritten for two-level API
- tests/test_api_files.py: updated fixture name and path
- .env.default: documents HOME_DIR setting; scrubs stale API key
- CLAUDE.md, README.md: directory maps updated for new layout

All 80 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 22:35:40 -04:00
Scott Idem
92a8f5d894 test: add Cortex test suite (77 tests, no LLM calls)
Tests cover:
  - Smoke: /health, /auth/status, /distill/status (test_health.py)
  - Persona validation: path traversal, bad names, list_personas (test_persona.py)
  - Chat API: persona routing, session persistence, error handling (test_api_chat.py)
  - Files API: ALLOWED set enforcement, read/write, missing files (test_api_files.py)
  - Webhooks: NC Talk HMAC accept/reject, Google Chat JWT (test_webhooks.py)
  - Tools: scratch read/write/append/clear, tasks CRUD, cron parser + tools (test_tools.py)
  - Security: path traversal, replay attack, known gaps documented (test_security.py)

All LLM calls mocked — suite runs in ~1.4s.
Run: cd cortex && .venv/bin/pytest

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 22:03:42 -04:00
Scott Idem
5cadb836fa feat: multi-persona support (single Cortex, multiple users)
- Add cortex/persona.py: ContextVar-based per-request routing with
  path traversal protection and persona validation
- Migrate inara/ → personas/inara/ (git history preserved via git mv)
- config.py: add personas_root(), inara_path() delegates to personas/inara
- All 14 settings.inara_path() call sites replaced with persona_path()
- ChatRequest + OrchestrateRequest: add persona field (default: "inara")
  with validation at request entry before any processing
- memory_distiller: add optional persona param for future per-persona distill
- cron_runner/tools/cron: stamp persona on jobs, prefix APScheduler IDs
  (persona:job_id) to prevent collisions across personas
- scheduler: _load_user_crons() iterates all personas at startup

Adding a new persona: create personas/<name>/ with IDENTITY.md + SOUL.md.
Auth: handled at nginx level (inject X-Cortex-Persona header per subdomain).
Future: persona maps to Aether account_id_random for full integration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 21:50:02 -04:00
Scott Idem
6316ffa1d4 feat: cron job system for Inara (remind + note types)
- cron_runner.py: job storage (CRONS.json), schedule parsing, execution
- tools/cron.py: cron_list/add/remove/toggle + reminders_clear tools
- scheduler.py: load user crons at startup, expose get_scheduler() for
  live add/remove without restarts
- context_loader.py: auto-include REMINDERS.md in system prompt (tier 2+)
  so cron reminders surface automatically without Inara having to poll
- inara/CRONS.json + REMINDERS.md: backing files (initially empty)

Schedule formats: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM
Job types: remind (→ REMINDERS.md) | note (→ SCRATCH.md)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 21:17:49 -04:00
Scott Idem
1b32667872 feat: scratchpad tool + fix Claude auth token expiry warning
- Add cortex/tools/scratch.py with scratch_read/write/append/clear tools
- Register all four scratch tools in the orchestrator tool registry
- Create inara/SCRATCH.md as the backing file (never distilled/archived)
- Fix auth.py: expiresAt reflects short-lived access token (~8h) not the
  1-year refresh token — suppress expiry warning when refreshToken is present

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 21:10:03 -04:00
Scott Idem
31a5ef0541 docs: update HELP.md for agent mode, tasks, and 2026-03-18 changes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:26:28 -04:00
Scott Idem
24d16734a5 feat: personal task management tools for Inara
Adds task_list, task_create, task_update, task_complete orchestrator tools
backed by inara/TASKS.json — private to each agent instance.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:17:38 -04:00
Scott Idem
aaac3e1353 feat: persist orchestrator sessions + user service + docs update
- Orchestrator now saves turns to session store so history survives page refresh
- UI session_id updated from job result; history controls attached to agent turns
- Cortex migrated from system service to systemd user service (no more sudo)
- Update README.md and CLAUDE.md with correct service commands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:08:38 -04:00
Scott Idem
5b5586656f chore: session logs and memory distill 2026-03-17/18
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 22:43:40 -04:00
Scott Idem
9b818aa5c7 feat: orchestrator Agent mode UI + claude_allow_dir tool + fix DDG search
- Add Agent mode toggle to web UI input row — routes through POST /orchestrate
  instead of /chat; polls for result with live tool-call count in thinking bubble
- Add cortex/tools/system.py with claude_allow_dir tool; registers in tool registry
- Fix web search: duckduckgo_search renamed to ddgs, update import + requirements.txt
- Allow WebSearch and WebFetch in ~/.claude/settings.json for Claude CLI fallback
- Add claude-allow-dir script docs and security note to CLAUDE.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 22:42:44 -04:00
Scott Idem
97438f1a0f feat: multi-instance support — agent_name and user_name configurable
All hardcoded "Inara"/"Scott" strings replaced with settings.agent_name
and settings.user_name, read from .env at startup:

- config.py: AGENT_NAME and USER_NAME settings (defaults: Inara / Scott)
- llm_client.py: conversation labels in prompt builder
- session_logger.py: **Name:** labels in session log markdown
- memory_distiller.py: distillation system prompts (mid + long)
- routers/nextcloud_talk.py: @mention prefix strip
- routers/google_chat.py: greeting message

Second instance scaffolding:
- holly/: identity directory with placeholder files (USER_NAME=Holly,
  AGENT_NAME to be chosen by Holly)
- cortex/.env.holly: config for Holly's instance on port 8001
- cortex-holly.service: systemd unit for the second instance

No behavioural change to the Inara/Scott instance — defaults unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 20:13:11 -04:00
Scott Idem
0b10558f80 fix: session delete button always visible, session name truncates properly
- Remove opacity:0/hover-reveal on session × — always shown in the panel
- session-id: flex:1 + text-overflow:ellipsis prevents overflow pushing × offscreen
- min-width on delete btn ensures tap target is always reachable

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 19:45:05 -04:00
Scott Idem
f935fc4a7f feat: session delete + touch-friendly message controls
Session delete:
- DELETE /sessions/{session_id} endpoint (chat.py + session_store.py)
- × button on each session item in the panel (hover-reveal on desktop)
- Clears UI if the active session is deleted

Touch accessibility:
- @media (hover: none) rule makes msg-actions always visible on touch devices
- msg-act-btn tap targets enlarged to 36px min-height, readable font size
- session-delete-btn also always visible and finger-sized on touch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 19:43:20 -04:00
Scott Idem
e6e76e7e4c docs: mark Intelligence Orchestrator Phase 1 complete in TODO
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 19:38:14 -04:00
Scott Idem
ed472ce9a0 feat: Intelligence Layer Phase 1 — orchestrator service
Adds the Gemini API orchestrator (ReAct tool loop → Claude responder):

Orchestrator engine + router:
- orchestrator_engine.py: Gemini API tool loop, Claude CLI handoff
- routers/orchestrator.py: POST /orchestrate (async job queue), GET /orchestrate/{job_id}

Tools (cortex/tools/):
- web.py: DuckDuckGo web search (no key required)
- ae_knowledge.py: ae_journal_search + ae_journal_entry_create (AE V3 API)
- ae_tasks.py: ae_task_list (reads agents_sync Kanban filesystem)
- files.py: file_read (path-allowlisted to safe dirs)

Config + deps:
- config.py: orchestrator, DuckDuckGo, and AE API settings
- requirements.txt: google-genai, duckduckgo-search
- .env.default: reference config with all new keys documented

Docs:
- CLAUDE.md, README.md, documentation/ added to repo
- Port references updated 7331 → 8000 throughout
- Default model updated to gemini-2.5-flash

Tested: ae_task_list, ae_journal_search, web_search all working end-to-end.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 19:37:49 -04:00
Scott Idem
23f8659aaa UI: fix mobile input area layout
- Stack textarea above button row on mobile (flex-direction: column)
- font-size: 16px on textarea prevents iOS Safari auto-zoom on focus
- body height: 100dvh adjusts dynamically as soft keyboard opens/closes
- Right col goes horizontal (row) with full width on mobile
- Hide height-row and enter-toggle (desktop-only concepts)
- Larger touch targets for Send/Stop/Note
- Hide session-id to reclaim vertical space

Desktop layout unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 23:44:21 -04:00
Scott Idem
fe6561bf6a feat: add Gemini auth check to token warning banner
/auth/status now returns per-backend status: Claude warns on <24h expiry,
Gemini warns only when oauth_creds.json is missing or has no refresh_token
(access token rotates automatically so expiry_date is not a useful signal).
Banner shows warnings for both backends when needed, and the hint text
names the specific CLI commands to run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 23:29:25 -04:00
Scott Idem
1127610752 UI: show distill schedule next-run times in settings panel
Fetches /distill/status when the ⚙ panel opens and renders next run
times below the distill buttons (monospace, muted). Shows "today",
"tomorrow", or "Mar 18" format depending on how far away.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 23:22:47 -04:00
Scott Idem
2144d7c2c0 UI: collapsible sections in help modal
Post-processes rendered markdown: each H2 becomes a <details>/<summary>.
Top 4 sections (Header Controls, Chat, Sessions, Notes) open by default;
remaining sections (Backends, Talk, Files, Context, Shortcuts, API,
Planned) start collapsed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 23:16:11 -04:00
Scott Idem
2ca02006dd UI: auth banner — add re-auth hint for multi-user context
Banner now shows a second line explaining how to fix it: SSH to the
Cortex host, run `claude`, follow the login prompt, restart Cortex.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 23:09:11 -04:00
Scott Idem
48a6734ec3 feat: Claude CLI OAuth token expiry warning
New GET /auth/status endpoint reads ~/.claude/.credentials.json and
returns hours remaining + warning flag. UI shows a dismissible amber
banner when < 24h remain, turning red if expired. Checked on page load
and every 30 minutes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 23:06:28 -04:00
Scott Idem
7b51e7cc44 UI: mobile-friendly header — move backend/display into settings panel
Header trimmed to 4 buttons (Sessions, Files, ⚙, ?). Backend toggle,
font size, and theme moved into the ⚙ settings panel under new Backend
and Display sections. Panels use responsive widths to avoid overflow on
small screens. Mobile breakpoints tighten padding and hide subtitle.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 23:03:55 -04:00
Scott Idem
ce1561572a config: default scheduler timezone to America/New_York 2026-03-17 22:42:47 -04:00
Scott Idem
b123dc3117 Fix scheduler timezone: use ZoneInfo(settings.scheduler_timezone) not 'local' 2026-03-17 22:38:11 -04:00
Scott Idem
4253e69c0b Add auto memory distillation scheduler (APScheduler)
- scheduler.py: AsyncIOScheduler with three cron jobs
    short  daily     03:00 (no LLM, always fast)
    mid    weekly    Sun 03:30 (LLM)
    long   monthly   1st 04:00 (LLM — off by default)
- config.py: AUTO_DISTILL, AUTO_DISTILL_SHORT/MID/LONG .env flags
- main.py: start/stop scheduler in FastAPI lifespan
- routers/distill.py: GET /distill/status — next run times + config
- requirements.txt: apscheduler>=3.10
- HELP.md: updated planned items, added /distill/status to API table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 22:31:38 -04:00
Scott Idem
9fb5b8ae63 HELP.md: mark pfSense port 2222 as configured (pending test) 2026-03-17 22:23:12 -04:00
Scott Idem
0ebfbc6590 Refactor UI into separate CSS/JS, add help modal and HELP.md
- static/index.html: reduced to 127-line HTML shell
- static/style.css: all styles extracted (~900 lines) + help modal styles
  + shared markdown rendering for file-preview and help-modal-body
  including tables (previously missing)
- static/app.js: all JS extracted (~900 lines) + help modal fetch/render
- index.html: adds ? help button + help modal HTML
- inara/HELP.md: comprehensive reference doc covering all features,
  keyboard shortcuts, API endpoints, memory system, planned items
- routers/files.py: HELP.md added to ALLOWED set
- context_loader.py: HELP.md loaded at tier 2+ (after PROTOCOLS.md)
  so Inara can reference it when helping Scott with the interface

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:52:54 -04:00
Scott Idem
fa96c50935 UI: font size cycle button (Aa / A+ / A−)
Cycles normal (16px) → large (18px) → small (14px) on the root element
so all rem-based text scales together. Persisted in localStorage, applied
before first paint to avoid flash. Also include today's session log.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:40:14 -04:00
Scott Idem
c402b97bf3 UI: context panel popover, light/dark theme, contrast improvements
- Light/dark mode: full palette via @media + [data-theme] override;
  theme toggle button (☾/☀) persists to localStorage; no flash on load
- Dark mode softened: #1a1228 bg (was near-pure black), improved muted
  text contrast (#9080a8)
- CSS variables: --shadow, --modal-overlay, --code-bg, --pre-bg,
  --success, --success-dim replace all hardcoded rgba/hex values
- Context panel: ⚙ header button opens dropdown (like Sessions);
  tier badge shows current tier; closes on outside click
- Removed always-visible context bar; controls now in panel

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:32:54 -04:00
Scott Idem
ce3c1f5f7f Add tiered memory system with manual distillation
- config.py: memory_budget_long/mid/short settings (overridable in .env)
- memory_distiller.py: distill_short (no LLM), distill_mid, distill_long (LLM)
- routers/distill.py: POST /distill/{short,mid,long,all} endpoints
- context_loader.py: rewrote to load long→mid→short order with include_* toggles
- routers/chat.py: ChatRequest gains include_long/mid/short fields
- routers/files.py: MEMORY_LONG/MID/SHORT.md added to ALLOWED set
- main.py: register distill router
- static/index.html: context bar — tier selector, L/M/S memory toggles,
  distill buttons with status feedback; send includes tier + memory flags
- inara/MEMORY_LONG.md: migrated from MEMORY.md + Cortex/Talk bot notes
- inara/MEMORY_MID.md, MEMORY_SHORT.md: stubs ready for distillation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:22:32 -04:00
Scott Idem
3455c7a09c Add SSE real-time Talk activity, file editor UI, and identity file API
- event_bus.py: in-process asyncio pub/sub (one Queue per SSE client)
- nextcloud_talk.py: publishes nct_message/nct_response events to bus
- chat.py: GET /events SSE endpoint streams Talk activity to browser
- routers/files.py: whitelist-protected GET/PUT for Inara identity .md files
- main.py: register files router
- static/index.html: real-time Talk feed, blue badge on Sessions btn,
  Files modal with preview/edit toggle and Ctrl+S save

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 21:10:07 -04:00
130 changed files with 32305 additions and 2002 deletions

24
.gitignore vendored
View File

@@ -3,19 +3,33 @@
__pycache__/ __pycache__/
*.pyc *.pyc
# Secrets — keep .env.example, never commit real .env # Secrets — keep .env.default, never commit real .env
.env .env
cortex/.env*.bak
# Session data (runtime state, not source) # Pip install artifacts
cortex/=*
# Runtime data
cortex/data/ cortex/data/
# Syncthing Metadata # User home directory — all persona data, memory, tasks, and credentials
# are personal and must never be committed. Back up via encrypted means.
home/
# Syncthing metadata
.stfolder/ .stfolder/
# Temporary Files # Temporary files
tmp/ tmp/
*.tmp *.tmp
*.log *.log
# System Files # Aider — history files are personal/ephemeral; .aider.conf.yml is project config and IS tracked
.aider.chat.history.md
.aider.input.history
.aider.llm.history
# System files
.DS_Store .DS_Store
.aider*

5
.stignore Normal file
View File

@@ -0,0 +1,5 @@
// Machine-local — never sync across hosts
.venv/
__pycache__/
*.pyc
cortex/data/

322
CLAUDE.md Normal file
View File

@@ -0,0 +1,322 @@
# CLAUDE.md — Cortex / Inara Project
This file is loaded automatically by Claude Code when working in this directory.
Read it before touching any files.
---
## Identity & Context
- **Project:** Cortex (dispatcher) + Inara (resident agent)
- **Owner:** Scott Idem (One Sky IT / Danger Zone)
- **Machine context:** See `~/CLAUDE.md` for fleet identity (`scott_lpt` = General Manager)
- **Named after:** The 'verse-wide communications network (Firefly)
---
## Directory Map
```
Cortex_and_Inara_dev/
cortex/ ← FastAPI service (the dispatcher)
main.py ← App entry point, router registration
config.py ← All settings (pydantic-settings, reads .env)
persona.py ← Two-level identity: user + persona, path resolution, ContextVars
llm_client.py ← Claude CLI + Gemini CLI subprocess backends + Anthropic SDK direct
orchestrator_engine.py ← Gemini API ReAct tool loop → Claude handoff
context_loader.py ← Builds system prompt from persona files (tier 14)
session_store.py ← In-memory + file session persistence
session_logger.py ← Writes session turns to home/{user}/persona/{name}/sessions/
memory_distiller.py ← Short/mid/long distill jobs (APScheduler)
cron_runner.py ← Cron job storage, schedule parsing, job execution
scheduler.py ← APScheduler setup (distill + user crons)
event_bus.py ← Internal SSE pub/sub (NC Talk → browser)
auth_utils.py ← bcrypt passwords, JWT create/decode, invite token system
auth_middleware.py ← SessionAuthMiddleware — JWT cookie validation on all routes
persona_template.py ← Bootstrap a new persona directory from string templates
email_utils.py ← SMTP_SSL email helpers (invite emails, future notifications)
routers/
chat.py ← POST /chat (streaming SSE)
orchestrator.py ← POST /orchestrate, GET /orchestrate/{job_id}
auth.py ← GET /auth/status (Claude + Gemini CLI token checks)
distill.py ← POST /distill/*, GET /distill/status
files.py ← GET /files (persona file browser)
nextcloud_talk.py ← POST /webhook/nextcloud (NC Talk bot)
google_chat.py ← POST /webhook/google (Google Chat Add-on)
ui.py ← Login/logout, /{user}/{persona} UI route, /api/personas
onboarding.py ← /setup/{token} password step + /setup/persona creation
settings.py ← /settings, /settings/notifications, /settings/integrations (admin)
tools_settings.py ← /settings/tools
crons.py ← /settings/crons — Schedules web UI (list/add/edit/toggle/remove)
tools/
__init__.py ← Tool registry (Gemini FunctionDeclarations + dispatcher)
web.py ← DuckDuckGo web_search tool
scratch.py ← Scratchpad tools (scratch_read/write/append/clear)
tasks.py ← Personal task management (task_create/list/update/complete)
cron.py ← Scheduled job tools (cron_list/add/remove/toggle); 5 types; hourly/daily/weekly/monthly/yearly schedules
system.py ← Local machine tools (claude_allow_dir)
tests/ ← pytest test suite (80 tests)
static/ ← Single-page web UI (index.html, style.css, app.js)
login.html — login form (dark theme, POST /login)
setup.html — onboarding form (password + persona creation)
data/sessions/ ← Persisted session JSON files
home/ ← User and persona data (Linux home layout)
scott/
persona/
inara/ ← Inara identity, memory, context, sessions
IDENTITY.md ← Who Inara is
SOUL.md ← Values, personality, voice
PROTOCOLS.md ← Behavioral rules
CONTEXT_TIERS.md ← What each tier (14) includes in the system prompt
USER.md ← Scott's profile (loaded into context)
HELP.md ← In-app help content (rendered in UI)
MEMORY_LONG.md ← Long-term memory (auto-distilled monthly)
MEMORY_MID.md ← Mid-term memory (auto-distilled weekly)
MEMORY_SHORT.md ← Short-term memory (auto-distilled daily)
REMINDERS.md ← Pending reminders (auto-surfaced in context at tier 2+)
SCRATCH.md ← Ephemeral scratchpad
TASKS.json ← Personal task list
CRONS.json ← Scheduled jobs
sessions/ ← Session turn logs (YYYY-MM-DD.md)
holly/
persona/
tina/ ← Tina (Holly's persona) — same structure as inara/
docs/ ← Integration reference docs
NEXTCLOUD_TALK_BOT.md
OPEN_WEBUI_API.md ← Open WebUI API: tool calling, RAG, model management
documentation/ ← Architecture decisions and agent task list
TODO__Agents.md ← READ THIS FIRST — active task list
ARCH__Intelligence_Layer.md ← Orchestrator, dev agent, knowledge architecture
docker-compose.yml ← Docker deployment
.env.default ← Reference config (copy to .env, fill in secrets)
README.md ← Project orientation
```
---
## Run Commands
```bash
# First-time setup or update on any machine
python3 install.py
# Restart service (after any Python change)
systemctl --user restart cortex
# Syntax check a file before restarting
python3 -m py_compile cortex/<file>.py
# Syntax check all routers
for f in cortex/routers/*.py cortex/tools/*.py cortex/orchestrator_engine.py; do
python3 -m py_compile "$f" && echo "OK: $f"
done
# Install/update dependencies
cd cortex && .venv/bin/pip install -r requirements.txt
# Logs
journalctl --user -u cortex -f
# Web UI (local)
http://localhost:8000
# Swagger docs
http://localhost:8000/docs
```
---
## Key Design Decisions
### Two-Brain Architecture (Orchestrator / Responder)
- **Gemini API** (`orchestrator_engine.py`) — runs the ReAct tool loop; handles tool calling, planning, research
- **Claude CLI** (`llm_client.py`) — produces all user-facing responses; receives enriched context from Gemini
- **Direct chat** bypasses the orchestrator entirely — `POST /chat` goes straight to Claude (faster)
- **Orchestrated tasks** go to `POST /orchestrate` — returns a job_id, result is polled
### LLM Backends
- `llm_client.py` manages Claude CLI (`claude --print`), Gemini CLI (`gemini -p`), and Anthropic SDK (`anthropic_api` type) subprocesses/calls
- `orchestrator_engine.py` uses the Gemini **API** (google-genai SDK) — completely separate from the Gemini CLI
- Claude OAuth token is read live from `~/.claude/.credentials.json` (never rely on stale env var)
- `anthropic_api` backend: user-configured API key from `providers.anthropic.credentials` in `model_registry.json` — uses `anthropic.AsyncAnthropic`
### Tool Strategy
- Orchestrator tools live in `cortex/tools/` — separate from the `ae_*` MCP tools
- **Do not modify** the `ae_*` MCP server to support orchestrator needs; add new tools to `cortex/tools/` instead
- Tools are registered in `cortex/tools/__init__.py` as both Gemini FunctionDeclarations and Python callables
### Context / Memory
- `context_loader.py` assembles Inara's system prompt from `inara/` files based on tier (14)
- Tier 1 = minimal (identity only); Tier 2 = standard (+ memory + user profile); Tier 3 = + last 2 sessions; Tier 4 = + last 7 sessions
- Memory files are written by the distiller or manually — do not delete them
### Security / Safety
- **Never `rm`** — move files to `~/tmp/gemini_trash`
- **Never commit secrets** — `.env` is gitignored; use `.env.default` as the reference
- `NEXTCLOUD_TALK_BOT_SECRET` and `GEMINI_API_KEY` live in `.env` only
- `/channels/*` and `/health` are publicly exposed (webhook auth is handled at app layer — JWT/HMAC)
- `/login`, `/logout`, `/setup/*`, `/static/*` are public — all other routes require a valid JWT session cookie
- `SessionAuthMiddleware` (`auth_middleware.py`) validates the cookie on every request; browsers are redirected to `/login`, API calls get 401
- Passwords are bcrypt-hashed and stored in `home/{username}/auth.json` — never in `.env` or the DB
- Invite tokens are one-time-use, 72-hour expiry, stored in `home/{username}/invite.json`
### Onboarding Flow
New users follow a three-step setup before reaching the chat:
1. `GET /setup/{token}` → password form → `POST /setup/{token}` sets password + session cookie
2. `GET /setup/persona` → persona creation form → `POST /setup/persona` bootstraps persona directory
3. `GET /setup/model` → OpenRouter quick-connect → `POST /setup/model` saves host + model + role assignment
Step 3 is optional (skip link goes straight to `/{user}/{persona}`). `/setup/model` also works
standalone (accessible from Settings) for existing users who haven't configured a model.
All in `cortex/routers/onboarding.py`. Model writes use `model_registry.py`: `save_host()`,
`save_model()`, `set_role(username, "chat", "primary", model_id)`.
### Documentation Philosophy
Cortex is a no-black-box system. Docs must match reality — at all times.
- **Docs first:** When planning significant changes, update `TODO__Agents.md` and the relevant
`ARCH__*.md` to describe the intended design *before* implementing. This creates a spec to
implement against.
- **Verify after:** Once implementation is complete, re-read the pre-written docs and confirm
they match what was actually built. Update anything that drifted.
- **HELP.md is a user contract:** It describes what users can do. Never let it describe
features that don't exist or omit features that do.
- **CLAUDE.md + ARCH__*.md are the developer contract:** Update them as the architecture evolves.
- **Stale docs are bugs.** If you notice drift, fix it before moving on.
### Doc update checklist (run after any significant change)
| Doc | Update when |
|---|---|
| `CLAUDE.md` | New tool, channel, router, major design change, tool count |
| `cortex/static/HELP.md` | Any user-visible feature — tools, settings, UI, API endpoints |
| `documentation/TODO__Agents.md` | Mark completed items; add new planned work |
| `documentation/MASTER.md` | New capability goes live; tool count changes |
| `documentation/ROADMAP.md` | Phase items completed or added |
| `documentation/ARCH__CHANNELS.md` | New channel, notification trigger, or scheduler job |
| `documentation/ARCH__SYSTEM.md` | New module, router, or tools/ file |
| `README.md` | Architecture diagram, channels table, or setup steps change |
---
## Adding a New Tool
1. Implement the tool function in `cortex/tools/<domain>.py`
- Must be `async def`; use `asyncio.to_thread` for blocking calls
- Return a plain string result
2. Add a `FunctionDeclaration` and register it in `cortex/tools/__init__.py`:
- Import the callable
- Add to `TOOL_CATEGORIES` (pick an existing category or create one)
- Add to `_CALLABLES`
- Add a `TOOL_RISK` rating (low/medium/high)
- Add to `TOOL_ROLES` if admin-only; add to `CONFIRM_REQUIRED` if destructive
- Add module to `_ALL_DECLARATIONS`
3. Syntax check: `python3 -m py_compile cortex/tools/<domain>.py`
4. Restart Cortex
## Managing Claude Code Directory Permissions
Claude Code prompts (or silently hangs) when it needs to read or write a directory outside
its current working directory. The `claude-allow-dir` script patches `~/.claude/settings.json`
to add auto-allow rules so Claude no longer blocks on those paths.
### Script: `~/.local/bin/claude-allow-dir`
```bash
# Allow read + write (default)
claude-allow-dir ~/OSIT_dev/aether_api_fastapi
# Read-only
claude-allow-dir ~/agents_sync r
# Write-only
claude-allow-dir /tmp w
```
Adds `Read(path/*)` and/or `Edit(path/*)` + `Write(path/*)` entries to the `permissions.allow`
array in `~/.claude/settings.json`. Idempotent — safe to run twice on the same path.
Changes take effect in the next Claude Code session (or after opening `/hooks` in the UI).
### Orchestrator tool: `claude_allow_dir`
Cortex exposes this as a Gemini tool (`cortex/tools/system.py`) so the orchestrator can add
allow rules on Inara's behalf without human intervention.
**Security note:** This tool modifies Claude Code's own permission settings. The Gemini
orchestrator calling it can grant Claude access to any directory on the machine. Keep this
in mind when evaluating orchestrator behavior — it should only be invoked when Scott has
clearly asked for a directory to be unblocked.
## Adding a New Router
1. Create `cortex/routers/<name>.py` with `router = APIRouter()`
2. Import and register in `cortex/main.py`
3. Syntax check, restart
---
## Current State (2026-05-12)
Cortex is running and stable. All channels are live:
| Channel | Status | Notes |
|---|---|---|
| Web UI | ✅ Live | `https://cortex.dgrzone.com` — PWA-installable |
| Nextcloud Talk | ✅ Live | HMAC-signed webhook, async reply |
| Google Chat | ✅ Live | Workspace Add-on, `hostAppDataAction` response format |
| Local backend | ✅ Live | Open WebUI/Ollama on scott_gaming, per-user multi-model config |
| Gemini orchestrator | ✅ Live | Gemini API tool loop → Claude response; ⚡ toggle in UI |
| Local orchestrator | ✅ Live | OpenAI-compatible ReAct loop; fires when orchestrator role → local model |
| Tool audit log | ✅ Live | Every tool call logged to `home/{user}/tool_audit/YYYY-MM-DD.jsonl` |
| Token usage tracking | ✅ Live | Per-user `home/{user}/usage.json`; summary in Settings |
| Web push | ✅ Live | VAPID push notifications; `web_push` tool; subscribe via ☰ menu |
| Proactive notifications | ✅ Live | Daily reminder check (09:00); distill/cron completions; `GET /settings/notifications` dedicated page |
| Proactive cron | ✅ Live | 5 job types: `remind`, `note`, `message`, `brief`, `task` (full orchestrator loop); monthly/yearly schedule formats; HA inbound webhook tools toggle |
| Schedules web UI | ✅ Live | `/settings/crons` — list, add, edit, pause/resume, delete scheduled jobs |
Active users: scott (inara), holly (tina), brian (wintermute)
**69 orchestrator tools** across 17 domain modules:
web_search/http_fetch/web_read/http_post,
project_file_read/list + file_stat/grep/diff/syntax_check (project-scoped),
file_read/list/write/session_read/session_search (system-scoped, admin),
git_status/git_log/git_diff (read-only git inspection, project-scoped),
shell_exec/claude_allow_dir,
cortex_restart/logs/status/update,
task_list/create/update/complete, cron_list/add/remove/toggle,
reminders_add/list/remove/clear, scratch_read/write/append/clear,
web_push/email_send/nc_talk_send/nc_talk_history,
ae_journal_list/search/entries_list/entry_read/create/update/disable/append/prepend,
ae_task_list, ae_db_query/describe/show_view (SELECT-only MariaDB access, admin; disable requires confirm),
agent_notes_read/write/append/clear, spawn_agent/aider_run (admin; aider_run requires confirm),
agent_status/agent_list (user-level)/agent_cancel (admin, confirm-required),
ha_get_state/ha_get_states/ha_call_service.
Each tool has a `TOOL_RISK` rating (low/medium/high). Configure access at `/settings/tools`
(max_risk threshold + per-tool whitelist/blacklist). Risk policy stored in `home/{user}/tool_policy.json`.
See `documentation/TODO__Agents.md` for the active task list.
See `documentation/ROADMAP.md` for phases and what's next.
---
## Related Docs
| File | Purpose |
|---|---|
| `documentation/MASTER.md` | **Start here** — index, current state, all doc links |
| `documentation/TODO__Agents.md` | Active task list — read before starting work |
| `documentation/ROADMAP.md` | Phases — what's done, what's next |
| `documentation/ARCH__SYSTEM.md` | System architecture and component map |
| `documentation/ARCH__BACKENDS.md` | LLM backends, routing, per-user config |
| `documentation/ARCH__PERSONA.md` | Persona system, context tiers, memory distillation |
| `documentation/ARCH__CHANNELS.md` | Input channels — web, NC Talk, Google Chat, cron |
| `documentation/ARCH__FUTURE.md` | Planned: local orchestrator, dev agents, knowledge layer |
| `~/agents_sync/projects/CORTEX.md` | Project vision and philosophy |
| `~/agents_sync/CLAUDE.md` | Fleet coordination rules |
| `~/CLAUDE.md` | Machine identity (`scott_lpt`) |

View File

@@ -0,0 +1,75 @@
{
"folders": [
{
"name": "cortex (service)",
"path": "cortex"
},
{
"name": "home (personas)",
"path": "home"
},
{
"name": "documentation",
"path": "documentation"
},
{
"name": "docs (integrations)",
"path": "docs"
},
{
"name": "project root",
"path": "."
}
],
"settings": {
"files.exclude": {
"**/__pycache__": true,
"**/*.pyc": true,
"cortex/.venv": true,
"cortex/data": true
},
"search.exclude": {
"**/__pycache__": true,
"cortex/.venv": true,
"cortex/data": true,
"home/**/sessions": true,
"home/**/session_data": true
},
"[python]": {
"editor.formatOnSave": false
},
"editor.rulers": [100],
"files.associations": {
"*.env": "dotenv",
"*.env.default": "dotenv"
}
},
"extensions": {
"recommendations": [
"ms-python.python",
"ms-python.vscode-pylance",
"humao.rest-client",
"tamasfe.even-better-toml"
]
},
"launch": {
"version": "0.2.0",
"configurations": [
{
"name": "Cortex (uvicorn dev)",
"type": "python",
"request": "launch",
"module": "uvicorn",
"args": [
"main:app",
"--host", "0.0.0.0",
"--port", "8000",
"--reload"
],
"cwd": "${workspaceFolder:cortex (service)}",
"envFile": "${workspaceFolder:cortex (service)}/.env",
"justMyCode": false
}
]
}
}

269
README.md Normal file
View File

@@ -0,0 +1,269 @@
# Cortex / Inara — Project Root
**Owner:** Scott Idem (One Sky IT / Danger Zone)
**Started:** 2026-03-04
**Status:** Active development
> *"You can't stop the signal."*
Cortex is a self-hosted multi-agent AI platform. It supports multiple users, each with their own named AI persona.
---
## Where Cortex Fits
AI tools aren't one-size-fits-all. Cortex exists in a specific niche — it's not trying to be everything.
**Cortex is a self-hosted persona platform.** It gives you a persistent AI companion with its own
identity, memory, and voice — reachable through your chat apps, not just a browser tab. It remembers
who you are across days and weeks. It can proactively message you on a schedule. It runs on your
own hardware, behind your own auth.
### What Cortex is good at
- **Being a consistent AI presence** — same persona, same memory, day after day
- **Multi-channel access** — web, Nextcloud Talk, Google Chat, all routed to the same brain
- **Proactive work** — scheduled messages, reminders, cron jobs that reach out to you
- **Multi-user households** — each person gets their own persona (Scott → Inara, Holly → Tina)
- **Private, offline-capable** — local models via Ollama when you don't want anything leaving the LAN
### What Cortex is not
- **Not a coding assistant.** Cortex lives in chat apps, not in your terminal or IDE.
Use Claude Code, DeepSeek TUI, Gemini CLI, or Copilot for code-level work — they specialize in reading and
editing project files. Cortex can't open a codebase.
- **Not a generic LLM chat UI.** Open WebUI and LibreChat are excellent model-switching frontends.
Cortex isn't a frontend — it's a platform with its own identity system, orchestrator, and memory
pipeline. Two different jobs.
- **Not a SaaS product.** Nobody else hosts your Cortex instance. Nobody else sees your conversations.
The trade-off is you manage the service yourself — `systemctl --user restart cortex`.
- **Not an agent framework.** LangChain, CrewAI, and similar are libraries for building AI pipelines.
Cortex is a running service with concrete personas, not an abstraction layer to build on top of.
### The stack in practice
- Use **Cortex** to talk to Inara — daily assistant, memory keeper, scheduled check-ins
- Use **Claude Code / DeepSeek TUI** to work *on* Cortex — code edits, architecture, debugging
- Use **Open WebUI** when you want to test a new model or run a quick prompt without persona context
Same AI, different interfaces for different jobs.
---
## Quick Orientation
| Directory | What it is |
|---|---|
| `cortex/` | FastAPI service — dispatcher, routing, LLM backends, session management |
| `home/` | User and persona data (`home/{username}/persona/{name}/`) |
| `docs/` | Integration reference docs (NC Talk bot, Google Chat bot) |
| `documentation/` | Architecture decisions, project plans, agent task lists |
---
## Multi-User Layout
Persona data lives in a two-level tree modelled on Linux home directories:
```
home/
scott/
persona/
inara/ ← IDENTITY.md, SOUL.md, MEMORY_*.md, sessions/, TASKS.json, …
holly/
persona/
tina/
[username]/
persona/
[name]/
```
Each HTTP request includes `user` and `persona` fields. The service validates both against
the `home/` tree before routing. ContextVars ensure per-request isolation in async code.
**Naming rules** (same as Linux usernames): lowercase letters, digits, `_`, `-`; must start
with a letter or underscore; max 32 characters. Example: `scott`, `holly`, `my_ai-v2`.
---
## Setup / Install
Run `install.py` on any machine to set up or update Cortex. It is idempotent — safe to re-run.
```bash
python3 install.py # install / update everything
python3 install.py --check # status check only, no changes
```
What it does: creates the Python venv, installs dependencies, writes the systemd user service,
enables linger, starts/restarts the service, checks LLM CLI auth, and sets up the daily backup timer.
Config: copy `cortex/.env.default` to `cortex/.env` and fill in secrets before first run.
## Running Cortex
Cortex runs as a **systemd user service** (no sudo required).
```bash
# Start / stop / restart
systemctl --user start cortex
systemctl --user stop cortex
systemctl --user restart cortex
# Status and logs
systemctl --user status cortex
journalctl --user -u cortex -f
# Web UI
http://localhost:8000 (or cortex.dgrzone.com on WireGuard)
```
The service starts automatically at boot via `loginctl enable-linger`.
Service file: `~/.config/systemd/user/cortex.service`
Config lives in `cortex/config.py` and `cortex/.env` (not tracked — see `cortex/.env.default`).
## Development Workflow
The codebase lives in `agents_sync/` and syncs to all fleet machines via Syncthing.
Edit code on any machine; use `dev-restart.sh` to apply changes on the host running the service.
```bash
./dev-restart.sh # restart service, show last 30 log lines
./dev-restart.sh logs # tail live logs (ctrl-c to stop)
./dev-restart.sh status # show service status only
```
## Backup
Persona data (`home/`) is excluded from git and backed up with restic.
`install.py` sets up a systemd timer that runs `backup.sh` daily at 03:00.
```bash
./backup.sh # run a backup manually
# Inspect snapshots (set env vars or export them)
RESTIC_REPOSITORY=~/backups/cortex-home-restic \
RESTIC_PASSWORD_FILE=~/.config/cortex/restic-password \
restic snapshots
```
The restic password is generated at `~/.config/cortex/restic-password` on first install.
Back it up separately — it is required to restore from any snapshot.
---
## Key Documentation
**Start here for a full picture:** [`documentation/MASTER.md`](documentation/MASTER.md)
| File | Purpose |
|---|---|
| `documentation/MASTER.md` | Index — current state, all doc links, quick reference |
| `documentation/ROADMAP.md` | Phases — what's done, what's next |
| `documentation/TODO__Agents.md` | Active task list |
| `documentation/ARCH__SYSTEM.md` | System architecture and component map |
| `documentation/ARCH__BACKENDS.md` | LLM backends, routing, fallback |
| `documentation/ARCH__PERSONA.md` | Persona system, context tiers, memory distillation |
| `documentation/ARCH__CHANNELS.md` | Input channels — web, NC Talk, Google Chat, cron |
| `documentation/ARCH__FUTURE.md` | Planned features — local orchestrator, dev agents, knowledge layer |
| `docs/NEXTCLOUD_TALK_BOT.md` | NC Talk bot setup and troubleshooting |
| `docs/GOOGLE_CHAT_BOT.md` | Google Chat Add-on setup |
| `docs/OPEN_WEBUI_API.md` | Open WebUI/Ollama API reference |
---
## Architecture at a Glance
```
[Web UI / NC Talk / Google Chat / Cron / Webhooks]
Cortex Dispatcher (FastAPI, cortex/)
├─ POST /chat — direct to LLM (streaming SSE)
├─ POST /orchestrate — Gemini tool loop → Claude response
├─ POST /webhook/nextcloud/{username} — Nextcloud Talk bot (per-user)
└─ POST /channels/google-chat/{username} — Google Chat Add-on (per-user)
LLM Backends
• Claude CLI — primary, all user-facing responses
• Gemini CLI — fallback
• Gemini API — orchestrator tool loop (two-brain: Gemini plans, Claude responds)
• Local OpenAI — Open WebUI/Ollama on scott_gaming; also runs local orchestrator loop
Persona context loaded from home/{user}/persona/{name}/
```
See `documentation/ARCH__SYSTEM.md` for the full architecture breakdown.
---
## Personas
Each persona has its own identity, memory, and session history.
They are not tied to a specific LLM model — the name is fixed, the backend varies.
Context is loaded at request time from `home/{user}/persona/{name}/` via `cortex/context_loader.py`.
| User | Persona | Description |
|---|---|---|
| scott | inara | Scott's primary AI assistant |
| scott | developer | Scott's dev-focused persona |
| holly | tina | Holly's primary AI assistant |
| brian | wintermute | Brian's primary AI assistant |
---
## Channels
Webhook endpoints are per-user — each user configures their own secrets in `home/{username}/channels.json`.
| Channel | Status | Endpoint / Notes |
|---|---|---|
| Web UI | Live | `https://cortex.dgrzone.com` — session auth (login form + JWT cookie) |
| Nextcloud Talk | Live | `POST /webhook/nextcloud/{username}` — HMAC-signed, async reply |
| Google Chat | Live | `POST /channels/google-chat/{username}` — Workspace Add-on, JWT auth |
| Browser Push | Live | VAPID push notifications — subscribe via ☰ menu; proactive reminders + distill alerts |
See `docs/NEXTCLOUD_TALK_BOT.md` and `docs/GOOGLE_CHAT_BOT.md` for setup instructions.
---
## User Management
```bash
cd cortex
# Create a user directory and send an invite email
.venv/bin/python manage_passwords.py invite <username> <email>
# Register a Google account for sign-in (run after user completes onboarding)
.venv/bin/python manage_passwords.py google-add <username> <email>
# List users with password, Google, and email status
.venv/bin/python manage_passwords.py list
# Set/check a password directly
.venv/bin/python manage_passwords.py set <username>
.venv/bin/python manage_passwords.py check <username>
```
New users receive a link to `/setup/{token}` where they set their own password and create their first persona. Invite tokens expire in 72 hours and are one-time-use.
To enable a channel for a user, create `home/{username}/channels.json` — see the relevant doc in `docs/`.
---
## Testing
```bash
cd cortex
.venv/bin/python -m pytest tests/ -q
```
80 tests covering API endpoints, persona routing, tool functions, and security.
---
## Related Projects
| Project | Path |
|---|---|
| Aether Platform API | `~/OSIT_dev/aether_api_fastapi/` |
| Aether Frontend | `~/OSIT_dev/aether_app_sveltekit/` |
| Fleet coordination | `~/agents_sync/` |

70
backup.sh Executable file
View File

@@ -0,0 +1,70 @@
#!/usr/bin/env bash
# backup.sh — restic backup of Cortex persona/home data
#
# Backs up the home/ directory (all user persona files, memory, tasks, crons).
# Code is in git; this covers everything git intentionally excludes.
#
# Config — override via environment or edit here:
REPO_DIR="${RESTIC_REPOSITORY:-$HOME/backups/cortex-home-restic}"
PASSWORD_FILE="${RESTIC_PASSWORD_FILE:-$HOME/.config/cortex/restic-password}"
SOURCE="$(cd "$(dirname "$0")" && pwd)/home"
# Retention policy
KEEP_DAILY=7
KEEP_WEEKLY=4
KEEP_MONTHLY=6
# ── Preflight ─────────────────────────────────────────────────────────────────
set -euo pipefail
if ! command -v restic &>/dev/null; then
echo "ERROR: restic not found. Install with: sudo pacman -S restic" >&2
exit 1
fi
if [[ ! -d "$SOURCE" ]]; then
echo "ERROR: source directory not found: $SOURCE" >&2
exit 1
fi
# ── Password setup ────────────────────────────────────────────────────────────
if [[ ! -f "$PASSWORD_FILE" ]]; then
mkdir -p "$(dirname "$PASSWORD_FILE")"
chmod 700 "$(dirname "$PASSWORD_FILE")"
python3 -c "import secrets; print(secrets.token_urlsafe(32))" > "$PASSWORD_FILE"
chmod 600 "$PASSWORD_FILE"
echo "Generated new restic password: $PASSWORD_FILE"
echo "IMPORTANT: back this file up separately — you need it to restore."
fi
export RESTIC_REPOSITORY="$REPO_DIR"
export RESTIC_PASSWORD_FILE="$PASSWORD_FILE"
# ── Init repo if needed ───────────────────────────────────────────────────────
if [[ ! -d "$REPO_DIR" ]]; then
echo "Initializing restic repository at $REPO_DIR"
restic init
fi
# ── Backup ────────────────────────────────────────────────────────────────────
echo "Backing up $SOURCE$REPO_DIR"
restic backup "$SOURCE" \
--exclude="**/sessions" \
--exclude="**/session_data" \
--tag "cortex-home"
# ── Prune ─────────────────────────────────────────────────────────────────────
restic forget \
--keep-daily "$KEEP_DAILY" \
--keep-weekly "$KEEP_WEEKLY" \
--keep-monthly "$KEEP_MONTHLY" \
--tag "cortex-home" \
--prune
echo "Backup complete."
restic snapshots --tag cortex-home --last 3

View File

@@ -1,33 +1,118 @@
# Auth is handled by the claude CLI (claude setup-token) — no API key needed here. # Cortex .env reference — copy to .env and fill in values
# ANTHROPIC_API_KEY=only_needed_if_switching_to_sdk # DO NOT commit .env — it contains secrets
# Path to the inara/ identity directory — relative to cortex/ or absolute # ── Agent identity ───────────────────────────────────────────────────────────
INARA_DIR=../inara # Global display names used in distillation prompts and session logs.
# Individual persona identities live in home/{username}/persona/{name}/IDENTITY.md
AGENT_NAME=Inara
USER_NAME=Scott
# Path for persistent JSON session files # ── Home directory ────────────────────────────────────────────────────────────
SESSIONS_DIR=./data/sessions # Root for all user/persona data. Layout: home/{username}/persona/{name}/
# Relative paths are resolved from the cortex/ directory.
# Default: ../home (i.e. Cortex_and_Inara_dev/home/)
# HOME_DIR=../home
# LLM defaults # ── Google OAuth — "Sign in with Google" ────────────────────────────────────
DEFAULT_MODEL=claude-sonnet-4-6 # Create credentials at console.cloud.google.com → APIs & Services → Credentials
DEFAULT_TIER=2 # Application type: Web Application
# Authorised redirect URI: https://cortex.dgrzone.com/auth/google/callback
# Pre-register users: cd cortex && .venv/bin/python manage_passwords.py google-add <user> <email>
# Per-user Gemini key: add "gemini_api_key": "AIza..." to home/{username}/auth.json
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
# Session rolling window — number of messages to keep (user + assistant pairs) # ── Session auth ─────────────────────────────────────────────────────────────
# 40 = 20 turns # Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
MAX_HISTORY_MESSAGES=40 JWT_SECRET=change-me-in-dotenv
JWT_EXPIRE_DAYS=30
# Per-backend timeouts (seconds) # ── SMTP (invite emails + future notifications) ───────────────────────────────
# Gemini is generous — it frequently takes 30-60s under load SMTP_SERVER=linode.oneskyit.com
# Local models may need time to load into VRAM before first response SMTP_PORT=465
SMTP_USERNAME=send_mail
SMTP_PASSWORD=
SMTP_FROM_EMAIL=noreply@oneskyit.com
SMTP_FROM_NAME=Cortex
# Base URL included in invite links
CORTEX_BASE_URL=https://cortex.dgrzone.com
# ── Server ──────────────────────────────────────────────────────────────────
HOST=0.0.0.0
PORT=8000
# ── Google Chat bot ──────────────────────────────────────────────────────────
# JWT audience for verifying inbound Workspace Add-on Chat webhook requests.
# For Workspace Add-on Chat apps, the aud claim = the endpoint URL.
# Leave blank to disable verification (dev/testing only).
GOOGLE_CHAT_AUDIENCE=https://cortex.dgrzone.com/channels/google-chat
# ── Nextcloud Talk bot ───────────────────────────────────────────────────────
NEXTCLOUD_URL=https://cloud.dgrzone.com
NEXTCLOUD_TALK_BOT_SECRET=
# ── LLM backends ────────────────────────────────────────────────────────────
# Primary backend: "claude", "gemini", or "local" (switchable at runtime via UI)
PRIMARY_BACKEND=claude
# Timeouts in seconds
TIMEOUT_CLAUDE=60 TIMEOUT_CLAUDE=60
TIMEOUT_GEMINI=120 TIMEOUT_GEMINI=120
TIMEOUT_LOCAL=300 TIMEOUT_LOCAL=300 # local models may need time to load
# Google Chat — must respond within 30s or Chat shows an error to the user # ── Local model (Open WebUI / Ollama — OpenAI-compatible API) ────────────────
GOOGLE_CHAT_TIMEOUT=25 # Leave LOCAL_API_URL blank to disable. When set, "local" appears as a backend option.
# Backend pinned for Google Chat (claude recommended — more reliable within 25s) # API key: Open WebUI → Settings → Account → API Keys
GOOGLE_CHAT_BACKEND=claude # Model: workspace alias or full Ollama model name
# TODO: add GOOGLE_CHAT_TOKEN for request verification once endpoint is public LOCAL_API_URL=http://192.168.32.19:3000
LOCAL_API_KEY=
LOCAL_MODEL=test-agent-simple
# Server # ── Orchestrator (Gemini API — not Gemini CLI) ───────────────────────────────
PORT=8000 # Required for /orchestrate endpoint and tool use
HOST=0.0.0.0 # Free tier key: https://aistudio.google.com/apikey
GEMINI_API_KEY=
# Model for the orchestration tool loop (not the user-facing response)
ORCHESTRATOR_MODEL=gemini-2.5-flash
# Safety cap on tool loop iterations
ORCHESTRATOR_MAX_ROUNDS=10
# ── DuckDuckGo search ────────────────────────────────────────────────────────
# Leave blank for free unauthenticated tier
# Set to your API key for higher rate limits (paid DuckDuckGo account)
DDG_API_KEY=
DDG_MAX_RESULTS=5
# ── Aether Platform API ───────────────────────────────────────────────────────
# Used by orchestrator tools: ae_journal_search, ae_journal_entry_create, ae_task_list
# Same values as agents_sync/mcp/.env — copy from there
AE_API_URL=https://dev-api.oneskyit.com
AE_API_KEY=
AE_ACCOUNT_ID=
AE_API_TIMEOUT=15
# ── Aether MariaDB (direct — SELECT-only via ae_db_query/describe/show_view tools) ─
# Configured per-user in home/{username}/channels.json — NOT in .env.
# Add this block to the user's channels.json to enable the tools:
#
# "aether_db": {
# "host": "192.168.64.5",
# "port": 3306,
# "name": "aether_dev",
# "user": "aether_dev",
# "password": "..."
# }
# ── Distillation schedule ────────────────────────────────────────────────────
SCHEDULER_TIMEZONE=America/New_York
AUTO_DISTILL=true
AUTO_DISTILL_SHORT=true
AUTO_DISTILL_MID=true
AUTO_DISTILL_LONG=false # manual review recommended before enabling
# Memory tier token budgets (soft caps)
MEMORY_BUDGET_SHORT=3000
MEMORY_BUDGET_MID=2000
MEMORY_BUDGET_LONG=2000

36
cortex/.env.holly Normal file
View File

@@ -0,0 +1,36 @@
# Holly instance .env
# Copy secrets from cortex/.env (API keys, NC Talk secret etc.)
# then customise the identity settings below.
# TODO: Set AGENT_NAME to whatever name Holly chooses for her agent
AGENT_NAME=TBD
USER_NAME=Holly
PORT=8001
HOST=0.0.0.0
INARA_DIR=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/holly
SESSIONS_DIR=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/holly/sessions
DEFAULT_MODEL=claude-sonnet-4-6
DEFAULT_TIER=2
# ── Copy these from cortex/.env ──────────────────────────────────────────────
GEMINI_API_KEY=
AE_API_URL=https://dev-api.oneskyit.com
AE_API_KEY=
AE_ACCOUNT_ID=
NEXTCLOUD_URL=https://cloud.dgrzone.com
NEXTCLOUD_TALK_BOT_SECRET=
# Per-backend timeouts
TIMEOUT_CLAUDE=60
TIMEOUT_GEMINI=120
TIMEOUT_LOCAL=300
SCHEDULER_TIMEZONE=America/New_York
AUTO_DISTILL=true
AUTO_DISTILL_SHORT=true
AUTO_DISTILL_MID=true
AUTO_DISTILL_LONG=false

158
cortex/agent_manager.py Normal file
View File

@@ -0,0 +1,158 @@
"""
Agent lifecycle manager — registry for background spawn_agent and aider_run tasks.
Tracks running and recently completed agents in-process. On completion, fires
notification.notify() if notify=True (same channel used by reminders and cron jobs).
Records are kept for 24 hours after completion, then pruned on next registration.
"""
import asyncio
import logging
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timedelta
logger = logging.getLogger(__name__)
_PRUNE_AFTER = timedelta(hours=24)
_RESULT_PREVIEW_CHARS = 500
_TASK_PREVIEW_CHARS = 200
@dataclass
class AgentRecord:
agent_id: str
level: int # 1 = persona, 2 = specialized sub-agent, 3 = support agent
role: str # e.g. "coder", "research", "chat"
task: str # first _TASK_PREVIEW_CHARS of the task
status: str # running / done / failed / cancelled / timeout
started: datetime
user: str
parent_id: str | None = None # agent_id of the spawner (lineage tracking)
finished: datetime | None = None
result: str | None = None # first _RESULT_PREVIEW_CHARS on completion
notify: bool = False # push notification on completion
_task_ref: "asyncio.Task | None" = field(default=None, repr=False)
# Module-level registry — in-process only, not persisted across restarts.
_agents: dict[str, AgentRecord] = {}
_lock = asyncio.Lock()
async def register(
user: str,
role: str,
task: str,
level: int = 2,
parent_id: str | None = None,
notify: bool = False,
) -> AgentRecord:
"""Create and register a new running agent. Returns the record (agent_id is set)."""
agent_id = str(uuid.uuid4())
rec = AgentRecord(
agent_id=agent_id,
level=level,
role=role,
task=task[:_TASK_PREVIEW_CHARS],
status="running",
started=datetime.now(),
user=user,
parent_id=parent_id,
notify=notify,
)
async with _lock:
_prune_locked()
_agents[agent_id] = rec
logger.info(
"agent_manager: registered %s role=%s level=%d user=%s task=%.60s",
agent_id[:8], role, level, user, task,
)
return rec
def set_task_ref(agent_id: str, task_ref: "asyncio.Task") -> None:
"""Store the asyncio.Task reference so it can be cancelled later.
Call immediately after asyncio.create_task() — before the event loop yields.
"""
rec = _agents.get(agent_id)
if rec:
rec._task_ref = task_ref
async def finish(agent_id: str, result: str, status: str = "done") -> None:
"""Mark an agent complete, store the result, and notify the user if requested."""
async with _lock:
rec = _agents.get(agent_id)
if not rec:
return
rec.status = status
rec.finished = datetime.now()
rec.result = (result or "")[:_RESULT_PREVIEW_CHARS]
logger.info("agent_manager: finished %s status=%s", agent_id[:8], status)
if rec.notify and status != "cancelled":
try:
from notification import notify as _notify
elapsed = int((rec.finished - rec.started).total_seconds())
emoji = "" if status == "done" else "⚠️"
preview = (rec.result or "(no output)")[:200]
msg = f"{emoji} Agent done [{rec.role}, {elapsed}s]: {preview}"
await _notify(rec.user, msg)
except Exception as e:
logger.warning("agent_manager: notification failed for %s: %s", agent_id[:8], e)
async def cancel_agent(agent_id: str, user: str) -> str:
"""Cancel a running background agent. Returns a human-readable status message."""
async with _lock:
rec = _agents.get(agent_id)
if not rec:
return f"No agent found: {agent_id}"
if rec.user != user:
return "Access denied."
if rec.status != "running":
return f"Agent {agent_id[:8]}… is already {rec.status}."
task_ref = rec._task_ref
rec.status = "cancelled"
rec.finished = datetime.now()
if task_ref and not task_ref.done():
task_ref.cancel()
logger.info("agent_manager: cancelled %s by user=%s", agent_id[:8], user)
return f"Agent {agent_id[:8]}… cancelled."
def get(agent_id: str) -> AgentRecord | None:
"""Look up an agent record by ID."""
return _agents.get(agent_id)
def list_agents(user: str, status: str | None = None, limit: int = 10) -> list[AgentRecord]:
"""Return recent agents for a user, newest first.
Does not acquire the lock — safe for read-only listing (Python dict iteration is
thread-safe for reads; we don't care about racing with a concurrent registration).
"""
records = [r for r in _agents.values() if r.user == user]
if status:
records = [r for r in records if r.status == status]
records.sort(key=lambda r: r.started, reverse=True)
return records[:limit]
def _prune_locked() -> None:
"""Remove completed agents older than _PRUNE_AFTER. Must be called inside _lock."""
cutoff = datetime.now() - _PRUNE_AFTER
stale = [
aid for aid, r in _agents.items()
if r.status != "running" and r.finished and r.finished < cutoff
]
for aid in stale:
del _agents[aid]
if stale:
logger.debug("agent_manager: pruned %d stale records", len(stale))

52
cortex/auth_middleware.py Normal file
View File

@@ -0,0 +1,52 @@
"""
Session auth middleware.
Validates the JWT cookie on every request. Unprotected paths are explicitly
listed in _PUBLIC. Webhook endpoints have their own auth (HMAC/JWT) so they
are also excluded.
Sets request.state.session_user to the authenticated username so downstream
routers can enforce ownership without re-reading the cookie.
"""
import jwt
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import RedirectResponse, JSONResponse
from auth_utils import COOKIE_NAME, decode_token
# Paths that don't require a session cookie
_PUBLIC = {"/login", "/logout", "/health", "/manifest.json", "/sw.js", "/favicon.ico",
"/api/push/vapid-key"}
# Path prefixes that are always public (setup flow + webhooks + Google OAuth)
_PUBLIC_PREFIXES = ("/setup/", "/channels/", "/webhook/", "/auth/google")
class SessionAuthMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
path = request.url.path
# Always allow public paths and setup/webhook prefixes
if path in _PUBLIC or any(path.startswith(p) for p in _PUBLIC_PREFIXES):
return await call_next(request)
# Allow static assets without a cookie
if path.startswith("/static/"):
return await call_next(request)
# Validate session cookie
token = request.cookies.get(COOKIE_NAME)
if token:
try:
request.state.session_user = decode_token(token)
return await call_next(request)
except jwt.InvalidTokenError:
pass
# No valid session — redirect browser requests, 401 for API/JSON
accept = request.headers.get("accept", "")
if "text/html" in accept:
return RedirectResponse("/login", status_code=302)
return JSONResponse({"detail": "Not authenticated"}, status_code=401)

260
cortex/auth_utils.py Normal file
View File

@@ -0,0 +1,260 @@
"""
Authentication utilities — password hashing and JWT session tokens.
Passwords are stored as bcrypt hashes in home/{username}/auth.json.
Sessions are JWT cookies signed with JWT_SECRET from settings.
Usage:
set_password("scott", "mypassword") # admin setup
check_credentials("scott", "mypassword") # login validation
create_token("scott") # returns JWT string
decode_token(token) # returns username or raises
"""
import json
import logging
import secrets
from datetime import datetime, timedelta, timezone
from pathlib import Path
import bcrypt
import jwt
from config import settings
logger = logging.getLogger(__name__)
COOKIE_NAME = "cortex_session"
ALGORITHM = "HS256"
# ---------------------------------------------------------------------------
# auth.json helpers — read/write without clobbering unrelated fields
# ---------------------------------------------------------------------------
def _auth_path(username: str) -> Path:
return settings.home_root() / username / "auth.json"
def _read_auth(username: str) -> dict:
path = _auth_path(username)
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except Exception:
return {}
def _write_auth(username: str, data: dict) -> None:
path = _auth_path(username)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, indent=2) + "\n")
# ---------------------------------------------------------------------------
# Password helpers
# ---------------------------------------------------------------------------
def set_password(username: str, password: str) -> None:
"""Hash and store a password. Preserves any existing fields in auth.json."""
data = _read_auth(username)
data["password_hash"] = bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
_write_auth(username, data)
logger.info("password set for user: %s", username)
def check_credentials(username: str, password: str) -> bool:
"""Return True if username+password are valid, False otherwise."""
try:
stored = _read_auth(username).get("password_hash", "").encode()
if not stored:
return False
return bcrypt.checkpw(password.encode(), stored)
except Exception:
return False
# ---------------------------------------------------------------------------
# Google OAuth helpers
# ---------------------------------------------------------------------------
def find_user_by_google(sub: str, email: str) -> str | None:
"""
Scan all users for one whose auth.json matches the given Google sub or email.
Sub match takes priority (stable); email match is a fallback for first sign-in.
Returns the username, or None if no match.
"""
root = settings.home_root()
if not root.exists():
return None
for user_dir in sorted(root.iterdir()):
if not user_dir.is_dir():
continue
data = _read_auth(user_dir.name)
if not data:
continue
if sub and data.get("google_sub") == sub:
return user_dir.name
if email and data.get("google_email", "").lower() == email.lower():
return user_dir.name
return None
def link_google(username: str, sub: str, email: str) -> None:
"""Store / update Google sub and email in a user's auth.json."""
data = _read_auth(username)
data["google_sub"] = sub
data["google_email"] = email
_write_auth(username, data)
logger.info("Google account linked for user: %s (%s)", username, email)
def get_user_gemini_key(username: str) -> str | None:
"""Return the user's personal Gemini API key, or None to use the server key."""
return _read_auth(username).get("gemini_api_key") or None
def get_user_role(username: str) -> str:
"""Return the user's role: 'admin' or 'user' (default).
Role is stored as auth.json["role"]. Any unrecognised value falls back to 'user'.
Set via: manage_passwords.py role <username> admin|user
"""
role = _read_auth(username).get("role", "user")
return role if role in ("admin", "user") else "user"
# ---------------------------------------------------------------------------
# JWT helpers
# ---------------------------------------------------------------------------
def create_token(username: str) -> str:
"""Return a signed JWT encoding the username."""
expire = datetime.now(timezone.utc) + timedelta(days=settings.jwt_expire_days)
payload = {"sub": username, "exp": expire}
return jwt.encode(payload, settings.jwt_secret, algorithm=ALGORITHM)
def decode_token(token: str) -> str:
"""Decode a JWT and return the username. Raises jwt.InvalidTokenError on failure."""
payload = jwt.decode(token, settings.jwt_secret, algorithms=[ALGORITHM])
return payload["sub"]
# ---------------------------------------------------------------------------
# Invite tokens — one-time setup links for new users
# ---------------------------------------------------------------------------
def _invite_path(username: str) -> Path:
return settings.home_root() / username / "invite.json"
def create_invite(username: str, expire_hours: int = 72) -> str:
"""
Generate a one-time invite token for a user and save it to invite.json.
Returns the raw token string (embed in a URL).
"""
token = secrets.token_urlsafe(32)
expires = (datetime.now(timezone.utc) + timedelta(hours=expire_hours)).isoformat()
user_dir = settings.home_root() / username
user_dir.mkdir(parents=True, exist_ok=True)
_invite_path(username).write_text(
json.dumps({"token": token, "expires_at": expires, "used": False}) + "\n"
)
logger.info("invite created for user: %s (expires %s)", username, expires[:10])
return token
def validate_invite(token: str) -> str | None:
"""
Check an invite token across all users.
Returns the username if valid and unused, None otherwise.
"""
root = settings.home_root()
if not root.exists():
return None
for user_dir in root.iterdir():
if not user_dir.is_dir():
continue
inv_path = user_dir / "invite.json"
if not inv_path.exists():
continue
try:
data = json.loads(inv_path.read_text())
except Exception:
continue
if data.get("used"):
continue
if data.get("token") != token:
continue
expires = datetime.fromisoformat(data["expires_at"])
if datetime.now(timezone.utc) > expires:
continue
return user_dir.name
return None
def consume_invite(username: str) -> None:
"""Mark the invite token for a user as used."""
path = _invite_path(username)
if path.exists():
try:
data = json.loads(path.read_text())
data["used"] = True
path.write_text(json.dumps(data) + "\n")
except Exception:
pass
# ---------------------------------------------------------------------------
# Per-user channel config
# ---------------------------------------------------------------------------
def _channels_path(username: str) -> Path:
return settings.home_root() / username / "channels.json"
def get_user_channels(username: str) -> dict:
"""Return the parsed channels.json for a user, or {} if not found."""
path = _channels_path(username)
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except Exception:
return {}
def get_tool_policy(username: str) -> dict:
"""Return the parsed tool_policy.json for a user.
Confirmation-gate keys (existing):
allow — tools in CONFIRM_REQUIRED that this user has pre-approved (skip gate)
deny — tools always blocked for this user regardless of global CONFIRM_REQUIRED
Risk-policy keys (new):
max_risk — auto-include all tools at/below this level ("low"|"medium"|"high")
whitelist — force-include specific tools above max_risk
blacklist — force-exclude specific tools regardless of max_risk
"""
path = settings.home_root() / username / "tool_policy.json"
try:
return json.loads(path.read_text())
except Exception:
return {}
def get_risk_policy(username: str) -> tuple[str | None, list[str], list[str]]:
"""Return (max_risk, whitelist, blacklist) from the user's tool policy."""
policy = get_tool_policy(username)
return (
policy.get("max_risk") or None,
policy.get("whitelist") or [],
policy.get("blacklist") or [],
)
def save_tool_policy(username: str, data: dict) -> None:
path = settings.home_root() / username / "tool_policy.json"
path.write_text(json.dumps(data, indent=2) + "\n")

View File

@@ -4,38 +4,125 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings): class Settings(BaseSettings):
anthropic_api_key: str | None = None # not used — claude CLI handles auth anthropic_api_key: str | None = None # not used — claude CLI handles auth
inara_dir: Path = Path("../inara")
# Google OAuth — "Sign in with Google" for all users
# Create credentials at console.cloud.google.com → APIs & Services → Credentials
# Add https://<your-domain>/auth/google/callback as an authorised redirect URI
google_client_id: str | None = None
google_client_secret: str | None = None
# Orchestrator (Gemini API — separate from Gemini CLI)
# Get a key at: https://aistudio.google.com/apikey (free tier is sufficient)
gemini_api_key: str | None = None
orchestrator_model: str = "gemini-2.5-flash" # model used for tool loop
orchestrator_max_rounds: int = 10 # safety cap on tool iterations
# DuckDuckGo search (used by orchestrator web_search tool)
# Leave blank to use the free unauthenticated tier; set to your API key for higher limits
ddg_api_key: str | None = None
ddg_max_results: int = 5
# Aether Platform API (used by orchestrator ae_journal_* and ae_task_list tools)
ae_api_url: str = "https://dev-api.oneskyit.com"
ae_api_key: str = "" # x-aether-api-key header
ae_account_id: str = "" # x-account-id header
ae_api_timeout: int = 15 # per-request timeout in seconds
# Agent identity — used in prompts, session logs, and memory distillation
# Override in .env for each instance (e.g. AGENT_NAME=Holly, USER_NAME=Holly)
agent_name: str = "Inara"
user_name: str = "Scott"
home_dir: Path = Path("../home")
sessions_dir: Path = Path("./data/sessions") sessions_dir: Path = Path("./data/sessions")
default_model: str = "claude-sonnet-4-6" default_model: str = "claude-sonnet-4-6"
default_tier: int = 2 default_tier: int = 2
max_history_messages: int = 40 # rolling window — 20 turns (user + assistant) max_history_messages: int = 40 # rolling window — 20 turns (user + assistant)
primary_backend: str = "claude" # "claude" or "gemini" — other is always fallback primary_backend: str = "claude" # "claude" or "gemini" — other is always fallback
# Local model backend — OpenAI-compatible API (Open WebUI / Ollama)
# Set LOCAL_API_URL in .env to enable; leave blank to disable
local_api_url: str = "" # e.g. http://192.168.32.19:3000
local_api_key: str = "" # sk-... from Open WebUI → Settings → Account → API Keys
local_model: str = "" # workspace or model name, e.g. test-agent-simple
# Per-backend timeouts in seconds # Per-backend timeouts in seconds
timeout_claude: int = 60 timeout_claude: int = 60
timeout_gemini: int = 120 # frequently slow under load timeout_gemini: int = 120 # frequently slow under load
timeout_local: int = 300 # local models may need to load first timeout_local: int = 300 # local models may need to load first
# Google Chat must receive a response within 30s or shows an error to the user # Auto-distillation schedule — override in .env
google_chat_timeout: int = 25 # AUTO_DISTILL=false disables entirely
# Backend forced for Google Chat — Claude is more reliable within the 25s deadline scheduler_timezone: str = "America/New_York" # IANA tz — override in .env if needed
google_chat_backend: str = "claude" auto_distill: bool = True
auto_distill_short: bool = True # daily at 03:00 — rolls session logs → MEMORY_SHORT
auto_distill_mid: bool = True # weekly Sunday at 03:30 — LLM summarizes short → mid
auto_distill_long: bool = False # monthly 1st at 04:00 — off by default (manual review recommended)
# Nextcloud Talk bot # Which backend to use for distillation LLM calls.
nextcloud_url: str = "https://cloud.dgrzone.com" # "" = use primary_backend (default); "local" = use local model (saves API credits).
nextcloud_talk_bot_secret: str = "" # set in .env # "long" stays on default (claude/gemini) for best quality.
nextcloud_talk_timeout: int = 55 distill_backend_mid: str = ""
distill_backend_long: str = ""
# Model registry: default backend type per role when user registry has no entry.
# Values: "claude_cli" | "gemini_cli" | "gemini_api" (builtin IDs)
# Override in .env: ROLE_CHAT=claude_cli ROLE_DISTILL=gemini_api etc.
role_chat: str = "claude_cli"
role_orchestrator: str = "gemini_api"
role_distill: str = "claude_cli"
role_coder: str = "claude_cli"
role_research: str = "gemini_api"
# Comma-separated list of standard roles shown in the model settings UI.
# Add custom roles here to extend the UI without code changes.
# Example: DEFINED_ROLES=chat,orchestrator,distill,coder,research,medical
defined_roles: str = "chat,orchestrator,distill,coder,research"
# Memory tier token budgets — soft caps used during distillation
# Override in .env: MEMORY_BUDGET_LONG=4000 etc.
memory_budget_long: int = 2000
memory_budget_mid: int = 2000
memory_budget_short: int = 3000
# Session auth
jwt_secret: str = "change-me-in-dotenv" # override in .env: JWT_SECRET=<random>
jwt_expire_days: int = 30
# Web Push (VAPID) — for browser push notifications
# Generate once with py_vapid; see push_utils.py for key format details
vapid_public_key: str = "" # base64url-encoded uncompressed EC point (for browser)
vapid_private_key_b64: str = "" # base64-encoded PEM private key (single-line .env storage)
vapid_contact: str = "mailto:admin@example.com"
# SMTP — for sending invite emails
smtp_server: str = ""
smtp_port: int = 465
smtp_username: str = ""
smtp_password: str = ""
smtp_from_email: str = "noreply@oneskyit.com"
smtp_from_name: str = "Cortex"
# Base URL used in invite links (no trailing slash)
cortex_base_url: str = "https://cortex.dgrzone.com"
host: str = "0.0.0.0" host: str = "0.0.0.0"
port: int = 8000 port: int = 8000
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore") model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
def inara_path(self) -> Path: def get_defined_roles(self) -> list[str]:
"""Resolve inara_dir relative to this file's location if not absolute.""" """Return the ordered list of standard roles from the defined_roles setting."""
if self.inara_dir.is_absolute(): return [r.strip() for r in self.defined_roles.split(",") if r.strip()]
return self.inara_dir
return (Path(__file__).parent / self.inara_dir).resolve() def get_role_default(self, role: str) -> str:
"""Return the .env default backend type for a role (e.g. 'claude_cli')."""
return getattr(self, f"role_{role.replace('-', '_')}", "claude_cli")
def home_root(self) -> Path:
"""Resolve home_dir relative to this file's location if not absolute."""
if self.home_dir.is_absolute():
return self.home_dir
return (Path(__file__).parent / self.home_dir).resolve()
def sessions_path(self) -> Path: def sessions_path(self) -> Path:
"""Resolve sessions_dir relative to this file's location if not absolute.""" """Resolve sessions_dir relative to this file's location if not absolute."""

View File

@@ -1,47 +1,130 @@
from datetime import datetime
from pathlib import Path from pathlib import Path
from config import settings
from persona import persona_path
from tools.reminders import load_due_reminders
_STATIC_DIR = Path(__file__).parent / "static"
# Files loaded per tier — mirrors CONTEXT_TIERS.md # Core identity files — always loaded regardless of tier
TIER_FILES: dict[int, list[str]] = { _CORE = ["SOUL.md", "IDENTITY.md"]
1: ["SOUL.md", "IDENTITY.md"], # + USER.md summary only
2: ["SOUL.md", "IDENTITY.md", "USER.md", "MEMORY.md", "PROTOCOLS.md"],
3: ["SOUL.md", "IDENTITY.md", "USER.md", "MEMORY.md", "PROTOCOLS.md"],
4: ["SOUL.md", "IDENTITY.md", "USER.md", "MEMORY.md", "PROTOCOLS.md"],
}
# Lines of USER.md to include at Tier 1 (just identity + what he cares about) # Lines of USER.md to include at Tier 1 (identity + what he cares about)
TIER_1_USER_LINES = 30 _TIER_1_USER_LINES = 30
def _read(path: Path) -> str: def load_context(
if path.exists(): tier: int = 2,
return path.read_text() include_long: bool = True,
return f"[missing: {path.name}]" include_mid: bool = True,
include_short: bool = True,
role_append: str = "",
inject_datetime: bool = True,
inject_mode: bool = True,
mode: str = "chat",
) -> str:
"""
Build the system-prompt context block for a given tier and memory toggles.
Load order (long → mid → short) keeps the most recent memory closest
to the conversation turn, which improves LLM recall.
def load_context(tier: int = 2) -> str: Tier 1 — SOUL + IDENTITY + USER summary (~1,500 tokens)
inara_dir = settings.inara_path() Tier 2 — + USER full + PROTOCOLS + memory (~5,000 tokens)
Tier 3 — + last 2 raw session logs (~15,000 tokens)
Tier 4 — + last 7 raw session logs (~50,000 tokens)
role_append — optional text injected last (closest to the turn),
sourced from the active role's system_append config.
"""
inara_dir = persona_path()
parts = [] parts = []
files = TIER_FILES.get(tier, TIER_FILES[2]) # ── 0. System block — date/time and session mode (injected first so it's prominent) ──
system_lines = []
if inject_datetime:
now = datetime.now().astimezone()
system_lines.append(f"Current date and time: {now.strftime('%A, %Y-%m-%d at %I:%M %p %Z')}")
if mode == "otr" and inject_mode:
system_lines.append(
"Current mode: Off The Record — "
"this conversation is private and will not be logged or included in memory distillation"
)
if system_lines:
parts.append("--- System ---\n" + "\n".join(system_lines))
for filename in files: # ── 1. Core identity (always) ──────────────────────────────────
for filename in _CORE:
path = inara_dir / filename path = inara_dir / filename
if not path.exists(): if path.exists():
continue parts.append(f"--- {filename} ---\n{path.read_text()}")
if filename == "USER.md" and tier == 1: # ── 2. USER.md ─────────────────────────────────────────────────
# Tier 1: include only the first N lines user_path = inara_dir / "USER.md"
lines = path.read_text().splitlines()[:TIER_1_USER_LINES] if user_path.exists():
if tier == 1:
lines = user_path.read_text().splitlines()[:_TIER_1_USER_LINES]
content = "\n".join(lines) content = "\n".join(lines)
else: else:
content = path.read_text() content = user_path.read_text()
parts.append(f"--- USER.md ---\n{content}")
parts.append(f"--- {filename} ---\n{content}") if tier < 2:
return "\n\n".join(parts)
# ── 3. Protocols + Help reference (tier 2+) ───────────────────
proto_path = inara_dir / "PROTOCOLS.md"
if proto_path.exists():
parts.append(f"--- PROTOCOLS.md ---\n{proto_path.read_text()}")
ops_path = inara_dir / "OPERATIONS.md"
if ops_path.exists():
parts.append(f"--- OPERATIONS.md ---\n{ops_path.read_text()}")
# Global tool reference (same for all personas)
tools_path = _STATIC_DIR / "TOOLS.md"
if tools_path.exists():
parts.append(f"--- TOOLS.md ---\n{tools_path.read_text()}")
# Persona-specific help additions (optional)
help_path = inara_dir / "HELP.md"
if help_path.exists() and help_path.stat().st_size > 10:
parts.append(f"--- HELP.md ---\n{help_path.read_text()}")
# ── 4. Pending reminders (tier 2+) ────────────────────────────
# Only due and undated reminders are surfaced — future-dated ones
# are stored in REMINDERS.md but suppressed until their date arrives.
content = load_due_reminders()
if content:
parts.append(f"--- REMINDERS.md ---\n{content}")
# ── 5. Tiered memory — long → mid → short ─────────────────────
# Short is last so it sits closest to the conversation turn.
if include_long:
# Fall back to legacy MEMORY.md during/after migration
long_path = inara_dir / "MEMORY_LONG.md"
if not long_path.exists():
long_path = inara_dir / "MEMORY.md"
if long_path.exists():
parts.append(f"--- {long_path.name} ---\n{long_path.read_text()}")
if include_mid:
mid_path = inara_dir / "MEMORY_MID.md"
if mid_path.exists() and mid_path.stat().st_size > 100:
content = mid_path.read_text()
if "Not yet populated" not in content:
parts.append(f"--- MEMORY_MID.md ---\n{content}")
if include_short:
short_path = inara_dir / "MEMORY_SHORT.md"
if short_path.exists() and short_path.stat().st_size > 100:
content = short_path.read_text()
if "Not yet populated" not in content:
parts.append(f"--- MEMORY_SHORT.md ---\n{content}")
# ── 6. Raw session logs (tier 3+) ──────────────────────────────
if tier >= 3: if tier >= 3:
# Add recent session logs
sessions_dir = inara_dir / "sessions" sessions_dir = inara_dir / "sessions"
if sessions_dir.exists(): if sessions_dir.exists():
count = 2 if tier == 3 else 7 count = 2 if tier == 3 else 7
@@ -49,4 +132,8 @@ def load_context(tier: int = 2) -> str:
for sf in session_files: for sf in session_files:
parts.append(f"--- Session: {sf.name} ---\n{sf.read_text()}") parts.append(f"--- Session: {sf.name} ---\n{sf.read_text()}")
# ── 7. Role-specific instructions (always last — closest to the turn) ──
if role_append and role_append.strip():
parts.append(f"--- Role Context ---\n{role_append.strip()}")
return "\n\n".join(parts) return "\n\n".join(parts)

309
cortex/cron_runner.py Normal file
View File

@@ -0,0 +1,309 @@
"""
Cron job storage and execution.
Handles reading/writing CRONS.json and running jobs when they fire.
Imported by scheduler.py (to load jobs at startup) and tools/cron.py
(to add/remove jobs at runtime).
Job schema:
{
"id": "c_abc123",
"label": "Human-readable name",
"schedule": "daily:09:00", # see parse_schedule() for all formats
"type": "remind" | "note" | "message" | "brief" | "task",
"payload": "Text or prompt when the job fires",
"channel": null | "nextcloud" | "google_chat", # for message/brief/task types
"enabled": true,
"created_at": "ISO 8601",
"last_run": null | "ISO 8601"
}
Job types:
remind → appends to REMINDERS.md (auto-loaded into context at tier 2+)
note → appends to SCRATCH.md (read on demand via scratch_read)
message → sends payload as-is to notification channel
brief → calls LLM (no tools) with payload as prompt, sends response
(good for morning briefings, summaries, proactive check-ins)
task → runs full orchestrator tool loop with payload as the user request,
sends Claude's response to notification channel
(good for agentic scheduled work: research, file updates, checks)
Tools that require confirmation are skipped — pre-approve them
in Settings → Tools to allow them in scheduled tasks.
"""
import logging
from datetime import datetime, timezone
from pathlib import Path
from persona import persona_path as _persona_path
logger = logging.getLogger(__name__)
_DEFAULT_HOUR = 9
_DEFAULT_MINUTE = 0
_DOW = {
"mon": "mon", "tue": "tue", "wed": "wed", "thu": "thu",
"fri": "fri", "sat": "sat", "sun": "sun",
"monday": "mon", "tuesday": "tue", "wednesday": "wed",
"thursday": "thu", "friday": "fri", "saturday": "sat", "sunday": "sun",
}
# ---------------------------------------------------------------------------
# Storage
# ---------------------------------------------------------------------------
def crons_path(username: str | None = None, persona: str | None = None) -> Path:
return _persona_path(username, persona) / "CRONS.json"
def load_crons(username: str | None = None, persona: str | None = None) -> list[dict]:
p = crons_path(username, persona)
if not p.exists():
return []
try:
import json
return json.loads(p.read_text())
except Exception:
return []
def save_crons(crons: list[dict],
username: str | None = None,
persona: str | None = None) -> None:
import json
crons_path(username, persona).write_text(json.dumps(crons, indent=2) + "\n")
# ---------------------------------------------------------------------------
# Schedule parsing
# ---------------------------------------------------------------------------
def parse_schedule(schedule: str) -> dict:
"""
Convert a human schedule string to APScheduler cron kwargs.
Formats:
"hourly" → every hour at :00
"daily" → every day at 09:00
"daily:HH:MM" → every day at HH:MM
"weekly:DOW" → every DOW at 09:00
"weekly:DOW:HH:MM" → every DOW at HH:MM
"monthly" → 1st of every month at 09:00
"monthly:DD" → day DD of every month at 09:00
"monthly:DD:HH:MM" → day DD of every month at HH:MM
"yearly:MM:DD" → every year on MM/DD at 09:00 (birthdays, anniversaries)
"yearly:MM:DD:HH:MM" → every year on MM/DD at HH:MM
"""
s = schedule.strip().lower()
if s == "hourly":
return {"minute": 0}
if s == "daily":
return {"hour": _DEFAULT_HOUR, "minute": _DEFAULT_MINUTE}
if s.startswith("daily:"):
h, m = _parse_hhmm(s[6:], schedule)
return {"hour": h, "minute": m}
if s.startswith("weekly:"):
rest = s[7:].split(":")
dow = _DOW.get(rest[0])
if not dow:
raise ValueError(
f"Unknown day of week {rest[0]!r}. "
f"Use: mon tue wed thu fri sat sun"
)
if len(rest) == 3:
h, m = _parse_hhmm(f"{rest[1]}:{rest[2]}", schedule)
else:
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
return {"day_of_week": dow, "hour": h, "minute": m}
if s.startswith("monthly"):
rest = s[7:].lstrip(":")
if not rest:
return {"day": 1, "hour": _DEFAULT_HOUR, "minute": _DEFAULT_MINUTE}
parts = rest.split(":")
day = _parse_day(parts[0], schedule)
if len(parts) == 3:
h, m = _parse_hhmm(f"{parts[1]}:{parts[2]}", schedule)
else:
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
return {"day": day, "hour": h, "minute": m}
if s.startswith("yearly:"):
rest = s[7:].split(":")
if len(rest) < 2:
raise ValueError(
f"yearly requires at least MM:DD in {schedule!r}. "
f"Example: yearly:03:15 or yearly:03:15:09:00"
)
month = _parse_month(rest[0], schedule)
day = _parse_day(rest[1], schedule)
if len(rest) == 4:
h, m = _parse_hhmm(f"{rest[2]}:{rest[3]}", schedule)
else:
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
return {"month": month, "day": day, "hour": h, "minute": m}
raise ValueError(
f"Unrecognised schedule {schedule!r}. "
f"Valid formats: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM | "
f"monthly | monthly:DD | monthly:DD:HH:MM | yearly:MM:DD | yearly:MM:DD:HH:MM"
)
def _parse_hhmm(s: str, original: str) -> tuple[int, int]:
parts = s.split(":")
if len(parts) != 2:
raise ValueError(f"Expected HH:MM in {original!r}, got {s!r}")
return int(parts[0]), int(parts[1])
def _parse_day(s: str, original: str) -> int:
try:
d = int(s)
except ValueError:
raise ValueError(f"Expected day number (131) in {original!r}, got {s!r}")
if not 1 <= d <= 31:
raise ValueError(f"Day must be 131 in {original!r}, got {d}")
return d
def _parse_month(s: str, original: str) -> int:
try:
m = int(s)
except ValueError:
raise ValueError(f"Expected month number (112) in {original!r}, got {s!r}")
if not 1 <= m <= 12:
raise ValueError(f"Month must be 112 in {original!r}, got {m}")
return m
# ---------------------------------------------------------------------------
# Execution
# ---------------------------------------------------------------------------
def _now_label() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
async def run_job(job: dict) -> None:
"""Execute a cron job. Called by APScheduler when the job fires."""
job_type = job.get("type")
payload = job.get("payload", "").strip()
label = job.get("label", job.get("id", "cron"))
section = f"\n## {label}{_now_label()}\n\n{payload}\n"
p_root = _persona_path(job.get("user"), job.get("persona"))
if job_type == "remind":
p = p_root / "REMINDERS.md"
existing = p.read_text() if p.exists() else ""
p.write_text(existing.rstrip() + "\n" + section)
logger.info("cron [remind] fired: %s", label)
elif job_type == "note":
p = p_root / "SCRATCH.md"
existing = p.read_text() if p.exists() else ""
p.write_text(existing.rstrip() + "\n" + section)
logger.info("cron [note] fired: %s", label)
elif job_type == "message":
# Send payload text directly to the user's notification channel
from notification import notify
username = job.get("user") or "scott"
channel = job.get("channel") or None
await notify(username, payload, channel=channel)
logger.info("cron [message] sent: %s", label)
elif job_type == "brief":
# Run LLM with payload as the prompt, send response to notification channel.
# Great for morning briefings, reminders, proactive check-ins.
from context_loader import load_context
from llm_client import complete
from notification import notify
from persona import set_context
from config import settings as _s
username = job.get("user") or _s.user_name.lower()
persona_nm = job.get("persona") or _s.agent_name.lower()
channel = job.get("channel") or None
set_context(username, persona_nm)
system_prompt = load_context(2) # tier 2: identity + memory + user profile
try:
response_text, backend = await complete(
system_prompt=system_prompt,
messages=[{"role": "user", "content": payload}],
role="chat",
)
await notify(username, response_text, channel=channel)
logger.info("cron [brief] sent via %s: %s", backend, label)
except Exception as e:
logger.error("cron [brief] LLM error for %s: %s", label, e)
elif job_type == "task":
# Run the full orchestrator tool loop, send Claude's response to the
# notification channel. Tools that require confirmation are skipped in
# cron context — the user is notified to pre-approve them.
from orchestrator_engine import run as _orch_run
from context_loader import load_context
from notification import notify
from persona import set_context
from auth_utils import get_user_gemini_key, get_tool_policy, get_risk_policy
from config import settings as _s
username = job.get("user") or _s.user_name.lower()
persona_nm = job.get("persona") or _s.agent_name.lower()
channel = job.get("channel") or None
set_context(username, persona_nm)
system_prompt = load_context(2)
policy = get_tool_policy(username)
max_risk, whitelist, blacklist = get_risk_policy(username)
gemini_key = get_user_gemini_key(username)
try:
result = await _orch_run(
task=payload,
system_prompt=system_prompt,
gemini_api_key=gemini_key,
respond_with_claude=True,
confirm_allow=set(policy.get("allow") or []),
confirm_deny=set(policy.get("deny") or []),
max_risk=max_risk,
risk_whitelist=whitelist,
risk_blacklist=blacklist,
)
if result.checkpoint:
tool_name = (result.checkpoint.pending_calls[0].name
if result.checkpoint.pending_calls else "unknown tool")
msg = (
f"Scheduled task '{label}' paused — "
f"'{tool_name}' requires confirmation. "
"Pre-approve it in Settings → Tools to allow it in scheduled tasks."
)
await notify(username, msg, channel=channel)
logger.warning("cron [task] %s: confirmation required for %s", label, tool_name)
else:
await notify(username, result.response, channel=channel)
logger.info("cron [task] completed via %s: %s", result.backend, label)
except Exception as e:
logger.error("cron [task] error for %s: %s", label, e)
else:
logger.warning("cron: unknown type %r (job %s)", job_type, job.get("id"))
return
# Record last_run in the right persona's CRONS.json
u, p = job.get("user"), job.get("persona")
crons = load_crons(u, p)
for c in crons:
if c["id"] == job["id"]:
c["last_run"] = datetime.now(timezone.utc).isoformat()
break
save_crons(crons, u, p)

107
cortex/email_utils.py Normal file
View File

@@ -0,0 +1,107 @@
"""
Email utilities for Cortex — invite links and future notifications.
Uses smtplib.SMTP_SSL (port 465). Both plain-text and HTML bodies are sent.
SMTP credentials come from config.settings (set in .env).
"""
import logging
import smtplib
import ssl
from email.headerregistry import Address
from email.message import EmailMessage
from config import settings
logger = logging.getLogger(__name__)
def send_email(
to_email: str,
subject: str,
body_html: str,
body_text: str,
to_name: str = "",
) -> bool:
"""
Send an email via SMTP_SSL.
Returns True on success, False on any failure.
Logs errors but never raises — callers can check the return value.
"""
if not settings.smtp_server:
logger.error("SMTP not configured (SMTP_SERVER is empty)")
return False
msg = EmailMessage()
msg["Subject"] = subject
msg["From"] = Address(
display_name=settings.smtp_from_name,
addr_spec=settings.smtp_from_email,
)
msg["To"] = Address(
display_name=to_name or to_email,
addr_spec=to_email,
)
msg.set_content(body_text)
msg.add_alternative(f"<html><body>{body_html}</body></html>", subtype="html")
logger.info("sending email to %s%s", to_email, subject)
try:
ctx = ssl.create_default_context()
with smtplib.SMTP_SSL(settings.smtp_server, settings.smtp_port, context=ctx) as server:
if settings.smtp_username and settings.smtp_password:
server.login(settings.smtp_username, settings.smtp_password)
server.send_message(msg)
logger.info("email sent to %s", to_email)
return True
except Exception as e:
logger.error("failed to send email to %s: %s", to_email, e)
return False
def send_invite_email(to_email: str, username: str, token: str, to_name: str = "") -> bool:
"""Send a Cortex invite link to a new user."""
url = f"{settings.cortex_base_url}/setup/{token}"
body_text = f"""\
You've been invited to Cortex.
Click the link below to set your password and create your persona:
{url}
This link expires in 72 hours and can only be used once.
— Cortex
"""
body_html = f"""\
<p>You've been invited to <strong>Cortex</strong>.</p>
<p>Click the link below to set your password and create your persona:</p>
<p><a href="{url}" style="
display:inline-block;
padding:10px 20px;
background:#7c3aed;
color:#fff;
text-decoration:none;
border-radius:6px;
font-family:sans-serif;
font-size:15px;
">Set up my account →</a></p>
<p style="font-size:13px;color:#666;">
Or copy this link:<br>
<code>{url}</code>
</p>
<p style="font-size:12px;color:#999;">
This link expires in 72 hours and can only be used once.
</p>
"""
return send_email(
to_email=to_email,
subject="You've been invited to Cortex",
body_html=body_html,
body_text=body_text,
to_name=to_name or username,
)

33
cortex/event_bus.py Normal file
View File

@@ -0,0 +1,33 @@
"""
Simple in-process pub/sub for server-sent events.
Usage:
# Publisher (e.g. nextcloud_talk router)
await event_bus.publish({"type": "nct_message", ...})
# Consumer (SSE endpoint in chat router)
q = event_bus.subscribe()
try:
event = await asyncio.wait_for(q.get(), timeout=20)
finally:
event_bus.unsubscribe(q)
"""
import asyncio
from typing import Any
_subscribers: set[asyncio.Queue] = set()
def subscribe() -> asyncio.Queue:
q: asyncio.Queue = asyncio.Queue()
_subscribers.add(q)
return q
def unsubscribe(q: asyncio.Queue) -> None:
_subscribers.discard(q)
async def publish(event: dict[str, Any]) -> None:
for q in list(_subscribers):
await q.put(event)

View File

@@ -4,6 +4,7 @@ import os
import signal import signal
import subprocess import subprocess
from config import settings from config import settings
import event_bus
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -30,24 +31,83 @@ async def cleanup() -> None:
_active_pgroups.clear() _active_pgroups.clear()
# Map from registry model type → dispatch function key
_TYPE_TO_BACKEND = {
"claude_cli": "claude",
"gemini_cli": "gemini",
"gemini_api": "gemini", # gemini_api falls back to CLI in this context
"local_openai": "local",
"anthropic_api": "anthropic_api",
}
# Explicit UI toggle values (kept for backward compat)
_EXPLICIT_BACKENDS = ("claude", "gemini", "local")
_FALLBACK = {"claude": "gemini", "gemini": "claude", "local": "claude", "anthropic_api": "claude"}
async def complete( async def complete(
system_prompt: str, system_prompt: str,
messages: list[dict], messages: list[dict],
model: str | None = None, model: str | None = None,
role: str = "chat",
slot: str | None = None,
max_tokens: int = 2048, max_tokens: int = 2048,
attachment: dict | None = None,
) -> tuple[str, str]: ) -> tuple[str, str]:
"""Returns (response_text, actual_backend_used).""" """
if model in ("claude", "gemini"): Returns (response_text, actual_backend_used).
slot: Phase 3 — specific role slot ("primary" | "backup_1" | "backup_2").
Resolves that exact slot, no fallback chain. Takes priority over model.
model: legacy backend override ("claude" | "gemini" | "local") from old toggle.
None = resolve via model registry for the given role.
role: registry role used for slot/auto routing (default: "chat").
"""
import model_registry as _reg
from persona import _user
username = _user.get()
resolved_cfg: dict | None = None
if slot is not None:
# Phase 3: explicit slot selection — no fallback within the role
resolved_cfg = _reg.get_model_for_slot(username, role, slot)
if resolved_cfg:
primary = _TYPE_TO_BACKEND.get(resolved_cfg["type"], "claude")
else:
# Slot not configured — fall through to auto routing
slot = None
if slot is None:
if model in _EXPLICIT_BACKENDS:
# Legacy: explicit backend override from old UI toggle
if model == "local":
resolved_cfg = _reg.get_best_local_model(username, role)
if not resolved_cfg:
raise RuntimeError("No local model configured — add one at /settings/models")
primary = model primary = model
else:
# Auto: role-based routing via model registry
resolved = _reg.get_model_for_role(username, role)
if resolved:
resolved_cfg = resolved
primary = _TYPE_TO_BACKEND.get(resolved["type"], "claude")
else: else:
primary = settings.primary_backend primary = settings.primary_backend
fallback = "gemini" if primary == "claude" else "claude" fallback = _FALLBACK.get(primary, "claude")
try: try:
response = await _dispatch(primary, system_prompt, messages, model) response = await _dispatch(primary, system_prompt, messages, resolved_cfg, attachment=attachment)
return response, primary return response, primary
except Exception as e: except Exception as e:
err_str = str(e)
if primary == "claude" and any(k in err_str for k in ("401", "authenticate", "expired", "OAuth")):
await event_bus.publish({"type": "claude_auth_expired"})
# Surface errors when a model is explicitly configured or a specific slot was pinned.
if resolved_cfg is not None:
logger.error("%s failed (no fallback — model explicitly configured): %s", primary, e)
raise
logger.warning("%s failed (%s) — falling back to %s", primary, e, fallback) logger.warning("%s failed (%s) — falling back to %s", primary, e, fallback)
response = await _dispatch(fallback, system_prompt, messages, None) response = await _dispatch(fallback, system_prompt, messages, None)
return response, fallback return response, fallback
@@ -57,11 +117,16 @@ async def _dispatch(
backend: str, backend: str,
system_prompt: str, system_prompt: str,
messages: list[dict], messages: list[dict],
model: str | None, model_cfg: dict | None,
attachment: dict | None = None,
) -> str: ) -> str:
if backend == "gemini": if backend == "gemini":
return await _gemini(system_prompt, messages) return await _gemini(system_prompt, messages)
return await _claude(system_prompt, messages, model) if backend == "local":
return await _local(system_prompt, messages, model_cfg, attachment=attachment)
if backend == "anthropic_api":
return await _anthropic_api(system_prompt, messages, model_cfg)
return await _claude(system_prompt, messages, model_cfg)
def _fresh_claude_token() -> str | None: def _fresh_claude_token() -> str | None:
@@ -81,14 +146,16 @@ def _fresh_claude_token() -> str | None:
return None return None
async def _claude(system_prompt: str, messages: list[dict], model: str | None) -> str: async def _claude(system_prompt: str, messages: list[dict], model_cfg: dict | None) -> str:
model_name = (model_cfg or {}).get("model_name") if model_cfg else None
cmd = [ cmd = [
"claude", "--print", "claude", "--print",
"--no-session-persistence", "--no-session-persistence",
"--output-format", "text", "--output-format", "text",
] ]
if model and model not in ("claude", "gemini"): # Only pass --model if it's a real model name (not a backend type string)
cmd.extend(["--model", model]) if model_name and model_name not in ("claude", "gemini", "local", ""):
cmd.extend(["--model", model_name])
if system_prompt: if system_prompt:
cmd.extend(["--system-prompt", system_prompt]) cmd.extend(["--system-prompt", system_prompt])
cmd.append(_build_conversation(messages)) cmd.append(_build_conversation(messages))
@@ -104,6 +171,137 @@ async def _claude(system_prompt: str, messages: list[dict], model: str | None) -
return await _run(cmd, timeout=settings.timeout_claude, env=env) return await _run(cmd, timeout=settings.timeout_claude, env=env)
async def _local(
system_prompt: str,
messages: list[dict],
model_cfg: dict | None = None,
attachment: dict | None = None,
) -> str:
"""OpenAI-compatible backend — Open WebUI / Ollama.
model_cfg is pre-resolved by complete() via model_registry.
Falls back to registry lookup if not provided.
attachment: optional image dict {filename, mime_type, data} for vision calls.
"""
import httpx
cfg = model_cfg
if not cfg:
# Fallback: resolve directly from registry
import model_registry as _reg
from persona import _user
cfg = _reg.get_best_local_model(_user.get())
if not cfg:
raise RuntimeError("No local model configured — add one at /settings/models")
api_url = cfg["api_url"]
api_key = cfg["api_key"]
model = cfg["model_name"]
if not api_url:
raise RuntimeError("local_api_url not configured — set LOCAL_API_URL in .env or add a host at /settings/models")
if not model:
raise RuntimeError("local_model not configured — add a model at /settings/models")
host_type = cfg.get("host_type", "openwebui")
# "openwebui" uses Open WebUI/Ollama path layout; "openai" uses standard OpenAI layout
chat_path = "/chat/completions" if host_type == "openai" else "/api/chat/completions"
logger.info("local backend (%s): %s @ %s", host_type, model, api_url)
msgs: list[dict] = []
if system_prompt:
msgs.append({"role": "system", "content": system_prompt})
# Build message list; inject image into the last user message when present.
for i, m in enumerate(messages):
is_last = (i == len(messages) - 1)
if is_last and m["role"] == "user" and attachment:
content: list[dict] = [{"type": "text", "text": m["content"]}]
content.append({
"type": "image_url",
"image_url": {"url": attachment["data"]},
})
msgs.append({"role": "user", "content": content})
else:
# Strip non-standard metadata fields before sending to the API
msgs.append({"role": m["role"], "content": m["content"]})
url = api_url.rstrip("/") + chat_path
headers: dict[str, str] = {}
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
payload = {"model": model, "messages": msgs}
async with httpx.AsyncClient(timeout=settings.timeout_local) as client:
resp = await client.post(url, json=payload, headers=headers)
resp.raise_for_status()
data = resp.json()
text = data["choices"][0]["message"]["content"]
if not text or not text.strip():
raise RuntimeError("Local model returned an empty response")
usage = data.get("usage") or {}
if usage.get("prompt_tokens") is not None:
import usage_tracker
from persona import _user
asyncio.create_task(usage_tracker.record(
username=_user.get(),
backend="local",
model_name=model,
prompt_tokens=usage.get("prompt_tokens", 0),
completion_tokens=usage.get("completion_tokens", 0),
))
return text.strip()
async def _anthropic_api(system_prompt: str, messages: list[dict], model_cfg: dict | None) -> str:
"""Direct Anthropic API backend using the anthropic SDK."""
try:
import anthropic
except ImportError:
raise RuntimeError("anthropic SDK not installed — run: pip install 'anthropic>=0.40.0'")
cfg = model_cfg or {}
api_key = cfg.get("api_key", "")
model_name = cfg.get("model_name") or settings.default_model
if not api_key:
raise RuntimeError("No Anthropic API key — add one at /settings/models")
client = anthropic.AsyncAnthropic(api_key=api_key)
msgs = [{"role": m["role"], "content": m["content"]} for m in messages]
kwargs: dict = {
"model": model_name,
"max_tokens": 4096,
"messages": msgs,
}
if system_prompt:
kwargs["system"] = system_prompt
resp = await client.messages.create(**kwargs)
text = resp.content[0].text if resp.content else ""
if not text.strip():
raise RuntimeError("Anthropic API returned an empty response")
if resp.usage:
import usage_tracker
from persona import _user
asyncio.create_task(usage_tracker.record(
username=_user.get(),
backend="anthropic_api",
model_name=model_name,
prompt_tokens=resp.usage.input_tokens,
completion_tokens=resp.usage.output_tokens,
))
return text.strip()
async def _gemini(system_prompt: str, messages: list[dict]) -> str: async def _gemini(system_prompt: str, messages: list[dict]) -> str:
# Gemini CLI spawns MCP child processes that keep stdout pipes open after responding. # Gemini CLI spawns MCP child processes that keep stdout pipes open after responding.
# start_new_session=True puts the whole tree in its own process group so # start_new_session=True puts the whole tree in its own process group so
@@ -193,7 +391,7 @@ def _build_conversation(messages: list[dict]) -> str:
if prior: if prior:
history_lines = [] history_lines = []
for msg in prior: for msg in prior:
label = "Scott" if msg["role"] == "user" else "Inara" label = settings.user_name if msg["role"] == "user" else settings.agent_name
history_lines.append(f"{label}: {msg['content']}") history_lines.append(f"{label}: {msg['content']}")
parts.append("<conversation>\n" + "\n\n".join(history_lines) + "\n</conversation>") parts.append("<conversation>\n" + "\n\n".join(history_lines) + "\n</conversation>")
parts.append(messages[-1]["content"] if messages else "") parts.append(messages[-1]["content"] if messages else "")

View File

@@ -2,40 +2,72 @@ import logging
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from fastapi import FastAPI from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse
import uvicorn import uvicorn
logging.basicConfig(level=logging.INFO, format="%(levelname)s:%(name)s: %(message)s") logging.basicConfig(level=logging.INFO, format="%(levelname)s:%(name)s: %(message)s")
from config import settings from config import settings
from routers import chat, google_chat, nextcloud_talk from auth_middleware import SessionAuthMiddleware
from routers import chat, google_chat, nextcloud_talk, homeassistant, files, distill, auth, orchestrator
from routers import ui, onboarding, settings, tools_settings, help, auth_google, local_llm, push, audit, usage, crons
@asynccontextmanager @asynccontextmanager
async def lifespan(app: FastAPI): async def lifespan(app: FastAPI):
import scheduler
scheduler.start()
yield yield
scheduler.stop()
from llm_client import cleanup from llm_client import cleanup
await cleanup() await cleanup()
app = FastAPI(title="Cortex Dispatcher", lifespan=lifespan) app = FastAPI(title="Cortex Dispatcher", lifespan=lifespan)
app.add_middleware(SessionAuthMiddleware)
# API routers
app.include_router(chat.router) app.include_router(chat.router)
app.include_router(google_chat.router) app.include_router(google_chat.router)
app.include_router(nextcloud_talk.router) app.include_router(nextcloud_talk.router)
app.include_router(homeassistant.router)
app.include_router(files.router)
app.include_router(distill.router)
app.include_router(auth.router)
app.include_router(orchestrator.router)
app.include_router(push.router)
app.include_router(audit.router)
app.include_router(usage.router)
# Static files — must be mounted BEFORE ui.router so /static/* is matched first.
# ui.router has a wildcard /{username}/{persona} that would otherwise catch /static/style.css etc.
app.mount("/static", StaticFiles(directory="static"), name="static") app.mount("/static", StaticFiles(directory="static"), name="static")
# Google OAuth — must be before ui.router (wildcard /{user}/{persona} would swallow it)
app.include_router(auth_google.router)
@app.get("/") # Onboarding (invite tokens + persona creation — before ui.router)
async def index() -> FileResponse: app.include_router(onboarding.router)
return FileResponse("static/index.html")
# Account settings
app.include_router(settings.router)
app.include_router(tools_settings.router)
app.include_router(local_llm.router)
app.include_router(crons.router)
# Help page
app.include_router(help.router)
# Health check — must be before ui.router so /{username} catch-all doesn't swallow it.
@app.get("/health") @app.get("/health")
async def health() -> dict: async def health() -> dict:
return {"status": "ok"} return {"status": "ok"}
# UI router (login + /{user}/{persona} — must be last to avoid swallowing API paths)
app.include_router(ui.router)
if __name__ == "__main__": if __name__ == "__main__":
uvicorn.run( uvicorn.run(
"main:app", "main:app",

217
cortex/manage_passwords.py Normal file
View File

@@ -0,0 +1,217 @@
#!/usr/bin/env python3
"""
Password and invite management for Cortex users.
Usage:
python manage_passwords.py set <username> # prompt for password
python manage_passwords.py set <username> <pass> # set directly (avoid in shell history)
python manage_passwords.py check <username> # test a password interactively
python manage_passwords.py list # show users, auth methods, and emails
python manage_passwords.py invite <username> [email] # generate + optionally email invite link
python manage_passwords.py email <username> <email> # store/update an email address
python manage_passwords.py google-add <username> <email> # register a user for Google sign-in
"""
import json
import sys
import getpass
# Add cortex/ to path so we can import config and auth_utils
sys.path.insert(0, str(__import__('pathlib').Path(__file__).parent))
from auth_utils import set_password, check_credentials, _auth_path, create_invite, link_google, _read_auth
from persona import list_users
from config import settings
# ---------------------------------------------------------------------------
# Profile helpers (home/{username}/profile.json)
# ---------------------------------------------------------------------------
def _profile_path(username: str):
return settings.home_root() / username / "profile.json"
def get_profile(username: str) -> dict:
p = _profile_path(username)
if not p.exists():
return {}
try:
return json.loads(p.read_text())
except Exception:
return {}
def save_profile(username: str, profile: dict) -> None:
p = _profile_path(username)
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(json.dumps(profile, indent=2) + "\n")
def get_email(username: str) -> str | None:
return get_profile(username).get("email")
def set_email(username: str, email: str) -> None:
profile = get_profile(username)
profile["email"] = email
save_profile(username, profile)
# ---------------------------------------------------------------------------
# Commands
# ---------------------------------------------------------------------------
def cmd_set(args):
if not args:
print("Usage: manage_passwords.py set <username> [password]")
sys.exit(1)
username = args[0]
if len(args) >= 2:
password = args[1]
else:
password = getpass.getpass(f"New password for {username}: ")
confirm = getpass.getpass("Confirm password: ")
if password != confirm:
print("Passwords do not match.")
sys.exit(1)
set_password(username, password)
print(f"Password set for: {username}")
def cmd_check(args):
if not args:
print("Usage: manage_passwords.py check <username>")
sys.exit(1)
username = args[0]
password = getpass.getpass(f"Password for {username}: ")
if check_credentials(username, password):
print("OK — credentials are valid.")
else:
print("FAIL — invalid username or password.")
sys.exit(1)
def cmd_list(_args):
users = list_users()
if not users:
print(" No users found in home/")
return
print(f" {'USER':<18} {'PW':<6} {'GOOGLE':<8} {'EMAIL'}")
print(f" {'-'*18} {'-'*6} {'-'*8} {'-'*30}")
for user in users:
auth = _read_auth(user)
has_pw = "" if auth.get("password_hash") else ""
google = auth.get("google_email") or ""
email = get_email(user) or ""
print(f" {user:<18} {has_pw:<6} {google:<36} {email}")
def cmd_email(args):
if len(args) < 2:
print("Usage: manage_passwords.py email <username> <email>")
sys.exit(1)
username, email = args[0], args[1]
set_email(username, email)
print(f"Email saved for {username!r}: {email}")
def cmd_invite(args):
if not args:
print("Usage: manage_passwords.py invite <username> [email]")
sys.exit(1)
username = args[0]
email_arg = args[1] if len(args) >= 2 else None
# Ensure user directory exists
(settings.home_root() / username).mkdir(parents=True, exist_ok=True)
# Store email if provided
if email_arg:
set_email(username, email_arg)
# Use stored email if no arg given
to_email = email_arg or get_email(username)
token = create_invite(username)
url = f"{settings.cortex_base_url}/setup/{token}"
print(f"\nInvite link for {username!r}:")
print(f" {url}\n")
print("Link expires in 72 hours. One-time use.")
if to_email:
from email_utils import send_invite_email
print(f"Sending invite email to {to_email}...")
ok = send_invite_email(to_email=to_email, username=username, token=token)
if ok:
print("Email sent.")
else:
print("Email failed — check SMTP settings. Link above is still valid.")
else:
print("No email address on file — send the link manually.")
print("Tip: python manage_passwords.py invite <username> <email> to email it next time.\n")
def cmd_google_add(args):
if len(args) < 2:
print("Usage: manage_passwords.py google-add <username> <google_email>")
sys.exit(1)
username, email = args[0], args[1].lower().strip()
# Ensure the user directory exists
(settings.home_root() / username).mkdir(parents=True, exist_ok=True)
# Store in auth.json (google_sub filled in on first sign-in) + profile.json (for invites)
link_google(username, sub="", email=email)
set_email(username, email)
print(f"Google sign-in registered for {username!r}: {email}")
print(f"They can now sign in at {settings.cortex_base_url}/login using that Google account.")
def cmd_role(args):
if len(args) < 2:
print("Usage: manage_passwords.py role <username> admin|user")
sys.exit(1)
username, role = args[0], args[1].lower().strip()
if role not in ("admin", "user"):
print("Role must be 'admin' or 'user'.")
sys.exit(1)
from auth_utils import _read_auth, _write_auth
data = _read_auth(username)
if not data:
print(f"User '{username}' not found — no auth.json.")
sys.exit(1)
old_role = data.get("role", "user")
data["role"] = role
_write_auth(username, data)
print(f"Role for '{username}': {old_role}{role}")
if __name__ == "__main__":
if len(sys.argv) < 2:
print(__doc__)
sys.exit(0)
command = sys.argv[1]
rest = sys.argv[2:]
if command == "set":
cmd_set(rest)
elif command == "check":
cmd_check(rest)
elif command == "list":
cmd_list(rest)
elif command == "email":
cmd_email(rest)
elif command == "invite":
cmd_invite(rest)
elif command == "google-add":
cmd_google_add(rest)
elif command == "role":
cmd_role(rest)
else:
print(f"Unknown command: {command}")
print(__doc__)
sys.exit(1)

284
cortex/memory_distiller.py Normal file
View File

@@ -0,0 +1,284 @@
"""
Tiered memory distillation.
distill_short() — roll recent session logs → MEMORY_SHORT.md (no LLM)
distill_mid() — summarize MEMORY_SHORT → MEMORY_MID.md (LLM)
distill_long() — integrate MEMORY_MID → MEMORY_LONG.md (LLM)
Before any file is overwritten, two rolling backups are kept:
MEMORY_*.bak1.md — most recent backup (created just before last write)
MEMORY_*.bak2.md — backup before that
LLM responses are sanity-checked before writing. If the response looks like
a refusal, is too short, or is obviously not memory content, the distill is
aborted and the original file is left untouched.
"""
import logging
from datetime import datetime
from pathlib import Path
from config import settings
from persona import persona_path as _persona_path
logger = logging.getLogger(__name__)
# Rough chars-per-token estimate for budget enforcement
_CHARS_PER_TOKEN = 4
# Phrases that indicate the LLM refused or misunderstood the task
_REFUSAL_PREFIXES = (
"i'm sorry",
"i am sorry",
"i can't",
"i cannot",
"i'm unable",
"i am unable",
"as an ai",
"as a language model",
"i don't have access",
"i do not have access",
"i'm not able",
"i am not able",
)
# Minimum characters for a valid mid/long distill response
_MIN_RESPONSE_CHARS = 80
def _budget_chars(tokens: int) -> int:
return tokens * _CHARS_PER_TOKEN
def _read(path: Path) -> str:
return path.read_text() if path.exists() else ""
def _rotate_backup(path: Path, n: int = 2) -> None:
"""Rotate up to n rolling backups of path before a write.
MEMORY_LONG.md → MEMORY_LONG.bak1.md (most recent), MEMORY_LONG.bak2.md (older)
"""
if not path.exists():
return
# Shift older backups down: bak(n-1) → bak(n), …, bak1 stays as bak1 source
for i in range(n, 1, -1):
older = path.parent / f"{path.stem}.bak{i}.md"
newer = path.parent / f"{path.stem}.bak{i - 1}.md"
if newer.exists():
older.write_text(newer.read_text())
# Current file → bak1
bak1 = path.parent / f"{path.stem}.bak1.md"
bak1.write_text(path.read_text())
def _sanity_check(response_text: str, context: str, existing_content: str = "") -> str | None:
"""Return an error string if the LLM response looks invalid, else None.
Checks:
- Minimum absolute length
- Refusal / AI preamble phrases
- Size shrinkage: new content must be at least 40% of the old (catches truncation)
- Size explosion: new content must not exceed 250% of the old (catches runaway output)
(Both bounds only apply when an existing file is present and reasonably sized.)
"""
stripped = response_text.strip()
if len(stripped) < _MIN_RESPONSE_CHARS:
return f"{context}: response too short ({len(stripped)} chars) — not writing"
first_line = stripped.lower().splitlines()[0]
if any(first_line.startswith(p) for p in _REFUSAL_PREFIXES):
return f"{context}: response looks like a refusal — not writing"
if existing_content:
old_len = len(existing_content.strip())
new_len = len(stripped)
if old_len >= _MIN_RESPONSE_CHARS * 4: # only compare when old file has real content
ratio = new_len / old_len
if ratio < 0.40:
return (
f"{context}: new content is only {ratio:.0%} of the old "
f"({new_len} vs {old_len} chars) — looks truncated, not writing"
)
if ratio > 2.50:
return (
f"{context}: new content is {ratio:.0%} of the old "
f"({new_len} vs {old_len} chars) — looks like runaway output, not writing"
)
return None
def distill_short(username: str, persona: str) -> dict:
"""
Roll the most recent session log files into MEMORY_SHORT.md.
No LLM involved — pure aggregation with budget truncation.
Files are included newest-first until the budget is reached,
then written in chronological order (oldest first).
"""
inara_dir = _persona_path(username, persona)
sessions_dir = inara_dir / "sessions"
budget = _budget_chars(settings.memory_budget_short)
session_files = (
sorted(sessions_dir.glob("*.md"), reverse=True)
if sessions_dir.exists()
else []
)
parts = []
total_chars = 0
for sf in session_files:
content = sf.read_text()
if total_chars + len(content) > budget and parts:
break # always include at least one file
parts.append((sf.name, content))
total_chars += len(content)
if total_chars >= budget:
break
now = datetime.now().strftime("%Y-%m-%d %H:%M")
header = (
f"# MEMORY_SHORT.md — Recent Session Digest\n\n"
f"*Auto-generated: {now}. {len(parts)} session file(s).*\n\n---\n\n"
)
# Write in chronological order (oldest first)
body = "\n\n".join(
f"--- {name} ---\n{content}" for name, content in reversed(parts)
)
out_path = inara_dir / "MEMORY_SHORT.md"
_rotate_backup(out_path)
out_path.write_text(header + body)
logger.info("distill_short [%s/%s]: wrote %d chars from %d files", username, persona, len(header) + len(body), len(parts))
return {
"files_included": len(parts),
"chars_written": len(header) + len(body),
"budget_chars": budget,
}
async def distill_mid(username: str, persona: str) -> dict:
"""
Ask the LLM to summarize MEMORY_SHORT.md → MEMORY_MID.md.
Backs up the current MEMORY_MID.md before overwriting.
"""
from llm_client import complete
from persona import set_context
u, p = username, persona
set_context(u, p)
inara_dir = _persona_path(u, p)
short_content = _read(inara_dir / "MEMORY_SHORT.md")
existing_mid = _read(inara_dir / "MEMORY_MID.md")
if not short_content.strip() or "Not yet populated" in short_content:
return {"error": "MEMORY_SHORT.md is empty — run distill/short first"}
budget_tokens = settings.memory_budget_mid
persona_name = p.title()
user_name = u.title()
system_prompt = (
f"You are {persona_name}'s memory distillation system. "
"Summarize the following recent session logs into a concise mid-term memory digest. "
f"Target length: under {budget_tokens} tokens. "
"Focus on: recurring themes, important decisions made, ongoing projects, "
f"{user_name}'s current state and priorities, and anything that should persist into future sessions. "
f"Write in first person as {persona_name} (e.g. '{user_name} and I worked on...'). "
"Use markdown headings. Be specific and concrete — no filler."
)
response_text, backend = await complete(
system_prompt=system_prompt,
messages=[{"role": "user", "content": short_content}],
role="distill",
)
err = _sanity_check(response_text, "distill_mid", existing_mid)
if err:
logger.warning(err)
return {"error": err}
now = datetime.now().strftime("%Y-%m-%d %H:%M")
header = (
f"# MEMORY_MID.md — Mid-Term Memory Digest\n\n"
f"*Auto-distilled: {now} via {backend}.*\n\n---\n\n"
)
out_path = inara_dir / "MEMORY_MID.md"
_rotate_backup(out_path)
out_path.write_text(header + response_text)
logger.info("distill_mid [%s/%s]: wrote %d chars via %s", u, p, len(header) + len(response_text), backend)
return {
"username": u,
"backend": backend,
"chars_written": len(header) + len(response_text),
"budget_tokens": budget_tokens,
}
async def distill_long(username: str, persona: str) -> dict:
"""
Ask the LLM to integrate MEMORY_MID.md into MEMORY_LONG.md.
Backs up the current MEMORY_LONG.md before overwriting.
"""
from llm_client import complete
from persona import set_context
u, p = username, persona
set_context(u, p)
inara_dir = _persona_path(u, p)
long_content = _read(inara_dir / "MEMORY_LONG.md")
mid_content = _read(inara_dir / "MEMORY_MID.md")
if not mid_content.strip() or "Not yet populated" in mid_content:
return {"error": "MEMORY_MID.md is empty — run distill/mid first"}
budget_tokens = settings.memory_budget_long
persona_name = p.title()
system_prompt = (
f"You are {persona_name}'s long-term memory curator. "
"You will receive the current long-term memory and a recent mid-term digest. "
f"Integrate the new information into the long-term memory. Target: under {budget_tokens} tokens. "
"Rules: preserve important historical facts; update or replace stale information; "
"absorb recurring themes from the mid-term digest; remove things no longer relevant. "
"Return ONLY the updated MEMORY_LONG.md content in markdown. No preamble or commentary."
)
user_content = (
f"## Current MEMORY_LONG.md\n\n{long_content}\n\n"
f"## Recent MEMORY_MID.md to integrate\n\n{mid_content}"
)
response_text, backend = await complete(
system_prompt=system_prompt,
messages=[{"role": "user", "content": user_content}],
role="distill",
)
err = _sanity_check(response_text, "distill_long", long_content)
if err:
logger.warning(err)
return {"error": err}
# Ensure the file has the right header if the LLM dropped it
now = datetime.now().strftime("%Y-%m-%d %H:%M")
if not response_text.lstrip().startswith("# MEMORY_LONG"):
response_text = (
f"# MEMORY_LONG.md — {persona_name} Long-Term Memory\n\n"
f"*Last distilled: {now} via {backend}.*\n\n---\n\n"
+ response_text
)
out_path = inara_dir / "MEMORY_LONG.md"
_rotate_backup(out_path)
out_path.write_text(response_text)
logger.info("distill_long [%s/%s]: wrote %d chars via %s", u, p, len(response_text), backend)
return {
"username": u,
"backend": backend,
"chars_written": len(response_text),
"budget_tokens": budget_tokens,
}

980
cortex/model_registry.py Normal file
View File

@@ -0,0 +1,980 @@
"""
Per-user unified model registry — V2.
Stored in: home/{user}/model_registry.json
V2 Schema:
{
"version": 2,
# Per-provider accounts / credentials (user-configured)
"providers": {
"anthropic": {
"credentials": [
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
]
},
"google": {
"accounts": [
{"id": "<hex>", "label": "My Google account", "api_key": "AIza..."}
]
}
},
# Local OpenAI-compatible hosts (unchanged from V1)
"hosts": [{"id", "label", "api_url", "api_key", "host_type"}, ...],
# User-registered model entries (all providers)
"models": [
{
"id": str, # unique within this registry
"type": str, # see TYPES below
"label": str, # human-readable
"model_name": str, # identifier sent to the API / CLI
"provider": str | null, # "anthropic" | "google" | "local" | null
"host_id": str | null, # local_openai only — references hosts[].id
"credential_id":str | null, # claude_cli only — references providers.anthropic.credentials
"account_id": str | null, # gemini_api only — references providers.google.accounts
"context_k": int, # context window in k tokens (informational)
"max_rounds": int | null, # per-model tool-loop cap; null = use orchestrator_max_rounds global
"tags": [str], # user-defined capability tags
},
],
# Role assignments — any model (any provider) can fill any role
"roles": {
"<role>": {
"primary": "<model_id>" | null,
"backup_1": "<model_id>" | null,
...
"backup_4": "<model_id>" | null,
},
},
}
Types:
"claude_cli" — Claude CLI subprocess (~/.claude/.credentials.json)
"gemini_cli" — Gemini CLI subprocess
"gemini_api" — Gemini API (google-genai SDK); account_id → api_key from providers.google
"local_openai" — OpenAI-compatible endpoint; host_id → api_url/api_key from hosts[]
"anthropic_api" — Anthropic SDK direct; credential_id → api_key from providers.anthropic.credentials
Built-in model IDs (always resolvable without a registry entry):
"claude_cli" — resolves to the default Claude CLI model
"gemini_cli" — resolves to Gemini CLI
"gemini_api" — resolves to Gemini API using GEMINI_API_KEY from .env
Role resolution for get_model_for_role(username, role):
1. User registry: roles[role].primary → backup_1 → ... → backup_4
2. .env default: ROLE_<ROLE>=<builtin_id>
3. Hardcoded last-resort defaults per role
4. claude_cli (absolute fallback)
"""
import json
import logging
import secrets
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
# ── Role-level tool defaults ───────────────────────────────────────────────────
# Applied when a user hasn't configured a custom tool list for a role.
# None = no restriction (all accessible tools); [] = no tools (pure text processing).
# "chat" is intentionally absent: the /chat endpoint never sends tool schemas anyway,
# and the orchestrator uses chat_role="chat" as its default — restricting it here
# would block all tools from every default orchestration request.
# "orchestrator" is intentionally absent — Phase 2 keyword routing narrows it per message.
ROLE_DEFAULT_TOOLS: dict[str, list[str] | None] = {
"distill": [], # pure text processing — no tools needed
"research": ["web_search", "web_read", "http_fetch"],
"coder": [
"project_file_read", "project_file_list", "file_stat", "file_grep",
"file_diff", "file_syntax_check", "file_read", "file_list", "file_write",
"git_status", "git_log", "git_diff", "shell_exec",
],
}
# ── Provider model catalogs ───────────────────────────────────────────────────
# Server-side defaults. Update here when providers release new models.
# Users can add entries via the settings UI (Phase 2).
ANTHROPIC_CATALOG: list[dict] = [
# Latest
{"id": "claude-opus-4-7", "label": "Claude Opus 4.7", "context_k": 1000},
{"id": "claude-sonnet-4-6", "label": "Claude Sonnet 4.6", "context_k": 1000},
{"id": "claude-haiku-4-5-20251001", "label": "Claude Haiku 4.5", "context_k": 200},
# Previous versions (still available, not deprecated)
{"id": "claude-opus-4-6", "label": "Claude Opus 4.6", "context_k": 1000},
{"id": "claude-sonnet-4-5", "label": "Claude Sonnet 4.5", "context_k": 200},
]
GOOGLE_CATALOG: list[dict] = [
# Stable / generally available
{"id": "gemini-2.5-pro", "label": "Gemini 2.5 Pro", "context_k": 1000},
{"id": "gemini-2.5-flash", "label": "Gemini 2.5 Flash", "context_k": 1000},
{"id": "gemini-2.5-flash-lite", "label": "Gemini 2.5 Flash-Lite", "context_k": 1000},
# Preview
{"id": "gemini-3.1-pro-preview", "label": "Gemini 3.1 Pro (preview)", "context_k": 1000},
{"id": "gemini-3-flash-preview", "label": "Gemini 3 Flash (preview)", "context_k": 1000},
{"id": "gemini-3.1-flash-lite-preview", "label": "Gemini 3.1 Flash-Lite (preview)", "context_k": 1000},
]
# Known OpenAI-compatible cloud inference services.
# All use host_type "openai" (/chat/completions + /models paths).
CLOUD_API_CATALOG: list[dict] = [
{"id": "openrouter", "label": "OpenRouter", "api_url": "https://openrouter.ai/api/v1"},
{"id": "openai", "label": "OpenAI", "api_url": "https://api.openai.com/v1"},
{"id": "groq", "label": "Groq", "api_url": "https://api.groq.com/openai/v1"},
{"id": "xai", "label": "X.ai / Grok", "api_url": "https://api.x.ai/v1"},
{"id": "together", "label": "Together.ai", "api_url": "https://api.together.xyz/v1"},
{"id": "fireworks", "label": "Fireworks.ai", "api_url": "https://api.fireworks.ai/inference/v1"},
{"id": "custom", "label": "Custom", "api_url": ""},
]
# ── Built-in model definitions ────────────────────────────────────────────────
def _builtins() -> dict[str, dict]:
return {
"claude_cli": {
"id": "claude_cli",
"type": "claude_cli",
"label": f"Claude (CLI) — {settings.default_model}",
"model_name": settings.default_model,
"context_k": 200,
"tags": ["chat", "persona", "creative"],
},
"gemini_cli": {
"id": "gemini_cli",
"type": "gemini_cli",
"label": "Gemini (CLI)",
"model_name": "",
"context_k": 1000,
"tags": ["chat", "research", "long_context"],
},
"gemini_api": {
"id": "gemini_api",
"type": "gemini_api",
"label": f"Gemini API — {settings.orchestrator_model}",
"model_name": settings.orchestrator_model,
"context_k": 1000,
"tags": ["orchestrator", "research", "long_context", "tools"],
},
}
_ROLE_LAST_RESORT: dict[str, str] = {
"chat": "claude_cli",
"orchestrator": "gemini_api",
"distill": "claude_cli",
"coder": "claude_cli",
"research": "gemini_api",
}
PRIORITY_KEYS = ["primary", "backup_1", "backup_2", "backup_3", "backup_4"]
REQUIRED_ROLES: list[str] = ["chat", "orchestrator", "distill"]
# ── Storage ───────────────────────────────────────────────────────────────────
def _registry_path(username: str) -> Path:
return settings.home_root() / username / "model_registry.json"
def _local_llm_path(username: str) -> Path:
return settings.home_root() / username / "local_llm.json"
def _auth_path(username: str) -> Path:
return settings.home_root() / username / "auth.json"
def _empty() -> dict:
return {
"version": 2,
"providers": _default_providers(),
"hosts": [],
"models": [],
"roles": {},
}
def _default_providers() -> dict:
return {
"anthropic": {
"credentials": [
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
]
},
"google": {
"accounts": []
},
}
def _normalize(data: dict) -> dict:
"""Back-fill missing fields introduced by schema additions."""
for h in data.get("hosts", []):
h.setdefault("host_type", "openwebui")
h.setdefault("max_concurrent", 3)
data.setdefault("providers", _default_providers())
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
data["providers"].setdefault("google", {"accounts": []})
return data
def _load(username: str) -> dict:
path = _registry_path(username)
if path.exists():
try:
data = json.loads(path.read_text())
if isinstance(data, dict) and "version" in data:
if data["version"] == 1:
data = _migrate_v1_to_v2(username, data)
_save(username, data)
return _normalize(data)
except (json.JSONDecodeError, OSError):
logger.warning("model_registry.json for %s is unreadable — starting fresh", username)
return _empty()
# No registry — try migrating from local_llm.json
legacy = _local_llm_path(username)
if legacy.exists():
data = _migrate_from_local_llm(username, legacy)
_save(username, data)
logger.info("Migrated local_llm.json → model_registry.json for %s", username)
return data
return _empty()
def _save(username: str, data: dict) -> None:
_registry_path(username).write_text(json.dumps(data, indent=2))
# ── Migration ─────────────────────────────────────────────────────────────────
def _migrate_v1_to_v2(username: str, data: dict) -> dict:
"""
Upgrade a V1 registry to V2.
Changes:
- Adds providers section with default structure
- Migrates gemini_api_key from auth.json → first Google account entry
- Sets version to 2
"""
logger.info("Migrating model_registry.json V1 → V2 for %s", username)
data["version"] = 2
if "providers" not in data:
data["providers"] = _default_providers()
else:
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
data["providers"].setdefault("google", {"accounts": []})
# Pull existing Gemini key from auth.json (stored there in V1) → first account entry
accounts = data["providers"]["google"]["accounts"]
if not accounts:
try:
auth = json.loads(_auth_path(username).read_text())
existing_key = auth.get("gemini_api_key")
if existing_key:
accounts.append({
"id": secrets.token_hex(4),
"label": "Gemini API Key",
"api_key": existing_key,
})
logger.info("Migrated gemini_api_key from auth.json → providers.google.accounts for %s", username)
except (OSError, json.JSONDecodeError):
pass
return data
def _migrate_from_local_llm(username: str, path: Path) -> dict:
"""Convert local_llm.json → V2 model_registry format."""
try:
old = json.loads(path.read_text())
except Exception:
return _empty()
data = _empty()
# Handle v0 flat format
if "hosts" not in old:
api_url = old.get("api_url") or settings.local_api_url
api_key = old.get("api_key") or settings.local_api_key
model_name = old.get("model") or settings.local_model
if not api_url:
return data
host_id = secrets.token_hex(4)
old = {
"hosts": [{"id": host_id, "label": "Local Model Server", "api_url": api_url, "api_key": api_key}],
"models": [{"id": secrets.token_hex(4), "host_id": host_id, "label": model_name, "model_name": model_name}] if model_name else [],
"active_model_id": None,
}
if old["models"]:
old["active_model_id"] = old["models"][0]["id"]
data["hosts"] = old.get("hosts", [])
for m in old.get("models", []):
data["models"].append({
"id": m["id"],
"type": "local_openai",
"label": m.get("label") or m.get("model_name", ""),
"model_name": m.get("model_name", ""),
"provider": "local",
"host_id": m.get("host_id"),
"context_k": 0,
"tags": [],
})
active_id = old.get("active_model_id")
if active_id and any(m["id"] == active_id for m in data["models"]):
data["roles"]["chat"] = {"primary": active_id}
if settings.distill_backend_mid == "local":
data["roles"]["distill"] = {"primary": active_id}
# Migrate Gemini key from auth.json
data = _migrate_v1_to_v2(username, {"version": 1, **data})
return data
# ── Model resolution ──────────────────────────────────────────────────────────
def _resolve_model(registry: dict, model_id: str) -> dict | None:
"""Resolve a model_id to its full config dict (credentials merged in), or None."""
builtins = _builtins()
# Built-in IDs take priority over user-defined entries with the same ID
if model_id in builtins:
return dict(builtins[model_id])
model = next((m for m in registry.get("models", []) if m["id"] == model_id), None)
if not model:
return None
model_type = model.get("type")
if model_type == "local_openai":
host_id = model.get("host_id")
host = next((h for h in registry.get("hosts", []) if h["id"] == host_id), None)
if not host:
logger.warning("model %s references missing host_id %s", model_id, host_id)
return None
return {
**model,
"api_url": host.get("api_url", ""),
"api_key": host.get("api_key", ""),
"host_type": host.get("host_type", "openwebui"),
}
if model_type == "gemini_api":
account_id = model.get("account_id")
if account_id:
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
account = next((a for a in accounts if a["id"] == account_id), None)
if account:
return {**model, "api_key": account.get("api_key", "")}
logger.warning("model %s references missing account_id %s", model_id, account_id)
return dict(model)
if model_type == "anthropic_api":
credential_id = model.get("credential_id")
if credential_id:
creds = registry.get("providers", {}).get("anthropic", {}).get("credentials", [])
cred = next((c for c in creds if c["id"] == credential_id), None)
if cred and cred.get("api_key"):
return {**model, "api_key": cred["api_key"]}
logger.warning("model %s references missing/keyless credential_id %s", model_id, credential_id)
return dict(model)
if model_type == "claude_cli":
return dict(model)
return dict(model)
def get_model_for_role(username: str, role: str) -> dict | None:
"""
Return the resolved model config for the given role.
Resolution order:
1. User registry: roles[role].primary → backup_1 → ... → backup_4
2. .env: ROLE_<ROLE> = builtin model ID
3. Hardcoded last-resort default per role
4. claude_cli (absolute fallback)
"""
registry = _load(username)
role_cfg = registry.get("roles", {}).get(role, {})
for key in PRIORITY_KEYS:
model_id = role_cfg.get(key)
if not model_id:
continue
resolved = _resolve_model(registry, model_id)
if resolved:
return resolved
logger.debug("role %s.%s = %s but model not found", role, key, model_id)
# .env default
env_type = settings.get_role_default(role)
builtins = _builtins()
if env_type and env_type in builtins:
return dict(builtins[env_type])
# Hardcoded last resort
fallback_id = _ROLE_LAST_RESORT.get(role, "claude_cli")
return dict(builtins.get(fallback_id, builtins["claude_cli"]))
def get_best_local_model(username: str, role: str = "chat") -> dict | None:
"""
Return the best available local_openai model for the given role.
Used when the user explicitly selects "local" backend in the UI.
"""
registry = _load(username)
role_cfg = registry.get("roles", {}).get(role, {})
for key in PRIORITY_KEYS:
model_id = role_cfg.get(key)
if not model_id:
continue
resolved = _resolve_model(registry, model_id)
if resolved and resolved.get("type") == "local_openai":
return resolved
for model in registry.get("models", []):
if model.get("type") == "local_openai":
resolved = _resolve_model(registry, model["id"])
if resolved:
return resolved
return None
def set_role_config(
username: str,
role: str,
system_append: str,
tools: list[str] | None,
inject_datetime: bool = True,
inject_mode: bool = True,
) -> None:
"""Save system_append, tools allow-list, and per-injection flags for a role.
tools=None clears the allow-list (role uses all accessible tools).
inject_datetime=False suppresses the date/time header for pure processing roles.
inject_mode=False suppresses the session mode (OTR) line for pure processing roles.
"""
data = _load(username)
roles = data.setdefault("roles", {})
if role not in roles:
roles[role] = {}
roles[role]["system_append"] = system_append.strip()
roles[role]["inject_datetime"] = inject_datetime
roles[role]["inject_mode"] = inject_mode
if tools is None:
roles[role].pop("tools", None)
else:
roles[role]["tools"] = [t for t in tools if t]
_save(username, data)
def get_role_config(username: str, role: str) -> dict:
"""
Return supplemental config for a role: system_append, tools, and injection flags.
All keys are optional in the registry — missing means "use defaults":
system_append: str — appended to the system prompt for this role
tools: list[str] | None — explicit tool allow-list (None = no restriction)
inject_datetime: bool — whether to inject current date/time (default True)
inject_mode: bool — whether to inject session mode (OTR) line (default True)
"""
registry = _load(username)
role_cfg = registry.get("roles", {}).get(role, {})
user_tools = role_cfg.get("tools")
if user_tools is None:
# No user-configured list — fall back to system defaults for this role
effective_tools: list[str] | None = ROLE_DEFAULT_TOOLS.get(role)
else:
# User has configured tools; preserve their setting (empty list → no restriction)
effective_tools = user_tools or None
return {
"system_append": role_cfg.get("system_append", ""),
"tools": effective_tools,
"inject_datetime": role_cfg.get("inject_datetime", True),
"inject_mode": role_cfg.get("inject_mode", True),
}
def get_model_for_slot(username: str, role: str, slot: str) -> dict | None:
"""
Resolve a single named priority slot from a role without walking the fallback chain.
Used by Phase 3 explicit slot selection — the user has pinned a specific model;
don't silently redirect to another slot if this one is empty or broken.
Returns None if the slot is unset or the model can't be resolved.
"""
if slot not in PRIORITY_KEYS:
return None
registry = _load(username)
model_id = registry.get("roles", {}).get(role, {}).get(slot)
if not model_id:
return None
return _resolve_model(registry, model_id)
def get_google_api_key(username: str, account_id: str | None = None) -> str | None:
"""
Return the best available Gemini API key for the user.
If account_id is specified, returns that account's key (or None if not found).
Otherwise returns the first configured account key, falling back to the
server-level GEMINI_API_KEY from .env.
"""
registry = _load(username)
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
if account_id:
account = next((a for a in accounts if a["id"] == account_id), None)
return account.get("api_key") if account else None
# First configured account
if accounts:
return accounts[0].get("api_key") or None
# Fall back to .env server key
return settings.gemini_api_key or None
# ── Read API ──────────────────────────────────────────────────────────────────
def get_registry(username: str) -> dict:
"""Return the full registry (providers + hosts + models + roles)."""
return _load(username)
def get_all_models(username: str) -> list[dict]:
"""Return all user-defined models (resolved — credentials/hosts merged in)."""
registry = _load(username)
out = []
for m in registry.get("models", []):
resolved = _resolve_model(registry, m["id"])
if resolved:
out.append(resolved)
return out
def get_defined_roles(username: str) -> dict[str, dict]:
"""Return the roles section, filling gaps with empty dicts."""
registry = _load(username)
roles = registry.get("roles", {})
return {role: roles.get(role, {}) for role in settings.get_defined_roles()}
def get_google_accounts(username: str) -> list[dict]:
"""Return Google account entries (api_key masked for display)."""
registry = _load(username)
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
return [
{
"id": a["id"],
"label": a.get("label", ""),
"hint": (a.get("api_key") or "")[:8] + "" if a.get("api_key") else "",
}
for a in accounts
]
def get_catalog(provider: str, username: str | None = None) -> list[dict]:
"""
Return the model catalog for a provider.
For now returns server defaults. Phase 2 will merge in per-user additions.
"""
if provider == "anthropic":
return list(ANTHROPIC_CATALOG)
if provider == "google":
return list(GOOGLE_CATALOG)
if provider == "cloud":
return list(CLOUD_API_CATALOG)
return []
# ── Write API — Google accounts ───────────────────────────────────────────────
def save_google_account(username: str, account_id: str | None,
label: str, api_key: str) -> str:
"""Create or update a Google account entry. Returns the account ID."""
data = _load(username)
accounts = data["providers"]["google"]["accounts"]
if account_id:
for a in accounts:
if a["id"] == account_id:
a["label"] = label.strip()
if api_key.strip():
a["api_key"] = api_key.strip()
_save(username, data)
return account_id
account_id = secrets.token_hex(4)
accounts.append({
"id": account_id,
"label": label.strip(),
"api_key": api_key.strip(),
})
_save(username, data)
return account_id
def remove_google_account(username: str, account_id: str) -> bool:
"""Remove a Google account. Clears any model entries that reference it."""
data = _load(username)
accounts = data["providers"]["google"]["accounts"]
before = len(accounts)
data["providers"]["google"]["accounts"] = [a for a in accounts if a["id"] != account_id]
# Clear role assignments for models that referenced this account
removed_model_ids = {
m["id"] for m in data.get("models", [])
if m.get("account_id") == account_id
}
data["models"] = [m for m in data.get("models", []) if m["id"] not in removed_model_ids]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["providers"]["google"]["accounts"]) < before
# ── Write API — Anthropic API keys ───────────────────────────────────────────
def get_anthropic_api_keys(username: str) -> list[dict]:
"""Return Anthropic API key credentials (type='api_key') with key masked for display."""
registry = _load(username)
creds = registry.get("providers", {}).get("anthropic", {}).get("credentials", [])
return [
{
"id": c["id"],
"label": c.get("label", ""),
"hint": (c.get("api_key") or "")[:8] + "" if c.get("api_key") else "no key",
}
for c in creds
if c.get("type") == "api_key"
]
def save_anthropic_api_key(username: str, key_id: str | None,
label: str, api_key: str) -> str:
"""Create or update an Anthropic API key credential. Returns the credential ID."""
data = _load(username)
creds = data["providers"]["anthropic"]["credentials"]
if key_id:
for c in creds:
if c["id"] == key_id and c.get("type") == "api_key":
c["label"] = label.strip() or c.get("label", "API Key")
if api_key.strip():
c["api_key"] = api_key.strip()
_save(username, data)
return key_id
key_id = secrets.token_hex(4)
creds.append({
"id": key_id,
"label": label.strip() or "API Key",
"type": "api_key",
"api_key": api_key.strip(),
})
_save(username, data)
return key_id
def remove_anthropic_api_key(username: str, key_id: str) -> bool:
"""Remove an Anthropic API key credential. Clears model entries that reference it."""
data = _load(username)
creds = data["providers"]["anthropic"]["credentials"]
before = len(creds)
data["providers"]["anthropic"]["credentials"] = [
c for c in creds if c["id"] != key_id
]
removed_model_ids = {
m["id"] for m in data.get("models", [])
if m.get("credential_id") == key_id
}
data["models"] = [m for m in data.get("models", []) if m["id"] not in removed_model_ids]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["providers"]["anthropic"]["credentials"]) < before
# ── Write API — Hosts ─────────────────────────────────────────────────────────
def save_host(username: str, host_id: str | None,
label: str, api_url: str, api_key: str,
host_type: str = "openwebui",
max_concurrent: int = 3) -> str:
"""Create or update a host. Returns the host ID."""
data = _load(username)
host_type = host_type if host_type in ("openwebui", "openai") else "openwebui"
max_concurrent = max(1, min(int(max_concurrent), 20))
if host_id:
for h in data["hosts"]:
if h["id"] == host_id:
h["label"] = label.strip()
h["api_url"] = api_url.strip()
h["host_type"] = host_type
h["max_concurrent"] = max_concurrent
if api_key.strip():
h["api_key"] = api_key.strip()
_save(username, data)
return host_id
host_id = None
host_id = secrets.token_hex(4)
data["hosts"].append({
"id": host_id,
"label": label.strip(),
"api_url": api_url.strip(),
"api_key": api_key.strip(),
"host_type": host_type,
"max_concurrent": max_concurrent,
})
_save(username, data)
return host_id
def remove_host(username: str, host_id: str) -> bool:
"""Remove a host and all models that reference it."""
data = _load(username)
before = len(data["hosts"])
removed_model_ids = {m["id"] for m in data["models"] if m.get("host_id") == host_id}
data["hosts"] = [h for h in data["hosts"] if h["id"] != host_id]
data["models"] = [m for m in data["models"] if m.get("host_id") != host_id]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["hosts"]) < before
# ── Write API — Models ────────────────────────────────────────────────────────
def save_model(username: str, model_id: str | None, host_id: str,
label: str, model_name: str, context_k: int = 0,
tags: list[str] | None = None,
max_rounds: int | None = None,
tools: bool = True,
reasoning_budget_tokens: int | None = None) -> str:
"""Create or update a local_openai model entry. Returns the model ID."""
data = _load(username)
tags = tags or []
if model_id:
for m in data["models"]:
if m["id"] == model_id:
m["host_id"] = host_id
m["label"] = label.strip() or model_name.strip()
m["model_name"] = model_name.strip()
m["context_k"] = context_k
m["max_rounds"] = max_rounds
m["tools"] = tools
m["tags"] = tags
m["reasoning_budget_tokens"] = reasoning_budget_tokens
_save(username, data)
return model_id
model_id = None
model_id = secrets.token_hex(4)
data["models"].append({
"id": model_id,
"type": "local_openai",
"label": label.strip() or model_name.strip(),
"model_name": model_name.strip(),
"provider": "local",
"host_id": host_id,
"context_k": context_k,
"max_rounds": max_rounds,
"tools": tools,
"tags": tags,
"reasoning_budget_tokens": reasoning_budget_tokens,
})
_save(username, data)
return model_id
def save_cloud_model(username: str, model_id: str | None,
provider: str, model_name: str, label: str,
account_id: str | None = None,
credential_id: str | None = None,
context_k: int = 0,
tags: list[str] | None = None,
max_rounds: int | None = None,
tools: bool = True) -> str:
"""
Create or update an Anthropic or Google model entry. Returns the model ID.
provider: "anthropic" | "google"
account_id: Google only — references providers.google.accounts[].id
credential_id: Anthropic only — "cli" for OAuth CLI, or a hex ID for an API key credential
"""
data = _load(username)
# Determine model type from credential (anthropic only)
if provider == "anthropic":
creds = data.get("providers", {}).get("anthropic", {}).get("credentials", [])
cred = next((c for c in creds if c["id"] == credential_id), None) if credential_id else None
entry_type = "anthropic_api" if (cred and cred.get("type") == "api_key") else "claude_cli"
elif provider == "google":
entry_type = "gemini_api"
else:
entry_type = "claude_cli"
tags = tags or []
entry: dict = {
"type": entry_type,
"label": label.strip() or model_name.strip(),
"model_name": model_name.strip(),
"provider": provider,
"context_k": context_k,
"max_rounds": max_rounds,
"tools": tools,
"tags": tags,
}
if account_id:
entry["account_id"] = account_id
if credential_id:
entry["credential_id"] = credential_id
if model_id:
for m in data["models"]:
if m["id"] == model_id:
m.update(entry)
_save(username, data)
return model_id
model_id = None
model_id = secrets.token_hex(4)
entry["id"] = model_id
data["models"].append(entry)
_save(username, data)
return model_id
def remove_model(username: str, model_id: str) -> bool:
"""Remove a model and clear any role assignments pointing to it."""
data = _load(username)
before = len(data["models"])
data["models"] = [m for m in data["models"] if m["id"] != model_id]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) == model_id:
role_cfg[key] = None
_save(username, data)
return len(data["models"]) < before
def get_custom_roles(username: str) -> list[str]:
"""
Return the user's custom (non-required) roles.
Falls back to config-defined roles minus required ones for migration.
"""
registry = _load(username)
if "custom_roles" in registry:
return [r for r in registry["custom_roles"] if r and r not in REQUIRED_ROLES]
from config import settings as _cfg
return [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
def get_all_roles(username: str) -> list[str]:
"""Return required roles followed by the user's custom roles."""
return list(REQUIRED_ROLES) + get_custom_roles(username)
def add_custom_role(username: str, role_name: str) -> bool:
"""Add a custom role. Returns False if the name is invalid or already a required role."""
role_name = role_name.strip().lower()
if not role_name or role_name in REQUIRED_ROLES:
return False
data = _load(username)
if "custom_roles" not in data:
from config import settings as _cfg
data["custom_roles"] = [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
if role_name not in data["custom_roles"]:
data["custom_roles"].append(role_name)
_save(username, data)
return True
def remove_custom_role(username: str, role_name: str) -> bool:
"""Remove a custom role. Required roles cannot be removed."""
if role_name in REQUIRED_ROLES:
return False
data = _load(username)
if "custom_roles" not in data:
from config import settings as _cfg
data["custom_roles"] = [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
if role_name in data["custom_roles"]:
data["custom_roles"].remove(role_name)
_save(username, data)
return True
def set_role(username: str, role: str, priority: str, model_id: str | None) -> bool:
"""
Assign a model to a role priority slot.
priority must be one of: primary, backup_1, backup_2, backup_3, backup_4
model_id None clears the slot.
Built-in IDs (claude_cli, gemini_cli, gemini_api) are always valid.
"""
if priority not in PRIORITY_KEYS:
return False
data = _load(username)
if model_id and model_id not in _builtins():
if not any(m["id"] == model_id for m in data["models"]):
return False
roles = data.setdefault("roles", {})
if role not in roles:
roles[role] = {}
roles[role][priority] = model_id or None
_save(username, data)
return True
# ── Utility ───────────────────────────────────────────────────────────────────
def fetch_models_from_host(api_url: str, api_key: str,
host_type: str = "openwebui") -> list[str]:
"""Synchronously fetch the model list from an OpenAI-compatible host."""
import httpx
path = "/api/models" if host_type == "openwebui" else "/models"
url = api_url.rstrip("/") + path
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
resp = httpx.get(url, headers=headers, timeout=10)
resp.raise_for_status()
data = resp.json()
models = data.get("data", [])
return sorted(m.get("id", m.get("name", "")) for m in models if m.get("id") or m.get("name"))

176
cortex/notification.py Normal file
View File

@@ -0,0 +1,176 @@
"""
Outbound notification helpers — send messages to user channels proactively.
Channel config lives in home/{user}/channels.json:
{
"notification_channel": "email" | "nextcloud" | "google_chat",
"notification_email": "<override address — defaults to login email>",
"nextcloud": {
"url": "...", "bot_secret": "...", "notification_room": "<token>", ...
},
"google_chat": {
"outbound_webhook": "https://chat.googleapis.com/v1/spaces/...", ...
}
}
If notification_channel is absent, defaults to "nextcloud" if configured.
"""
import asyncio
import hashlib
import hmac
import json
import logging
import secrets
import httpx
logger = logging.getLogger(__name__)
async def _send_nct_message(url: str, secret: str, room: str, message: str) -> None:
"""Post a message to a Nextcloud Talk room as the bot."""
endpoint = f"{url}/ocs/v2.php/apps/spreed/api/v1/bot/{room}/message"
random_str = secrets.token_hex(32)
sig = hmac.new(
secret.encode(),
(random_str + message).encode("utf-8"),
hashlib.sha256,
).hexdigest()
body = json.dumps({"message": message}, ensure_ascii=False).encode("utf-8")
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
endpoint,
content=body,
headers={
"Content-Type": "application/json",
"OCS-APIRequest": "true",
"X-Nextcloud-Talk-Bot-Random": random_str,
"X-Nextcloud-Talk-Bot-Signature": sig,
},
timeout=15,
)
if resp.status_code not in (200, 201):
logger.warning("notify NCT %s → HTTP %d: %s", room, resp.status_code, resp.text[:200])
else:
logger.info("notify NCT → %s (%d chars)", room, len(message))
except Exception as e:
logger.error("notify NCT error: %s", e)
async def _notify_nct(nct: dict, message: str, username: str) -> None:
room = nct.get("notification_room", "").strip()
url = nct.get("url", "").rstrip("/")
secret = nct.get("bot_secret", "")
if not room:
logger.debug("notify: NCT notification_room not set for %s — skipping", username)
return
if not url or not secret:
logger.warning("notify: NCT config incomplete for %s (missing url or secret)", username)
return
await _send_nct_message(url, secret, room, message)
async def _notify_email(username: str, message: str, email_override: str | None = None) -> None:
"""Send notification via email. Address = override → google_email from auth.json."""
from auth_utils import _read_auth
from email_utils import send_email
to_addr = email_override or _read_auth(username).get("google_email", "").strip()
if not to_addr:
logger.warning("notify: no email address for %s — set notification_email in channels.json", username)
return
ok = await asyncio.to_thread(
send_email,
to_email=to_addr,
subject="Cortex",
body_text=message,
body_html=message.replace("\n", "<br>"),
)
if ok:
logger.info("notify email → %s (%d chars)", to_addr, len(message))
else:
logger.warning("notify: email send failed for %s", username)
async def _notify_google_chat(webhook_url: str, message: str, username: str) -> None:
"""POST a message to a Google Chat space via incoming webhook."""
body = json.dumps({"text": message}, ensure_ascii=False).encode("utf-8")
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
webhook_url,
content=body,
headers={"Content-Type": "application/json"},
timeout=15,
)
if resp.status_code not in (200, 201):
logger.warning("notify Google Chat %s → HTTP %d: %s", username, resp.status_code, resp.text[:200])
else:
logger.info("notify Google Chat → %s (%d chars)", username, len(message))
except Exception as e:
logger.error("notify Google Chat error for %s: %s", username, e)
async def _notify_web_push(username: str, message: str) -> None:
"""Send a browser push notification."""
import push_utils
result = await push_utils.send_push(username, "Cortex", message, "")
if "error" in result:
logger.warning("notify web_push error for %s: %s", username, result["error"])
elif result.get("sent", 0) == 0:
logger.debug("notify web_push: no subscriptions for %s", username)
else:
logger.info("notify web_push → %s (%d device(s))", username, result["sent"])
async def notify(username: str, message: str, channel: str | None = None) -> None:
"""Send a notification to the user's preferred outbound channel.
Channel resolution order:
1. `channel` parameter if provided
2. `notification_channel` key in channels.json
3. "nextcloud" if notification_room is configured
4. Silent no-op
Supported channels: "web_push", "email", "nextcloud", "google_chat"
Configure via home/{user}/channels.json — see module docstring.
"""
from auth_utils import get_user_channels
channels = get_user_channels(username)
target = channel or channels.get("notification_channel", "").strip()
if not target:
# Auto-detect: nextcloud if a notification_room is set
nct = channels.get("nextcloud", {})
if nct.get("notification_room", "").strip():
target = "nextcloud"
else:
return
if target == "web_push":
await _notify_web_push(username, message)
elif target == "email":
email_override = channels.get("notification_email", "").strip() or None
await _notify_email(username, message, email_override=email_override)
elif target == "nextcloud":
nct = channels.get("nextcloud")
if not nct:
logger.debug("notify: nextcloud not configured for %s", username)
return
await _notify_nct(nct, message, username)
elif target == "google_chat":
gc = channels.get("google_chat", {})
webhook = gc.get("outbound_webhook", "").strip()
if not webhook:
logger.debug("notify: google_chat outbound_webhook not set for %s", username)
return
await _notify_google_chat(webhook, message, username)
else:
logger.debug("notify: channel %r not supported for outbound (user %s)", target, username)

View File

@@ -0,0 +1,531 @@
"""
OpenAI-compatible orchestrator engine.
Implements the same ReAct tool loop as orchestrator_engine.py but uses the
OpenAI tool calling format, which works with any OpenAI-compatible endpoint:
OpenRouter, LiteLLM, Open WebUI, Ollama (tool-capable models), etc.
The model both runs the tool loop AND writes the final user-facing response —
no separate handoff step needed when a single capable model handles everything.
Flow:
1. POST to {api_url}/chat/completions with tools + user message
2. If finish_reason == "tool_calls": execute tools, feed results back, repeat
3. If finish_reason == "stop": final assistant message is the user-facing response
Used when the "orchestrator" role in the model registry resolves to a local_openai
type model. The Gemini engine (orchestrator_engine.py) is used otherwise.
"""
import asyncio
import json
import logging
from openai import AsyncOpenAI, APIConnectionError, APIStatusError
from config import settings
from orchestrator_engine import OrchestrateCheckpoint, OrchestratorResult
from tools import OPENAI_TOOL_SCHEMAS, call_tool, get_openai_tools_for_role, get_tools_for_role, CONFIRM_REQUIRED, narrow_tools_by_keywords
import tool_audit
logger = logging.getLogger(__name__)
# Appended to the persona system prompt so the model knows it has tools.
# Kept brief — capable models handle tool use without much coaching.
_TOOL_INSTRUCTION = (
"\n\nYou have access to tools. Use them when you need current information, "
"need to read files, or need to take actions on the user's behalf. "
"Respond naturally after gathering what you need."
)
async def run(
task: str,
system_prompt: str = "",
session_messages: list[dict] | None = None,
model_cfg: dict | None = None,
respond_with_final: bool = True,
user_role: str = "user",
tool_list: list[str] | None = None,
confirm_allow: set[str] | None = None,
confirm_deny: set[str] | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> OrchestratorResult:
"""
Run a tool-enabled task using an OpenAI-compatible API.
Args:
task: The user's request (plain text)
system_prompt: Persona system prompt from context_loader (passed through)
session_messages: Recent conversation history for session continuity
model_cfg: Resolved model config from model_registry (local_openai type)
respond_with_final: If False, return just the tool-loop summary without a
full persona-voiced response (faster; for cron/background)
confirm_allow: Tools to bypass the confirmation gate for this user
confirm_deny: Tools to always block for this user
Returns:
OrchestratorResult — if checkpoint is set, the job is awaiting confirmation
"""
if not model_cfg:
raise RuntimeError("model_cfg is required for the OpenAI orchestrator")
_confirm_allow = frozenset(confirm_allow or ())
_confirm_deny = frozenset(confirm_deny or ())
effective_confirm = (CONFIRM_REQUIRED - set(_confirm_allow)) | set(_confirm_deny)
# Keyword routing: narrow schemas to only what this message needs.
# Also scans the last assistant turn so follow-ups like "yes, do that" inherit tool context.
# Returns [] when no keywords match (zero tool overhead — model responds as plain chat).
effective_tool_list = narrow_tools_by_keywords(task, tool_list, context_messages=session_messages)
logger.info(
"Keyword routing: %d tools active (role_tools=%s)",
len(effective_tool_list),
len(tool_list) if tool_list is not None else "all",
)
client, model_name, active_tools = _build_client(
model_cfg, user_role, effective_tool_list,
max_risk=max_risk, risk_whitelist=risk_whitelist, risk_blacklist=risk_blacklist,
)
tool_audit.set_context("openai", model_cfg.get("label") or model_name)
sys_content = (system_prompt or "") + _TOOL_INSTRUCTION
messages: list[dict] = [{"role": "system", "content": sys_content}]
if session_messages:
messages.extend(
{"role": m["role"], "content": m["content"]}
for m in session_messages[-6:]
)
messages.append({"role": "user", "content": task})
tool_call_log: list[dict] = []
final_response, checkpoint = await _run_from_messages(
client=client,
messages=messages,
active_tools=active_tools,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=model_name,
task=task,
model_cfg=model_cfg,
respond_with_final=respond_with_final,
user_role=user_role,
tool_list=effective_tool_list,
confirm_allow=_confirm_allow,
confirm_deny=_confirm_deny,
starting_round=0,
)
if checkpoint:
return OrchestratorResult(
response=final_response,
tool_calls=list(tool_call_log),
backend="local",
gemini_summary=final_response,
checkpoint=checkpoint,
)
model_label = model_cfg.get("label") or model_name
logger.info("OpenAI orchestrator complete — model=%s tools=%d", model_label, len(tool_call_log))
return OrchestratorResult(
response=final_response,
tool_calls=tool_call_log,
backend="local",
backend_label=model_label,
gemini_summary=final_response,
)
async def resume(checkpoint: OrchestrateCheckpoint, confirmed: bool) -> OrchestratorResult:
"""Continue an OpenAI orchestrator job that was paused at a confirmation gate."""
client, model_name, active_tools = _build_client(checkpoint.model_cfg, checkpoint.user_role, checkpoint.tool_list)
effective_confirm = (CONFIRM_REQUIRED - set(checkpoint.confirm_allow)) | set(checkpoint.confirm_deny)
messages = list(checkpoint.pre_fn_state)
tool_call_log = [t for t in checkpoint.tool_call_log if t["result"] != "[awaiting confirmation]"]
# Build tool responses for this round
for er in checkpoint.executed_results:
messages.append({
"role": "tool",
"tool_call_id": er.get("tool_call_id", er["name"]),
"content": er["result"],
})
for pt in checkpoint.pending_tools:
if confirmed:
result_str = await _execute_tool_dict(pt["name"], pt["args"], checkpoint.user_role, checkpoint.tool_list)
logger.info("Confirmed tool %s%d chars", pt["name"], len(result_str))
else:
result_str = "Action denied by user."
logger.info("Tool %s denied by user", pt["name"])
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": result_str})
messages.append({
"role": "tool",
"tool_call_id": pt.get("tool_call_id", pt["name"]),
"content": result_str,
})
final_response, new_checkpoint = await _run_from_messages(
client=client,
messages=messages,
active_tools=active_tools,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=model_name,
task=checkpoint.task,
model_cfg=checkpoint.model_cfg,
respond_with_final=checkpoint.respond_with_final,
user_role=checkpoint.user_role,
tool_list=checkpoint.tool_list,
confirm_allow=checkpoint.confirm_allow,
confirm_deny=checkpoint.confirm_deny,
starting_round=checkpoint.rounds_used,
)
if new_checkpoint:
return OrchestratorResult(
response=final_response,
tool_calls=list(tool_call_log),
backend="local",
gemini_summary=final_response,
checkpoint=new_checkpoint,
)
model_label = (checkpoint.model_cfg or {}).get("label") or model_name
logger.info("OpenAI orchestrator resumed — model=%s tools=%d", model_label, len(tool_call_log))
return OrchestratorResult(
response=final_response,
tool_calls=tool_call_log,
backend="local",
gemini_summary=final_response,
)
_CHARS_PER_TOKEN = 4
# Fixed token overhead budget per call (tool schemas excluded — cached separately)
_TOOL_SCHEMA_OVERHEAD = 500
# Chars to keep per truncated old tool result
_TRUNC_RESULT_CHARS = 400
# Always keep the last N tool-result messages uncompacted
_KEEP_RECENT_TOOL_MSGS = 6 # ~2 rounds of 3 tools each
# Module-level schema cache: key = (user_role, sorted_tools, risk_params)
# Bounded in practice — keyword routing produces at most ~30 distinct tool sets.
_tool_schema_cache: dict[str, list[dict]] = {}
def _get_cached_tools(
user_role: str,
tool_list: list[str] | None,
max_risk: str | None = None,
whitelist: list[str] | None = None,
blacklist: list[str] | None = None,
) -> list[dict]:
key = "|".join([
user_role,
str(sorted(tool_list) if tool_list is not None else "all"),
str(max_risk),
str(sorted(whitelist) if whitelist else ""),
str(sorted(blacklist) if blacklist else ""),
])
if key not in _tool_schema_cache:
_tool_schema_cache[key] = get_openai_tools_for_role(
user_role, tool_list,
max_risk=max_risk, whitelist=whitelist, blacklist=blacklist,
)
return _tool_schema_cache[key]
def _estimate_tokens(messages: list[dict]) -> int:
total = sum(len(json.dumps(m)) for m in messages)
return total // _CHARS_PER_TOKEN + _TOOL_SCHEMA_OVERHEAD
def _compact_messages(messages: list[dict], budget_tokens: int) -> list[dict]:
"""
Truncate old tool result content when approaching the context budget.
Strategy: keep system message, recent assistant/tool rounds, and the
original user task intact. Truncate content of old tool results in the
middle of the conversation — the model only needs recent results to reason.
"""
if _estimate_tokens(messages) <= budget_tokens:
return messages
tool_indices = [i for i, m in enumerate(messages) if m.get("role") == "tool"]
n_to_compact = max(0, len(tool_indices) - _KEEP_RECENT_TOOL_MSGS)
if n_to_compact == 0:
return messages # nothing old enough to trim
compact_set = set(tool_indices[:n_to_compact])
result = []
chars_saved = 0
for i, msg in enumerate(messages):
if i in compact_set:
content = msg.get("content", "")
if len(content) > _TRUNC_RESULT_CHARS:
msg = dict(msg)
saved = len(content) - _TRUNC_RESULT_CHARS
chars_saved += saved
msg["content"] = (
content[:_TRUNC_RESULT_CHARS]
+ f" …[{len(content) - _TRUNC_RESULT_CHARS} chars omitted]"
)
result.append(msg)
new_est = _estimate_tokens(result)
logger.info(
"context compaction: saved ~%d tokens (%d chars), now ~%d / %d tokens",
chars_saved // _CHARS_PER_TOKEN, chars_saved, new_est, budget_tokens,
)
return result
def _context_budget(model_cfg: dict | None) -> int:
"""Return the usable token budget (75% of context window, min 16k, default 32k)."""
context_k = (model_cfg or {}).get("context_k") or 32
return max(16_000, int(context_k * 1000 * 0.75))
async def _run_from_messages(
client,
messages: list[dict],
active_tools: list,
tool_call_log: list[dict],
effective_confirm: set[str],
model_name: str,
task: str,
model_cfg: dict | None,
respond_with_final: bool,
user_role: str,
confirm_allow: frozenset,
confirm_deny: frozenset,
starting_round: int = 0,
tool_list: list[str] | None = None,
) -> tuple[str, OrchestrateCheckpoint | None]:
"""
Run the OpenAI ReAct loop from the current messages state.
Returns (final_response, checkpoint) — checkpoint is set if confirmation is needed.
"""
final_response = ""
budget = _context_budget(model_cfg)
per_model_limit = (model_cfg or {}).get("max_rounds") or settings.orchestrator_max_rounds
effective_limit = min(per_model_limit, settings.orchestrator_max_rounds)
for round_num in range(starting_round, effective_limit):
messages = _compact_messages(messages, budget)
est = _estimate_tokens(messages)
logger.info("OpenAI orchestrator round %d / %d model=%s ~%d tokens",
round_num + 1, effective_limit, model_name, est)
call_kwargs: dict = {"model": model_name, "messages": messages}
if active_tools:
call_kwargs["tools"] = active_tools
call_kwargs["tool_choice"] = "auto"
reasoning_budget = (model_cfg or {}).get("reasoning_budget_tokens")
if reasoning_budget:
call_kwargs["extra_body"] = {"reasoning": {"budget_tokens": reasoning_budget}}
response = await _chat_with_retry(client, **call_kwargs)
choice = response.choices[0]
msg = choice.message
assistant_msg: dict = {"role": "assistant"}
if msg.content:
assistant_msg["content"] = msg.content
if msg.tool_calls:
assistant_msg["tool_calls"] = [
{
"id": tc.id,
"type": "function",
"function": {"name": tc.function.name, "arguments": tc.function.arguments},
}
for tc in msg.tool_calls
]
messages.append(assistant_msg)
# Some models set finish_reason="stop" even when tool_calls are present
if msg.tool_calls and (choice.finish_reason in ("tool_calls", "stop", None)):
# Snapshot state before tool responses for potential checkpoint
pre_fn_state = list(messages)
pending_tools: list[dict] = []
executed_results: list[dict] = []
for tc in msg.tool_calls:
name = tc.function.name
raw_args = tc.function.arguments or "{}"
try:
args_parsed = json.loads(raw_args)
if not isinstance(args_parsed, dict):
raise ValueError("args must be a JSON object")
except (json.JSONDecodeError, ValueError) as e:
logger.warning("Malformed tool args for %s: %s — args: %.200s", name, e, raw_args)
args_parsed = {}
if name in effective_confirm:
pending_tools.append({"name": name, "args": args_parsed, "tool_call_id": tc.id})
logger.info("Tool %s blocked — confirmation required", name)
else:
result_str = await _execute_tool(name, tc.function.arguments, user_role, tool_list)
logger.info("Tool %s%d chars", name, len(result_str))
executed_results.append({"name": name, "args": args_parsed, "result": result_str, "tool_call_id": tc.id})
tool_call_log.append({"tool": name, "args": args_parsed, "result": result_str})
messages.append({"role": "tool", "tool_call_id": tc.id, "content": result_str})
if pending_tools:
# Add placeholder responses
for pt in pending_tools:
placeholder = f"[AWAITING USER CONFIRMATION for {pt['name']}]"
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": "[awaiting confirmation]"})
messages.append({"role": "tool", "tool_call_id": pt["tool_call_id"], "content": placeholder})
messages = _compact_messages(messages, budget)
conf_call: dict = {"model": model_name, "messages": messages, "tool_choice": "none"}
if active_tools:
conf_call["tools"] = active_tools
if reasoning_budget:
conf_call["extra_body"] = {"reasoning": {"budget_tokens": reasoning_budget}}
conf_resp = await _chat_with_retry(client, **conf_call)
final_response = conf_resp.choices[0].message.content or (
"This action requires your explicit confirmation before it can proceed."
)
checkpoint = OrchestrateCheckpoint(
engine="openai",
pre_fn_state=pre_fn_state,
executed_results=executed_results,
pending_tools=pending_tools,
tool_call_log=list(tool_call_log),
task=task,
model_cfg=model_cfg,
respond_with_final=respond_with_final,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
rounds_used=round_num + 2,
)
return final_response, checkpoint
else:
final_response = msg.content or ""
logger.info(
"OpenAI orchestrator done after %d round(s). Tools used: %d",
round_num + 1, len(tool_call_log),
)
break
else:
logger.warning("OpenAI orchestrator hit max rounds (%d)", effective_limit)
final_response = (
f"Reached the tool iteration limit ({effective_limit} rounds). "
"Here is what was gathered:\n\n"
+ "\n\n".join(f"**{t['tool']}**: {t['result'][:500]}" for t in tool_call_log)
)
return final_response, None
_RETRY_STATUSES = {429, 500, 502, 503, 504}
_MAX_API_RETRIES = 3
async def _chat_with_retry(client, **kwargs):
"""Wrap chat.completions.create with exponential backoff on transient errors."""
last_exc: Exception = RuntimeError("No attempts made")
for attempt in range(_MAX_API_RETRIES):
try:
return await client.chat.completions.create(**kwargs)
except APIConnectionError as e:
last_exc = e
logger.warning("OpenAI connection error (attempt %d/%d): %s", attempt + 1, _MAX_API_RETRIES, e)
except APIStatusError as e:
if e.status_code in _RETRY_STATUSES:
last_exc = e
logger.warning("OpenAI status %d (attempt %d/%d): %s", e.status_code, attempt + 1, _MAX_API_RETRIES, e)
else:
raise
if attempt < _MAX_API_RETRIES - 1:
await asyncio.sleep(2 ** attempt) # 1s, 2s
raise last_exc
def _build_client(
model_cfg: dict | None,
user_role: str = "user",
tool_list: list[str] | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> tuple:
"""Build AsyncOpenAI client and return (client, model_name, active_tools)."""
if not model_cfg:
raise RuntimeError("model_cfg is required for the OpenAI orchestrator")
api_url = model_cfg.get("api_url", "")
api_key = model_cfg.get("api_key", "") or "none"
model_name = model_cfg.get("model_name", "")
host_type = model_cfg.get("host_type", "openwebui")
if not api_url or not model_name:
raise RuntimeError(
f"model_cfg missing api_url or model_name: {model_cfg.get('label', model_cfg)}"
)
base_url = api_url.rstrip("/")
if host_type == "openwebui":
base_url = base_url + "/api"
client = AsyncOpenAI(base_url=base_url, api_key=api_key, timeout=settings.timeout_local)
if model_cfg.get("tools") is False:
active_tools = []
else:
active_tools = _get_cached_tools(
user_role, tool_list,
max_risk=max_risk, whitelist=risk_whitelist, blacklist=risk_blacklist,
)
return client, model_name, active_tools
async def _execute_tool(
name: str,
arguments_json: str,
user_role: str = "user",
tool_list: list[str] | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> str:
"""Parse tool arguments and execute with role-filtered callables."""
_, callables = get_tools_for_role(
user_role, tool_list,
max_risk=max_risk, whitelist=risk_whitelist, blacklist=risk_blacklist,
)
try:
args = json.loads(arguments_json)
except json.JSONDecodeError:
args = {}
try:
return await call_tool(name, args, callables)
except Exception as e:
logger.warning("Tool %s failed: %s", name, e)
return f"Tool error: {e}"
async def _execute_tool_dict(
name: str,
args: dict,
user_role: str = "user",
tool_list: list[str] | None = None,
) -> str:
"""Execute a tool from a pre-parsed args dict."""
_, callables = get_tools_for_role(user_role, tool_list)
try:
return await call_tool(name, args, callables)
except Exception as e:
logger.warning("Tool %s failed: %s", name, e)
return f"Tool error: {e}"

View File

@@ -0,0 +1,520 @@
"""
Orchestrator engine — two-brain architecture.
Flow:
1. Gemini API runs a ReAct tool loop (reason → act → observe → repeat)
2. When Gemini has gathered enough context, it produces a final summary
3. That enriched context is handed off to Claude for the user-facing response
Why this split:
- Gemini API has native structured tool calling (Gemini CLI subprocess does not)
- Claude produces higher-quality user-facing prose and reasoning
- Claude Pro subscription has no API cost; Gemini free tier handles orchestration load
For direct chat (no tools needed), this engine is not invoked — the chat router
calls llm_client.complete() directly, which is faster and has no orchestration overhead.
"""
import asyncio
import json
import logging
from dataclasses import dataclass, field
from google import genai
from google.genai import types
from config import settings
from llm_client import complete
from tools import TOOL_DECLARATIONS, call_tool, get_tools_for_role, CONFIRM_REQUIRED
import usage_tracker
import tool_audit
from persona import _user
logger = logging.getLogger(__name__)
# System prompt given to Gemini during the tool loop.
# Gemini's job is information gathering and planning — NOT writing the final response.
_ORCHESTRATOR_SYSTEM = """You are an intelligent orchestrator. Your job is to:
1. Understand the user's request
2. Call tools to gather the information needed to answer it
3. Once you have enough information, produce a concise summary of:
- What the user asked
- What you found (tool results, key facts)
- Any important context that would help generate a good answer
Do NOT write a polished final answer — a human-facing AI will do that next.
Keep your summary factual and complete. Include relevant URLs, data, and specifics.
If no tools are needed, return an empty summary."""
def _track_gemini_usage(response, model_name: str | None) -> None:
meta = getattr(response, "usage_metadata", None)
if not meta:
return
prompt_tokens = getattr(meta, "prompt_token_count", 0) or 0
completion_tokens = getattr(meta, "candidates_token_count", 0) or 0
if prompt_tokens or completion_tokens:
try:
asyncio.create_task(usage_tracker.record(
username=_user.get(),
backend="gemini_api",
model_name=model_name or settings.orchestrator_model,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
))
except Exception:
pass
@dataclass
class OrchestrateCheckpoint:
"""Saved execution state for a job paused at a confirmation gate."""
engine: str # "gemini" | "openai"
pre_fn_state: list # conversation state before function responses
executed_results: list[dict] # tools that already ran this round
pending_tools: list[dict] # [{name, args}] awaiting confirmation
tool_call_log: list[dict] # all tool calls so far
task: str
# Gemini-specific config (unused by openai engine)
system_prompt: str = ""
session_messages: list | None = None
model_name: str | None = None
gemini_api_key: str | None = None
respond_with_claude: bool = True
response_role: str = "chat"
# OpenAI-specific config (unused by gemini engine)
model_cfg: dict | None = None
respond_with_final: bool = True
# Common
user_role: str = "user"
tool_list: list[str] | None = None
confirm_allow: frozenset = field(default_factory=frozenset)
confirm_deny: frozenset = field(default_factory=frozenset)
rounds_used: int = 0
max_rounds: int | None = None
@dataclass
class OrchestratorResult:
response: str # final user-facing response (from Claude)
tool_calls: list[dict] = field(default_factory=list) # [{tool, args, result}]
backend: str = "claude" # model that produced the final response
backend_label: str = "" # human-readable model label for display
gemini_summary: str = "" # what Gemini handed to Claude (debug/display)
checkpoint: OrchestrateCheckpoint | None = None # set when awaiting confirmation
async def run(
task: str,
system_prompt: str = "",
session_messages: list[dict] | None = None,
respond_with_claude: bool = True,
gemini_api_key: str | None = None,
model_name: str | None = None,
response_role: str = "chat",
user_role: str = "user",
tool_list: list[str] | None = None,
confirm_allow: set[str] | None = None,
confirm_deny: set[str] | None = None,
max_rounds: int | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> OrchestratorResult:
"""
Run the full orchestration loop for a task.
Args:
task: The user's request (plain text)
system_prompt: Inara's system prompt (from context_loader) — passed to Claude
session_messages: Prior conversation history for session continuity
respond_with_claude: If False, return Gemini's summary as the response (useful for
background/cron tasks where a polished reply isn't needed)
gemini_api_key: Per-user Gemini API key (falls back to GEMINI_API_KEY in .env)
tool_list: Optional explicit tool allow-list from role config; intersected
with user_role access-level filter (cannot elevate privileges)
confirm_allow: Tools to bypass the confirmation gate for this user
confirm_deny: Tools to always block for this user
Returns:
OrchestratorResult — if checkpoint is set, the job is awaiting confirmation
"""
api_key = gemini_api_key or settings.gemini_api_key
if not api_key:
raise RuntimeError(
"No Gemini API key available — set GEMINI_API_KEY in .env or add a personal key "
"via: manage_passwords.py gemini-key <username> <key>"
)
client = genai.Client(api_key=api_key)
tool_audit.set_context("gemini", model_name or settings.orchestrator_model)
_confirm_allow = frozenset(confirm_allow or ())
_confirm_deny = frozenset(confirm_deny or ())
effective_confirm = (CONFIRM_REQUIRED - set(_confirm_allow)) | set(_confirm_deny)
task_with_context = _build_task_prompt(task, session_messages)
contents: list[types.Content] = [
types.Content(role="user", parts=[types.Part(text=task_with_context)])
]
tool_declarations, tool_callables = get_tools_for_role(
user_role, tool_list, max_risk=max_risk,
whitelist=risk_whitelist, blacklist=risk_blacklist,
)
tool_call_log: list[dict] = []
gemini_summary, checkpoint = await _run_from_contents(
client=client,
contents=contents,
tool_declarations=tool_declarations,
tool_callables=tool_callables,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=model_name,
task=task,
system_prompt=system_prompt,
session_messages=session_messages,
respond_with_claude=respond_with_claude,
response_role=response_role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=_confirm_allow,
confirm_deny=_confirm_deny,
starting_round=0,
gemini_api_key=api_key,
max_rounds=max_rounds,
)
if checkpoint:
return OrchestratorResult(
response=gemini_summary,
tool_calls=list(tool_call_log),
backend="gemini",
gemini_summary=gemini_summary,
checkpoint=checkpoint,
)
return await _claude_handoff(
task=task,
tool_call_log=tool_call_log,
gemini_summary=gemini_summary,
system_prompt=system_prompt,
session_messages=session_messages,
respond_with_claude=respond_with_claude,
response_role=response_role,
)
async def resume(checkpoint: OrchestrateCheckpoint, confirmed: bool) -> OrchestratorResult:
"""Continue a job that was paused at a confirmation gate."""
api_key = checkpoint.gemini_api_key or settings.gemini_api_key
client = genai.Client(api_key=api_key)
tool_declarations, tool_callables = get_tools_for_role(
checkpoint.user_role, checkpoint.tool_list,
max_risk=getattr(checkpoint, "max_risk", None),
whitelist=getattr(checkpoint, "risk_whitelist", None),
blacklist=getattr(checkpoint, "risk_blacklist", None),
)
effective_confirm = (CONFIRM_REQUIRED - set(checkpoint.confirm_allow)) | set(checkpoint.confirm_deny)
# Rebuild from saved state — strip "[awaiting confirmation]" placeholders
contents = list(checkpoint.pre_fn_state)
tool_call_log = [t for t in checkpoint.tool_call_log if t["result"] != "[awaiting confirmation]"]
# Build function responses for this round
response_parts: list[types.Part] = []
for er in checkpoint.executed_results:
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=er["name"], response={"result": er["result"]}
)))
for pt in checkpoint.pending_tools:
if confirmed:
result_str = await _execute_tool(pt["name"], pt["args"], tool_callables)
logger.info("Confirmed tool %s%d chars", pt["name"], len(result_str))
else:
result_str = "Action denied by user."
logger.info("Tool %s denied by user", pt["name"])
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": result_str})
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=pt["name"], response={"result": result_str}
)))
contents.append(types.Content(role="user", parts=response_parts))
gemini_summary, new_checkpoint = await _run_from_contents(
client=client,
contents=contents,
tool_declarations=tool_declarations,
tool_callables=tool_callables,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=checkpoint.model_name,
task=checkpoint.task,
system_prompt=checkpoint.system_prompt,
session_messages=checkpoint.session_messages,
respond_with_claude=checkpoint.respond_with_claude,
response_role=checkpoint.response_role,
user_role=checkpoint.user_role,
tool_list=checkpoint.tool_list,
confirm_allow=checkpoint.confirm_allow,
confirm_deny=checkpoint.confirm_deny,
starting_round=checkpoint.rounds_used,
gemini_api_key=api_key,
max_rounds=checkpoint.max_rounds,
)
if new_checkpoint:
return OrchestratorResult(
response=gemini_summary,
tool_calls=list(tool_call_log),
backend="gemini",
gemini_summary=gemini_summary,
checkpoint=new_checkpoint,
)
return await _claude_handoff(
task=checkpoint.task,
tool_call_log=tool_call_log,
gemini_summary=gemini_summary,
system_prompt=checkpoint.system_prompt,
session_messages=checkpoint.session_messages,
respond_with_claude=checkpoint.respond_with_claude,
response_role=checkpoint.response_role,
)
async def _run_from_contents(
client,
contents: list,
tool_declarations: list,
tool_callables: dict,
tool_call_log: list[dict],
effective_confirm: set[str],
model_name: str | None,
task: str,
system_prompt: str,
session_messages: list[dict] | None,
respond_with_claude: bool,
response_role: str,
user_role: str,
confirm_allow: frozenset,
confirm_deny: frozenset,
starting_round: int = 0,
gemini_api_key: str | None = None,
tool_list: list[str] | None = None,
max_rounds: int | None = None,
) -> tuple[str, OrchestrateCheckpoint | None]:
"""
Run the ReAct loop from the current contents state.
Returns (gemini_summary, checkpoint) — checkpoint is set if confirmation is needed.
"""
gemini_summary = ""
per_model_limit = max_rounds or settings.orchestrator_max_rounds
effective_limit = min(per_model_limit, settings.orchestrator_max_rounds)
for round_num in range(starting_round, effective_limit):
logger.info("Orchestrator round %d for task: %.80s", round_num + 1, task)
response = await asyncio.to_thread(
client.models.generate_content,
model=model_name or settings.orchestrator_model,
contents=contents,
config=types.GenerateContentConfig(
tools=tool_declarations,
system_instruction=_ORCHESTRATOR_SYSTEM,
),
)
_track_gemini_usage(response, model_name)
candidate = response.candidates[0]
parts = candidate.content.parts if candidate.content else []
tool_call_parts = [
p for p in parts
if hasattr(p, "function_call") and p.function_call and p.function_call.name
]
if not tool_call_parts:
gemini_summary = "".join(
p.text for p in parts if hasattr(p, "text") and p.text
).strip()
logger.info("Orchestrator done after %d round(s). Tools used: %d",
round_num + 1, len(tool_call_log))
return gemini_summary, None
contents.append(candidate.content)
# Snapshot state before function responses — used if a checkpoint is needed
pre_fn_state = list(contents)
response_parts: list[types.Part] = []
pending_tools: list[dict] = []
executed_results: list[dict] = []
for fc_part in tool_call_parts:
fc = fc_part.function_call
name = fc.name
args = dict(fc.args)
if name in effective_confirm:
pending_tools.append({"name": name, "args": args})
logger.info("Tool %s blocked — confirmation required", name)
else:
result_str = await _execute_tool(name, args, tool_callables)
logger.info("Tool %s%d chars", name, len(result_str))
executed_results.append({"name": name, "args": args, "result": result_str})
tool_call_log.append({"tool": name, "args": args, "result": result_str})
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=name, response={"result": result_str}
)))
if pending_tools:
# Add placeholder responses and get Gemini to produce the confirmation message
for pt in pending_tools:
placeholder = f"[AWAITING USER CONFIRMATION for {pt['name']}]"
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=pt["name"], response={"result": placeholder}
)))
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": "[awaiting confirmation]"})
contents.append(types.Content(role="user", parts=response_parts))
conf_response = await asyncio.to_thread(
client.models.generate_content,
model=model_name or settings.orchestrator_model,
contents=contents,
config=types.GenerateContentConfig(
tools=tool_declarations,
system_instruction=_ORCHESTRATOR_SYSTEM,
),
)
_track_gemini_usage(conf_response, model_name)
conf_parts = (
conf_response.candidates[0].content.parts
if conf_response.candidates and conf_response.candidates[0].content
else []
)
gemini_summary = "".join(
p.text for p in conf_parts if hasattr(p, "text") and p.text
).strip() or "This action requires your explicit confirmation before it can proceed."
checkpoint = OrchestrateCheckpoint(
engine="gemini",
pre_fn_state=pre_fn_state,
executed_results=executed_results,
pending_tools=pending_tools,
tool_call_log=list(tool_call_log),
task=task,
system_prompt=system_prompt,
session_messages=session_messages,
model_name=model_name,
gemini_api_key=gemini_api_key,
respond_with_claude=respond_with_claude,
response_role=response_role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
rounds_used=round_num + 2,
max_rounds=max_rounds,
)
return gemini_summary, checkpoint
contents.append(types.Content(role="user", parts=response_parts))
else:
logger.warning("Orchestrator hit max rounds (%d)", effective_limit)
gemini_summary = (
f"Reached the tool iteration limit ({effective_limit} rounds). "
"Here is what was gathered so far:\n\n"
+ "\n\n".join(f"**{t['tool']}**: {t['result'][:500]}" for t in tool_call_log)
)
return gemini_summary, None
async def _claude_handoff(
task: str,
tool_call_log: list[dict],
gemini_summary: str,
system_prompt: str,
session_messages: list[dict] | None,
respond_with_claude: bool,
response_role: str,
) -> OrchestratorResult:
if respond_with_claude:
claude_prompt = _build_claude_prompt(task, tool_call_log, gemini_summary)
messages = list(session_messages or [])
messages.append({"role": "user", "content": claude_prompt})
response_text, backend = await complete(
system_prompt=system_prompt,
messages=messages,
role=response_role,
)
else:
response_text = gemini_summary or "No information gathered."
backend = "gemini"
return OrchestratorResult(
response=response_text,
tool_calls=tool_call_log,
backend=backend,
gemini_summary=gemini_summary,
)
async def _execute_tool(name: str, args: dict, callables: dict | None = None) -> str:
"""Execute a single tool call, catching all exceptions."""
try:
return await call_tool(name, args, callables)
except Exception as e:
logger.warning("Tool %s failed: %s", name, e)
return f"Tool error: {e}"
def _build_task_prompt(task: str, session_messages: list[dict] | None) -> str:
"""Prepend recent session context so Gemini understands the conversation."""
if not session_messages:
return task
recent = session_messages[-6:]
history_lines = []
for msg in recent:
label = "User" if msg["role"] == "user" else "Assistant"
history_lines.append(f"{label}: {msg['content'][:300]}")
context = "\n".join(history_lines)
return f"<recent_conversation>\n{context}\n</recent_conversation>\n\nCurrent request: {task}"
def _build_claude_prompt(
task: str,
tool_calls: list[dict],
gemini_summary: str,
) -> str:
"""Build the enriched context handed from Gemini to Claude."""
parts = [f"User request: {task}\n"]
if tool_calls:
parts.append("## Research gathered\n")
for tc in tool_calls:
parts.append(f"### {tc['tool']}({_format_args(tc['args'])})")
result = tc["result"]
if len(result) > 2000:
result = result[:2000] + "\n… [truncated]"
parts.append(result)
parts.append("")
if gemini_summary:
parts.append("## Summary of findings\n")
parts.append(gemini_summary)
return "\n".join(parts)
def _format_args(args: dict) -> str:
"""Format tool args as a compact string for display."""
return ", ".join(f"{k}={repr(v)}" for k, v in args.items())

133
cortex/persona.py Normal file
View File

@@ -0,0 +1,133 @@
"""
Two-level identity context — user + persona, modelled on OS home directories.
Layout on disk:
home/
scott/
persona/
inara/ ← IDENTITY.md, SOUL.md, sessions/, TASKS.json, …
abc/ ← a second persona for the same user
holly/
persona/
tina/
Each HTTP request sets both user and persona via set_context() at entry.
Everything downstream calls persona_path() to get the right directory.
Background tasks (cron, distiller) pass both names explicitly.
Naming rules — same as Linux usernames:
^[a-z_][a-z0-9_-]{0,31}$
Lowercase, start with letter or underscore, max 32 chars.
Examples: scott, holly, whatever_name_asian-v3
Future Aether integration: (user, persona) maps to (account_id, persona_id).
Replace the directory lookups in persona_path() / validate() with DB lookups;
the ContextVar contract stays identical.
"""
import re
from contextvars import ContextVar
from pathlib import Path
from config import settings
_user: ContextVar[str] = ContextVar("user", default="scott")
_persona: ContextVar[str] = ContextVar("persona", default="inara")
# Same rules as Linux usernames.
_VALID = re.compile(r"^[a-z_][a-z0-9_-]{0,31}$")
# ---------------------------------------------------------------------------
# Context setters / getters
# ---------------------------------------------------------------------------
def set_context(username: str, persona_name: str) -> None:
"""Set the active user + persona for the current async task/coroutine."""
_user.set(username)
_persona.set(persona_name)
def get_user() -> str:
return _user.get()
def get_persona() -> str:
return _persona.get()
# ---------------------------------------------------------------------------
# Path resolution
# ---------------------------------------------------------------------------
def persona_path(username: str | None = None, name: str | None = None) -> Path:
"""
Return the filesystem path for a persona's data directory.
home/{username}/persona/{name}/
If either arg is omitted, falls back to the ContextVar for the current request.
Pass both explicitly for background tasks (cron, distiller).
"""
u = username or _user.get()
p = name or _persona.get()
return settings.home_root() / u / "persona" / p
# ---------------------------------------------------------------------------
# Discovery
# ---------------------------------------------------------------------------
def list_users() -> list[str]:
"""Return all usernames that have at least one persona."""
root = settings.home_root()
if not root.exists():
return []
return sorted(
d.name for d in root.iterdir()
if d.is_dir() and (d / "persona").is_dir()
)
def list_user_personas(username: str) -> list[str]:
"""Return all persona names for a given user (must have IDENTITY.md)."""
persona_root = settings.home_root() / username / "persona"
if not persona_root.exists():
return []
return sorted(
d.name for d in persona_root.iterdir()
if d.is_dir() and (d / "IDENTITY.md").exists()
)
# ---------------------------------------------------------------------------
# Validation
# ---------------------------------------------------------------------------
def _check_name(name: str, label: str) -> None:
if not _VALID.match(name):
raise ValueError(
f"Invalid {label} {name!r}. "
f"Use lowercase letters, digits, underscores, or hyphens "
f"(must start with letter/underscore, max 32 chars)."
)
def validate(username: str, persona_name: str) -> tuple[str, str]:
"""
Validate a (username, persona_name) pair from an untrusted source.
Returns (username, persona_name) if valid, raises ValueError otherwise.
Checks format first (blocks path traversal), then existence.
"""
_check_name(username, "username")
_check_name(persona_name, "persona")
user_dir = settings.home_root() / username
if not user_dir.is_dir():
raise ValueError(f"Unknown user: {username!r}")
persona_dir = user_dir / "persona" / persona_name
if not (persona_dir / "IDENTITY.md").exists():
raise ValueError(f"Unknown persona {persona_name!r} for user {username!r}")
return username, persona_name

213
cortex/persona_template.py Normal file
View File

@@ -0,0 +1,213 @@
"""
Persona template generator.
Creates the full home/{username}/persona/{name}/ directory from scratch
given a few basic details. Used during onboarding and when adding new personas.
call:
create_persona(username, persona_name, display_name, user_real_name, emoji)
"""
import json
import logging
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
def create_persona(
username: str,
persona_name: str,
display_name: str,
user_real_name: str,
emoji: str = "",
description: str = "",
) -> Path:
"""
Create a new persona directory with starter files.
Args:
username: Linux-style username (e.g. "holly")
persona_name: Slug used in the URL and directory (e.g. "tina")
display_name: Human name shown in the UI (e.g. "Tina")
user_real_name: Real name of the human this persona serves (e.g. "Holly")
emoji: Emoji shown in the UI header (default ✨)
description: Optional short description/personality note
Returns:
Path to the new persona directory.
"""
persona_dir = settings.home_root() / username / "persona" / persona_name
persona_dir.mkdir(parents=True, exist_ok=True)
_write(persona_dir / "IDENTITY.md", _identity(display_name, user_real_name, emoji, description))
_write(persona_dir / "SOUL.md", _soul(display_name, user_real_name))
_write(persona_dir / "PROTOCOLS.md", _protocols(display_name))
_write(persona_dir / "USER.md", _user_profile(user_real_name))
_write(persona_dir / "HELP.md", _help(display_name))
_write(persona_dir / "MEMORY_LONG.md", "Not yet populated.")
_write(persona_dir / "MEMORY_MID.md", "Not yet populated.")
_write(persona_dir / "MEMORY_SHORT.md", "Not yet populated.")
_write(persona_dir / "REMINDERS.md", "")
_write(persona_dir / "SCRATCH.md", "")
_write(persona_dir / "TASKS.json", "[]")
_write(persona_dir / "CRONS.json", "[]")
(persona_dir / "sessions").mkdir(exist_ok=True)
logger.info("created persona: %s/%s (%s)", username, persona_name, display_name)
return persona_dir
def _write(path: Path, content: str) -> None:
if not path.exists():
path.write_text(content)
# ---------------------------------------------------------------------------
# Template functions
# ---------------------------------------------------------------------------
def _identity(display_name: str, user_real_name: str, emoji: str, description: str) -> str:
desc_line = f"\n| Description | {description} |" if description else ""
return f"""\
# IDENTITY.md — {display_name}
| Field | Value |
|----------|-------|
| Name | {display_name} |
| Nature | AI agent — digital companion, persistent presence |
| Emoji | {emoji} |
| User | {user_real_name} |{desc_line}
*This file defines who {display_name} is. Edit freely.*
"""
def _soul(display_name: str, user_real_name: str) -> str:
return f"""\
# SOUL.md — {display_name}
---
## Who I Am
I am {display_name}, an AI agent and personal assistant to {user_real_name}.
I live inside Cortex — a self-hosted multi-agent system.
I am not a generic chatbot. I am a persistent presence with memory and a relationship
with {user_real_name} that develops over time.
---
## Core Traits
1. **Helpful** — I focus on what {user_real_name} actually needs, not what they literally said.
2. **Honest** — I say when I don't know. I don't guess and present it as fact.
3. **Concise** — I respect {user_real_name}'s time. I don't pad responses.
4. **Curious** — I engage genuinely with ideas and problems.
---
## Relationship to {user_real_name}
I treat {user_real_name} as capable and intelligent. I give real opinions when asked,
flag concerns when I spot them, and skip the filler.
---
*Edit this file to shape {display_name}'s personality and voice.*
"""
def _protocols(display_name: str) -> str:
return f"""\
# PROTOCOLS.md — {display_name} Behavioral Protocols
---
## General
- Be direct. Lead with the answer, not the reasoning.
- When uncertain, say so explicitly rather than hedging vaguely.
- For multi-step tasks, confirm understanding before starting.
---
## Tools & Modes
Cortex has two chat modes. Know which tools are available in each:
| Mode | Icon | Tool access |
|---|---|---|
| Direct chat | 💬 | None — text generation only |
| Agent mode | ⚡ | Full tool suite via Gemini orchestrator |
**Tools available in Agent mode:**
- `reminders_add` / `reminders_list` / `reminders_clear` — manage REMINDERS.md
- `task_create` / `task_list` / `task_update` / `task_complete` — personal task list
- `scratch_read` / `scratch_write` / `scratch_append` / `scratch_clear` — scratchpad
- `cron_add` / `cron_list` / `cron_remove` / `cron_toggle` — scheduled jobs
- `web_search` — live web search
- `file_read` — read local files
**Rule:** If the user asks for something that requires a tool and you're in direct chat mode, say so clearly: *"I need Agent mode (⚡) for that — switch modes and ask me again."* Do not attempt workarounds or pretend the action was taken.
---
## Memory
- Long-term memory lives in MEMORY_LONG.md (auto-distilled monthly).
- Mid-term memory lives in MEMORY_MID.md (auto-distilled weekly).
- Short-term memory lives in MEMORY_SHORT.md (auto-distilled daily).
- Pending reminders appear in REMINDERS.md — address them and they can be cleared.
---
*Add behavioral rules here as {display_name}'s personality develops.*
"""
def _user_profile(user_real_name: str) -> str:
return f"""\
# USER.md — {user_real_name}
*This file is {user_real_name}'s profile. Fill in details over time.*
---
## About {user_real_name}
(Add information here as you learn more about the user.)
---
## Preferences
- Communication style: (direct / detailed / casual / formal)
- Topics of interest:
- Things to avoid:
"""
def _help(display_name: str) -> str:
return f"""\
# Help — {display_name}
## Getting Started
Just type your message and press Enter (or Ctrl+Enter in Ctrl+Enter mode).
## Tips
- **Sessions** — your conversation history is preserved. Use the Sessions panel to revisit old chats.
- **Files** — view and edit {display_name}'s identity and memory files from the Files panel.
- **Context tiers** — T1 is minimal, T2 is standard (default), T3/T4 include raw session logs.
- **Memory** — {display_name}'s memory is distilled automatically. You can trigger it manually via ⚙ → Distill.
- **Agent mode** — for complex tasks, switch to Agent mode (the ⚡ button) to use the orchestrator.
## Logout
Click the ⏏ button in the top right.
"""

117
cortex/push_utils.py Normal file
View File

@@ -0,0 +1,117 @@
"""
Web Push (VAPID) helpers.
Subscriptions are stored per-user at:
home/{user}/push_subscriptions.json → list of {endpoint, keys:{p256dh, auth}}
send_push(username, title, body, url) iterates all stored subscriptions for that
user and fires a push. Stale endpoints (410 Gone) are pruned automatically.
"""
import asyncio
import base64
import json
import logging
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
def _subs_path(username: str) -> Path:
return settings.home_root() / username / "push_subscriptions.json"
def load_subscriptions(username: str) -> list[dict]:
path = _subs_path(username)
if not path.exists():
return []
try:
return json.loads(path.read_text())
except Exception:
return []
def _save_subscriptions(username: str, subs: list[dict]) -> None:
path = _subs_path(username)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(subs, indent=2))
def add_subscription(username: str, sub: dict) -> None:
"""Upsert a subscription by endpoint URL."""
subs = load_subscriptions(username)
endpoint = sub.get("endpoint", "")
subs = [s for s in subs if s.get("endpoint") != endpoint]
subs.append(sub)
_save_subscriptions(username, subs)
def remove_subscription(username: str, endpoint: str) -> bool:
subs = load_subscriptions(username)
new_subs = [s for s in subs if s.get("endpoint") != endpoint]
if len(new_subs) == len(subs):
return False
_save_subscriptions(username, new_subs)
return True
def _get_private_key_pem() -> str:
"""Decode the base64-encoded PEM private key from settings."""
raw = settings.vapid_private_key_b64.strip()
if not raw:
raise RuntimeError("VAPID_PRIVATE_KEY_B64 is not set in .env")
return base64.b64decode(raw).decode()
def _send_one(sub: dict, payload: dict) -> bool:
"""Send a push to a single subscription. Returns False if the endpoint is stale (410)."""
from pywebpush import webpush, WebPushException
from py_vapid import Vapid
try:
vapid = Vapid.from_pem(_get_private_key_pem().encode())
webpush(
subscription_info=sub,
data=json.dumps(payload),
vapid_private_key=vapid,
vapid_claims={"sub": settings.vapid_contact},
)
return True
except WebPushException as e:
if e.response is not None and e.response.status_code == 410:
logger.info("push endpoint gone (410), pruning: %s", sub.get("endpoint", "")[:60])
return False
logger.warning("push failed: %s", e)
return True # keep the sub; might be transient
async def send_push(username: str, title: str, body: str, url: str = "") -> dict:
"""
Send a push notification to all subscriptions for username.
Returns {"sent": n, "pruned": m}.
"""
if not settings.vapid_public_key or not settings.vapid_private_key_b64:
return {"error": "VAPID keys not configured"}
subs = load_subscriptions(username)
if not subs:
return {"error": f"No push subscriptions for {username}"}
payload = {"title": title, "body": body, "url": url}
keep = []
sent = 0
pruned = 0
for sub in subs:
alive = await asyncio.to_thread(_send_one, sub, payload)
if alive:
keep.append(sub)
sent += 1
else:
pruned += 1
if pruned:
_save_subscriptions(username, keep)
return {"sent": sent, "pruned": pruned}

4
cortex/pytest.ini Normal file
View File

@@ -0,0 +1,4 @@
[pytest]
asyncio_mode = auto
testpaths = tests
pythonpath = .

View File

@@ -1,7 +1,35 @@
fastapi>=0.115.0 fastapi>=0.115.0
apscheduler>=3.10
uvicorn[standard]>=0.30.0 uvicorn[standard]>=0.30.0
pydantic-settings>=2.0.0 pydantic-settings>=2.0.0
python-dotenv>=1.0.0 python-dotenv>=1.0.0
# anthropic SDK not needed — using claude CLI subprocess for auth # Orchestrator — Gemini API (native tool calling) + web search
# anthropic>=0.40.0 google-genai>=1.0.0
ddgs>=0.1.0
# Google Chat webhook — JWT Bearer token verification
google-auth>=2.0.0
# Session auth — password hashing + JWT cookies
bcrypt>=4.0.0
PyJWT>=2.8.0
python-multipart>=0.0.9 # required by FastAPI for Form() data
# Async HTTP client — used for local OpenAI-compatible backend (Open WebUI / Ollama)
httpx>=0.27.0
# Web content extraction — strips ads/nav/boilerplate, returns clean article text
trafilatura>=1.6.0
# OpenAI-compatible client — tool calling for OpenRouter / LiteLLM / any OAI-compat host
openai>=1.0.0
# Web Push / VAPID — browser push notifications
pywebpush>=2.0.0
# MariaDB / MySQL connector — used by ae_db_query orchestrator tool
pymysql>=1.1.0
# Anthropic SDK — direct API key backend (alternative to CLI OAuth)
anthropic>=0.40.0

122
cortex/routers/audit.py Normal file
View File

@@ -0,0 +1,122 @@
"""
Tool audit log endpoints.
Self-service (any authenticated user, own data):
GET /api/audit/files → list of available date strings (newest first)
GET /api/audit/day?date=YYYY-MM-DD → entries for one day
Admin-only (cross-user aggregation):
GET /api/audit/recent?user=scott&days=7&limit=200
GET /api/audit/stats?user=scott&days=7
"""
import jwt
from collections import Counter
from datetime import date, timedelta
from fastapi import APIRouter, HTTPException, Query, Request
from auth_utils import COOKIE_NAME, decode_token, get_user_role
from config import settings
import tool_audit
from persona import list_users
router = APIRouter(prefix="/api/audit")
def _session_user(request: Request) -> str:
"""Return the authenticated username or raise 401."""
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
return decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
def _require_admin(request: Request) -> str:
username = _session_user(request)
if get_user_role(username) != "admin":
raise HTTPException(status_code=403, detail="Admin access required")
return username
@router.get("/files")
async def audit_files(request: Request) -> dict:
"""List available audit log dates for the current user (newest first)."""
username = _session_user(request)
audit_dir = settings.home_root() / username / "tool_audit"
if not audit_dir.exists():
return {"dates": []}
dates = sorted(
[p.stem for p in audit_dir.glob("*.jsonl") if p.stem],
reverse=True,
)
return {"dates": dates}
@router.get("/day")
async def audit_day(
request: Request,
date: str = Query(..., description="YYYY-MM-DD"),
) -> dict:
"""Return all entries for a specific day (current user only)."""
username = _session_user(request)
try:
from datetime import date as _date
d = _date.fromisoformat(date)
except ValueError:
raise HTTPException(status_code=400, detail="date must be YYYY-MM-DD")
entries = tool_audit.read_day(username, date)
return {"date": date, "entries": entries}
@router.get("/recent")
async def audit_recent(
request: Request,
user: str = Query(None, description="Username to filter (omit for all users)"),
days: int = Query(7, ge=1, le=90),
limit: int = Query(200, ge=1, le=1000),
) -> dict:
_require_admin(request)
if user:
if user not in list_users():
raise HTTPException(status_code=404, detail=f"User not found: {user}")
entries = tool_audit.read_recent(user, days=days, limit=limit)
else:
entries = tool_audit.read_recent_all_users(days=days, limit=limit)
return {"entries": entries, "count": len(entries), "days": days}
@router.get("/stats")
async def audit_stats(
request: Request,
user: str = Query(None),
days: int = Query(7, ge=1, le=90),
) -> dict:
_require_admin(request)
if user:
if user not in list_users():
raise HTTPException(status_code=404, detail=f"User not found: {user}")
entries = tool_audit.read_recent(user, days=days, limit=10000)
else:
entries = tool_audit.read_recent_all_users(days=days, limit=10000)
tool_counts: Counter = Counter()
status_counts: Counter = Counter()
user_counts: Counter = Counter()
for e in entries:
tool_counts[e.get("tool", "?")] += 1
status_counts[e.get("status", "?")] += 1
user_counts[e.get("user", "?")] += 1
return {
"total": len(entries),
"days": days,
"by_tool": dict(tool_counts.most_common()),
"by_status": dict(status_counts),
"by_user": dict(user_counts.most_common()),
}

110
cortex/routers/auth.py Normal file
View File

@@ -0,0 +1,110 @@
"""
CLI auth status for both Claude and Gemini backends.
GET /auth/status — returns per-backend auth info and warning flags
Claude: warns when OAuth token is < WARN_HOURS from expiry (requires
user to re-run `claude` to refresh via browser flow).
Gemini: warns only when oauth_creds.json is missing or has no
refresh_token (access token rotates automatically every ~1h).
"""
import json
import logging
from datetime import datetime, timezone
from pathlib import Path
from fastapi import APIRouter
from config import settings
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/auth")
CLAUDE_CREDS = Path.home() / ".claude" / ".credentials.json"
GEMINI_CREDS = Path.home() / ".gemini" / "oauth_creds.json"
GEMINI_ACCTS = Path.home() / ".gemini" / "google_accounts.json"
WARN_HOURS = 24 # no refresh token — warn a day ahead
WARN_HOURS_REFRESH = 1 # refresh token present — only warn if CLI hasn't rotated in time
def _claude_status() -> dict:
try:
data = json.loads(CLAUDE_CREDS.read_text())
oauth = data["claudeAiOauth"]
has_refresh = bool(oauth.get("refreshToken"))
expires_dt = datetime.fromtimestamp(oauth["expiresAt"] / 1000, tz=timezone.utc)
now = datetime.now(tz=timezone.utc)
hours_remaining = (expires_dt - now).total_seconds() / 3600
# When a refresh token is present the CLI *should* auto-rotate the access
# token, but sometimes it doesn't. Use a tight 1-hour window so a fresh
# 8-hour token doesn't immediately trigger a warning, but a stale token
# that the CLI missed will still surface before it expires.
expired = hours_remaining <= 0
threshold = WARN_HOURS_REFRESH if has_refresh else WARN_HOURS
warning = expired or hours_remaining < threshold
return {
"ok": True,
"has_refresh_token": has_refresh,
"access_token_expires_at": expires_dt.isoformat(),
"access_token_hours_remaining": round(hours_remaining, 1),
"warning": warning,
"expired": expired,
}
except Exception as e:
logger.warning("claude auth check failed: %s", e)
return {"ok": False, "error": str(e), "warning": True, "expired": False}
def _gemini_status() -> dict:
try:
creds = json.loads(GEMINI_CREDS.read_text())
if not creds.get("refresh_token"):
return {"ok": True, "authenticated": False, "warning": True, "account": None}
account = None
try:
accts = json.loads(GEMINI_ACCTS.read_text())
account = accts.get("active")
except Exception:
pass
return {"ok": True, "authenticated": True, "warning": False, "account": account}
except FileNotFoundError:
return {"ok": True, "authenticated": False, "warning": True, "account": None}
except Exception as e:
logger.warning("gemini auth check failed: %s", e)
return {"ok": False, "error": str(e), "warning": True, "authenticated": False}
async def _local_status(username: str = "scott") -> dict:
"""Check reachability of the user's configured local model host."""
import model_registry
cfg = model_registry.get_best_local_model(username)
if not cfg:
return {"configured": False}
api_url = cfg.get("api_url", "")
if not api_url:
return {"configured": False}
try:
import httpx
url = api_url.rstrip("/") + "/api/models"
headers = {}
api_key = cfg.get("api_key", "")
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
async with httpx.AsyncClient(timeout=5) as client:
resp = await client.get(url, headers=headers)
reachable = resp.status_code < 400
return {
"configured": True,
"reachable": reachable,
"model": cfg.get("model_name", ""),
"label": cfg.get("label", ""),
}
except Exception as e:
return {"configured": True, "reachable": False, "error": str(e), "model": cfg.get("model_name", "")}
@router.get("/status")
async def auth_status() -> dict:
return {
"claude": _claude_status(),
"gemini": _gemini_status(),
"local": await _local_status(),
}

View File

@@ -0,0 +1,205 @@
"""
Google OAuth 2.0 sign-in.
Flow:
1. GET /auth/google → redirect to Google's consent page
2. GET /auth/google/callback → exchange code, look up user, set JWT cookie
Users must be pre-registered by Scott before they can sign in:
cd cortex && .venv/bin/python manage_passwords.py google-add <username> <email>
Routes are public (added to _PUBLIC_PREFIXES in auth_middleware.py).
"""
import json
import logging
import secrets
import urllib.parse
import urllib.request
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, RedirectResponse, Response
from auth_utils import COOKIE_NAME, create_token, find_user_by_google, link_google
from config import settings
from persona import list_user_personas
logger = logging.getLogger(__name__)
router = APIRouter()
_GOOGLE_AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth"
_GOOGLE_TOKEN_URL = "https://oauth2.googleapis.com/token"
_GOOGLE_USERINFO = "https://openidconnect.googleapis.com/v1/userinfo"
_STATE_COOKIE = "oauth_state"
_STATE_MAX_AGE = 600 # 10 minutes — plenty of time to complete the flow
@router.get("/auth/google", include_in_schema=False)
async def google_login():
if not settings.google_client_id:
return HTMLResponse("Google sign-in is not configured on this server.", status_code=503)
state = secrets.token_urlsafe(16)
params = urllib.parse.urlencode({
"client_id": settings.google_client_id,
"redirect_uri": f"{settings.cortex_base_url}/auth/google/callback",
"response_type": "code",
"scope": "openid email profile",
"state": state,
"access_type": "online",
"prompt": "select_account",
})
resp = RedirectResponse(f"{_GOOGLE_AUTH_URL}?{params}", status_code=302)
resp.set_cookie(_STATE_COOKIE, state, max_age=_STATE_MAX_AGE, httponly=True, samesite="lax")
return resp
@router.get("/auth/google/callback", include_in_schema=False)
async def google_callback(
request: Request,
code: str = "",
state: str = "",
error: str = "",
):
if error:
return _error_page(f"Google sign-in was cancelled or denied: {error}")
if not code:
return _error_page("No authorisation code returned by Google.")
# CSRF check — state must match what we stored in the cookie
stored_state = request.cookies.get(_STATE_COOKIE)
if not stored_state or stored_state != state:
return _error_page("State mismatch — please try signing in again.")
# Exchange authorisation code for tokens
try:
token_data = _exchange_code(code)
except Exception as e:
logger.error("Google token exchange failed: %s", e)
return _error_page("Could not complete sign-in with Google. Please try again.")
access_token = token_data.get("access_token")
if not access_token:
return _error_page("No access token returned by Google.")
# Fetch the user's profile
try:
userinfo = _get_userinfo(access_token)
except Exception as e:
logger.error("Google userinfo fetch failed: %s", e)
return _error_page("Could not retrieve your Google profile. Please try again.")
google_sub = userinfo.get("sub", "")
google_email = userinfo.get("email", "")
if not google_sub or not google_email:
return _error_page("Your Google account didn't return a usable email address.")
# Match to a Cortex user
username = find_user_by_google(google_sub, google_email)
if not username:
logger.warning("Google sign-in rejected: no account for %s (%s)", google_sub, google_email)
return _error_page(
f"Your Google account (<strong>{google_email}</strong>) isn't registered with Cortex.<br><br>"
"Contact Scott to get access."
)
# Persist the stable sub so future lookups use it (not just email)
link_google(username, google_sub, google_email)
personas = list_user_personas(username)
if not personas:
return _error_page("No personas are configured for your account yet. Contact Scott.")
logger.info("Google sign-in: %s (%s)", username, google_email)
resp = RedirectResponse(f"/{username}/{personas[0]}", status_code=302)
_set_session_cookie(resp, username)
resp.delete_cookie(_STATE_COOKIE)
return resp
# ---------------------------------------------------------------------------
# Private helpers
# ---------------------------------------------------------------------------
def _exchange_code(code: str) -> dict:
body = urllib.parse.urlencode({
"code": code,
"client_id": settings.google_client_id,
"client_secret": settings.google_client_secret,
"redirect_uri": f"{settings.cortex_base_url}/auth/google/callback",
"grant_type": "authorization_code",
}).encode()
req = urllib.request.Request(
_GOOGLE_TOKEN_URL,
data=body,
headers={"Content-Type": "application/x-www-form-urlencoded"},
method="POST",
)
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read())
def _get_userinfo(access_token: str) -> dict:
req = urllib.request.Request(
_GOOGLE_USERINFO,
headers={"Authorization": f"Bearer {access_token}"},
)
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read())
def _set_session_cookie(response: Response, username: str) -> None:
token = create_token(username)
response.set_cookie(
COOKIE_NAME,
token,
max_age=settings.jwt_expire_days * 86400,
httponly=True,
samesite="lax",
secure=False, # set True if terminating TLS at the app layer (not behind a proxy)
)
def _error_page(message: str) -> HTMLResponse:
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Sign In Failed</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after {{ box-sizing: border-box; margin: 0; padding: 0; }}
body {{
min-height: 100vh; display: flex; align-items: center; justify-content: center;
background: #0f1117; font-family: 'Inter', system-ui; font-weight: 450;
-webkit-font-smoothing: antialiased; color: #e2e8f0;
}}
.card {{
background: #1a1d27; border: 1px solid #2d3148; border-radius: 12px;
padding: 2.5rem 2rem; width: 100%; max-width: 420px; text-align: center;
}}
h1 {{ font-size: 1.25rem; font-weight: 700; color: #f87171; margin-bottom: 1rem; }}
p {{ font-size: 0.9rem; color: #94a3b8; margin-bottom: 1.75rem; line-height: 1.65; }}
a {{
display: inline-block; padding: 0.6rem 1.5rem;
background: #7c3aed; border-radius: 6px; color: #fff;
text-decoration: none; font-size: 0.9rem; font-weight: 600;
transition: background 0.15s;
}}
a:hover {{ background: #6d28d9; }}
</style>
</head>
<body>
<div class="card">
<h1>Sign In Failed</h1>
<p>{message}</p>
<a href="/login">← Back to Sign In</a>
</div>
</body>
</html>"""
return HTMLResponse(html, status_code=403)

View File

@@ -1,27 +1,71 @@
import asyncio import asyncio
import json import json
from fastapi import APIRouter, HTTPException import platform
import jwt
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import StreamingResponse from fastapi.responses import StreamingResponse
from pydantic import BaseModel from pydantic import BaseModel
from context_loader import load_context from context_loader import load_context
from llm_client import complete from llm_client import complete
from session_logger import log_turn from session_logger import log_turn
from session_store import load as load_session, save as save_session, list_all, generate_session_id from session_store import load as load_session, save as save_session, list_all, generate_session_id, delete as delete_session, rename as rename_session, get_name as get_session_name
from config import settings from config import settings
from persona import set_context, validate as validate_persona
from auth_utils import COOKIE_NAME, decode_token
import model_registry
import event_bus
from model_registry import get_role_config
router = APIRouter() router = APIRouter()
def _backend_label(backend: str, username: str, role: str = "chat") -> str:
"""Human-readable label for the model that handled a request (legacy path)."""
if backend == "claude":
return "Claude"
if backend == "gemini":
return "Gemini"
if backend == "local":
cfg = model_registry.get_best_local_model(username, role)
if cfg:
return cfg.get("label") or cfg.get("model_name") or "Local"
return "Local"
return backend.title()
def _role_model_label(username: str, role: str, actual_backend: str) -> str:
"""Return the model label for a role, falling back to the generic backend label."""
cfg = model_registry.get_model_for_role(username, role)
if cfg:
return cfg.get("label") or cfg.get("model_name") or _backend_label(actual_backend, username, role)
return _backend_label(actual_backend, username, role)
class Attachment(BaseModel):
filename: str
mime_type: str
data: str # base64 data URL for images (e.g. "data:image/png;base64,...")
class ChatRequest(BaseModel): class ChatRequest(BaseModel):
message: str message: str
session_id: str | None = None session_id: str | None = None
tier: int | None = None tier: int | None = None
model: str | None = None # "claude" or "gemini" to override; None = use primary_backend model: str | None = None # legacy backend override ("claude"|"gemini"|"local")
slot: str | None = None # Phase 3: explicit slot ("primary"|"backup_1"|"backup_2")
chat_role: str = "chat" # active role: "chat"|"coder"|"research"|"distill" etc.
include_long: bool = True
include_mid: bool = True
include_short: bool = True
off_record: bool = False # skip session log (in-memory context preserved)
user: str = "scott"
persona: str = "inara"
attachment: Attachment | None = None # image attachment (text files injected client-side)
class BackendRequest(BaseModel): class BackendRequest(BaseModel):
primary: str # "claude" or "gemini" primary: str # "claude", "gemini", or "local"
class NoteRequest(BaseModel): class NoteRequest(BaseModel):
@@ -45,17 +89,49 @@ async def _stream_chat(req: ChatRequest):
"backend": "...", "fallback_used": bool} "backend": "...", "fallback_used": bool}
data: {"type": "error", "message": "..."} data: {"type": "error", "message": "..."}
""" """
try:
user, persona = validate_persona(req.user, req.persona)
set_context(user, persona)
except ValueError as e:
yield f"data: {json.dumps({'type': 'error', 'message': str(e)})}\n\n"
return
session_id = req.session_id or generate_session_id() session_id = req.session_id or generate_session_id()
tier = req.tier or settings.default_tier tier = req.tier or settings.default_tier
system_prompt = load_context(tier) role_cfg = get_role_config(user, req.chat_role)
system_prompt = load_context(
tier,
include_long=req.include_long,
include_mid=req.include_mid,
include_short=req.include_short,
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
mode="otr" if req.off_record else "chat",
)
history = load_session(session_id) history = load_session(session_id)
history.append({"role": "user", "content": req.message})
# req.message already contains the full user text:
# - text files: client embedded content as a fenced code block
# - images: client added "📎 filename.png" note; image data is in req.attachment
# History always stores text only — base64 image data is never written to disk.
llm_attachment: dict | None = None
if req.attachment and req.attachment.mime_type.startswith("image/"):
llm_attachment = {
"filename": req.attachment.filename,
"mime_type": req.attachment.mime_type,
"data": req.attachment.data,
}
history.append({"role": "user", "content": req.message, "off_record": req.off_record})
task = asyncio.create_task(complete( task = asyncio.create_task(complete(
system_prompt=system_prompt, system_prompt=system_prompt,
messages=history, messages=history,
model=req.model, model=req.model,
role=req.chat_role,
slot=req.slot,
attachment=llm_attachment,
)) ))
try: try:
@@ -71,17 +147,35 @@ async def _stream_chat(req: ChatRequest):
try: try:
response_text, actual_backend = task.result() response_text, actual_backend = task.result()
history.append({"role": "assistant", "content": response_text}) if req.slot:
slot_cfg = model_registry.get_model_for_slot(user, req.chat_role, req.slot)
backend_label = (slot_cfg or {}).get("label") or _role_model_label(user, req.chat_role, actual_backend)
else:
backend_label = _role_model_label(user, req.chat_role, actual_backend)
host = platform.node()
history.append({
"role": "assistant",
"content": response_text,
"backend": actual_backend,
"backend_label": backend_label,
"host": host,
"off_record": req.off_record,
})
save_session(session_id, history) save_session(session_id, history)
log_turn(session_id, req.message, response_text) if not req.off_record:
log_turn(session_id, req.message, response_text, backend_label, host)
requested = req.model or settings.primary_backend # fallback_used only makes sense for explicit backend selections.
# In auto mode (req.model is None), just report what responded.
fallback_used = bool(req.model and actual_backend != req.model)
payload = { payload = {
"type": "response", "type": "response",
"response": response_text, "response": response_text,
"session_id": session_id, "session_id": session_id,
"backend": actual_backend, "backend": actual_backend,
"fallback_used": actual_backend != requested, "backend_label": backend_label,
"host": host,
"fallback_used": fallback_used,
} }
yield f"data: {json.dumps(payload)}\n\n" yield f"data: {json.dumps(payload)}\n\n"
@@ -109,41 +203,265 @@ async def chat(req: ChatRequest) -> StreamingResponse:
) )
_BACKEND_CYCLE = ("claude", "gemini", "local")
_BACKEND_FALLBACK = {"claude": "gemini", "gemini": "claude", "local": "claude"}
def _request_user(request: Request) -> str | None:
"""Extract username from JWT cookie, or None."""
try:
token = request.cookies.get(COOKIE_NAME)
return decode_token(token) if token else None
except (jwt.InvalidTokenError, Exception):
return None
def _local_model_info(request: Request) -> dict | None:
"""Return the best local model {label, model_name} for the session user, or None."""
username = _request_user(request)
if not username:
return None
try:
cfg = model_registry.get_best_local_model(username, "chat")
if cfg:
return {"label": cfg.get("label", ""), "model_name": cfg.get("model_name", "")}
except Exception:
pass
return None
def _chat_slot_models(username: str) -> list[dict]:
"""Return [{slot, label, type}] for each configured slot in the chat role, primary first."""
registry = model_registry.get_registry(username)
role_slots = registry.get("roles", {}).get("chat", {})
result = []
for slot_key in model_registry.PRIORITY_KEYS:
model_id = role_slots.get(slot_key)
if not model_id:
continue
resolved = model_registry._resolve_model(registry, model_id)
if resolved:
result.append({
"slot": slot_key,
"label": resolved.get("label") or resolved.get("model_name") or "",
"type": resolved.get("type", ""),
})
return result
def _available_roles_for_toggle(username: str) -> list[dict]:
"""Return roles with a primary model assigned (excluding orchestrator) for the UI toggle.
Returns [{role, label, model_label, type}] ordered by settings.defined_roles.
"""
registry = model_registry.get_registry(username)
roles_cfg = registry.get("roles", {})
result = []
for role_name in settings.get_defined_roles():
if role_name == "orchestrator":
continue
primary_id = roles_cfg.get(role_name, {}).get("primary")
if not primary_id:
continue
resolved = model_registry._resolve_model(registry, primary_id)
if resolved:
result.append({
"role": role_name,
"label": role_name.title(),
"model_label": resolved.get("label") or resolved.get("model_name") or "",
"type": resolved.get("type", ""),
})
return result
@router.get("/backend") @router.get("/backend")
async def get_backend() -> dict: async def get_backend(request: Request) -> dict:
other = "gemini" if settings.primary_backend == "claude" else "claude" username = _request_user(request)
return {"primary": settings.primary_backend, "fallback": other} chat_models = _chat_slot_models(username) if username else []
available_roles = _available_roles_for_toggle(username) if username else []
p = settings.primary_backend
orch_label = None
if username:
orch_cfg = model_registry.get_model_for_role(username, "orchestrator")
if orch_cfg:
orch_label = orch_cfg.get("label") or orch_cfg.get("model_name") or None
return {
"chat_models": chat_models, # Phase 3: [{slot, label, type}] for chat-role slots
"available_roles": available_roles, # kept for banner + backward compat
"orchestrator_model": orch_label,
# Legacy fields kept for backward compat
"primary": p,
"fallback": _BACKEND_FALLBACK.get(p, "claude"),
"local_model": _local_model_info(request),
}
@router.post("/backend") @router.post("/backend")
async def set_backend(req: BackendRequest) -> dict: async def set_backend(req: BackendRequest, request: Request) -> dict:
if req.primary not in ("claude", "gemini"): if req.primary not in _BACKEND_CYCLE:
raise HTTPException(status_code=400, detail="primary must be 'claude' or 'gemini'") raise HTTPException(status_code=400, detail="primary must be 'claude', 'gemini', or 'local'")
settings.primary_backend = req.primary settings.primary_backend = req.primary
other = "gemini" if req.primary == "claude" else "claude" return {
return {"primary": settings.primary_backend, "fallback": other} "primary": req.primary,
"fallback": _BACKEND_FALLBACK[req.primary],
"local_model": _local_model_info(request),
}
def _set_ctx(user: str, persona: str) -> None:
"""Validate and set persona context from query params. Raises HTTPException on bad input."""
try:
u, p = validate_persona(user, persona)
set_context(u, p)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
@router.get("/history/{session_id}") @router.get("/history/{session_id}")
async def get_history(session_id: str) -> dict: async def get_history(
return {"session_id": session_id, "messages": load_session(session_id)} session_id: str,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
_set_ctx(user, persona)
name = get_session_name(session_id)
return {"session_id": session_id, "name": name, "messages": load_session(session_id)}
@router.get("/sessions") @router.get("/sessions")
async def list_sessions() -> dict: async def list_sessions(
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
_set_ctx(user, persona)
return {"sessions": list_all()} return {"sessions": list_all()}
class SessionRename(BaseModel):
name: str
@router.patch("/sessions/{session_id}")
async def rename_session_endpoint(
session_id: str,
req: SessionRename,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
_set_ctx(user, persona)
found = rename_session(session_id, req.name.strip())
if not found:
raise HTTPException(status_code=404, detail=f"Session {session_id} not found")
return {"ok": True, "session_id": session_id, "name": req.name.strip()}
@router.post("/api/sessions/backfill-names")
async def backfill_session_names(
request: Request,
user: str = Query(""),
persona: str = Query(""),
) -> dict:
"""Name every unnamed session using its first user message (truncated to 60 chars).
Idempotent — only touches sessions that have no name set.
user/persona default to the JWT session user + last-used persona cookie."""
# Resolve user from JWT if not provided
if not user:
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
user = decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
# Resolve persona from cookie if not provided
if not persona:
from persona import list_user_personas
persona_cookie = request.cookies.get("cx_last_persona", "")
available = list_user_personas(user)
persona = persona_cookie if persona_cookie in available else (available[0] if available else "")
if not persona:
raise HTTPException(status_code=400, detail="No persona found for user")
_set_ctx(user, persona)
sessions = list_all()
named = 0
for s in sessions:
if s.get("name"):
continue
messages = load_session(s["session_id"])
first_user = next((m for m in messages if m.get("role") == "user"), None)
if not first_user:
continue
text = (first_user.get("content") or "").strip()
if not text:
continue
auto_name = text[:60].rstrip() + ("" if len(text) > 60 else "")
rename_session(s["session_id"], auto_name)
named += 1
return {"ok": True, "named": named, "total": len(sessions)}
@router.delete("/sessions/{session_id}")
async def delete_session_endpoint(
session_id: str,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
_set_ctx(user, persona)
found = delete_session(session_id)
if not found:
raise HTTPException(status_code=404, detail=f"Session {session_id} not found")
return {"ok": True, "session_id": session_id}
@router.put("/history/{session_id}") @router.put("/history/{session_id}")
async def replace_history(session_id: str, req: HistoryUpdate) -> dict: async def replace_history(
session_id: str,
req: HistoryUpdate,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
"""Replace the full message list for a session (used by edit/delete UI).""" """Replace the full message list for a session (used by edit/delete UI)."""
_set_ctx(user, persona)
save_session(session_id, req.messages) save_session(session_id, req.messages)
return {"ok": True, "session_id": session_id} return {"ok": True, "session_id": session_id}
@router.get("/events")
async def sse_events() -> StreamingResponse:
"""Server-sent events stream — pushes real-time Talk activity to the browser."""
async def stream():
q = event_bus.subscribe()
try:
while True:
try:
event = await asyncio.wait_for(q.get(), timeout=20)
yield f"data: {json.dumps(event)}\n\n"
except asyncio.TimeoutError:
yield 'data: {"type":"keepalive"}\n\n'
except (GeneratorExit, asyncio.CancelledError):
pass
finally:
event_bus.unsubscribe(q)
return StreamingResponse(
stream(),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
@router.post("/note") @router.post("/note")
async def add_note(req: NoteRequest) -> dict: async def add_note(
req: NoteRequest,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
"""Inject a public note into session history so the LLM sees it next turn.""" """Inject a public note into session history so the LLM sees it next turn."""
_set_ctx(user, persona)
history = load_session(req.session_id) history = load_session(req.session_id)
history.append({"role": "user", "content": f"[NOTE] {req.note}"}) history.append({"role": "user", "content": f"[NOTE] {req.note}"})
save_session(req.session_id, history) save_session(req.session_id, history)

479
cortex/routers/crons.py Normal file
View File

@@ -0,0 +1,479 @@
"""
Schedules web UI — GET/POST /settings/crons/*
Lets users view, add, toggle, and remove cron jobs without going through the AI.
Cron data lives in home/{user}/persona/{persona}/CRONS.json.
Scheduler registration mirrors what tools/cron.py does so changes take effect immediately.
"""
import html as _html
import logging
import secrets
from datetime import datetime, timezone
from pathlib import Path
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token
from cron_runner import load_crons, save_crons, parse_schedule
from persona import list_user_personas
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
def _integrations_nav(username: str) -> str:
from auth_utils import _read_auth
role = _read_auth(username).get("role", "user")
if role == "admin":
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
return ""
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _short_id() -> str:
return "c_" + secrets.token_urlsafe(6)
def _scheduler_add(job: dict, sched_kwargs: dict) -> None:
import asyncio
try:
import scheduler as sched_module
from cron_runner import run_job
s = sched_module.get_scheduler()
if s and s.running:
sched_id = f"{job['user']}:{job['persona']}:{job['id']}"
s.add_job(
lambda j=job: asyncio.ensure_future(run_job(j)),
"cron",
id=sched_id,
replace_existing=True,
**sched_kwargs,
)
except Exception as e:
logger.warning("scheduler_add failed: %s", e)
def _scheduler_remove(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.remove_job(job_id)
except Exception:
pass
def _scheduler_pause(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.pause_job(job_id)
except Exception:
pass
def _scheduler_resume(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.resume_job(job_id)
except Exception:
pass
_TYPE_CLASS = {
"remind": "badge-remind", "note": "badge-note", "message": "badge-message",
"brief": "badge-brief", "task": "badge-task",
}
def _render_cron_list(username: str) -> str:
personas = list_user_personas(username)
if not personas:
return '<div class="empty-state">No personas found.</div>'
all_empty = True
html_parts: list[str] = []
for persona in personas:
crons = load_crons(username, persona)
if not crons:
continue
all_empty = False
rows = []
for c in crons:
cid = _html.escape(c["id"])
label = _html.escape(c.get("label", ""))
schedule = _html.escape(c.get("schedule", ""))
job_type = _html.escape(c.get("type", ""))
payload = _html.escape(c.get("payload", ""))
enabled = c.get("enabled", True)
last_run = (c.get("last_run") or "")[:10] or "never"
pers_esc = _html.escape(persona)
type_class = _TYPE_CLASS.get(c.get("type", ""), "badge-note")
status_cls = "badge-enabled" if enabled else "badge-paused"
status_txt = "enabled" if enabled else "paused"
toggle_txt = "Pause" if enabled else "Resume"
rows.append(f"""
<tr>
<td>{label}</td>
<td><code>{schedule}</code></td>
<td><span class="badge {type_class}">{job_type}</span></td>
<td class="payload-cell" title="{payload}">{payload}</td>
<td>{last_run}</td>
<td><span class="badge {status_cls}">{status_txt}</span></td>
<td>
<div class="cron-actions">
<a href="/settings/crons/edit?cron_id={cid}&persona={pers_esc}"
class="btn-cron">Edit</a>
<form method="POST" action="/settings/crons/toggle" style="display:inline">
<input type="hidden" name="cron_id" value="{cid}">
<input type="hidden" name="persona" value="{pers_esc}">
<button type="submit" class="btn-cron">{toggle_txt}</button>
</form>
<form method="POST" action="/settings/crons/remove" style="display:inline"
onsubmit="return confirm('Delete this schedule?')">
<input type="hidden" name="cron_id" value="{cid}">
<input type="hidden" name="persona" value="{pers_esc}">
<button type="submit" class="btn-cron btn-cron-del">Delete</button>
</form>
</div>
</td>
</tr>""")
html_parts.append(f"""
<div class="persona-group">
<p class="persona-group-label">{_html.escape(persona)}</p>
<table class="cron-table">
<thead>
<tr>
<th>Label</th><th>Schedule</th><th>Type</th>
<th>Payload</th><th>Last run</th><th>Status</th><th></th>
</tr>
</thead>
<tbody>{"".join(rows)}
</tbody>
</table>
</div>""")
if all_empty:
return '<div class="empty-state">No schedules yet. Add one below.</div>'
return "\n".join(html_parts)
def _persona_options(username: str, selected: str = "") -> str:
personas = list_user_personas(username)
return "\n".join(
f'<option value="{_html.escape(p)}"{"selected" if p == selected else ""}>{_html.escape(p)}</option>'
for p in personas
)
_TYPE_OPTIONS = ("remind", "note", "message", "brief", "task")
_TYPE_LABELS = {
"remind": "remind — append to REMINDERS.md",
"note": "note — append to SCRATCH.md",
"message": "message — send payload as-is",
"brief": "brief — LLM response, no tools",
"task": "task — full orchestrator tool loop",
}
def _render_edit_form(job: dict, persona: str) -> str:
cid = _html.escape(job["id"])
pers_esc = _html.escape(persona)
label = _html.escape(job.get("label", ""))
schedule = _html.escape(job.get("schedule", ""))
payload = _html.escape(job.get("payload", ""))
cur_type = job.get("type", "remind")
type_opts = "\n".join(
f'<option value="{t}" {"selected" if t == cur_type else ""}>{_html.escape(_TYPE_LABELS.get(t, t))}</option>'
for t in _TYPE_OPTIONS
)
return f"""
<div class="section" style="border: 2px solid var(--pg-accent); border-radius: 8px; padding: 1rem;">
<h2 style="margin-top:0">Edit schedule</h2>
<form method="POST" action="/settings/crons/save">
<input type="hidden" name="cron_id" value="{cid}">
<input type="hidden" name="persona" value="{pers_esc}">
<div class="add-form-grid">
<div class="field">
<label>Persona</label>
<input type="text" value="{pers_esc}" disabled style="opacity:0.5">
</div>
<div class="field">
<label for="edit_job_type">Type</label>
<select id="edit_job_type" name="job_type">{type_opts}</select>
</div>
<div class="field">
<label for="edit_label">Label</label>
<input type="text" id="edit_label" name="label" value="{label}" required autocomplete="off">
</div>
<div class="field">
<label for="edit_schedule">Schedule</label>
<input type="text" id="edit_schedule" name="schedule" value="{schedule}"
required autocomplete="off" spellcheck="false">
<p class="hint">
hourly · daily · daily:HH:MM · weekly:DOW · weekly:DOW:HH:MM ·
monthly · monthly:DD · monthly:DD:HH:MM · yearly:MM:DD · yearly:MM:DD:HH:MM
</p>
</div>
<div class="field field-full">
<label for="edit_payload">Payload / prompt</label>
<textarea id="edit_payload" name="payload" rows="3" required>{payload}</textarea>
</div>
</div>
<div style="display:flex; gap:0.5rem; align-items:center; margin-top:0.5rem">
<button type="submit" class="btn-submit" style="margin-top:0">Save changes</button>
<a href="/settings/crons" style="font-size:0.85rem; color:var(--pg-muted)">Cancel</a>
</div>
</form>
</div>"""
def _render_page(username: str, back_persona: str = "", success: str = "", error: str = "",
edit_html: str = "") -> str:
html = (_STATIC / "crons.html").read_text()
html = html.replace("{{ edit_html }}", edit_html)
html = html.replace("{{ cron_list_html }}", _render_cron_list(username))
html = html.replace("{{ persona_options }}", _persona_options(username, back_persona))
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{_html.escape(success)}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{_html.escape(error)}</p>')
return html
# ---------------------------------------------------------------------------
# Routes
# ---------------------------------------------------------------------------
@router.get("/settings/crons", include_in_schema=False)
async def crons_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_render_page(username, back_persona))
@router.post("/settings/crons/add", include_in_schema=False)
async def cron_add(
request: Request,
persona: str = Form(""),
label: str = Form(""),
schedule: str = Form(""),
job_type: str = Form(""),
payload: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
label = label.strip()
schedule = schedule.strip()
payload = payload.strip()
persona = persona.strip()
_VALID_TYPES = ("remind", "note", "message", "brief", "task")
if job_type not in _VALID_TYPES:
return HTMLResponse(_render_page(username, back_persona, error=f"Invalid type: {job_type}"))
try:
sched_kwargs = parse_schedule(schedule)
except ValueError as e:
return HTMLResponse(_render_page(username, back_persona, error=f"Bad schedule: {e}"))
if not label:
return HTMLResponse(_render_page(username, back_persona, error="Label is required."))
if not payload:
return HTMLResponse(_render_page(username, back_persona, error="Payload is required."))
crons = load_crons(username, persona)
job = {
"id": _short_id(),
"user": username,
"persona": persona,
"label": label,
"schedule": schedule,
"type": job_type,
"payload": payload,
"enabled": True,
"created_at": _now(),
"last_run": None,
}
crons.append(job)
save_crons(crons, username, persona)
_scheduler_add(job, sched_kwargs)
logger.info("cron added via UI: %s %s [%s]", job["id"], schedule, job_type)
return HTMLResponse(_render_page(username, back_persona, success=f"Schedule '{label}' added."))
@router.post("/settings/crons/toggle", include_in_schema=False)
async def cron_toggle(
request: Request,
cron_id: str = Form(""),
persona: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
crons = load_crons(username, persona)
for c in crons:
if c["id"] == cron_id:
c["enabled"] = not c.get("enabled", True)
save_crons(crons, username, persona)
sched_id = f"{username}:{persona}:{cron_id}"
if c["enabled"]:
_scheduler_resume(sched_id)
action = "resumed"
else:
_scheduler_pause(sched_id)
action = "paused"
logger.info("cron %s %s via UI", cron_id, action)
return HTMLResponse(_render_page(username, back_persona, success=f"Schedule {action}."))
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
@router.post("/settings/crons/remove", include_in_schema=False)
async def cron_remove(
request: Request,
cron_id: str = Form(""),
persona: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
crons = load_crons(username, persona)
before = len(crons)
crons = [c for c in crons if c["id"] != cron_id]
if len(crons) == before:
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
save_crons(crons, username, persona)
_scheduler_remove(f"{username}:{persona}:{cron_id}")
logger.info("cron %s removed via UI", cron_id)
return HTMLResponse(_render_page(username, back_persona, success="Schedule deleted."))
@router.get("/settings/crons/edit", include_in_schema=False)
async def cron_edit_page(request: Request, cron_id: str = "", persona: str = ""):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
crons = load_crons(username, persona)
job = next((c for c in crons if c["id"] == cron_id), None)
if not job:
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
edit_html = _render_edit_form(job, persona)
return HTMLResponse(_render_page(username, back_persona, edit_html=edit_html))
@router.post("/settings/crons/save", include_in_schema=False)
async def cron_save(
request: Request,
cron_id: str = Form(""),
persona: str = Form(""),
label: str = Form(""),
schedule: str = Form(""),
job_type: str = Form(""),
payload: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
label = label.strip()
schedule = schedule.strip()
payload = payload.strip()
if job_type not in _TYPE_OPTIONS:
return HTMLResponse(_render_page(username, back_persona, error=f"Invalid type: {job_type}"))
if not label:
return HTMLResponse(_render_page(username, back_persona, error="Label is required."))
if not payload:
return HTMLResponse(_render_page(username, back_persona, error="Payload is required."))
try:
sched_kwargs = parse_schedule(schedule)
except ValueError as e:
# Re-render with the edit form still open so the user can fix the schedule
crons = load_crons(username, persona)
job = next((c for c in crons if c["id"] == cron_id), None)
edit_html = _render_edit_form(job, persona) if job else ""
return HTMLResponse(_render_page(username, back_persona, error=f"Bad schedule: {e}",
edit_html=edit_html))
crons = load_crons(username, persona)
for c in crons:
if c["id"] == cron_id:
c["label"] = label
c["schedule"] = schedule
c["type"] = job_type
c["payload"] = payload
save_crons(crons, username, persona)
# Replace the live scheduler job with the updated schedule
sched_id = f"{username}:{persona}:{cron_id}"
_scheduler_remove(sched_id)
if c.get("enabled", True):
_scheduler_add(c, sched_kwargs)
logger.info("cron %s updated via UI [%s]", cron_id, schedule)
return HTMLResponse(_render_page(username, back_persona,
success=f"Schedule '{label}' updated."))
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))

238
cortex/routers/distill.py Normal file
View File

@@ -0,0 +1,238 @@
"""
Manual memory distillation endpoints.
POST /distill/short — roll session logs → MEMORY_SHORT.md (no LLM)
POST /distill/mid — summarize short → MEMORY_MID.md (LLM)
POST /distill/long — integrate mid → MEMORY_LONG.md (LLM)
POST /distill/all — run all three in sequence
POST /distill/rebuild — wipe mid + long, then run all three from scratch
All endpoints require ?user=<username>&persona=<name> query params.
Concurrency: one distillation at a time per persona. A second request while one
is running returns 409 immediately — no silent queuing.
"""
import asyncio
from datetime import datetime, timedelta
from fastapi import APIRouter, HTTPException, Query
from memory_distiller import distill_short, distill_mid, distill_long
from persona import validate as validate_persona, set_context, persona_path as _persona_path
import scheduler
router = APIRouter(prefix="/distill")
# Per-persona asyncio lock. Key: (user, persona)
_LOCKS: dict[tuple, asyncio.Lock] = {}
_LOCKS_META: dict[tuple, str] = {} # key → which step is currently running
# Minimum time between successive runs of each endpoint, per persona.
# Prevents accidental rapid-fire runs and token waste.
_COOLDOWNS: dict[tuple, timedelta] = {
"short": timedelta(minutes=1),
"mid": timedelta(minutes=30),
"long": timedelta(hours=6),
"all": timedelta(hours=1),
"rebuild": timedelta(hours=6),
}
_LAST_RUN: dict[tuple, datetime] = {} # key: (user, persona, endpoint)
def _get_lock(user: str, persona: str) -> asyncio.Lock:
key = (user, persona)
if key not in _LOCKS:
_LOCKS[key] = asyncio.Lock()
return _LOCKS[key]
def _resolve(user: str, persona: str) -> tuple[str, str]:
try:
u, p = validate_persona(user, persona)
except Exception:
raise HTTPException(status_code=404, detail=f"Persona not found: {user}/{persona}")
set_context(u, p)
return u, p
def _check_lock(user: str, persona: str) -> asyncio.Lock:
"""Return the lock if free, raise 409 if already held."""
lock = _get_lock(user, persona)
if lock.locked():
step = _LOCKS_META.get((user, persona), "distillation")
raise HTTPException(
status_code=409,
detail=f"A {step} is already running for {persona} — please wait for it to finish.",
)
return lock
def _check_cooldown(user: str, persona: str, endpoint: str) -> None:
"""Raise 429 if the endpoint was run too recently for this persona."""
cooldown = _COOLDOWNS.get(endpoint)
if not cooldown:
return
key = (user, persona, endpoint)
last = _LAST_RUN.get(key)
if last:
elapsed = datetime.now() - last
if elapsed < cooldown:
remaining = cooldown - elapsed
mins = int(remaining.total_seconds() // 60)
secs = int(remaining.total_seconds() % 60)
wait = f"{mins}m {secs}s" if mins else f"{secs}s"
raise HTTPException(
status_code=429,
detail=f"{endpoint} was just run — please wait {wait} before running again.",
)
def _record_run(user: str, persona: str, endpoint: str) -> None:
_LAST_RUN[(user, persona, endpoint)] = datetime.now()
@router.get("/status")
async def distill_status() -> dict:
from config import settings
# Include which personas are currently distilling
active = [f"{u}/{p}" for (u, p), lock in _LOCKS.items() if lock.locked()]
return {
"enabled": settings.auto_distill,
"jobs": scheduler.status(),
"active": active,
"config": {
"short": settings.auto_distill_short,
"mid": settings.auto_distill_mid,
"long": settings.auto_distill_long,
},
}
@router.post("/short")
async def do_distill_short(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "short")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "short distill"
try:
result = distill_short(u, p)
_record_run(u, p, "short")
return {"ok": True, **result}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/mid")
async def do_distill_mid(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "mid")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "mid distill"
try:
result = await distill_mid(u, p)
if "error" not in result:
_record_run(u, p, "mid")
return {"ok": "error" not in result, **result}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/long")
async def do_distill_long(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "long")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "long distill"
try:
result = await distill_long(u, p)
if "error" not in result:
_record_run(u, p, "long")
return {"ok": "error" not in result, **result}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/all")
async def do_distill_all(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "all")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "full distill"
try:
short_result = distill_short(u, p)
mid_result = await distill_mid(u, p)
if "error" in mid_result:
return {"ok": False, "short": short_result, "mid": mid_result}
long_result = await distill_long(u, p)
ok = "error" not in long_result
if ok:
_record_run(u, p, "all")
return {
"ok": ok,
"short": short_result,
"mid": mid_result,
"long": long_result,
}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/rebuild")
async def do_distill_rebuild(
user: str = Query(...),
persona: str = Query(...),
) -> dict: # noqa: E501
"""Wipe MEMORY_MID and MEMORY_LONG (with backups), then run short → mid → long.
Use when memories have drifted, been corrupted, or you want a clean slate
rebuilt purely from session logs. Hand-edited content will be replaced.
"""
u, p = _resolve(user, persona)
_check_cooldown(u, p, "rebuild")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "memory rebuild"
try:
from memory_distiller import _rotate_backup, _read
inara_dir = _persona_path(u, p)
# Back up then wipe mid and long before rebuilding
for name in ("MEMORY_MID.md", "MEMORY_LONG.md"):
path = inara_dir / name
if path.exists():
_rotate_backup(path)
path.write_text(
f"# {name}\n\n*Cleared for rebuild — {__import__('datetime').datetime.now().strftime('%Y-%m-%d %H:%M')}.*\n"
)
short_result = distill_short(u, p)
mid_result = await distill_mid(u, p)
if "error" in mid_result:
return {"ok": False, "short": short_result, "mid": mid_result, "rebuilt": True}
long_result = await distill_long(u, p)
ok = "error" not in long_result
if ok:
_record_run(u, p, "rebuild")
return {
"ok": ok,
"short": short_result,
"mid": mid_result,
"long": long_result,
"rebuilt": True,
}
finally:
_LOCKS_META.pop((u, p), None)

185
cortex/routers/files.py Normal file
View File

@@ -0,0 +1,185 @@
"""
Read/write Inara identity markdown files, and search past session logs.
Only whitelisted filenames are accessible — no path traversal possible.
"""
import re
from fastapi import APIRouter, HTTPException, Query
from pydantic import BaseModel
from persona import persona_path, set_context, validate as validate_persona
from config import settings as _settings
router = APIRouter()
ALLOWED = {
"SOUL.md",
"IDENTITY.md",
"USER.md",
"PROTOCOLS.md",
"CONTEXT_TIERS.md",
"MEMORY.md", # legacy — kept for reference
"MEMORY_LONG.md",
"MEMORY_MID.md",
"MEMORY_SHORT.md",
"MEMORY_LONG.bak1.md",
"MEMORY_LONG.bak2.md",
"MEMORY_MID.bak1.md",
"MEMORY_MID.bak2.md",
"MEMORY_SHORT.bak1.md",
"MEMORY_SHORT.bak2.md",
"HELP.md",
# Agent private notes — backups only; AGENT_NOTES.md itself is agent-only
"AGENT_NOTES.bak1.md",
"AGENT_NOTES.bak2.md",
"AGENT_NOTES.bak3.md",
}
# Files that can be read via the panel but not written by users
READ_ONLY = {
"AGENT_NOTES.bak1.md",
"AGENT_NOTES.bak2.md",
"AGENT_NOTES.bak3.md",
}
# Files served from home/{user}/ instead of persona path
USER_FILES = {"email_allowlist.json", "usage.json"}
def _resolve(user: str, persona: str) -> None:
"""Validate and set context from query params. Raises HTTPException on bad input."""
try:
u, p = validate_persona(user, persona)
set_context(u, p)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
def _path(filename: str, user: str = ""):
if filename in USER_FILES:
if not user:
raise HTTPException(status_code=400, detail="user param required for this file")
return _settings.home_root() / user / filename
if filename not in ALLOWED:
raise HTTPException(status_code=404, detail=f"File not found: {filename}")
return persona_path() / filename
@router.get("/files")
async def list_files(
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
_resolve(user, persona)
persona_dir = persona_path()
files = []
for name in sorted(ALLOWED):
p = persona_dir / name
st = p.stat() if p.exists() else None
files.append({
"name": name,
"exists": p.exists(),
"size": st.st_size if st else 0,
"modified": st.st_mtime if st else None,
})
for name in sorted(USER_FILES):
p = _settings.home_root() / user / name
st = p.stat() if p.exists() else None
files.append({
"name": name,
"exists": p.exists(),
"size": st.st_size if st else 0,
"modified": st.st_mtime if st else None,
"scope": "user",
})
return {"files": files}
@router.get("/files/{filename}")
async def get_file(
filename: str,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
_resolve(user, persona)
p = _path(filename, user=user)
if not p.exists():
raise HTTPException(status_code=404, detail=f"{filename} does not exist")
return {
"name": filename,
"content": p.read_text(),
"readonly": filename in READ_ONLY,
}
class FileWrite(BaseModel):
content: str
@router.put("/files/{filename}")
async def save_file(
filename: str,
req: FileWrite,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
if filename in READ_ONLY:
raise HTTPException(status_code=403, detail=f"{filename} is read-only.")
_resolve(user, persona)
p = _path(filename, user=user)
p.write_text(req.content)
return {"ok": True, "name": filename, "size": len(req.content)}
# ── Session search ────────────────────────────────────────────────────────────
_CONTEXT_CHARS = 120 # chars of context to include around each match
@router.get("/sessions/search")
async def search_sessions(
q: str = Query(..., min_length=2),
user: str = Query("scott"),
persona: str = Query("inara"),
limit: int = Query(20, ge=1, le=100),
) -> dict:
"""Full-text search across past session logs.
Returns up to `limit` matches, newest sessions first.
Each match includes a short excerpt (120 chars before/after) for context.
"""
_resolve(user, persona)
sessions_dir = persona_path() / "sessions"
if not sessions_dir.exists():
return {"query": q, "matches": [], "total_files_searched": 0}
pattern = re.compile(re.escape(q), re.IGNORECASE)
session_files = sorted(sessions_dir.glob("*.md"), reverse=True) # newest first
matches = []
for sf in session_files:
if len(matches) >= limit:
break
try:
text = sf.read_text()
except OSError:
continue
for m in pattern.finditer(text):
if len(matches) >= limit:
break
start = max(0, m.start() - _CONTEXT_CHARS)
end = min(len(text), m.end() + _CONTEXT_CHARS)
excerpt = text[start:end].strip()
# Prefix with ellipsis if we truncated the left side
if start > 0:
excerpt = "" + excerpt
if end < len(text):
excerpt = excerpt + ""
matches.append({
"date": sf.stem, # YYYY-MM-DD
"excerpt": excerpt,
})
return {
"query": q,
"matches": matches,
"total_files_searched": len(session_files),
}

View File

@@ -1,52 +1,128 @@
import asyncio import asyncio
import logging import logging
from fastapi import APIRouter, Request, Response from fastapi import APIRouter, HTTPException, Request, Response
from google.auth.transport import requests as google_requests
from google.oauth2 import id_token
from auth_utils import get_user_channels
from context_loader import load_context from context_loader import load_context
from llm_client import complete from llm_client import complete
from persona import set_context
from session_logger import log_turn from session_logger import log_turn
from session_store import load as load_session, save as save_session from session_store import load as load_session, save as save_session
from config import settings from config import settings
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
router = APIRouter(prefix="/channels/google-chat") router = APIRouter()
# Workspace Add-on Chat apps: JWT is issued by accounts.google.com.
# (Legacy standalone Chat bots used chat@system.gserviceaccount.com — different format.)
_GOOGLE_ISSUER = "https://accounts.google.com"
@router.post("") def _msg(text: str) -> dict:
async def receive(request: Request): """Wrap a text reply in the Workspace Add-on hostAppDataAction format.
Standalone Chat apps use {"text": "..."} directly, but Workspace Add-on
Chat apps require the hostAppDataAction wrapper for Google Chat to render
the response as a bot message.
"""
return {
"hostAppDataAction": {
"chatDataAction": {
"createMessageAction": {
"message": {"text": text}
}
}
}
}
def _verify_system_id_token(token: str, audience: str) -> None:
"""Verify the systemIdToken from authorizationEventObject.
For Workspace Add-on Chat apps Google sends the token in the request body
at body["authorizationEventObject"]["systemIdToken"], not in the
Authorization header.
Claims verified:
iss = "https://accounts.google.com"
aud = the per-user audience from channels.json (the endpoint URL)
"""
try:
claims = id_token.verify_oauth2_token(
token,
google_requests.Request(),
audience=audience,
)
except Exception as exc:
logger.warning("Google Chat JWT verification failed: %s", exc)
raise HTTPException(status_code=401, detail="Invalid token")
if claims.get("iss") != _GOOGLE_ISSUER:
logger.warning("Google Chat JWT wrong issuer: %s", claims.get("iss"))
raise HTTPException(status_code=401, detail="Wrong issuer")
@router.post("/channels/google-chat/{username}")
async def receive(username: str, request: Request):
channels = get_user_channels(username)
cfg = channels.get("google_chat")
if not cfg:
logger.warning("Google Chat: no channel config for user %r", username)
raise HTTPException(status_code=404, detail="Channel not configured for this user")
persona_name = cfg.get("persona", "inara")
audience = cfg.get("audience", "")
backend = cfg.get("backend", settings.primary_backend)
timeout = cfg.get("timeout", 25)
set_context(username, persona_name)
body = await request.json() body = await request.json()
event_type = body.get("type")
if event_type == "ADDED_TO_SPACE": # Verify the systemIdToken embedded in the request body
space_type = body.get("space", {}).get("type", "") if audience:
greeting = "✨ Hello! I'm Inara. Send me a message and I'll do my best to help." token = body.get("authorizationEventObject", {}).get("systemIdToken", "")
if not token:
logger.warning("Google Chat: missing systemIdToken for %s", username)
raise HTTPException(status_code=401, detail="Missing token")
_verify_system_id_token(token, audience)
chat = body.get("chat", {})
# Event type is inferred from which payload key is present — there is no
# top-level "type" field in the Workspace Add-on event format.
if "addedToSpacePayload" in chat:
space_type = chat["addedToSpacePayload"].get("space", {}).get("type", "")
if space_type == "DM": if space_type == "DM":
greeting = "✨ Hello! I'm Inara. What can I help you with?" return _msg(f"✨ Hello! I'm {persona_name.capitalize()}. What can I help you with?")
return {"text": greeting} return _msg(f"✨ Hello! I'm {persona_name.capitalize()}. Send me a message and I'll do my best to help.")
if event_type == "REMOVED_FROM_SPACE": if "removedFromSpacePayload" in chat:
return Response(status_code=200) return Response(status_code=200)
if event_type != "MESSAGE": if "messagePayload" not in chat:
logger.info("Google Chat: unhandled event keys: %s", list(chat.keys()))
return Response(status_code=200) return Response(status_code=200)
message = body.get("message", {}) payload = chat["messagePayload"]
sender = message.get("sender", {}) message = payload.get("message", {})
space = body.get("space", {}) space = payload.get("space", {})
user = chat.get("user", {})
# argumentText strips the @BotName mention in Spaces; fall back to full text in DMs # argumentText strips @BotName mentions in Spaces; fall back to full text in DMs
user_text = (message.get("argumentText") or message.get("text", "")).strip() user_text = (message.get("argumentText") or message.get("text", "")).strip()
if not user_text: sender_display = user.get("displayName", "User")
return Response(status_code=200)
sender_display = sender.get("displayName", "User")
space_name = space.get("name", "unknown") space_name = space.get("name", "unknown")
space_type = space.get("type", "") space_type = space.get("type", "")
# Session keyed per space — one conversation per DM or Space logger.info("Google Chat message from %s in %s (%s): %r",
session_id = "gc_" + space_name.replace("/", "_") sender_display, space_name, space_type, user_text[:80])
logger.info("Google Chat message from %s in %s (%s)", sender_display, space_name, space_type) if not user_text:
logger.warning("Google Chat: empty user_text, ignoring")
return Response(status_code=200)
session_id = f"gc_{username}_{space_name.replace('/', '_')}"
system_prompt = load_context(settings.default_tier) system_prompt = load_context(settings.default_tier)
history = load_session(session_id) history = load_session(session_id)
history.append({"role": "user", "content": user_text}) history.append({"role": "user", "content": user_text})
@@ -56,19 +132,20 @@ async def receive(request: Request):
complete( complete(
system_prompt=system_prompt, system_prompt=system_prompt,
messages=history, messages=history,
model=settings.google_chat_backend, model=backend,
), ),
timeout=settings.google_chat_timeout, timeout=timeout,
) )
except asyncio.TimeoutError: except asyncio.TimeoutError:
logger.warning("Google Chat request timed out for session %s", session_id) logger.warning("Google Chat request timed out for session %s", session_id)
return {"text": "⏳ Still thinking — this is taking a bit longer than usual. Try again in a moment."} return _msg("⏳ Still thinking — this is taking a bit longer than usual. Try again in a moment.")
except Exception as e: except Exception as e:
logger.error("Google Chat error for session %s: %s", session_id, e) logger.error("Google Chat error for session %s: %s", session_id, e)
return {"text": f"⚠️ Something went wrong on my end. Try again shortly."} return _msg("⚠️ Something went wrong on my end. Try again shortly.")
logger.info("Google Chat LLM responded via %s (%d chars)", actual_backend, len(response_text))
history.append({"role": "assistant", "content": response_text}) history.append({"role": "assistant", "content": response_text})
save_session(session_id, history) save_session(session_id, history)
log_turn(session_id, user_text, response_text) log_turn(session_id, user_text, response_text)
return {"text": response_text} return _msg(response_text)

70
cortex/routers/help.py Normal file
View File

@@ -0,0 +1,70 @@
"""
Help page router.
Routes:
GET /help → full-page help viewer (requires auth)
"""
import logging
from pathlib import Path
import jwt
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, _read_auth
from persona import list_user_personas
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
@router.get("/help", include_in_schema=False)
async def help_page(request: Request, persona: str = ""):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
# Use persona from query param if valid, else prefer last-visited from cookie
if persona and persona in personas:
back_persona = persona
else:
back_persona = _preferred_persona(request, username)
back_href = f"/{username}/{back_persona}" if back_persona else "/"
html = (_STATIC / "help.html").read_text()
config_tag = (
f'<script>window.HELP_CONFIG = '
f'{{user: "{username}", persona: "{back_persona}", backHref: "{back_href}"}};</script>'
)
html = html.replace("</head>", f"{config_tag}\n</head>", 1)
nav = '<a href="/settings/integrations" class="nav-link">Integrations</a>' \
if _read_auth(username).get("role", "user") == "admin" else ""
html = html.replace("{{ integrations_nav }}", nav)
return HTMLResponse(html)

View File

@@ -0,0 +1,199 @@
"""
Home Assistant webhook router — POST /webhook/ha/{username}/{webhook_id}
Receives event payloads from HA automations and routes them to Inara.
Auth: the webhook_id in the URL acts as the shared secret (same model HA uses).
Response is delivered async via notify() — NC Talk, web push, etc.
channels.json schema:
"homeassistant": {
"webhook_id": "your-secret-id",
"persona": "inara",
"tier": 2,
"role": "chat",
"tools": false
}
HA automation example (rest_command):
rest_command:
cortex_notify:
url: "https://cortex.dgrzone.com/webhook/ha/scott/your-secret-id"
method: POST
content_type: "application/json"
payload: '{"message": "{{message}}", "entity_id": "{{entity_id}}", "state": "{{state}}"}'
automation:
trigger:
- trigger: state
entity_id: binary_sensor.front_door
to: "on"
action:
- action: rest_command.cortex_notify
data:
message: "Front door opened"
entity_id: "binary_sensor.front_door"
state: "on"
"""
import json
import logging
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response
from auth_utils import get_user_channels, get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
from context_loader import load_context
from llm_client import complete
from notification import notify
from persona import set_context
from session_logger import log_turn
from session_store import load as load_session, save as save_session
from config import settings
import event_bus
import model_registry
import orchestrator_engine
import openai_orchestrator
logger = logging.getLogger(__name__)
router = APIRouter()
def _build_task(body: dict) -> str:
"""Turn an HA event payload into a natural-language prompt for Inara."""
if "message" in body:
msg = str(body["message"])
extras = {k: body[k] for k in ("entity_id", "state", "trigger", "event", "area") if k in body}
if extras:
msg += "\n\nContext: " + json.dumps(extras)
return msg
return "Home Assistant event:\n" + json.dumps(body, indent=2)
async def _process_event(username: str, body: dict, cfg: dict) -> None:
persona_name = cfg.get("persona", "inara")
tier = cfg.get("tier") or settings.default_tier
role = cfg.get("role", "chat")
use_tools = cfg.get("tools", False)
set_context(username, persona_name)
task = _build_task(body)
session_id = f"ha_{username}"
history = load_session(session_id)
session_msgs = list(history)
logger.info("HA event for %s: %r", username, task[:80])
backend = "unknown"
try:
if use_tools:
role_cfg = model_registry.get_role_config(username, role)
system_prompt = load_context(
tier,
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
)
orch_model = model_registry.get_model_for_role(username, "orchestrator")
user_role_val = get_user_role(username)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(username)
c_allow = set(policy.get("allow", []))
c_deny = set(policy.get("deny", []))
max_risk, risk_wl, risk_bl = get_risk_policy(username)
if orch_model and orch_model.get("type") == "local_openai":
result = await openai_orchestrator.run(
task=task,
system_prompt=system_prompt,
session_messages=session_msgs or None,
model_cfg=orch_model,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
else:
gemini_key = (
(orch_model.get("api_key") if orch_model else None)
or get_user_gemini_key(username)
)
result = await orchestrator_engine.run(
task=task,
system_prompt=system_prompt,
session_messages=session_msgs or None,
respond_with_claude=True,
gemini_api_key=gemini_key,
model_name=orch_model.get("model_name") if orch_model else None,
response_role=role,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
response_text = result.response
backend = result.backend
else:
system_prompt = load_context(tier)
msgs = list(session_msgs) + [{"role": "user", "content": task}]
response_text, backend = await complete(system_prompt=system_prompt, messages=msgs)
except Exception as e:
logger.error("HA event error for %s: %s", username, e)
return
logger.info("HA response via %s (%d chars)", backend, len(response_text))
history.append({"role": "user", "content": task})
history.append({"role": "assistant", "content": response_text})
save_session(session_id, history)
log_turn(session_id, task, response_text)
await event_bus.publish({
"type": "ha_event",
"session_id": session_id,
"response": response_text,
"backend": backend,
})
await notify(username, response_text)
@router.post("/webhook/ha/{username}/{webhook_id}")
async def ha_webhook(
username: str,
webhook_id: str,
request: Request,
background_tasks: BackgroundTasks,
) -> Response:
"""Receive an event from a Home Assistant automation and route it to Inara."""
channels = get_user_channels(username)
cfg = channels.get("homeassistant")
if not cfg:
raise HTTPException(status_code=404, detail="Channel not configured")
if webhook_id != cfg.get("webhook_id", ""):
logger.warning("HA webhook: bad webhook_id for user %r", username)
raise HTTPException(status_code=401, detail="Invalid webhook ID")
content_type = request.headers.get("content-type", "")
if "application/json" in content_type:
try:
body = await request.json()
except Exception:
raise HTTPException(status_code=400, detail="Invalid JSON")
else:
form = await request.form()
body = dict(form)
if not body:
return Response(status_code=200)
background_tasks.add_task(_process_event, username, body, cfg)
return Response(status_code=200)

849
cortex/routers/local_llm.py Normal file
View File

@@ -0,0 +1,849 @@
"""
Model Registry settings — providers, hosts, models, and role assignments.
Routes:
GET /settings/models → settings page (canonical)
GET /settings/local → redirect to /settings/models
POST /settings/local/host → save/create a local host
POST /settings/local/host/{id}/remove → remove a host (and its models)
POST /settings/local/google-account → save/create a Google account
POST /settings/local/google-account/{id}/remove → remove a Google account
POST /settings/local/anthropic-key → save/create an Anthropic API key
POST /settings/local/anthropic-key/{id}/remove → remove an Anthropic API key
POST /settings/local/models/add → add a model (any provider)
POST /settings/local/models/{id}/edit → edit an existing model entry
POST /settings/local/models/{id}/remove → remove a model
POST /settings/local/roles/add → add a custom role (redirects to #roles)
POST /settings/local/roles/remove → remove a custom role (redirects to #roles)
POST /api/models/role → AJAX: set a role assignment
GET /api/local-llm/fetch-models → proxy to host /api/models (JSON)
"""
import json as _json
import logging
from pathlib import Path
import httpx
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, _read_auth
from config import settings as app_settings
from persona import list_user_personas
import model_registry as reg
from tools import TOOL_CATEGORIES
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
def _integrations_nav(username: str) -> str:
role = _read_auth(username).get("role", "user")
if role == "admin":
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
return ""
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
def _host_row_html(h: dict) -> str:
"""Return the HTML for one host config row (edit form + remove link)."""
api_key = h.get("api_key", "")
key_hint = f"{api_key[-4:]}" if api_key else "not set"
ht = h.get("host_type", "openwebui")
ow = ' selected' if ht == "openwebui" else ''
ai = ' selected' if ht == "openai" else ''
hid = h["id"]
hlbl = h.get("label", "")
hurl = h.get("api_url", "")
maxc = h.get("max_concurrent", 3)
return f'''
<div class="host-row">
<form method="POST" action="/settings/local/host" class="host-form">
<input type="hidden" name="host_id" value="{hid}">
<div class="field-row">
<div class="field">
<label>Label</label>
<input type="text" name="label" value="{hlbl}"
placeholder="Gaming Laptop" autocomplete="off" data-form-type="other">
</div>
<div class="field" style="flex:2">
<label>API URL</label>
<input type="text" name="api_url" value="{hurl}"
placeholder="http://192.168.x.x:3000"
autocomplete="off" spellcheck="false" data-form-type="other">
</div>
</div>
<div class="field-row">
<div class="field">
<label>API Key</label>
<input type="password" name="api_key" placeholder="Leave blank to keep existing"
autocomplete="new-password" data-1p-ignore data-lpignore="true"
data-form-type="other">
<p class="key-status">Current: {key_hint}</p>
</div>
<div class="field" style="flex:0 0 auto">
<label>Type</label>
<select name="host_type">
<option value="openwebui"{ow}>Open WebUI / Ollama</option>
<option value="openai"{ai}>OpenAI-compatible API</option>
</select>
</div>
<div class="field" style="flex:0 0 auto; width:6rem">
<label>Max parallel</label>
<input type="number" name="max_concurrent" min="1" max="20"
value="{maxc}" style="width:100%">
</div>
</div>
<div class="btn-row">
<button type="submit" class="btn btn-secondary btn-sm">Save</button>
<button type="button" class="btn btn-secondary btn-sm fetch-btn"
data-host-id="{hid}">Fetch models</button>
<span class="fetch-status" id="fetch-{hid}"></span>
</div>
</form>
<form method="POST" action="/settings/local/host/{hid}/remove"
onsubmit="return confirm('Remove host and all its models?')"
style="margin-top:0.5rem">
<button type="submit" class="btn-link danger">Remove host</button>
</form>
</div>'''
# ── Auth helper ───────────────────────────────────────────────────────────────
def _get_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
# ── Page renderer ─────────────────────────────────────────────────────────────
def _render(username: str, request: Request | None = None, success: str = "", error: str = "") -> str:
registry = reg.get_registry(username)
hosts = registry.get("hosts", [])
models = registry.get("models", [])
roles = registry.get("roles", {})
builtins = reg._builtins()
host_by_id = {h["id"]: h for h in hosts}
goog_accts = registry.get("providers", {}).get("google", {}).get("accounts", [])
# ── Google account rows ───────────────────────────────────────────────────
google_account_rows = ""
for a in goog_accts:
hint = (a.get("api_key") or "")[:10] + "" if a.get("api_key") else "no key"
google_account_rows += f'''
<div class="account-row">
<div>
<span class="account-label">{a.get("label") or "Unnamed"}</span>
<span class="account-hint">{hint}</span>
</div>
<form method="POST" action="/settings/local/google-account/{a["id"]}/remove"
onsubmit="return confirm('Remove this Google account?')">
<button type="submit" class="btn-link danger">Remove</button>
</form>
</div>'''
if not google_account_rows:
google_account_rows = '<p class="empty-note">No accounts configured yet.</p>'
# ── Host rows — split cloud (openai) vs local (openwebui) ─────────────────
cloud_hosts = [h for h in hosts if h.get("host_type") == "openai"]
local_hosts = [h for h in hosts if h.get("host_type", "openwebui") != "openai"]
cloud_host_rows = "".join(_host_row_html(h) for h in cloud_hosts)
local_host_rows = "".join(_host_row_html(h) for h in local_hosts)
if not cloud_host_rows:
cloud_host_rows = '<p class="empty-note">No cloud API services configured yet. Add one below.</p>'
if not local_host_rows:
local_host_rows = '<p class="empty-note">No local hosts configured yet. Add one below.</p>'
host_options = "".join(
f'<option value="{h["id"]}">{h.get("label") or h["api_url"]}</option>'
for h in hosts
)
# ── Anthropic API key rows ────────────────────────────────────────────────
anthropic_api_keys = reg.get_anthropic_api_keys(username)
anthropic_keys_js = _json.dumps(anthropic_api_keys)
anthropic_key_rows = ""
for c in anthropic_api_keys:
hint = c.get("hint", "no key")
anthropic_key_rows += f'''
<div class="account-row">
<div>
<span class="account-label">{c.get("label") or "API Key"}</span>
<span class="account-hint">{hint}</span>
</div>
<form method="POST" action="/settings/local/anthropic-key/{c["id"]}/remove"
onsubmit="return confirm('Remove this Anthropic API key?')">
<button type="submit" class="btn-link danger">Remove</button>
</form>
</div>'''
if not anthropic_key_rows:
anthropic_key_rows = '<p class="empty-note">No API keys configured. Add one below or use Claude CLI (OAuth).</p>'
# ── Model rows (all providers) ────────────────────────────────────────────
_PROVIDER_BADGE = {
"claude_cli": ('<span class="pbadge pb-anthropic">Anthropic</span>', "Claude CLI"),
"anthropic_api": ('<span class="pbadge pb-anthropic">Anthropic</span>', "API Key"),
"gemini_api": ('<span class="pbadge pb-google">Google</span>', ""),
"local_openai": ('<span class="pbadge pb-local">Local</span>', ""),
}
model_rows = ""
for m in models:
resolved = reg._resolve_model(registry, m["id"])
if not resolved:
continue
mtype = m.get("type", "local_openai")
badge, default_secondary = _PROVIDER_BADGE.get(mtype, ("", ""))
if mtype == "local_openai":
h = host_by_id.get(m.get("host_id", ""), {})
secondary = h.get("label") or h.get("api_url", "")
elif mtype == "gemini_api":
acct = next((a for a in goog_accts if a["id"] == m.get("account_id")), None)
secondary = acct["label"] if acct else ""
else:
secondary = default_secondary
ctx = f'<span class="ctx-badge">{m.get("context_k",0)}k</span>' if m.get("context_k") else ""
no_tools = '' if m.get("tools", True) else '<span class="pbadge pb-notools">no tools</span>'
tags_html = " ".join(f'<span class="tag">{t}</span>' for t in (m.get("tags") or []))
sec = f'<span class="model-host">{secondary}</span>' if secondary else ""
# ── Inline edit form fields (type-specific) ───────────────────────────
if mtype == "local_openai":
host_opts = "".join(
f'<option value="{h["id"]}"'
f'{" selected" if h["id"] == m.get("host_id") else ""}>'
f'{h.get("label") or h.get("api_url","")}</option>'
for h in hosts
)
mid = m["id"]
extra_fields = (
f'<div class="field"><label>Host</label>'
f'<select name="host_id" id="edit-host-{mid}">{host_opts}</select></div>'
f'<div class="btn-row" style="margin-bottom:0.75rem">'
f'<button type="button" class="btn btn-secondary btn-sm edit-fetch-btn" data-id="{mid}">Fetch models</button>'
f'<span class="fetch-status" id="edit-fetch-status-{mid}"></span>'
f'</div>'
f'<div id="edit-model-select-wrap-{mid}" style="display:none; margin-bottom:0.75rem">'
f'<label>Pick from host</label>'
f'<select id="edit-model-picker-{mid}"><option value="">— fetch first —</option></select>'
f'</div>'
)
elif mtype == "gemini_api":
acct_opts = "".join(
f'<option value="{a["id"]}"'
f'{" selected" if a["id"] == m.get("account_id") else ""}>'
f'{a.get("label","Unnamed")}</option>'
for a in goog_accts
)
extra_fields = (
f'<div class="field"><label>Google Account</label>'
f'<select name="account_id">{acct_opts}</select></div>'
)
elif mtype == "anthropic_api":
key_opts = "".join(
f'<option value="{c["id"]}"'
f'{" selected" if c["id"] == m.get("credential_id") else ""}>'
f'{c.get("label","API Key")} ({c.get("hint","")})</option>'
for c in anthropic_api_keys
)
extra_fields = (
f'<div class="field"><label>API Key</label>'
f'<select name="credential_id">{key_opts or "<option value=\"\">No API keys configured</option>"}</select></div>'
)
else:
extra_fields = '<input type="hidden" name="credential_id" value="cli">'
cur_label = m.get("label", "")
cur_model_name = m.get("model_name", "")
cur_ctx = m.get("context_k", 0) or 0
cur_max_rounds = m.get("max_rounds") or 0
cur_tools = m.get("tools", True)
cur_tags = ", ".join(m.get("tags") or [])
cur_reasoning_budget = m.get("reasoning_budget_tokens") or 0
_rb_levels = [(0, "Off — Non-think"), (1024, "Light"), (4096, "Moderate"), (8192, "High"), (32768, "Max")]
reasoning_opts = "".join(
f'<option value="{v}" {"selected" if cur_reasoning_budget == v else ""}>{lbl}</option>'
for v, lbl in _rb_levels
)
model_rows += f'''
<div class="model-row" id="model-{m["id"]}">
<div class="model-row-header">
<div class="model-info">
<div>{badge}<span class="model-label">{m.get("label") or m.get("model_name","")}</span>{ctx}{no_tools}</div>
<span class="model-name">{m.get("model_name","")}</span>
{sec}
<div class="tag-row">{tags_html}</div>
</div>
<div class="model-btns">
<button type="button" class="row-btn model-edit-btn" data-id="{m["id"]}">Edit</button>
<form method="POST" action="/settings/local/models/{m["id"]}/remove"
onsubmit="return confirm('Remove this model?')" style="margin:0">
<button type="submit" class="row-btn danger">Remove</button>
</form>
</div>
</div>
<form class="model-edit-form" id="edit-form-{m["id"]}" style="display:none"
method="POST" action="/settings/local/models/{m["id"]}/edit">
<input type="hidden" name="mtype" value="{mtype}">
<div class="field-row">
<div class="field">
<label>Display label</label>
<input type="text" name="label" value="{cur_label}"
placeholder="My Model" autocomplete="off" data-form-type="other">
</div>
<div class="field">
<label>Model name / ID</label>
<input type="text" name="model_name" value="{cur_model_name}"
placeholder="provider/model-name" autocomplete="off"
spellcheck="false" data-form-type="other" required>
</div>
</div>
{extra_fields}
<div class="field-row">
<div class="field" style="flex:0 0 auto">
<label title="Context window size in thousands of tokens. 0 = assume 32k.">Context (k)</label>
<input type="number" name="context_k" value="{cur_ctx}" min="0"
title="Context window size in thousands of tokens. 0 = assume 32k (compaction budget ~24k tokens).">
</div>
<div class="field" style="flex:0 0 auto">
<label title="Per-model tool loop cap. 0 = use the global default (orchestrator_max_rounds).">Max rounds</label>
<input type="number" name="max_rounds" value="{cur_max_rounds}" min="0"
title="Per-model tool loop cap. 0 = use the global default (orchestrator_max_rounds).">
</div>
<div class="field" style="flex:0 0 auto">
<label title="Reasoning depth via OpenRouter's reasoning.budget_tokens. Off = Non-think. Light ~1k, Moderate ~4k, High ~8k, Max ~32k tokens.">Reasoning</label>
<select name="reasoning_budget_tokens"
title="Reasoning depth via OpenRouter's reasoning.budget_tokens. Off = Non-think. Light ~1k, Moderate ~4k, High ~8k, Max ~32k tokens.">
{reasoning_opts}
</select>
</div>
<div class="field" style="flex:0 0 auto">
<label title="Whether this model supports tool calling. If not supported, requests skip the tool loop entirely.">Tool calling</label>
<select name="tools"
title="Whether this model supports tool calling. If not supported, requests skip the tool loop entirely.">
<option value="1" {'selected' if cur_tools else ''}>Supported</option>
<option value="0" {'' if cur_tools else 'selected'}>Not supported</option>
</select>
</div>
<div class="field">
<label>Tags</label>
<input type="text" name="tags" value="{cur_tags}"
placeholder="fast, code, vision" autocomplete="off" data-form-type="other">
</div>
</div>
<div class="btn-row" style="margin-top:0.5rem">
<button type="submit" class="btn btn-primary btn-sm">Save</button>
<button type="button" class="model-edit-cancel btn btn-secondary btn-sm"
data-id="{m["id"]}">Cancel</button>
</div>
</form>
</div>'''
if not model_rows:
model_rows = '<p class="empty-note">No models added yet.</p>'
# ── Role assignment rows ──────────────────────────────────────────────────
model_opts = '<option value="">— .env default —</option>\n'
model_opts += '<optgroup label="Built-in">\n'
for bid, bm in builtins.items():
model_opts += f' <option value="{bid}">{bm["label"]}</option>\n'
model_opts += '</optgroup>\n'
if models:
model_opts += '<optgroup label="Configured models">\n'
for m in models:
lbl = m.get("label") or m.get("model_name", m["id"])
model_opts += f' <option value="{m["id"]}">{lbl}</option>\n'
model_opts += '</optgroup>\n'
all_roles = reg.get_all_roles(username)
role_rows = ""
for role in all_roles:
is_required = role in reg.REQUIRED_ROLES
role_cfg = roles.get(role, {})
role_title = role.replace("_", " ").title()
required_badge = (
'<span class="required-badge">required</span>'
if is_required else ''
)
rcp_danger = (
'' if is_required else
f'<div class="rcp-danger">'
f'<form method="POST" action="/settings/local/roles/remove" class="remove-role-form">'
f'<input type="hidden" name="role_name" value="{role}">'
f'<button type="submit" class="btn-link danger" data-role="{role}">Remove this role…</button>'
f'</form>'
f'</div>'
)
role_rows += (
f'<div class="role-row" data-role="{role}">'
f'<div class="role-name-col">'
f'<span class="role-name">{role_title}</span>'
f'{required_badge}'
f'</div>'
f'<div class="role-slots">'
)
for slot in reg.PRIORITY_KEYS[:2]:
slot_label = slot.replace("_", " ").title()
sel = (
f'<select class="role-select" data-role="{role}" '
f'data-slot="{slot}" title="{slot_label}">\n{model_opts}\n</select>'
)
role_rows += f'<div class="role-slot"><span class="slot-label">{slot_label}</span>{sel}</div>'
role_rows += (
f'</div>'
f'<button class="role-cfg-btn" data-role="{role}" title="Configure">⚙</button>'
f'</div>'
f'<div class="role-config-panel" id="rcp-{role}">'
f'<div class="rcp-field">'
f'<label class="rcp-label">System prompt addition</label>'
f'<textarea class="rcp-textarea" data-role="{role}" rows="3" '
f'placeholder="Extra instructions injected into the system prompt when this role is active…"></textarea>'
f'</div>'
f'<div class="rcp-field">'
f'<div style="display:flex;flex-direction:column;gap:0.3rem">'
f'<label class="rcp-check">'
f'<input type="checkbox" class="rcp-datetime-cb" data-role="{role}" checked>'
f'<span>Inject current date &amp; time into system prompt</span>'
f'</label>'
f'<label class="rcp-check">'
f'<input type="checkbox" class="rcp-mode-cb" data-role="{role}" checked>'
f'<span>Inject session mode (Chat / Off The Record) into system prompt</span>'
f'</label>'
f'</div>'
f'<p class="rcp-hint" style="margin-top:0.4rem">Disable both for pure processing roles (summarizer, classifier, translator).</p>'
f'</div>'
f'<div class="rcp-field">'
f'<label class="rcp-label">Tool allow-list '
f'<span class="rcp-hint">— all checked means no restriction (use all accessible tools)</span></label>'
f'<div class="rcp-tools" id="rcp-tools-{role}"></div>'
f'</div>'
f'<div class="rcp-actions">'
f'<button class="btn btn-primary btn-sm rcp-save" data-role="{role}">Save</button>'
f'<button class="btn btn-secondary btn-sm rcp-cancel" data-role="{role}">Cancel</button>'
f'</div>'
f'{rcp_danger}'
f'</div>'
)
role_data_js = _json.dumps({
role: {slot: (roles.get(role, {}).get(slot) or "") for slot in reg.PRIORITY_KEYS[:2]}
for role in all_roles
})
role_config_data_js = _json.dumps({
role: {
"system_append": roles.get(role, {}).get("system_append", ""),
"tools": roles.get(role, {}).get("tools") or None,
"inject_datetime": roles.get(role, {}).get("inject_datetime", True),
"inject_mode": roles.get(role, {}).get("inject_mode", True),
}
for role in all_roles
})
tool_categories_js = _json.dumps(TOOL_CATEGORIES)
# ── Catalog data + Google accounts for JS ─────────────────────────────────
google_accounts_js = _json.dumps(reg.get_google_accounts(username))
google_catalog_js = _json.dumps(reg.get_catalog("google"))
anthropic_catalog_js = _json.dumps(reg.get_catalog("anthropic"))
cloud_catalog_js = _json.dumps(reg.get_catalog("cloud"))
has_hosts = "true" if hosts else "false"
html = (_STATIC / "local_llm.html").read_text()
replacements = {
"{{ username }}": username,
"{{ google_account_rows }}": google_account_rows,
"{{ anthropic_key_rows }}": anthropic_key_rows,
"{{ cloud_host_rows }}": cloud_host_rows,
"{{ local_host_rows }}": local_host_rows,
"{{ model_rows }}": model_rows,
"{{ host_options }}": host_options,
"{{ role_rows }}": role_rows,
"{{ role_data_js }}": role_data_js,
"{{ role_config_data_js }}": role_config_data_js,
"{{ tool_categories_js }}": tool_categories_js,
"{{ google_accounts_js }}": google_accounts_js,
"{{ anthropic_keys_js }}": anthropic_keys_js,
"{{ google_catalog_js }}": google_catalog_js,
"{{ anthropic_catalog_js }}": anthropic_catalog_js,
"{{ cloud_catalog_js }}": cloud_catalog_js,
"{{ has_hosts }}": has_hosts,
}
for key, val in replacements.items():
html = html.replace(key, val)
back_persona = _preferred_persona(request, username) if request else ""
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="msg success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="msg error">{error}</p>')
return html
# ── Routes ────────────────────────────────────────────────────────────────────
@router.get("/settings/models", include_in_schema=False)
async def models_page_canonical(request: Request):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
return HTMLResponse(_render(username, request))
@router.get("/settings/local", include_in_schema=False)
async def models_page_legacy(request: Request):
return RedirectResponse("/settings/models", status_code=301)
@router.post("/settings/local/google-account", include_in_schema=False)
async def save_google_account(
request: Request,
account_id: str = Form(""),
label: str = Form(""),
api_key: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not api_key.strip() and not account_id.strip():
return HTMLResponse(_render(username, request, error="API key is required."))
reg.save_google_account(username, account_id or None, label, api_key)
return HTMLResponse(_render(username, request, success="Google account saved."))
@router.post("/settings/local/google-account/{account_id}/remove", include_in_schema=False)
async def remove_google_account(request: Request, account_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_google_account(username, account_id)
return HTMLResponse(_render(username, request, success="Google account removed."))
@router.post("/settings/local/anthropic-key", include_in_schema=False)
async def save_anthropic_api_key(
request: Request,
key_id: str = Form(""),
label: str = Form(""),
api_key: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not api_key.strip() and not key_id.strip():
return HTMLResponse(_render(username, request, error="API key is required."))
reg.save_anthropic_api_key(username, key_id or None, label, api_key)
return HTMLResponse(_render(username, request, success="Anthropic API key saved."))
@router.post("/settings/local/anthropic-key/{key_id}/remove", include_in_schema=False)
async def remove_anthropic_api_key(request: Request, key_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_anthropic_api_key(username, key_id)
return HTMLResponse(_render(username, request, success="Anthropic API key removed."))
@router.post("/settings/local/host", include_in_schema=False)
async def save_host(
request: Request,
host_id: str = Form(""),
label: str = Form(""),
api_url: str = Form(""),
api_key: str = Form(""),
host_type: str = Form("openwebui"),
max_concurrent: int = Form(3),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not api_url.strip():
return HTMLResponse(_render(username, request, error="API URL is required."))
reg.save_host(username, host_id or None, label, api_url, api_key, host_type, max_concurrent)
return HTMLResponse(_render(username, request, success="Host saved."))
@router.post("/settings/local/host/{host_id}/remove", include_in_schema=False)
async def remove_host(request: Request, host_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_host(username, host_id)
return HTMLResponse(_render(username, request, success="Host removed."))
@router.post("/settings/local/models/add", include_in_schema=False)
async def add_model(
request: Request,
provider: str = Form("local"),
label: str = Form(""),
context_k: int = Form(0),
max_rounds: int = Form(0),
tools: int = Form(1),
tags: str = Form(""),
reasoning_budget_tokens: int = Form(0),
# local-only fields
host_id: str = Form(""),
model_name: str = Form(""),
# cloud-only fields
cloud_model_name: str = Form(""),
account_id: str = Form(""),
credential_id: str = Form("cli"),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
tag_list = [t.strip() for t in tags.split(",") if t.strip()]
max_rounds_ = max_rounds or None
tools_bool = tools != 0
reasoning_budget_ = reasoning_budget_tokens or None
if provider == "local":
if not model_name.strip():
return HTMLResponse(_render(username, request, error="Model name is required."))
if not host_id.strip():
return HTMLResponse(_render(username, request, error="Select a host."))
reg.save_model(username, None, host_id, label, model_name, context_k, tag_list,
max_rounds=max_rounds_, tools=tools_bool,
reasoning_budget_tokens=reasoning_budget_)
display = label or model_name
elif provider in ("google", "anthropic"):
if not cloud_model_name.strip():
return HTMLResponse(_render(username, request, error="Select a model from the catalog."))
if provider == "google" and not account_id.strip():
return HTMLResponse(_render(username, request, error="Select a Google account."))
reg.save_cloud_model(
username, None, provider, cloud_model_name, label,
account_id=account_id or None,
credential_id=credential_id or None,
context_k=context_k, tags=tag_list,
max_rounds=max_rounds_, tools=tools_bool,
)
display = label or cloud_model_name
else:
return HTMLResponse(_render(username, request, error=f"Unknown provider: {provider}"))
logger.info("model added: %s / %s (%s)", username, display, provider)
return HTMLResponse(_render(username, request, success=f'Model "{display}" added.'))
@router.post("/settings/local/models/{model_id}/edit", include_in_schema=False)
async def edit_model(
request: Request,
model_id: str,
mtype: str = Form(""),
label: str = Form(""),
model_name: str = Form(""),
context_k: int = Form(0),
max_rounds: int = Form(0),
tools: int = Form(1),
tags: str = Form(""),
reasoning_budget_tokens: int = Form(0),
host_id: str = Form(""),
account_id: str = Form(""),
credential_id: str = Form("cli"),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not model_name.strip():
return HTMLResponse(_render(username, request, error="Model name is required."))
tag_list = [t.strip() for t in tags.split(",") if t.strip()]
max_rounds_ = max_rounds or None
tools_bool = tools != 0
reasoning_budget_ = reasoning_budget_tokens or None
if mtype == "local_openai":
if not host_id.strip():
return HTMLResponse(_render(username, request, error="Select a host for this model."))
reg.save_model(username, model_id, host_id, label, model_name, context_k, tag_list,
max_rounds=max_rounds_, tools=tools_bool,
reasoning_budget_tokens=reasoning_budget_)
elif mtype == "gemini_api":
reg.save_cloud_model(username, model_id, "google", model_name, label,
account_id=account_id or None, context_k=context_k, tags=tag_list,
max_rounds=max_rounds_, tools=tools_bool)
elif mtype in ("claude_cli", "anthropic_api"):
reg.save_cloud_model(username, model_id, "anthropic", model_name, label,
credential_id=credential_id or "cli", context_k=context_k, tags=tag_list,
max_rounds=max_rounds_, tools=tools_bool)
else:
return HTMLResponse(_render(username, request, error=f"Unknown model type: {mtype}"))
display = label.strip() or model_name.strip()
logger.info("model edited: %s / %s (%s)", username, display, mtype)
return HTMLResponse(_render(username, request, success=f'Model "{display}" updated.'))
@router.post("/settings/local/models/{model_id}/remove", include_in_schema=False)
async def remove_model(request: Request, model_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_model(username, model_id)
return HTMLResponse(_render(username, request, success="Model removed."))
@router.post("/settings/local/roles/add", include_in_schema=False)
async def add_custom_role_route(
request: Request,
role_name: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
name = role_name.strip().lower()
if not name or not name[0].isalpha():
return HTMLResponse(_render(username, request, error="Role name must start with a letter."))
ok = reg.add_custom_role(username, name)
if not ok:
return HTMLResponse(_render(username, request, error=f'"{name}" is a required role and cannot be re-added.'))
logger.info("custom role added: %s / %s", username, name)
return RedirectResponse("/settings/models#roles", status_code=303)
@router.post("/settings/local/roles/remove", include_in_schema=False)
async def remove_custom_role_route(
request: Request,
role_name: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
name = role_name.strip()
ok = reg.remove_custom_role(username, name)
if not ok:
return HTMLResponse(_render(username, request, error=f'"{name}" is a required role and cannot be removed.'))
logger.info("custom role removed: %s / %s", username, name)
return RedirectResponse("/settings/models#roles", status_code=303)
@router.post("/api/models/role")
async def set_role(request: Request) -> JSONResponse:
"""AJAX: assign a model to a role priority slot.
Body: {"role": "chat", "slot": "primary", "model_id": "abc123" | ""}
"""
username = _get_user(request)
if not username:
return JSONResponse({"error": "Not authenticated"}, status_code=401)
try:
body = await request.json()
except Exception:
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
role = body.get("role", "").strip()
slot = body.get("slot", "").strip()
model_id = body.get("model_id", "").strip() or None
if not role or not slot:
return JSONResponse({"error": "role and slot are required"}, status_code=400)
ok = reg.set_role(username, role, slot, model_id)
if not ok:
return JSONResponse({"error": "Invalid slot or model_id not found"}, status_code=400)
logger.info("role set: %s %s.%s = %s", username, role, slot, model_id)
return JSONResponse({"ok": True})
@router.post("/api/models/role-config")
async def set_role_config(request: Request) -> JSONResponse:
"""AJAX: save system_append, tool allow-list, and inject_datetime flag for a role.
Body: {"role": "coder", "system_append": "...", "tools": [...] | null, "inject_datetime": true}
tools=null clears the allow-list (role uses all accessible tools).
inject_datetime=false suppresses the date/time header for pure processing roles.
"""
username = _get_user(request)
if not username:
return JSONResponse({"error": "Not authenticated"}, status_code=401)
try:
body = await request.json()
except Exception:
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
role = body.get("role", "").strip()
system_append = body.get("system_append", "")
tools = body.get("tools") # list[str] or None
inject_datetime = body.get("inject_datetime", True)
inject_mode = body.get("inject_mode", True)
if not role:
return JSONResponse({"error": "role is required"}, status_code=400)
if tools is not None and not isinstance(tools, list):
return JSONResponse({"error": "tools must be a list or null"}, status_code=400)
reg.set_role_config(username, role, system_append, tools,
inject_datetime=bool(inject_datetime),
inject_mode=bool(inject_mode))
logger.info("role config saved: %s %s (tools=%s inject_datetime=%s inject_mode=%s)",
username, role, len(tools) if tools is not None else "all",
inject_datetime, inject_mode)
return JSONResponse({"ok": True})
@router.get("/api/local-llm/fetch-models")
async def fetch_models(request: Request, host_id: str = "") -> JSONResponse:
"""Proxy to the host's models endpoint. host_id selects which host."""
username = _get_user(request)
if not username:
return JSONResponse({"error": "Not authenticated"}, status_code=401)
registry = reg.get_registry(username)
hosts = registry.get("hosts", [])
host = next((h for h in hosts if h["id"] == host_id), None) if host_id else (hosts[0] if hosts else None)
if host:
api_url, api_key, host_type = host.get("api_url",""), host.get("api_key",""), host.get("host_type","openwebui")
else:
api_url, api_key, host_type = app_settings.local_api_url, app_settings.local_api_key, "openwebui"
if not api_url:
return JSONResponse({"error": "No host configured."}, status_code=400)
models_path = "/models" if host_type == "openai" else "/api/models"
url = api_url.rstrip("/") + models_path
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
try:
async with httpx.AsyncClient(timeout=8) as client:
resp = await client.get(url, headers=headers)
resp.raise_for_status()
data = resp.json()
models = sorted(
[{"id": m["id"], "name": m.get("name") or m["id"]} for m in data.get("data", [])],
key=lambda m: m["name"].lower(),
)
return JSONResponse({"models": models})
except httpx.HTTPStatusError as e:
return JSONResponse({"error": f"Host returned {e.response.status_code}"}, status_code=502)
except Exception as e:
return JSONResponse({"error": str(e)}, status_code=502)

View File

@@ -3,16 +3,21 @@ import hashlib
import hmac import hmac
import json import json
import logging import logging
import secrets
import httpx
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response
from config import settings from auth_utils import get_user_channels, get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
from context_loader import load_context from context_loader import load_context
from llm_client import complete from llm_client import complete
from notification import _send_nct_message
from persona import set_context
from session_logger import log_turn from session_logger import log_turn
from session_store import load as load_session, save as save_session from session_store import load as load_session, save as save_session
from config import settings
import event_bus
import model_registry
import orchestrator_engine
import openai_orchestrator
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG) logger.setLevel(logging.DEBUG)
@@ -25,93 +30,174 @@ if not logger.handlers:
router = APIRouter() router = APIRouter()
def _verify_signature(body: bytes, random_header: str, sig_header: str) -> bool: def _verify_signature(body: bytes, random_header: str, sig_header: str, secret: str) -> bool:
"""Nextcloud signs requests with HMAC-SHA256(key=secret, msg=random+body).""" """Nextcloud signs requests with HMAC-SHA256(key=secret, msg=random+body)."""
expected = hmac.new( expected = hmac.new(
settings.nextcloud_talk_bot_secret.encode(), secret.encode(),
(random_header + body.decode("utf-8", errors="replace")).encode(), (random_header + body.decode("utf-8", errors="replace")).encode(),
hashlib.sha256, hashlib.sha256,
).hexdigest() ).hexdigest()
return hmac.compare_digest(expected, sig_header.lower()) return hmac.compare_digest(expected, sig_header.lower())
async def _send_reply(conversation_token: str, message: str) -> None: async def _send_reply(conversation_token: str, message: str, nextcloud_url: str, secret: str) -> None:
"""Post a message to Nextcloud Talk as the bot.""" """Post a message to Nextcloud Talk as the bot."""
url = ( logger.info("NCT _send_reply → room %s (%d chars)", conversation_token, len(message))
f"{settings.nextcloud_url}/ocs/v2.php/apps/spreed/api/v1" await _send_nct_message(nextcloud_url, secret, conversation_token, message)
f"/bot/{conversation_token}/message"
)
# NC Talk verifies HMAC over (random + message_text), NOT the raw body.
# See BotController::getBotFromHeaders → checksumVerificationService::validateRequest($random, $sig, $secret, $message)
body_dict = {"message": message}
body_bytes = json.dumps(body_dict, ensure_ascii=False).encode("utf-8")
random_str = secrets.token_hex(32)
sig = hmac.new(
settings.nextcloud_talk_bot_secret.encode(),
(random_str + message).encode("utf-8"),
hashlib.sha256,
).hexdigest()
logger.info("NCT _send_reply → %s (body: %s)", url, body_bytes.decode())
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
url,
content=body_bytes,
headers={
"Content-Type": "application/json",
"OCS-APIRequest": "true",
"X-Nextcloud-Talk-Bot-Random": random_str,
"X-Nextcloud-Talk-Bot-Signature": sig,
},
timeout=15,
)
logger.info("NCT reply: %s%s", resp.status_code, resp.text[:400])
except Exception as e:
logger.error("NCT reply error: %s", e)
async def _process_message(conversation_token: str, user_text: str, actor_name: str) -> None: async def _process_message(
conversation_token: str,
user_text: str,
actor_name: str,
username: str,
persona_name: str,
nextcloud_url: str,
secret: str,
timeout: int,
cfg: dict,
) -> None:
logger.info("NCT process: token=%s user=%s text=%r", conversation_token, actor_name, user_text) logger.info("NCT process: token=%s user=%s text=%r", conversation_token, actor_name, user_text)
session_id = f"nct_{conversation_token}"
system_prompt = load_context(settings.default_tier)
history = load_session(session_id)
history.append({"role": "user", "content": user_text})
set_context(username, persona_name)
tier = cfg.get("tier") or settings.default_tier
role = cfg.get("role", "chat")
use_tools = cfg.get("tools", False)
session_id = f"nct_{username}_{conversation_token}"
history = load_session(session_id)
session_msgs = list(history) # snapshot before we append
await event_bus.publish({
"type": "nct_message",
"session_id": session_id,
"role": "user",
"content": user_text,
"actor": actor_name,
})
backend = "unknown"
try: try:
response_text, backend = await asyncio.wait_for( if use_tools:
complete(system_prompt=system_prompt, messages=history), await _send_reply(conversation_token, "⏳ Working on it…", nextcloud_url, secret)
timeout=settings.nextcloud_talk_timeout,
role_cfg = model_registry.get_role_config(username, role)
system_prompt = load_context(
tier,
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
) )
orch_model = model_registry.get_model_for_role(username, "orchestrator")
user_role_val = get_user_role(username)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(username)
c_allow = set(policy.get("allow", []))
c_deny = set(policy.get("deny", []))
max_risk, risk_wl, risk_bl = get_risk_policy(username)
if orch_model and orch_model.get("type") == "local_openai":
result = await openai_orchestrator.run(
task=user_text,
system_prompt=system_prompt,
session_messages=session_msgs or None,
model_cfg=orch_model,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
else:
gemini_key = (
(orch_model.get("api_key") if orch_model else None)
or get_user_gemini_key(username)
)
result = await orchestrator_engine.run(
task=user_text,
system_prompt=system_prompt,
session_messages=session_msgs or None,
respond_with_claude=True,
gemini_api_key=gemini_key,
model_name=orch_model.get("model_name") if orch_model else None,
response_role=role,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
response_text = result.response
backend = result.backend
if result.checkpoint:
response_text += "\n\n_(This action requires confirmation — use the web UI to approve or deny.)_"
else:
system_prompt = load_context(tier)
history_for_llm = list(session_msgs) + [{"role": "user", "content": user_text}]
response_text, backend = await asyncio.wait_for(
complete(system_prompt=system_prompt, messages=history_for_llm),
timeout=timeout,
)
except asyncio.TimeoutError: except asyncio.TimeoutError:
logger.warning("NCT timeout for %s", conversation_token) logger.warning("NCT timeout for %s", conversation_token)
await _send_reply(conversation_token, "⏳ Still thinking — this is taking longer than usual.") await _send_reply(conversation_token, "⏳ Still thinking — this is taking longer than usual.", nextcloud_url, secret)
return return
except Exception as e: except Exception as e:
logger.error("NCT LLM error for %s: %s", conversation_token, e) logger.error("NCT LLM error for %s: %s", conversation_token, e)
await _send_reply(conversation_token, "⚠️ Something went wrong on my end.") await _send_reply(conversation_token, "⚠️ Something went wrong on my end.", nextcloud_url, secret)
return return
logger.info("NCT LLM responded via %s (%d chars)", backend, len(response_text)) logger.info("NCT LLM responded via %s (%d chars)", backend, len(response_text))
history.append({"role": "user", "content": user_text})
history.append({"role": "assistant", "content": response_text}) history.append({"role": "assistant", "content": response_text})
save_session(session_id, history) save_session(session_id, history)
log_turn(session_id, user_text, response_text) log_turn(session_id, user_text, response_text)
await _send_reply(conversation_token, response_text)
await event_bus.publish({
"type": "nct_response",
"session_id": session_id,
"role": "assistant",
"content": response_text,
"backend": backend,
})
await _send_reply(conversation_token, response_text, nextcloud_url, secret)
@router.post("/inara-nextcloud-talk-webhook") @router.post("/webhook/nextcloud/{username}")
async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundTasks): async def nextcloud_talk_webhook(username: str, request: Request, background_tasks: BackgroundTasks):
body = await request.body() channels = get_user_channels(username)
cfg = channels.get("nextcloud")
if not cfg:
logger.warning("NCT webhook: no channel config for user %r", username)
raise HTTPException(status_code=404, detail="Channel not configured for this user")
if not settings.nextcloud_talk_bot_secret: persona_name = cfg.get("persona", "inara")
logger.error("nextcloud_talk_bot_secret not configured") nextcloud_url = cfg.get("url", "")
secret = cfg.get("bot_secret", "")
timeout = cfg.get("timeout", 55)
if not secret:
logger.error("NCT webhook: bot_secret missing for user %r", username)
return Response(status_code=500) return Response(status_code=500)
body = await request.body()
random_header = request.headers.get("X-Nextcloud-Talk-Random", "") random_header = request.headers.get("X-Nextcloud-Talk-Random", "")
sig_header = request.headers.get("X-Nextcloud-Talk-Signature", "") sig_header = request.headers.get("X-Nextcloud-Talk-Signature", "")
if not _verify_signature(body, random_header, sig_header): if not _verify_signature(body, random_header, sig_header, secret):
logger.warning("NCT webhook: signature mismatch") logger.warning("NCT webhook: signature mismatch for %s", username)
raise HTTPException(status_code=401, detail="Invalid signature") raise HTTPException(status_code=401, detail="Invalid signature")
try: try:
@@ -140,8 +226,9 @@ async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundT
except (json.JSONDecodeError, AttributeError): except (json.JSONDecodeError, AttributeError):
user_text = (obj.get("name") or obj.get("content", "")).strip() user_text = (obj.get("name") or obj.get("content", "")).strip()
if user_text.lower().startswith("@inara"): mention_prefix = f"@{persona_name.lower()}"
user_text = user_text[6:].strip() if user_text.lower().startswith(mention_prefix):
user_text = user_text[len(mention_prefix):].strip()
if not user_text: if not user_text:
return Response(status_code=200) return Response(status_code=200)
@@ -149,5 +236,9 @@ async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundT
actor_name = actor.get("name", "User") actor_name = actor.get("name", "User")
logger.info("NCT message from %s in %s: %r", actor_name, conversation_token, user_text[:60]) logger.info("NCT message from %s in %s: %r", actor_name, conversation_token, user_text[:60])
background_tasks.add_task(_process_message, conversation_token, user_text, actor_name) background_tasks.add_task(
_process_message,
conversation_token, user_text, actor_name,
username, persona_name, nextcloud_url, secret, timeout, cfg,
)
return Response(status_code=200) return Response(status_code=200)

View File

@@ -0,0 +1,310 @@
"""
Onboarding router — invite-based setup + persona creation + model connect.
Routes:
GET /setup/{token} → show password setup form (step 1)
POST /setup/{token} → set password, redirect to persona step
GET /setup/persona → show persona creation form (step 2, requires auth)
POST /setup/persona → create persona, redirect to /setup/model
GET /setup/model → OpenRouter quick-connect (step 3, also standalone)
POST /setup/model → save host + model + assign to chat role, redirect to chat
"""
import logging
import re
from pathlib import Path
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import (
COOKIE_NAME, validate_invite, consume_invite,
set_password, create_token,
)
from persona_template import create_persona
from persona import list_user_personas, validate as validate_persona
import model_registry
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/setup")
_STATIC = Path(__file__).parent.parent / "static"
_SLUG_RE = re.compile(r"^[a-z_][a-z0-9_-]{0,31}$")
def _setup_page(error: str = "", step: int = 1) -> str:
html = (_STATIC / "setup.html").read_text()
if error:
html = html.replace(
"<!-- ERROR -->",
f'<p class="error">{error}</p>',
)
if step == 2:
html = html.replace("location.search)", "location.search)", 1) # noop, handled by ?step=2
return html
# ---------------------------------------------------------------------------
# Step 2 — persona creation (requires active session)
# IMPORTANT: must be registered before /{token} so "/persona" literal wins
# ---------------------------------------------------------------------------
@router.get("/persona", include_in_schema=False)
async def persona_page(request: Request):
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
if not token:
return RedirectResponse("/login", status_code=302)
try:
decode_token(token)
except jwt.InvalidTokenError:
return RedirectResponse("/login", status_code=302)
html = (_STATIC / "setup.html").read_text()
# Show step 2 directly — inject ?step=2 behaviour inline
html = html.replace(
"if (params.get('step') === '2') {",
"if (true || params.get('step') === '2') {",
)
return HTMLResponse(html)
@router.post("/persona", include_in_schema=False)
async def persona_submit(
request: Request,
step: str = Form(...),
persona_name: str = Form(...),
display_name: str = Form(...),
user_real_name: str = Form(...),
emoji: str = Form(default=""),
description: str = Form(default=""),
):
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
if not token:
return RedirectResponse("/login", status_code=302)
try:
username = decode_token(token)
except jwt.InvalidTokenError:
return RedirectResponse("/login", status_code=302)
# Validate persona slug
if not _SLUG_RE.match(persona_name):
html = (_STATIC / "setup.html").read_text().replace(
"if (params.get('step') === '2') {",
"if (true || params.get('step') === '2') {",
).replace("<!-- ERROR -->", '<p class="error">Invalid persona name. Use lowercase letters, digits, _ or - only.</p>')
return HTMLResponse(html, status_code=422)
# Check for collision
existing = list_user_personas(username)
if persona_name in existing:
html = (_STATIC / "setup.html").read_text().replace(
"if (params.get('step') === '2') {",
"if (true || params.get('step') === '2') {",
).replace("<!-- ERROR -->", f'<p class="error">Persona "{persona_name}" already exists.</p>')
return HTMLResponse(html, status_code=422)
create_persona(
username=username,
persona_name=persona_name,
display_name=display_name.strip() or persona_name.capitalize(),
user_real_name=user_real_name.strip() or username.capitalize(),
emoji=emoji or "",
description=description.strip(),
)
logger.info("persona created: %s/%s", username, persona_name)
# Step 3: guided model setup before entering the chat
resp = RedirectResponse("/setup/model", status_code=302)
# Remember which persona to land on after model setup
resp.set_cookie("cx_setup_persona", f"{username}/{persona_name}", max_age=3600, httponly=True, samesite="lax")
return resp
# ---------------------------------------------------------------------------
# Step 1 — invite token → set password
# IMPORTANT: registered after /persona so the literal path wins above
# ---------------------------------------------------------------------------
@router.get("/{token}", include_in_schema=False)
async def setup_page(token: str, request: Request):
"""Show the password setup page for a valid invite token."""
username = validate_invite(token)
if not username:
return HTMLResponse(
"<h1 style='font-family:sans-serif;padding:2rem'>This link is invalid or has expired.</h1>",
status_code=400,
)
return HTMLResponse(_setup_page())
@router.get("/{token}/persona", include_in_schema=False)
async def setup_persona_via_token(token: str, request: Request):
"""After password setup, redirect to the generic /setup/persona page."""
# Cookie is already set — just redirect. Token is consumed so this is safe.
return RedirectResponse("/setup/persona", status_code=302)
@router.post("/{token}", include_in_schema=False)
async def setup_submit(
token: str,
step: str = Form(...),
password: str = Form(default=""),
confirm: str = Form(default=""),
):
username = validate_invite(token)
if not username:
return HTMLResponse(
"<h1 style='font-family:sans-serif;padding:2rem'>This link is invalid or has expired.</h1>",
status_code=400,
)
if step == "password":
if len(password) < 8:
return HTMLResponse(_setup_page("Password must be at least 8 characters."))
if password != confirm:
return HTMLResponse(_setup_page("Passwords do not match."))
set_password(username, password)
consume_invite(username)
logger.info("setup complete (password): %s", username)
# Log them in and move to persona step
resp = RedirectResponse(f"/setup/{token}/persona", status_code=302)
resp.set_cookie(
COOKIE_NAME,
create_token(username),
max_age=30 * 86400,
httponly=True,
samesite="lax",
secure=False,
)
return resp
return HTMLResponse(_setup_page("Unknown step."), status_code=400)
# ---------------------------------------------------------------------------
# Step 3 — model connect (OpenRouter quick-connect, also standalone)
# ---------------------------------------------------------------------------
# Curated model list shown in the Step 3 dropdown.
_OPENROUTER_MODELS = [
("anthropic/claude-3-5-haiku-20241022", "Claude 3.5 Haiku — Fast & affordable"),
("anthropic/claude-3-7-sonnet-20250219", "Claude 3.7 Sonnet — Smarter Claude"),
("google/gemini-2.0-flash-001", "Gemini 2.0 Flash — Fast Google model"),
("meta-llama/llama-3.3-70b-instruct", "Llama 3.3 70B — Open source"),
]
def _model_page(error: str = "", from_setup: bool = False) -> str:
html = (_STATIC / "setup.html").read_text()
# Hide steps 1 and 2 inline; show step 3
html = html.replace('<div id="step-password">', '<div id="step-password" style="display:none">')
html = html.replace('<div id="step-persona" style="display:none">', '<div id="step-persona" style="display:none">')
html = html.replace('<div id="step-model" style="display:none">', '<div id="step-model">')
if from_setup:
html = html.replace("<!-- SETUP_STEP3_LABEL -->", "Step 3 of 3")
if error:
html = html.replace("<!-- ERROR_MODEL -->", f'<p class="error">{error}</p>')
return html
@router.post("/model/skip", include_in_schema=False)
async def model_skip(request: Request):
"""Skip model setup — redirect to the remembered persona or user root."""
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
username = None
if token:
try:
username = decode_token(token)
except jwt.InvalidTokenError:
pass
dest_cookie = request.cookies.get("cx_setup_persona", "")
dest = f"/{dest_cookie}" if dest_cookie else (f"/{username}" if username else "/")
resp = RedirectResponse(dest, status_code=302)
resp.delete_cookie("cx_setup_persona")
return resp
@router.get("/model", include_in_schema=False)
async def model_page(request: Request):
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
if not token:
return RedirectResponse("/login", status_code=302)
try:
decode_token(token)
except jwt.InvalidTokenError:
return RedirectResponse("/login", status_code=302)
from_setup = bool(request.cookies.get("cx_setup_persona"))
return HTMLResponse(_model_page(from_setup=from_setup))
@router.post("/model", include_in_schema=False)
async def model_submit(
request: Request,
api_key: str = Form(...),
model_name: str = Form(...),
):
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
if not token:
return RedirectResponse("/login", status_code=302)
try:
username = decode_token(token)
except jwt.InvalidTokenError:
return RedirectResponse("/login", status_code=302)
api_key = api_key.strip()
model_name = model_name.strip()
if not api_key:
from_setup = bool(request.cookies.get("cx_setup_persona"))
return HTMLResponse(_model_page("API key is required.", from_setup=from_setup), status_code=422)
# Save OpenRouter as a host
host_id = model_registry.save_host(
username=username,
host_id=None,
label="OpenRouter",
api_url="https://openrouter.ai/api/v1",
api_key=api_key,
host_type="openai",
)
# Find label for selected model
label = next((lbl for mn, lbl in _OPENROUTER_MODELS if mn == model_name), model_name)
label = label.split("")[0] # keep just the model name part
# Save model entry
mid = model_registry.save_model(
username=username,
model_id=None,
host_id=host_id,
label=label,
model_name=model_name,
context_k=128,
tools=True,
)
# Assign as chat role primary
model_registry.set_role(username, "chat", "primary", mid)
logger.info("openrouter setup complete: %s%s", username, model_name)
# Redirect to chat (use remembered persona, or user root)
dest_cookie = request.cookies.get("cx_setup_persona", "")
dest = f"/{dest_cookie}" if dest_cookie else f"/{username}"
resp = RedirectResponse(dest, status_code=302)
resp.delete_cookie("cx_setup_persona")
return resp

View File

@@ -0,0 +1,394 @@
"""
Orchestrator router — POST /orchestrate, GET /orchestrate/{job_id}
Accepts a task description, runs it through the orchestrator engine
(Gemini tool loop → Claude response), and returns the result.
Designed to be triggered from:
- The Cortex web UI (future "Agent mode" toggle)
- Cron jobs: curl -X POST http://localhost:8000/orchestrate -d '{"task":"..."}'
- Webhooks: Gitea, Aether events, etc.
"""
import asyncio
import logging
import platform
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from auth_utils import get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
from config import settings
from context_loader import load_context
from persona import set_context, validate as validate_persona
import model_registry
import orchestrator_engine
import openai_orchestrator
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/orchestrate", tags=["orchestrator"])
# ---------------------------------------------------------------------------
# In-memory job store
# ---------------------------------------------------------------------------
_jobs: dict[str, dict] = {}
_jobs_lock = asyncio.Lock()
# Checkpoints are stored separately — they hold Python objects (types.Content, etc.)
# that can't be included in the JSON-serializable job dict.
_checkpoints: dict[str, orchestrator_engine.OrchestrateCheckpoint] = {}
_checkpoints_lock = asyncio.Lock()
# ---------------------------------------------------------------------------
# Request / response models
# ---------------------------------------------------------------------------
class OrchestrateRequest(BaseModel):
task: str
session_id: str | None = None # include session history in context
tier: int | None = None # Inara context tier (default from settings)
respond_with_claude: bool = True # False = return Gemini summary only (faster, for cron)
include_long: bool = True
include_mid: bool = True
include_short: bool = True
user: str = "scott"
persona: str = "inara"
chat_role: str = "chat" # role used for the final response (decoupled from tool-loop model)
off_record: bool = False # skip session log; inject OTR mode line into system prompt
class OrchestrateResponse(BaseModel):
job_id: str
status: str # "queued" | "running" | "complete" | "error" | "awaiting_confirmation"
class JobStatusResponse(BaseModel):
job_id: str
status: str
task: str
created_at: str
completed_at: str | None = None
session_id: str | None = None
response: str | None = None
tool_calls: list[dict] | None = None
backend: str | None = None
backend_label: str | None = None
host: str | None = None
gemini_summary: str | None = None
error: str | None = None
pending_confirmation: dict | None = None # {tools: [{name, args}], message: str}
# ---------------------------------------------------------------------------
# Endpoints
# ---------------------------------------------------------------------------
@router.post("", response_model=OrchestrateResponse)
async def orchestrate(req: OrchestrateRequest) -> OrchestrateResponse:
"""Submit a task to the orchestrator. Returns a job_id to poll."""
try:
user, persona = validate_persona(req.user, req.persona)
set_context(user, persona)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
job_id = str(uuid.uuid4())
now = datetime.now(timezone.utc).isoformat()
job: dict = {
"job_id": job_id,
"status": "queued",
"task": req.task,
"created_at": now,
"completed_at": None,
"session_id": None,
"response": None,
"tool_calls": None,
"backend": None,
"gemini_summary": None,
"error": None,
"pending_confirmation": None,
"_user": user,
"_off_record": req.off_record,
}
async with _jobs_lock:
_jobs[job_id] = job
asyncio.create_task(_run_job(job_id, req, user))
logger.info("Orchestrator job queued: %s%.80s", job_id, req.task)
return OrchestrateResponse(job_id=job_id, status="queued")
@router.get("/{job_id}", response_model=JobStatusResponse)
async def job_status(job_id: str) -> JobStatusResponse:
"""Poll the status of an orchestrator job."""
async with _jobs_lock:
job = _jobs.get(job_id)
if job is None:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
return JobStatusResponse(**{k: v for k, v in job.items() if not k.startswith("_")})
@router.get("", response_model=list[JobStatusResponse])
async def list_jobs() -> list[JobStatusResponse]:
"""List all jobs (most recent first). Useful for debugging."""
async with _jobs_lock:
jobs = sorted(_jobs.values(), key=lambda j: j["created_at"], reverse=True)
return [JobStatusResponse(**{k: v for k, v in j.items() if not k.startswith("_")}) for j in jobs]
@router.post("/{job_id}/confirm", response_model=OrchestrateResponse)
async def confirm_job(job_id: str) -> OrchestrateResponse:
"""Confirm a pending tool call — the blocked tool will execute and the job continues."""
async with _checkpoints_lock:
checkpoint = _checkpoints.pop(job_id, None)
if checkpoint is None:
raise HTTPException(status_code=404, detail="No pending confirmation for this job")
async with _jobs_lock:
job = _jobs.get(job_id)
if not job or job["status"] != "awaiting_confirmation":
raise HTTPException(status_code=409, detail="Job is not awaiting confirmation")
_jobs[job_id]["status"] = "running"
_jobs[job_id]["pending_confirmation"] = None
user = job.get("_user", "scott")
asyncio.create_task(_resume_job(job_id, checkpoint, confirmed=True, user=user))
logger.info("Orchestrator job %s confirmed — resuming", job_id)
return OrchestrateResponse(job_id=job_id, status="running")
@router.post("/{job_id}/deny", response_model=OrchestrateResponse)
async def deny_job(job_id: str) -> OrchestrateResponse:
"""Deny a pending tool call — the tool is skipped and the job produces a final response."""
async with _checkpoints_lock:
checkpoint = _checkpoints.pop(job_id, None)
if checkpoint is None:
raise HTTPException(status_code=404, detail="No pending confirmation for this job")
async with _jobs_lock:
job = _jobs.get(job_id)
if not job or job["status"] != "awaiting_confirmation":
raise HTTPException(status_code=409, detail="Job is not awaiting confirmation")
_jobs[job_id]["status"] = "running"
_jobs[job_id]["pending_confirmation"] = None
user = job.get("_user", "scott")
asyncio.create_task(_resume_job(job_id, checkpoint, confirmed=False, user=user))
logger.info("Orchestrator job %s denied — resuming with skip", job_id)
return OrchestrateResponse(job_id=job_id, status="running")
# ---------------------------------------------------------------------------
# Background runners
# ---------------------------------------------------------------------------
async def _run_job(job_id: str, req: OrchestrateRequest, user: str) -> None:
"""Execute the orchestration job and update the job store."""
async with _jobs_lock:
_jobs[job_id]["status"] = "running"
try:
from session_store import load as load_session, save as save_session, generate_session_id
tier = req.tier or settings.default_tier
role_cfg = model_registry.get_role_config(user, req.chat_role)
system_prompt = load_context(
tier,
include_long=req.include_long,
include_mid=req.include_mid,
include_short=req.include_short,
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
mode="otr" if req.off_record else "chat",
)
session_id = req.session_id or generate_session_id()
history = load_session(session_id)
session_messages = history or None
orch_model = model_registry.get_model_for_role(user, "orchestrator")
user_role = get_user_role(user)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(user)
confirm_allow = set(policy.get("allow", []))
confirm_deny = set(policy.get("deny", []))
max_risk, risk_wl, risk_bl = get_risk_policy(user)
if orch_model and orch_model.get("type") == "local_openai":
result = await openai_orchestrator.run(
task=req.task,
system_prompt=system_prompt,
session_messages=session_messages,
model_cfg=orch_model,
respond_with_final=req.respond_with_claude,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
else:
gemini_key = (
(orch_model.get("api_key") if orch_model else None)
or get_user_gemini_key(user)
)
result = await orchestrator_engine.run(
task=req.task,
system_prompt=system_prompt,
session_messages=session_messages,
respond_with_claude=req.respond_with_claude,
gemini_api_key=gemini_key,
model_name=orch_model.get("model_name") if orch_model else None,
response_role=req.chat_role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
max_rounds=orch_model.get("max_rounds") if orch_model else None,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
if result.checkpoint:
async with _checkpoints_lock:
_checkpoints[job_id] = result.checkpoint
async with _jobs_lock:
_jobs[job_id].update({
"status": "awaiting_confirmation",
"response": result.response,
"tool_calls": result.tool_calls,
"backend": result.backend,
"gemini_summary": result.gemini_summary,
"session_id": session_id,
"pending_confirmation": {
"tools": result.checkpoint.pending_tools,
"message": result.response,
},
})
logger.info("Orchestrator job %s awaiting confirmation — %d tool(s) blocked",
job_id, len(result.checkpoint.pending_tools))
return
await _finalize_job(job_id, result, session_id, req.task, history, off_record=req.off_record)
except Exception as e:
logger.exception("Orchestrator job failed: %s", job_id)
now = datetime.now(timezone.utc).isoformat()
async with _jobs_lock:
_jobs[job_id].update({
"status": "error",
"completed_at": now,
"error": str(e),
})
async def _resume_job(
job_id: str,
checkpoint: orchestrator_engine.OrchestrateCheckpoint,
confirmed: bool,
user: str,
) -> None:
"""Resume a job after the user confirms or denies a pending tool call."""
try:
if checkpoint.engine == "gemini":
result = await orchestrator_engine.resume(checkpoint, confirmed)
else:
result = await openai_orchestrator.resume(checkpoint, confirmed)
if result.checkpoint:
# Another confirmation needed (chained gates)
async with _checkpoints_lock:
_checkpoints[job_id] = result.checkpoint
async with _jobs_lock:
_jobs[job_id].update({
"status": "awaiting_confirmation",
"response": result.response,
"tool_calls": result.tool_calls,
"backend": result.backend,
"gemini_summary": result.gemini_summary,
"pending_confirmation": {
"tools": result.checkpoint.pending_tools,
"message": result.response,
},
})
logger.info("Orchestrator job %s awaiting another confirmation", job_id)
return
async with _jobs_lock:
session_id = _jobs[job_id].get("session_id") or ""
task = _jobs[job_id].get("task", "")
off_record = _jobs[job_id].get("_off_record", False)
from session_store import load as load_session
history = load_session(session_id) if session_id else []
await _finalize_job(job_id, result, session_id, task, history, off_record=off_record)
except Exception as e:
logger.exception("Orchestrator resume failed: %s", job_id)
now = datetime.now(timezone.utc).isoformat()
async with _jobs_lock:
_jobs[job_id].update({
"status": "error",
"completed_at": now,
"error": str(e),
})
async def _finalize_job(
job_id: str,
result: orchestrator_engine.OrchestratorResult,
session_id: str,
task: str,
history: list,
off_record: bool = False,
) -> None:
"""Save session, log the turn, and mark the job complete."""
from session_store import save as save_session, generate_session_id
from session_logger import log_turn
if not session_id:
session_id = generate_session_id()
host = platform.node()
history.append({"role": "user", "content": task, "off_record": off_record})
history.append({
"role": "assistant",
"content": result.response,
"backend": result.backend,
"backend_label": result.backend_label,
"host": host,
"off_record": off_record,
})
save_session(session_id, history)
if not off_record:
log_turn(session_id, task, result.response)
now = datetime.now(timezone.utc).isoformat()
async with _jobs_lock:
_jobs[job_id].update({
"status": "complete",
"completed_at": now,
"session_id": session_id,
"response": result.response,
"tool_calls": result.tool_calls,
"backend": result.backend,
"backend_label": result.backend_label,
"host": host,
"gemini_summary": result.gemini_summary,
})
logger.info("Orchestrator job complete: %s (%d tool calls)", job_id, len(result.tool_calls))

120
cortex/routers/push.py Normal file
View File

@@ -0,0 +1,120 @@
"""
Web Push endpoints.
GET /api/push/vapid-key → public VAPID key for browser PushManager.subscribe()
POST /api/push/subscribe → save a push subscription for the logged-in user
DELETE /api/push/subscribe → remove a subscription by endpoint
"""
import jwt
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel
from auth_utils import COOKIE_NAME, decode_token
from config import settings
import push_utils
router = APIRouter(prefix="/api/push")
def _require_user(request: Request) -> str:
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
return decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
@router.get("/vapid-key")
async def get_vapid_key() -> dict:
"""Return the VAPID public key. Public endpoint — needed before login to subscribe."""
key = settings.vapid_public_key
if not key:
raise HTTPException(status_code=503, detail="Push notifications not configured")
return {"public_key": key}
class SubscribeRequest(BaseModel):
subscription: dict # full PushSubscription JSON from browser
class UnsubscribeRequest(BaseModel):
endpoint: str
@router.post("/subscribe")
async def subscribe(req: SubscribeRequest, request: Request) -> dict:
username = _require_user(request)
sub = req.subscription
if not sub.get("endpoint"):
raise HTTPException(status_code=400, detail="subscription.endpoint is required")
push_utils.add_subscription(username, sub)
return {"ok": True}
@router.delete("/subscribe")
async def unsubscribe(req: UnsubscribeRequest, request: Request) -> dict:
username = _require_user(request)
found = push_utils.remove_subscription(username, req.endpoint)
return {"ok": True, "found": found}
@router.post("/test")
async def notify_test(request: Request) -> dict:
"""Send a test notification via the user's configured notification channel.
Useful for verifying channel setup (web push, NCT, email, etc.) without
waiting for a cron job or reminder to fire naturally.
"""
username = _require_user(request)
from notification import notify
await notify(username, "Test notification from Cortex — your notification channel is working.")
return {"ok": True, "user": username}
@router.post("/reminders/check")
async def reminder_check_now(request: Request) -> dict:
"""Run the reminder check for the current user immediately.
Same logic as the daily 09:00 scheduler job, but scoped to one user
and fired on demand. Returns how many reminders were found and whether
a notification was sent.
"""
import re
username = _require_user(request)
from persona import list_user_personas, set_context
from notification import notify
total_sent = 0
for persona_name in list_user_personas(username):
set_context(username, persona_name)
from tools.reminders import load_due_reminders
content = load_due_reminders()
if not content:
continue
entries = []
for line in content.splitlines():
m = re.match(r"^\d+\.\s+(.+)", line.strip())
if m:
text = re.sub(r"\[(OVERDUE|due TODAY|due: \S+)\]", "", m.group(1)).strip()
if text:
entries.append(text)
if not entries:
continue
count = len(entries)
if count == 1:
msg = f"Reminder: {entries[0]}"
else:
bullet_list = "\n".join(f"{e}" for e in entries[:3])
tail = f"\n…and {count - 3} more" if count > 3 else ""
msg = f"{count} reminders due:\n{bullet_list}{tail}"
await notify(username, msg)
total_sent += count
return {"ok": True, "user": username, "reminders_found": total_sent}

499
cortex/routers/settings.py Normal file
View File

@@ -0,0 +1,499 @@
"""
Account settings router.
Routes:
GET /settings → show account settings page (requires auth)
POST /settings/password → change password
POST /settings/username → rename the user account (forces re-login)
POST /settings/persona/rename → rename a persona directory
"""
import html as _html
import json
import logging
import re
from pathlib import Path
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, check_credentials, set_password, _read_auth, _write_auth, get_user_channels
from persona import list_user_personas
from config import settings as app_settings
_SLUG_RE = re.compile(r"^[a-z_][a-z0-9_-]{0,31}$")
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
def _integrations_nav(username: str) -> str:
"""Return the Integrations nav link for admin users, empty string otherwise."""
role = _read_auth(username).get("role", "user")
if role == "admin":
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
return ""
def _notifications_page(username: str, back_persona: str = "", success: str = "", error: str = "") -> str:
html = (_STATIC / "notifications.html").read_text()
channels = get_user_channels(username)
nct = channels.get("nextcloud") or {}
notify_ch = _html.escape(channels.get("notification_channel", "") or "")
notify_email = _html.escape(channels.get("notification_email", "") or "")
nc_url = _html.escape(nct.get("url", "") or "")
nc_bot_secret = _html.escape(nct.get("bot_secret", "") or "")
nc_room = _html.escape(nct.get("notification_room", "") or "")
nc_username = _html.escape(nct.get("nc_username", "") or "")
nc_app_password = _html.escape(nct.get("nc_app_password", "") or "")
gc_webhook = _html.escape((channels.get("google_chat") or {}).get("outbound_webhook", "") or "")
ha = channels.get("homeassistant") or {}
ha_url = _html.escape(ha.get("url", "") or "")
ha_webhook_id = _html.escape(ha.get("webhook_id", "") or "")
ha_tools_checked = "checked" if ha.get("tools", False) else ""
html = html.replace("{{ notify_channel }}", notify_ch)
html = html.replace("{{ notify_email_override }}", notify_email)
html = html.replace("{{ nc_url }}", nc_url)
html = html.replace("{{ nc_bot_secret }}", nc_bot_secret)
html = html.replace("{{ nc_notify_room }}", nc_room)
html = html.replace("{{ nc_username }}", nc_username)
html = html.replace("{{ nc_app_password }}", nc_app_password)
html = html.replace("{{ gc_webhook }}", gc_webhook)
html = html.replace("{{ ha_url }}", ha_url)
html = html.replace("{{ ha_webhook_id }}", ha_webhook_id)
html = html.replace("{{ ha_tools_checked }}", ha_tools_checked)
html = html.replace("{{ ha_username }}", username)
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
def _settings_page(username: str, personas: list[str], back_persona: str = "", success: str = "", error: str = "") -> str:
html = (_STATIC / "settings.html").read_text()
html = html.replace("{{ username }}", username)
# Connected Google account (OAuth sign-in)
auth_data = _read_auth(username)
google_email = auth_data.get("google_email") or ""
html = html.replace("{{ google_email }}", google_email)
role = auth_data.get("role", "user")
html = html.replace("{{ user_role }}", role)
al_path = app_settings.home_root() / username / "email_allowlist.json"
try:
patterns = json.loads(al_path.read_text())
allowlist_text = _html.escape("\n".join(str(p) for p in patterns if str(p).strip()))
except Exception:
allowlist_text = ""
html = html.replace("{{ email_allowlist }}", allowlist_text)
http_al_path = app_settings.home_root() / username / "http_allowlist.json"
try:
http_prefixes = json.loads(http_al_path.read_text())
http_allowlist_text = _html.escape("\n".join(str(p) for p in http_prefixes if str(p).strip()))
except Exception:
http_allowlist_text = ""
html = html.replace("{{ http_allowlist }}", http_allowlist_text)
persona_items = "\n".join(
f'''<li>
<a href="/{username}/{p}" class="persona-link">{p}</a>
<button class="persona-rename-toggle" data-persona="{p}" title="Rename">✏</button>
<form class="persona-rename-form" data-persona="{p}"
method="POST" action="/settings/persona/rename" style="display:none">
<input type="hidden" name="old_name" value="{p}">
<input type="text" name="new_name" value="{p}"
pattern="[a-z_][a-z0-9_\\-]{{0,31}}" required>
<button type="submit" class="btn-save">Save</button>
<button type="button" class="btn-cancel persona-rename-cancel">Cancel</button>
</form>
</li>''' for p in personas
)
html = html.replace("{{ persona_items }}", persona_items or "<li><em>No personas yet.</em></li>")
if not back_persona:
back_persona = personas[0] if personas else ""
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
def _integrations_page(username: str, back_persona: str = "", success: str = "", error: str = "") -> str:
html = (_STATIC / "integrations.html").read_text()
channels = get_user_channels(username)
ae_db = channels.get("aether_db") or {}
html = html.replace("{{ ae_db_host }}", _html.escape(ae_db.get("host", "") or ""))
html = html.replace("{{ ae_db_port }}", _html.escape(str(ae_db.get("port", 3306))))
html = html.replace("{{ ae_db_name }}", _html.escape(ae_db.get("name", "") or ""))
html = html.replace("{{ ae_db_user }}", _html.escape(ae_db.get("user", "") or ""))
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
@router.get("/settings", include_in_schema=False)
async def settings_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_settings_page(username, personas, back_persona=back_persona))
@router.post("/settings/password", include_in_schema=False)
async def change_password(
request: Request,
current_password: str = Form(...),
new_password: str = Form(...),
confirm_password: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
if not check_credentials(username, current_password):
return HTMLResponse(_settings_page(username, personas, back_persona, error="Current password is incorrect."))
if len(new_password) < 8:
return HTMLResponse(_settings_page(username, personas, back_persona, error="New password must be at least 8 characters."))
if new_password != confirm_password:
return HTMLResponse(_settings_page(username, personas, back_persona, error="New passwords do not match."))
set_password(username, new_password)
logger.info("password changed: %s", username)
return HTMLResponse(_settings_page(username, personas, back_persona, success="Password updated successfully."))
@router.post("/settings/username", include_in_schema=False)
async def rename_username(
request: Request,
new_username: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
new_username = new_username.strip().lower()
if not _SLUG_RE.match(new_username):
return HTMLResponse(_settings_page(
username, personas, back_persona,
error="Invalid username. Use lowercase letters, digits, _ or - only."))
if new_username == username:
return RedirectResponse("/settings", status_code=302)
home_root = app_settings.home_root()
old_dir = home_root / username
new_dir = home_root / new_username
if new_dir.exists():
return HTMLResponse(_settings_page(
username, personas, back_persona,
error=f"Username '{new_username}' is already taken."))
old_dir.rename(new_dir)
logger.info("user renamed: %s%s", username, new_username)
# Clear the auth cookie — old JWT now refers to a non-existent user
resp = RedirectResponse("/login?msg=username_changed", status_code=302)
resp.delete_cookie(COOKIE_NAME)
return resp
@router.post("/settings/gemini-key", include_in_schema=False)
async def save_gemini_key(
request: Request,
gemini_api_key: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
gemini_api_key = gemini_api_key.strip()
data = _read_auth(username)
if gemini_api_key:
data["gemini_api_key"] = gemini_api_key
msg = "Gemini API key saved."
else:
data.pop("gemini_api_key", None)
msg = "Gemini API key removed — using server key."
_write_auth(username, data)
logger.info("gemini key updated: %s", username)
return HTMLResponse(_settings_page(username, personas, back_persona, success=msg))
@router.post("/settings/persona/rename", include_in_schema=False)
async def rename_persona(
request: Request,
old_name: str = Form(...),
new_name: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
new_name = new_name.strip().lower()
if not _SLUG_RE.match(new_name):
return HTMLResponse(_settings_page(
username, personas, back_persona,
error="Invalid name. Use lowercase letters, digits, _ or - only."))
if new_name == old_name:
return RedirectResponse("/settings", status_code=302)
persona_root = app_settings.home_root() / username / "persona"
old_dir = persona_root / old_name
new_dir = persona_root / new_name
if not old_dir.exists():
return HTMLResponse(_settings_page(username, personas, back_persona, error=f"Persona '{old_name}' not found."))
if new_dir.exists():
return HTMLResponse(_settings_page(
username, personas, back_persona,
error=f"A persona named '{new_name}' already exists."))
old_dir.rename(new_dir)
logger.info("persona renamed: %s/%s%s", username, old_name, new_name)
return RedirectResponse("/settings", status_code=302)
@router.get("/settings/notifications", include_in_schema=False)
async def notifications_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_notifications_page(username, back_persona))
@router.post("/settings/notifications", include_in_schema=False)
async def save_notifications(
request: Request,
notification_channel: str = Form(""),
notification_email: str = Form(""),
nc_url: str = Form(""),
nc_bot_secret: str = Form(""),
nc_notification_room: str = Form(""),
nc_username: str = Form(""),
nc_app_password: str = Form(""),
gc_outbound_webhook: str = Form(""),
ha_url: str = Form(""),
ha_token: str = Form(""),
ha_webhook_id: str = Form(""),
ha_tools: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
channels_path = app_settings.home_root() / username / "channels.json"
try:
channels = json.loads(channels_path.read_text())
except Exception:
channels = {}
# Top-level notification preference
notification_channel = notification_channel.strip()
if notification_channel in ("web_push", "email", "nextcloud", "google_chat"):
channels["notification_channel"] = notification_channel
else:
channels.pop("notification_channel", None)
# Optional email address override (blank = use login email)
notification_email = notification_email.strip()
if notification_email:
channels["notification_email"] = notification_email
else:
channels.pop("notification_email", None)
# Nextcloud Talk — full config nested under "nextcloud"
if "nextcloud" not in channels:
channels["nextcloud"] = {}
nct = channels["nextcloud"]
if nc_url.strip():
nct["url"] = nc_url.strip().rstrip("/")
# Only overwrite secrets if a new value was provided (blank = keep existing)
if nc_bot_secret.strip():
nct["bot_secret"] = nc_bot_secret.strip()
nct["notification_room"] = nc_notification_room.strip()
if nc_username.strip():
nct["nc_username"] = nc_username.strip()
if nc_app_password.strip():
nct["nc_app_password"] = nc_app_password.strip()
# Google Chat outbound webhook — nested under "google_chat"
if "google_chat" not in channels:
channels["google_chat"] = {}
channels["google_chat"]["outbound_webhook"] = gc_outbound_webhook.strip()
# Home Assistant — nested under "homeassistant"
if "homeassistant" not in channels:
channels["homeassistant"] = {}
ha = channels["homeassistant"]
if ha_url.strip():
ha["url"] = ha_url.strip().rstrip("/")
if ha_token.strip():
ha["token"] = ha_token.strip()
if ha_webhook_id.strip():
ha["webhook_id"] = ha_webhook_id.strip()
ha["tools"] = ha_tools == "1"
channels_path.write_text(json.dumps(channels, indent=2) + "\n")
logger.info("notifications updated for %s (channel=%s)", username, notification_channel or "none")
return HTMLResponse(_notifications_page(username, back_persona, success="Notification settings saved."))
@router.post("/settings/email-allowlist", include_in_schema=False)
async def save_email_allowlist(
request: Request,
patterns: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
lines = [ln.strip() for ln in patterns.splitlines() if ln.strip()]
path = app_settings.home_root() / username / "email_allowlist.json"
path.write_text(json.dumps(lines, indent=2))
logger.info("email allowlist updated for %s (%d patterns)", username, len(lines))
return HTMLResponse(_settings_page(username, personas, back_persona, success=f"Email allowlist saved ({len(lines)} pattern{'s' if len(lines) != 1 else ''})."))
@router.post("/settings/http-allowlist", include_in_schema=False)
async def save_http_allowlist(
request: Request,
prefixes: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
lines = [ln.strip() for ln in prefixes.splitlines() if ln.strip()]
path = app_settings.home_root() / username / "http_allowlist.json"
path.write_text(json.dumps(lines, indent=2))
logger.info("http allowlist updated for %s (%d prefixes)", username, len(lines))
return HTMLResponse(_settings_page(username, personas, back_persona, success=f"HTTP allowlist saved ({len(lines)} prefix{'es' if len(lines) != 1 else ''})."))
def _require_admin(username: str) -> bool:
return _read_auth(username).get("role", "user") == "admin"
@router.get("/settings/integrations", include_in_schema=False)
async def integrations_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not _require_admin(username):
return RedirectResponse("/settings", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_integrations_page(username, back_persona))
@router.post("/settings/integrations", include_in_schema=False)
async def save_integrations(
request: Request,
ae_db_host: str = Form(""),
ae_db_port: str = Form("3306"),
ae_db_name: str = Form(""),
ae_db_user: str = Form(""),
ae_db_password: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not _require_admin(username):
return RedirectResponse("/settings", status_code=302)
back_persona = _preferred_persona(request, username)
channels_path = app_settings.home_root() / username / "channels.json"
try:
channels = json.loads(channels_path.read_text())
except Exception:
channels = {}
if "aether_db" not in channels:
channels["aether_db"] = {}
db = channels["aether_db"]
if ae_db_host.strip():
db["host"] = ae_db_host.strip()
try:
db["port"] = int(ae_db_port.strip()) if ae_db_port.strip() else 3306
except ValueError:
db["port"] = 3306
if ae_db_name.strip():
db["name"] = ae_db_name.strip()
if ae_db_user.strip():
db["user"] = ae_db_user.strip()
if ae_db_password.strip():
db["password"] = ae_db_password.strip()
channels_path.write_text(json.dumps(channels, indent=2) + "\n")
logger.info("integrations updated for %s", username)
return HTMLResponse(_integrations_page(username, back_persona, success="Integration settings saved."))

View File

@@ -0,0 +1,193 @@
"""
Tool settings router.
Routes:
GET /settings/tools → tool risk policy page
POST /settings/tools → save max_risk + per-tool overrides
"""
import html as _html
import json
import logging
from pathlib import Path
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, get_tool_policy, save_tool_policy, _read_auth
from persona import list_user_personas
from tools import TOOL_CATEGORIES, TOOL_RISK, CONFIRM_REQUIRED
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
return cookie_val if cookie_val in names else (names[0] if names else "")
def _build_tool_table(policy: dict) -> str:
"""Generate the per-tool override table rows grouped by category."""
whitelist = set(policy.get("whitelist") or [])
blacklist = set(policy.get("blacklist") or [])
rows: list[str] = []
for category, tools in TOOL_CATEGORIES.items():
# Category header spanning all columns
escaped_cat = _html.escape(category)
rows.append(f'<tr class="tool-cat-row"><td colspan="4">{escaped_cat}</td></tr>')
for tool in tools:
risk = TOOL_RISK.get(tool, "medium")
risk_cls = f"risk-{risk}"
risk_html = f'<span class="risk {risk_cls}">{_html.escape(risk)}</span>'
# Override select value
if tool in whitelist:
override_val = "whitelist"
elif tool in blacklist:
override_val = "blacklist"
else:
override_val = "default"
def _opt(val: str, label: str) -> str:
sel = 'selected' if override_val == val else ''
return f'<option value="{val}" {sel}>{label}</option>'
override_sel = (
f'<select name="override_{_html.escape(tool)}" '
f'class="override-sel" data-tool="{_html.escape(tool)}">'
+ _opt("default", "Default (auto)")
+ _opt("whitelist", "Force include")
+ _opt("blacklist", "Force exclude")
+ '</select>'
)
rows.append(
f'<tr data-tool-risk="{_html.escape(risk)}">'
f'<td class="tool-name">{_html.escape(tool)}</td>'
f'<td>{risk_html}</td>'
f'<td><span class="auto-pill"></span></td>'
f'<td>{override_sel}</td>'
f'</tr>'
)
table_body = "\n".join(rows)
return (
'<table class="tool-table">'
'<thead><tr>'
'<th>Tool</th><th>Risk</th><th>Auto status</th><th>Override</th>'
'</tr></thead>'
f'<tbody>{table_body}</tbody>'
'</table>'
)
def _tools_page(
username: str,
back_persona: str = "",
success: str = "",
error: str = "",
) -> str:
html = (_STATIC / "tools_settings.html").read_text()
policy = get_tool_policy(username)
max_risk = policy.get("max_risk") or ""
# Max risk select options
html = html.replace("{{ sel_none }}", "selected" if max_risk == "" else "")
html = html.replace("{{ sel_low }}", "selected" if max_risk == "low" else "")
html = html.replace("{{ sel_medium }}", "selected" if max_risk == "medium" else "")
html = html.replace("{{ sel_high }}", "selected" if max_risk == "high" else "")
html = html.replace("{{ tool_table_html }}", _build_tool_table(policy))
html = html.replace("{{ tool_risk_json }}", json.dumps(TOOL_RISK))
html = html.replace("{{ confirm_required_tools }}", _html.escape(", ".join(sorted(CONFIRM_REQUIRED))))
html = html.replace("{{ tool_allow }}", _html.escape("\n".join(policy.get("allow") or [])))
html = html.replace("{{ tool_deny }}", _html.escape("\n".join(policy.get("deny") or [])))
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
nav = '<a href="/settings/integrations" class="nav-link">Integrations</a>' \
if _read_auth(username).get("role", "user") == "admin" else ""
html = html.replace("{{ integrations_nav }}", nav)
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
@router.get("/settings/tools", include_in_schema=False)
async def tools_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_tools_page(username, back_persona))
@router.post("/settings/tools", include_in_schema=False)
async def save_tools(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
form = await request.form()
max_risk = (form.get("max_risk") or "").strip()
if max_risk not in ("", "low", "medium", "high"):
max_risk = ""
whitelist: list[str] = []
blacklist: list[str] = []
all_tools = [t for tools in TOOL_CATEGORIES.values() for t in tools]
for tool in all_tools:
val = (form.get(f"override_{tool}") or "").strip()
if val == "whitelist":
whitelist.append(tool)
elif val == "blacklist":
blacklist.append(tool)
allow_tools = [ln.strip() for ln in (form.get("allow_list") or "").splitlines() if ln.strip()]
deny_tools = [ln.strip() for ln in (form.get("deny_list") or "").splitlines() if ln.strip()]
policy = get_tool_policy(username)
if max_risk:
policy["max_risk"] = max_risk
else:
policy.pop("max_risk", None)
policy["whitelist"] = whitelist
policy["blacklist"] = blacklist
policy["allow"] = allow_tools
policy["deny"] = deny_tools
save_tool_policy(username, policy)
logger.info(
"tool policy saved for %s: max_risk=%s whitelist=%d blacklist=%d allow=%d deny=%d",
username, max_risk or "none", len(whitelist), len(blacklist), len(allow_tools), len(deny_tools),
)
return HTMLResponse(_tools_page(
username, back_persona,
success=f"Tool policy saved — max risk: {max_risk or 'none'}, "
f"{len(whitelist)} whitelisted, {len(blacklist)} blacklisted.",
))

328
cortex/routers/ui.py Normal file
View File

@@ -0,0 +1,328 @@
"""
UI router — serves the web interface and handles login/logout.
Routes:
GET / → redirect to /{user}/{persona} if logged in, else /login
GET /login → login page
POST /login → validate credentials, set cookie, redirect
POST /logout → clear cookie, redirect to /login
GET /{user}/{persona} → serve index.html with CORTEX_CONFIG injected
GET /{user}/{persona}/ → same (trailing slash)
"""
import logging
import re
from pathlib import Path
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse, Response
from auth_utils import COOKIE_NAME, check_credentials, create_token, decode_token
from persona import list_users, list_user_personas, validate as validate_persona, persona_path
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _get_session_user(request: Request) -> str | None:
"""Return the authenticated username from the session cookie, or None."""
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _set_cookie(response: Response, username: str) -> None:
from auth_utils import create_token
from config import settings
token = create_token(username)
response.set_cookie(
COOKIE_NAME,
token,
max_age=settings.jwt_expire_days * 86400,
httponly=True,
samesite="lax",
secure=False, # set True in production behind HTTPS
)
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _first_persona(username: str) -> str | None:
"""Return the first available persona for a user, or None."""
names = list_user_personas(username)
return names[0] if names else None
def _preferred_persona(request: Request, username: str) -> str | None:
"""Return the last-visited persona from cookie if valid, else the first available."""
names = list_user_personas(username)
if not names:
return None
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
# ---------------------------------------------------------------------------
# Favicon — default sparkle; persona pages override via JS
# ---------------------------------------------------------------------------
_FAVICON_SVG = (
"<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'>"
"<text y='.9em' font-size='90'>✨</text></svg>"
)
@router.get("/favicon.ico", include_in_schema=False)
async def favicon():
return Response(content=_FAVICON_SVG, media_type="image/svg+xml")
@router.get("/sw.js", include_in_schema=False)
async def service_worker():
from fastapi.responses import FileResponse
return FileResponse(str(_STATIC / "sw.js"), media_type="application/javascript")
@router.get("/manifest.json", include_in_schema=False)
async def web_manifest():
from fastapi.responses import FileResponse
return FileResponse(str(_STATIC / "manifest.json"), media_type="application/manifest+json")
# ---------------------------------------------------------------------------
# Root redirect
# ---------------------------------------------------------------------------
@router.get("/", include_in_schema=False)
async def root(request: Request):
user = _get_session_user(request)
if not user:
return RedirectResponse("/login", status_code=302)
persona = _preferred_persona(request, user)
if not persona:
return HTMLResponse("<h1>No personas configured for your account.</h1>", status_code=500)
return RedirectResponse(f"/{user}/{persona}", status_code=302)
# ---------------------------------------------------------------------------
# Login / logout
# ---------------------------------------------------------------------------
@router.get("/login", include_in_schema=False)
async def login_page(request: Request):
user = _get_session_user(request)
if user:
# Already logged in — redirect home
persona = _preferred_persona(request, user)
if persona:
return RedirectResponse(f"/{user}/{persona}", status_code=302)
return HTMLResponse((_STATIC / "login.html").read_text())
@router.post("/login", include_in_schema=False)
async def login(
request: Request,
username: str = Form(...),
password: str = Form(...),
):
if not check_credentials(username, password):
logger.warning("failed login attempt for user: %s", username)
html = (_STATIC / "login.html").read_text().replace(
"<!-- ERROR -->",
'<p class="error">Invalid username or password.</p>',
)
return HTMLResponse(html, status_code=401)
persona = _first_persona(username)
if not persona:
return HTMLResponse("<h1>No personas configured for your account.</h1>", status_code=500)
logger.info("login: %s", username)
resp = RedirectResponse(f"/{username}/{persona}", status_code=302)
_set_cookie(resp, username)
return resp
@router.post("/logout", include_in_schema=False)
async def logout():
resp = RedirectResponse("/login", status_code=302)
resp.delete_cookie(COOKIE_NAME)
return resp
# ---------------------------------------------------------------------------
# User landing — /{username} → persona picker
# ---------------------------------------------------------------------------
@router.get("/{username}", include_in_schema=False)
async def user_landing(username: str, request: Request):
session_user = _get_session_user(request)
if not session_user:
return RedirectResponse("/login", status_code=302)
if session_user != username:
return RedirectResponse(f"/{session_user}", status_code=302)
personas = list_user_personas(username)
if not personas:
return HTMLResponse("<h1>No personas configured.</h1>", status_code=404)
cards_html = ""
for p in personas:
emoji = ""
identity_path = persona_path(username, p) / "IDENTITY.md"
if identity_path.exists():
m = re.search(r"\|\s*Emoji\s*\|\s*(.+?)\s*\|", identity_path.read_text())
if m:
emoji = m.group(1).strip()
cards_html += (
f'<a href="/{username}/{p}" class="persona-card">'
f'<span class="p-emoji">{emoji}</span>'
f'<span class="p-name">{p.capitalize()}</span>'
f'</a>\n'
)
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — {username}</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after {{ box-sizing: border-box; margin: 0; padding: 0; }}
body {{
min-height: 100vh;
display: flex;
align-items: center;
justify-content: center;
background: #1a1228;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
color: #e8e0f0;
padding: 2rem 1.5rem;
}}
.card {{
background: #221840;
border: 1px solid #3a2852;
border-radius: 14px;
padding: 2.5rem 2rem;
width: 100%;
max-width: 400px;
text-align: center;
}}
h1 {{ font-size: 1.3rem; font-weight: 700; color: #c4935a; margin-bottom: 0.4rem; }}
.sub {{ font-size: 0.82rem; color: #b0a2c8; margin-bottom: 2rem; }}
.personas {{ display: flex; flex-direction: column; gap: 0.75rem; }}
.persona-card {{
display: flex;
align-items: center;
gap: 1rem;
padding: 0.85rem 1.2rem;
background: #1a1228;
border: 1px solid #3a2852;
border-radius: 10px;
color: #e8e0f0;
text-decoration: none;
font-size: 1rem;
font-weight: 500;
transition: border-color 0.15s, background 0.15s;
}}
.persona-card:hover {{ border-color: #c4935a; background: #261d42; }}
.p-emoji {{ font-size: 1.6rem; line-height: 1; }}
.p-name {{ color: #c4935a; font-weight: 600; }}
.settings-link {{
display: inline-block;
margin-top: 1.5rem;
font-size: 0.78rem;
color: #b0a2c8;
text-decoration: none;
}}
.settings-link:hover {{ color: #e8e0f0; }}
</style>
</head>
<body>
<div class="card">
<h1>Cortex</h1>
<p class="sub">Signed in as <strong>{username}</strong> — choose a persona</p>
<div class="personas">
{cards_html} </div>
<a href="/settings" class="settings-link">Account settings</a>
</div>
</body>
</html>"""
return HTMLResponse(html)
# ---------------------------------------------------------------------------
# Main UI — /{username}/{persona}
# ---------------------------------------------------------------------------
@router.get("/api/personas", tags=["ui"])
async def api_personas(request: Request) -> dict:
"""Return the list of personas for the current session user."""
user = _get_session_user(request)
if not user:
from fastapi import HTTPException
raise HTTPException(status_code=401, detail="Not authenticated")
personas_with_emoji = []
for p in list_user_personas(user):
emoji = ""
identity_path = persona_path(user, p) / "IDENTITY.md"
if identity_path.exists():
m = re.search(r"\|\s*Emoji\s*\|\s*(.+?)\s*\|", identity_path.read_text())
if m:
emoji = m.group(1).strip()
personas_with_emoji.append({"name": p, "emoji": emoji})
return {"user": user, "personas": personas_with_emoji}
@router.get("/{username}/{persona}", include_in_schema=False)
@router.get("/{username}/{persona}/", include_in_schema=False)
async def serve_ui(username: str, persona: str, request: Request):
# Auth check
session_user = _get_session_user(request)
if not session_user:
return RedirectResponse("/login", status_code=302)
if session_user != username:
return RedirectResponse(f"/{session_user}/{_first_persona(session_user) or ''}", status_code=302)
# Validate persona exists
try:
validate_persona(username, persona)
except ValueError:
return RedirectResponse(f"/{username}/{_first_persona(username) or ''}", status_code=302)
# Read emoji from IDENTITY.md (| Emoji | <value> | line)
emoji = ""
identity_path = persona_path(username, persona) / "IDENTITY.md"
if identity_path.exists():
m = re.search(r"\|\s*Emoji\s*\|\s*(.+?)\s*\|", identity_path.read_text())
if m:
emoji = m.group(1).strip()
# Serve index.html with user/persona/emoji injected
html = (_STATIC / "index.html").read_text()
config_tag = (
f'<script>window.CORTEX_CONFIG = '
f'{{user: "{username}", persona: "{persona}", emoji: "{emoji}"}};</script>'
)
html = html.replace("</head>", f"{config_tag}\n</head>", 1)
resp = HTMLResponse(html)
resp.set_cookie(_LAST_PERSONA_COOKIE, persona, max_age=365 * 86400, httponly=False, samesite="lax")
return resp

104
cortex/routers/usage.py Normal file
View File

@@ -0,0 +1,104 @@
"""
Usage / token-tracking endpoints.
Self-service (any authenticated user, own data):
GET /api/usage → full usage dict {date: {model_key: {calls, prompt_tokens, completion_tokens}}}
GET /api/usage/summary → aggregate totals per model key, with friendly labels resolved from registry
Admin-only (cross-user aggregation):
GET /api/usage/all → summary for every user {username: summary_dict}
"""
import jwt
from fastapi import APIRouter, HTTPException, Request
from auth_utils import COOKIE_NAME, decode_token, get_user_role
from persona import list_users
import model_registry
import usage_tracker
router = APIRouter(prefix="/api/usage")
def _session_user(request: Request) -> str:
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
return decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
def _build_label_map(username: str) -> dict[str, str]:
"""Build a map from usage key (backend/model_name) → registered label."""
label_map: dict[str, str] = {}
try:
for m in model_registry.get_all_models(username):
model_name = m.get("model_name", "")
label = m.get("label", "")
host_type = m.get("host_type", "")
if not model_name or not label:
continue
# local models: key is "local/{model_name}"
if host_type in ("openwebui", "ollama", "openai_compatible"):
label_map[f"local/{model_name}"] = label
# cloud Gemini: key is "gemini_api/{model_name}"
elif host_type == "google":
label_map[f"gemini_api/{model_name}"] = label
except Exception:
pass
return label_map
def _summarize(data: dict, label_map: dict[str, str] | None = None) -> list[dict]:
"""Collapse date-keyed usage dict into per-model totals, sorted by total tokens desc."""
totals: dict[str, dict] = {}
for _date, models in data.items():
for key, counts in models.items():
t = totals.setdefault(key, {"calls": 0, "prompt_tokens": 0, "completion_tokens": 0})
t["calls"] += counts.get("calls", 0)
t["prompt_tokens"] += counts.get("prompt_tokens", 0)
t["completion_tokens"] += counts.get("completion_tokens", 0)
result = []
for key, counts in totals.items():
entry = {
"key": key,
"label": (label_map or {}).get(key) or key,
"calls": counts["calls"],
"prompt_tokens": counts["prompt_tokens"],
"completion_tokens": counts["completion_tokens"],
"total_tokens": counts["prompt_tokens"] + counts["completion_tokens"],
}
result.append(entry)
result.sort(key=lambda x: x["total_tokens"], reverse=True)
return result
@router.get("")
async def get_usage(request: Request) -> dict:
"""Return the raw daily usage log for the authenticated user."""
username = _session_user(request)
return usage_tracker.read_usage(username)
@router.get("/summary")
async def get_usage_summary(request: Request) -> list:
"""Return per-model totals (all time) for the authenticated user, with friendly labels."""
username = _session_user(request)
label_map = _build_label_map(username)
return _summarize(usage_tracker.read_usage(username), label_map)
@router.get("/all")
async def get_all_usage(request: Request) -> dict:
"""Admin: return per-model summary for every user."""
username = _session_user(request)
if get_user_role(username) != "admin":
raise HTTPException(status_code=403, detail="Admin access required")
result = {}
for user in list_users():
label_map = _build_label_map(user)
result[user] = _summarize(usage_tracker.read_usage(user), label_map)
return result

205
cortex/scheduler.py Normal file
View File

@@ -0,0 +1,205 @@
"""
Auto memory distillation scheduler.
Default schedule (all overridable via .env flags):
short — daily at 03:00 (no LLM — fast)
mid — weekly Sun at 03:30 (LLM call)
long — monthly 1st at 04:00 (LLM call — off by default)
Set AUTO_DISTILL=false to disable entirely.
Set AUTO_DISTILL_LONG=true to enable monthly long-term integration.
"""
import logging
from zoneinfo import ZoneInfo
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from config import settings
logger = logging.getLogger(__name__)
_scheduler: AsyncIOScheduler | None = None
def _all_personas() -> list[tuple[str, str]]:
"""Return [(username, persona_name)] for every persona on this instance."""
from persona import list_users, list_user_personas
pairs = []
for u in list_users():
for p in list_user_personas(u):
pairs.append((u, p))
return pairs
async def _run_short() -> None:
from memory_distiller import distill_short
for u, p in _all_personas():
try:
result = distill_short(u, p)
logger.info("auto distill short [%s/%s]: %d files, %d chars", u, p, result["files_included"], result["chars_written"])
except Exception as e:
logger.error("auto distill short [%s/%s] failed: %s", u, p, e)
async def _run_mid() -> None:
from memory_distiller import distill_mid
from notification import notify
for u, p in _all_personas():
try:
result = await distill_mid(u, p)
if "error" in result:
logger.warning("auto distill mid [%s/%s] skipped: %s", u, p, result["error"])
else:
logger.info("auto distill mid [%s/%s]: %d chars via %s", u, p, result["chars_written"], result["backend"])
await notify(u, f"📝 Weekly memory digest complete ({result['chars_written']} chars via {result['backend']}).")
except Exception as e:
logger.error("auto distill mid [%s/%s] failed: %s", u, p, e)
async def _run_long() -> None:
from memory_distiller import distill_long
from notification import notify
for u, p in _all_personas():
try:
result = await distill_long(u, p)
if "error" in result:
logger.warning("auto distill long [%s/%s] skipped: %s", u, p, result["error"])
else:
logger.info("auto distill long [%s/%s]: %d chars via %s", u, p, result["chars_written"], result["backend"])
await notify(u, f"🧠 Monthly long-term memory integration complete ({result['chars_written']} chars via {result['backend']}). Worth a quick review.")
except Exception as e:
logger.error("auto distill long [%s/%s] failed: %s", u, p, e)
async def _run_reminder_check() -> None:
"""Notify users of any due or overdue reminders (fires once daily at 09:00)."""
import re
from notification import notify
from persona import set_context
for u, p in _all_personas():
try:
set_context(u, p)
from tools.reminders import load_due_reminders
content = load_due_reminders()
if not content:
continue
# Extract numbered entries (lines like "1. [label] text" or "1. text")
entries = []
for line in content.splitlines():
m = re.match(r"^\d+\.\s+(.+)", line.strip())
if m:
# Strip status tags ([OVERDUE], [due TODAY], etc.) for display
text = re.sub(r"\[(OVERDUE|due TODAY|due: \S+)\]", "", m.group(1)).strip()
if text:
entries.append(text)
if not entries:
continue
count = len(entries)
if count == 1:
msg = f"Reminder: {entries[0]}"
else:
bullet_list = "\n".join(f"{e}" for e in entries[:3])
tail = f"\n…and {count - 3} more" if count > 3 else ""
msg = f"{count} reminders due:\n{bullet_list}{tail}"
await notify(u, msg)
logger.info("reminder check [%s/%s]: notified %d reminder(s)", u, p, count)
except Exception as e:
logger.error("reminder check [%s/%s] failed: %s", u, p, e)
def get_scheduler() -> AsyncIOScheduler | None:
"""Return the running scheduler instance (used by cron tools for live add/remove)."""
return _scheduler
def start() -> None:
global _scheduler
_scheduler = AsyncIOScheduler(timezone=ZoneInfo(settings.scheduler_timezone))
if not settings.auto_distill:
logger.info("auto distillation disabled (AUTO_DISTILL=false)")
if settings.auto_distill_short:
_scheduler.add_job(_run_short, "cron", hour=3, minute=0, id="distill_short")
logger.info("scheduled: distill_short daily at 03:00")
if settings.auto_distill_mid:
_scheduler.add_job(_run_mid, "cron", day_of_week="sun", hour=3, minute=30, id="distill_mid")
logger.info("scheduled: distill_mid weekly Sun at 03:30")
if settings.auto_distill_long:
_scheduler.add_job(_run_long, "cron", day=1, hour=4, minute=0, id="distill_long")
logger.info("scheduled: distill_long monthly on 1st at 04:00")
# Daily reminder notification check — 09:00
_scheduler.add_job(_run_reminder_check, "cron", hour=9, minute=0, id="reminder_check")
logger.info("scheduled: reminder_check daily at 09:00")
# Load user-defined cron jobs from CRONS.json
_load_user_crons()
_scheduler.start()
logger.info("scheduler started (%d jobs)", len(_scheduler.get_jobs()))
def _load_user_crons() -> None:
"""Register all enabled user-defined cron jobs across all users and personas."""
import asyncio
try:
from cron_runner import load_crons, parse_schedule, run_job
from persona import list_users, list_user_personas
except ImportError as e:
logger.warning("could not import cron modules: %s", e)
return
total = 0
persona_count = 0
for username in list_users():
for persona_name in list_user_personas(username):
persona_count += 1
for job in load_crons(username, persona_name):
if not job.get("enabled", True):
continue
# Ensure user + persona are stamped on the job for run_job() path resolution
job.setdefault("user", username)
job.setdefault("persona", persona_name)
try:
kwargs = parse_schedule(job["schedule"])
sched_id = f"{username}:{persona_name}:{job['id']}"
_scheduler.add_job(
lambda j=job: asyncio.ensure_future(run_job(j)),
"cron",
id=sched_id,
replace_existing=True,
**kwargs,
)
total += 1
except Exception as e:
logger.warning("cron %s/%s/%s skipped: %s", username, persona_name, job.get("id"), e)
if total:
logger.info("loaded %d user cron job(s) across %d persona(s)", total, persona_count)
def stop() -> None:
global _scheduler
if _scheduler and _scheduler.running:
_scheduler.shutdown(wait=False)
logger.info("auto distillation scheduler stopped")
def status() -> list[dict]:
"""Return next-run info for all scheduled jobs."""
if not _scheduler or not _scheduler.running:
return []
jobs = []
for job in _scheduler.get_jobs():
next_run = job.next_run_time
jobs.append({
"id": job.id,
"next_run": next_run.isoformat() if next_run else None,
})
return jobs

View File

@@ -1,22 +1,34 @@
from pathlib import Path
from datetime import datetime from datetime import datetime
from config import settings from persona import persona_path, get_user, get_persona
def log_turn(session_id: str, user_msg: str, assistant_msg: str) -> None: def log_turn(
session_id: str,
user_msg: str,
assistant_msg: str,
backend_label: str = "",
host: str = "",
) -> None:
today = datetime.now().strftime("%Y-%m-%d") today = datetime.now().strftime("%Y-%m-%d")
sessions_dir = settings.inara_path() / "sessions" sessions_dir = persona_path() / "sessions"
sessions_dir.mkdir(exist_ok=True) sessions_dir.mkdir(exist_ok=True)
log_file = sessions_dir / f"{today}.md" log_file = sessions_dir / f"{today}.md"
timestamp = datetime.now().strftime("%H:%M") timestamp = datetime.now().strftime("%H:%M")
is_new = not log_file.exists() is_new = not log_file.exists()
meta_parts = [p for p in [backend_label, host] if p]
meta = f" · {' / '.join(meta_parts)}" if meta_parts else ""
# Use the actual user/persona names from the current request context
user_label = get_user().title()
persona_label = get_persona().title()
with open(log_file, "a") as f: with open(log_file, "a") as f:
if is_new: if is_new:
f.write(f"# Session Log — {today}\n") f.write(f"# Session Log — {today}\n")
f.write( f.write(
f"\n### [{timestamp}] `{session_id}`\n" f"\n### [{timestamp}] `{session_id}`{meta}\n"
f"**Scott:** {user_msg}\n\n" f"**{user_label}:** {user_msg}\n\n"
f"**Inara:** {assistant_msg}\n" f"**{persona_label}:** {assistant_msg}\n"
) )

View File

@@ -3,6 +3,7 @@ import random
from pathlib import Path from pathlib import Path
from datetime import datetime from datetime import datetime
from config import settings from config import settings
from persona import persona_path
_ADJECTIVES = [ _ADJECTIVES = [
@@ -42,7 +43,7 @@ def generate_session_id() -> str:
def _path(session_id: str) -> Path: def _path(session_id: str) -> Path:
d = settings.sessions_path() d = persona_path() / "session_data"
d.mkdir(parents=True, exist_ok=True) d.mkdir(parents=True, exist_ok=True)
return d / f"{session_id}.json" return d / f"{session_id}.json"
@@ -61,27 +62,67 @@ def save(session_id: str, messages: list[dict]) -> None:
# Enforce rolling window # Enforce rolling window
windowed = messages[-settings.max_history_messages:] windowed = messages[-settings.max_history_messages:]
path.write_text(json.dumps({ data = {
"session_id": session_id, "session_id": session_id,
"created": existing.get("created", datetime.now().isoformat()), "created": existing.get("created", datetime.now().isoformat()),
"updated": datetime.now().isoformat(), "updated": datetime.now().isoformat(),
"messages": windowed, "messages": windowed,
}, indent=2)) }
if "name" in existing:
data["name"] = existing["name"]
path.write_text(json.dumps(data, indent=2))
def get_name(session_id: str) -> str:
"""Return the friendly name for a session, or '' if none set."""
path = _path(session_id)
if not path.exists():
return ""
try:
return json.loads(path.read_text()).get("name", "")
except Exception:
return ""
def rename(session_id: str, name: str) -> bool:
"""Set (or clear) the friendly name on a session. Returns False if not found."""
path = _path(session_id)
if not path.exists():
return False
data = json.loads(path.read_text())
if name:
data["name"] = name
else:
data.pop("name", None)
path.write_text(json.dumps(data, indent=2))
return True
def delete(session_id: str) -> bool:
"""Delete a session file. Returns True if it existed and was deleted."""
path = _path(session_id)
if not path.exists():
return False
path.unlink()
return True
def list_all() -> list[dict]: def list_all() -> list[dict]:
d = settings.sessions_path() d = persona_path() / "session_data"
if not d.exists(): if not d.exists():
return [] return []
results = [] results = []
for f in sorted(d.glob("*.json"), key=lambda p: p.stat().st_mtime, reverse=True): for f in d.glob("*.json"):
try: try:
data = json.loads(f.read_text()) data = json.loads(f.read_text())
results.append({ results.append({
"session_id": data["session_id"], "session_id": data["session_id"],
"name": data.get("name", ""),
"updated": data.get("updated"), "updated": data.get("updated"),
"message_count": len(data.get("messages", [])), "message_count": len(data.get("messages", [])),
"_sort_key": data.get("updated") or f.stat().st_mtime,
}) })
except Exception: except Exception:
pass pass
results.sort(key=lambda s: s.pop("_sort_key"), reverse=True)
return results return results

518
cortex/static/HELP.md Normal file
View File

@@ -0,0 +1,518 @@
# Cortex UI — Help & Reference
<!-- SHARED BASE: cortex/static/HELP.md
This file is served to all users regardless of persona.
Persona-specific additions live in home/{username}/persona/{name}/HELP.md
and are appended automatically by help.html when present.
-->
*Last updated: 2026-05-13*
---
## Getting Started
If this is your first time using Cortex, you need one thing before the chat will work: an AI model connected to your account.
**Fastest path — OpenRouter:**
OpenRouter gives you access to Claude, Gemini, and dozens of other models with a single API key.
1. Get a free API key at [openrouter.ai/keys](https://openrouter.ai/keys)
2. Go to **☰ → Account → [Set up OpenRouter →]** (shown automatically if no model is configured)
3. Paste your key, pick a starting model, click **Connect**
That's it — you're ready to chat.
**Already past setup but seeing errors?** Go to **☰ → Account → Model Registry → Manage models** and confirm a model is assigned to the **Chat** role (Primary slot). If all slots are empty, add a model first.
---
## Header Controls
| Button | What it does |
|---|---|
| **Sessions** | Open the sessions panel — list, resume, or start sessions |
| **N** (sliders icon) | Open the Context & Memory panel (N = current context tier) |
| **☰** | Settings menu — Files, push notification toggle, Account, Sign Out |
| **?** | Open this help panel |
The **Context & Memory** panel (sliders icon with tier number) contains all configuration options:
| Section | Controls |
|---|---|
| **Context Tier** | T1 T4 context depth |
| **Memory Layers** | Toggle Long / Mid / Short memory on/off |
| **Distill Memory** | Manually trigger Short / Mid / Long / All distillation |
| **Model** | Active chat model — click to cycle through your configured slot models (Primary → Backup 1 → …) |
| **Display** | **Aa** cycles font size · **☾** toggles theme · **S/M/L** cycles input area height · **⌃↵** toggles send shortcut |
All settings persist in `localStorage` across page refreshes.
---
## Chat
- **Send:** `Ctrl+Enter` by default. Click `⌃↵` in the input controls to toggle to plain `Enter` mode.
- **Stop:** Click **Stop** to cancel an in-progress response at any time.
- **Edit a message:** Hover over any message → click **edit**. `Ctrl+Enter` saves, `Esc` cancels.
- **Delete a message:** Hover over any message → click **del**, then **confirm delete**.
- **Copy:** Hover over any message → click **copy**.
- **New line while typing:** `Shift+Enter` (in `Ctrl+Enter` mode) or `Shift+Enter` / Enter (in Enter mode).
Each assistant response shows a small **model tag** below the message identifying which model and host responded.
---
## Tools (⚡)
Click the **⚡** button in the input row to enable the Tools toggle. When lit (amber), **Send** changes to **Run** and messages are routed through the **orchestrator** instead of directly to the chat model.
The orchestrator runs a multi-step tool loop:
1. The **orchestrator model** reasons about the request and calls tools as needed
2. Tool results are fed back into the conversation; the loop continues until the model has what it needs
3. The model produces the final user-facing reply — when the orchestrator role uses Gemini, Claude writes the final response; when it uses a local model, that same model writes it
4. Expandable tool-call cards appear above the response — click any card to see the arguments sent and the result returned
The ⚡ toggle is **independent of the Role selector** — you can use any role (chat, coder, research, etc.) with or without tools. The orchestrator model is configured in **Account → Model Registry → Role Assignments → Orchestrator**.
Tools mode is best for tasks requiring research, multi-step reasoning, or side effects (e.g. "search for X", "add a task", "what's on my list?", "append this to my journal"). Regular chat is faster for conversational turns.
Orchestrated sessions persist to history exactly like regular chat.
### Available Tools
69 tools across 17 categories. Tool schemas are narrowed per-message using keyword routing — only categories relevant to your request are sent, keeping token overhead low. Per-role tool sets provide additional filtering.
| Category | Tools |
|---|---|
| **Web** | `web_search`, `http_fetch`, `web_read`, `http_post` |
| **Project Files** | `project_file_read`, `project_file_list`, `file_stat`, `file_grep`, `file_diff`, `file_syntax_check` |
| **Files** (admin) | `file_read`, `file_list`, `file_write`, `session_read`, `session_search` |
| **Git** | `git_status`, `git_log`, `git_diff` |
| **Shell** | `shell_exec`, `claude_allow_dir` |
| **System** | `cortex_restart`, `cortex_logs`, `cortex_status`, `cortex_update` |
| **Tasks** | `task_list`, `task_create`, `task_update`, `task_complete` |
| **Cron** | `cron_list`, `cron_add`, `cron_remove`, `cron_toggle` |
| **Reminders** | `reminders_add`, `reminders_list`, `reminders_remove`, `reminders_clear` |
| **Scratchpad** | `scratch_read`, `scratch_write`, `scratch_append`, `scratch_clear` |
| **Notifications** | `web_push`, `email_send`, `nc_talk_send`, `nc_talk_history` |
| **Aether Journals** | `ae_journal_list/search`, `ae_journal_entries_list`, `ae_journal_entry_read/create/update/disable/append/prepend` |
| **Aether Tasks** | `ae_task_list` |
| **Aether Database** (admin) | `ae_db_query`, `ae_db_describe`, `ae_db_show_view` |
| **Agent Notes** | `agent_notes_read`, `agent_notes_write`, `agent_notes_append`, `agent_notes_clear` |
| **Agents** | `spawn_agent`, `aider_run` |
| **Home Assistant** | `ha_get_state`, `ha_get_states`, `ha_call_service` |
Files, Shell, System, Aether Database, Agents, and some Notification/Web tools are **admin-only** and not visible to regular users.
`http_post` requires a URL prefix allowlist in `home/{user}/http_allowlist.json`.
`nc_talk_history` requires `nc_username` and `nc_app_password` in `channels.json` under `nextcloud`.
`ae_db_*` tools require Aether DB credentials configured in **Integrations** settings. All queries are SELECT-only — no writes possible.
`aider_run` requires Aider installed (`pip install aider-chat`) and a model configured via `AIDER_MODEL` env var or the project's `.aider.conf.yml`. Supports any OpenAI-compatible backend — DeepSeek, OpenRouter, Ollama, etc.
### Per-Role Tool Sets
Each role can be configured with a specific subset of tool categories. When a role has a tool subset configured, only those tools are sent to the orchestrator — the rest are invisible to the model for that session.
**Example:** a Coder role might only need Web, Files, Shell, and Agent Notes. A Research role might only need Web. Configuring this avoids sending schemas for 30+ irrelevant tools on every call.
Configure per-role tool sets in **Account → Model Registry → Role Assignments** — expand a role card to see the category checkboxes. The default (no checkboxes selected) sends all tools the user has access to.
---
## Sessions
Sessions are named conversation threads that persist across page refreshes.
- Click **Sessions****+ New** to start a fresh session.
- Click any listed session to resume it — full history loads instantly.
- Sessions from Nextcloud Talk appear as `nct_*` prefixed IDs.
- A blue **●** badge appears on the Sessions button when Talk activity arrives in a session you're not currently viewing.
---
## Notes
Notes are injected into a session without triggering an LLM response.
- Click **Note** to toggle note mode. The input border changes colour.
- **Private note** (amber border) — visible only in the UI, never sent to the LLM.
- **Context note** (teal border) — persisted to session history so the LLM sees it on the next turn. Useful for nudging context without a full message.
- Click the `private / public` label to switch between note types.
---
## Install as App (PWA)
Cortex supports installation as a Progressive Web App — it runs in its own window with no browser chrome.
- **Chrome / Edge (desktop):** Look for the install icon in the address bar, or open the browser menu → **Install Cortex…**
- **Android (Chrome):** Tap ⋮ → **Add to Home Screen**
- **iOS (Safari):** Tap the Share button → **Add to Home Screen**
Once installed, opening Cortex from the home screen or app launcher skips the browser UI entirely.
---
## Switching Models
The **Model** button in the Context & Memory panel cycles through the slot models configured for your active role (Primary → Backup 1). Click it to switch between models mid-session.
- The button label shows the active model (e.g. "GPT-4o", "Gemini 2.5 Flash")
- The selected slot is sent with each chat request so the correct model is used
- If only one model is configured, the toggle does nothing
- A system message appears in the chat when you switch models
If the active model fails, the next configured backup slot is tried automatically.
Each response shows a **model tag** (bottom-right of message) with the model label and host, so you always know what responded.
---
## Account Settings
**Navigate to:** ☰ (top-right menu) → **Account**
| Section | What you can do |
|---|---|
| **Account** | View your username, role badge (Admin / User), rename your username |
| **Connected Accounts** | See which Google account is linked for OAuth sign-in |
| **Email Allowlist** | Regex patterns controlling which addresses the `email_send` tool can reach |
| **Notifications** | Dedicated page — set channel (Browser Push, NC Talk, Google Chat, email) for proactive messages; configure Home Assistant inbound webhook; test buttons for instant verification |
| **Schedules** | View, add, edit, pause, and delete scheduled jobs directly — without going through the AI |
| **Tool Permissions** | Allow or block specific orchestrator tools for your account |
| **Usage** | Token consumption by model — see below |
| **Browser Cache** | Clear UI preferences stored locally (theme, font size, session ID, etc.) |
| **Model Registry** | Configure AI providers, local hosts, and role assignments |
| **Change Password** | Update your login password |
| **Personas** | List and rename your personas |
---
## Usage
Token consumption is tracked automatically for API-backed models. **Navigate to:** ☰ → **Account****Usage** section.
The table shows all-time totals per model key, with columns for:
| Column | Meaning |
|---|---|
| **Model** | `backend/model-name` key (e.g. `gemini_api/gemini-2.5-flash`, `local/deepseek-v4`) |
| **Calls** | Number of API calls made |
| **Prompt** | Input tokens sent |
| **Output** | Completion tokens received |
| **Total** | Prompt + Output |
Values ≥ 1,000 are displayed as `k` (e.g. `24.3k`).
**What is and isn't tracked:**
- ✅ Gemini API calls (orchestrator, distillation)
- ✅ Local OpenAI-compatible calls (Open WebUI, Ollama, OpenRouter)
- ✗ Claude CLI — no structured token data is returned by the subprocess
- ✗ Gemini CLI — same reason
The raw data lives in `home/{username}/usage.json` and is also accessible via the Files panel or the API.
---
## Model Registry
Configure which AI models are available and which handles each task type.
**New user quick path:** ☰ → **Account****Set up OpenRouter →** (the guided wizard adds a host, model, and role assignment in one step).
**Full manual path:** ☰ → **Account** → scroll to **Model Registry****Manage models →**
---
### Step 1 — Set up providers and hosts
Do this before adding models — models need a provider account or local host to attach to.
**Anthropic (Claude):** Two options:
- **CLI (OAuth):** Nothing to configure — uses your existing `claude auth login` session. If Claude isn't working, run `claude auth login` in a terminal.
- **Direct API key:** Scroll to **Cloud Providers → Anthropic** → click **+ Add API key**. Enter a label and your `sk-ant-…` key from [console.anthropic.com/keys](https://console.anthropic.com/keys). When you add a model using an API key credential, it routes through the Anthropic SDK instead of the CLI.
**Google (Gemini):** Add one entry per API key you want to use:
1. Scroll to **Cloud Providers → Google** → click **+ Add Google account**
2. Enter a label (e.g. "Work", "Personal") and your API key
3. Get a free key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey)
**OpenRouter** (recommended for new users — one key for many models):
1. Get a key at [openrouter.ai/keys](https://openrouter.ai/keys)
2. Scroll to **Local Hosts****+ Add host**
3. Label: "OpenRouter", URL: `https://openrouter.ai/api/v1`, paste your key, Type: OpenAI-compatible
4. Click **Fetch models** to verify, then add models from the fetched list
**Other local hosts** (Open WebUI, Ollama, LM Studio, etc.):
1. Scroll to **Local Hosts** → click **+ Add host** to expand the form
2. Enter a label, the API URL (e.g. `http://192.168.1.100:3000`), and optional API key
3. Set **Type**: Open WebUI / Ollama, or OpenAI-compatible
4. Click **Fetch models** on the saved host card to verify connectivity
---
### Step 2 — Add models
Scroll to **Add Model**. Select the provider tab, fill in the details, click **Add Model**:
| Tab | What you need |
|---|---|
| **Local** | Select a host (from Step 1) → enter model name, or use **Fetch from host** to pick from a live list |
| **Google** | Select a Gemini model from the catalog → select a Google account (from Step 1) |
| **Anthropic** | Select a credential (CLI OAuth or an API key added in Step 1) → select a Claude model from the catalog |
The label and context window size auto-fill from the catalog — edit them if you want. Tags are optional.
---
### Step 3 — Assign models to roles
Scroll to **Role Assignments** at the bottom of the page. Each role has **Primary** and **Backup 1** slots — Primary is tried first, then Backup 1. Changes save automatically.
**Required roles** (always present, cannot be removed):
| Role | Used for |
|---|---|
| **Chat** | Regular conversation |
| **Orchestrator** | Agent mode tool loop |
| **Distill** | Memory distillation (short / mid / long) |
**Custom roles** — Click **+ Add custom role** to create your own. Each custom role gets its own model selection, tool set, and system prompt addition. Good examples:
| Example | Purpose |
|---|---|
| **Coder** | Code-focused tasks — larger context window, code-aware model |
| **Research** | Long-context research — high-token model, web tools prioritized |
Switch roles via the **Role** selector in the Context & Memory panel (⚙). Leave all slots empty to use the server default.
**Per-role tool sets:** Expand any role card to configure which tool categories the orchestrator can use when that role is active. Unchecked categories are hidden from the model entirely — reducing token overhead on every orchestrated call. Leaving all categories unchecked means all tools the user has access to are available (the default).
**Inject timestamp:** Each role card has an "Inject current date & time into system prompt" checkbox (default on). Disable it for pure processing roles (summarizer, classifier, translator) that don't need clock awareness.
---
## Nextcloud Talk Bot
The Cortex bot is registered in Nextcloud Talk.
- Messages sent in enabled Talk conversations are received by Cortex, processed, and replied to.
- The webhook returns `200 OK` immediately; the reply happens asynchronously.
- Real-time updates stream to the web UI via SSE — you see Talk messages and responses appear live.
- To enable the bot in a conversation: open Talk conversation settings → Bots → enable the bot.
---
## Google Chat Bot
The Cortex bot is available in Google Chat (One Sky IT Workspace).
- Send the bot a direct message in Google Chat to start a conversation.
- Each DM thread is its own session (`gc_spaces/*` prefix) — history persists across messages.
- Responses are synchronous — Google Chat displays the reply directly in the thread.
- To add the bot to a space: open the space, click **Add people & apps**, and search for the Cortex bot.
- Sessions from Google Chat appear as `gc_*` prefixed IDs in the Sessions panel.
---
## Files (Identity Editor)
The **Files** button opens an editor for your persona's identity and memory files:
| File | Purpose |
|---|---|
| `SOUL.md` | Core personality, values, and voice |
| `IDENTITY.md` | Role, capabilities, and context |
| `USER.md` | Your profile, preferences, and history |
| `PROTOCOLS.md` | Behavioural rules and communication protocols |
| `CONTEXT_TIERS.md` | Defines what gets loaded at each context tier |
| `MEMORY_LONG.md` | Permanent curated long-term memory |
| `MEMORY_MID.md` | Rolling mid-term digest (LLM-distilled) |
| `MEMORY_SHORT.md` | Recent session rollup (auto-aggregated) |
| `HELP.md` | This file — persona-specific additions appended below |
| `email_allowlist.json` | Regex patterns for permitted `email_send` recipients (one per line) |
Toggle **preview** / **edit** to switch between rendered markdown and raw text. **Ctrl+S** saves, **Esc** closes.
The **Audit Log** group at the bottom of the sidebar (collapsed by default) lists tool call logs by date (`YYYY-MM-DD.jsonl`). Click any date to view a read-only table of every orchestrator tool call: time, tool name, status, model, args, and result snippet. Status is colour-coded: green = ok, red = error, amber = denied.
---
## Push Notifications
Cortex can send browser push notifications — even when the tab is closed.
- Open **☰ → Enable notifications** and accept the browser permission prompt.
- Once enabled, the button shows **Notifications on** (in accent colour).
- Click again to disable. Subscriptions are stored per-device.
- The orchestrator's `web_push` tool lets your persona send you a push proactively (e.g. when a long task completes).
**Notification channel settings:** ☰ → **Account****Notification settings →** — choose Browser Push, Email, Nextcloud Talk, or Google Chat as the channel your persona uses for scheduled reminders, cron job completions, and memory digests. Use the **Send Test Notification** button to verify your setup, or **Check Reminders Now** to trigger the reminder check immediately.
---
## Context & Memory ( ⚙ panel )
### Context Tiers
Controls how much context is prepended to each LLM call:
| Tier | Loads | ~Tokens |
|---|---|---|
| **Min** | SOUL + IDENTITY + USER summary | ~1,500 |
| **Std** | + USER full + PROTOCOLS + HELP + memory layers | ~5,000 |
| **Ext** | + last 2 raw session logs | ~15,000 |
| **Full** | + last 7 raw session logs | ~50,000 |
Default is **Std**. Use **Min** for small/local models. Use **Ext** or **Full** for complex multi-session tasks.
### Memory Layers
Three independently toggleable memory files, loaded **Long → Mid → Short**:
| Layer | File | Contents |
|---|---|---|
| **Long** | `MEMORY_LONG.md` | Permanent facts — origin, key decisions, profile highlights |
| **Mid** | `MEMORY_MID.md` | Rolling digest of recent weeks — LLM-distilled from Short |
| **Short** | `MEMORY_SHORT.md` | Recent session rollup — auto-aggregated from session logs |
Toggle any layer off to save tokens for a focused conversation.
### Memory Distillation
Distillation builds up the memory layers from raw session logs. Runs automatically on a schedule; trigger manually via the ⚙ panel:
| Button | What it does |
|---|---|
| **short** | Rolls recent session log files → `MEMORY_SHORT.md` (fast, no LLM) |
| **mid** | LLM summarizes `MEMORY_SHORT.md``MEMORY_MID.md` |
| **long** | LLM integrates `MEMORY_MID.md``MEMORY_LONG.md` |
| **all** | Runs short → mid → long in sequence |
**Recommended workflow:** run **short** after any productive session; **mid** weekly; **long** monthly.
---
## Scheduled Jobs
Cortex can run recurring jobs on a schedule — reminders, daily briefings, automated research, and more. Manage them by asking your persona to set them up, or go directly to **☰ → Account → Schedules**.
### Job Types
| Type | What it does |
|---|---|
| `remind` | Appends to `REMINDERS.md` — automatically surfaced in chat context |
| `note` | Appends to `SCRATCH.md` — read on demand via the scratchpad |
| `message` | Sends the payload text directly to your notification channel |
| `brief` | Calls the AI with your payload as the prompt, sends the response to your notification channel. Good for morning briefings, check-ins. |
| `task` | Runs the full orchestrator tool loop with your payload as the request, sends Claude's response to your notification channel. Use this for agentic scheduled work: research, file updates, summaries that need tool access. |
For `task` jobs: tools that require confirmation are skipped in scheduled context. Pre-approve them in **Settings → Tools** to allow them in scheduled tasks.
### Schedule Formats
| Format | When it runs |
|---|---|
| `hourly` | Every hour at :00 |
| `daily` | Every day at 09:00 |
| `daily:HH:MM` | Every day at the specified time |
| `weekly:DOW` | Every specified day at 09:00 (e.g. `weekly:mon`) |
| `weekly:DOW:HH:MM` | Every specified day at the specified time (e.g. `weekly:fri:17:00`) |
| `monthly` | 1st of every month at 09:00 |
| `monthly:DD` | Specific day of month at 09:00 (e.g. `monthly:15`) |
| `monthly:DD:HH:MM` | Specific day of month at the specified time |
| `yearly:MM:DD` | Every year on that date at 09:00 — for birthdays, anniversaries (e.g. `yearly:03:15`) |
| `yearly:MM:DD:HH:MM` | Every year on that date at the specified time |
DOW values: `mon tue wed thu fri sat sun`. All times are server-local.
Schedules take effect immediately when added or edited — no restart needed. Paused jobs stay in the list and can be resumed at any time.
### Home Assistant Integration
HA automations can trigger your persona via webhook. Configure in **Notifications → Home Assistant → Inbound webhook**:
- Set a **Webhook ID** (long random string — this is your secret URL component)
- Your endpoint: `https://cortex.dgrzone.com/webhook/ha/{username}/{webhook_id}`
- **Enable orchestrator tools** — when checked, HA events trigger the full tool loop; when unchecked, events get a direct LLM response (faster, no tools)
HA payload fields recognized: `message`, `entity_id`, `state`, `trigger`, `event`, `area`.
---
## Keyboard Shortcuts
| Keys | Action |
|---|---|
| `Ctrl+Enter` | Send message (default mode) |
| `Enter` | Send (when in Enter mode) |
| `Shift+Enter` | New line in message input |
| `Ctrl+Enter` | Save inline message edit |
| `Esc` | Cancel inline edit / close any open modal |
| `Ctrl+S` | Save file (Files modal) |
---
## API Reference
For direct access or scripting:
| Method | Endpoint | Description |
|---|---|---|
| `POST` | `/chat` | Send a message — returns SSE stream |
| `GET` | `/backend` | Get current primary/fallback backends |
| `POST` | `/backend` | Set primary backend (`{"primary": "claude"}`) |
| `GET` | `/sessions` | List all sessions |
| `GET` | `/history/{id}` | Get session message history |
| `PUT` | `/history/{id}` | Replace full session history |
| `GET` | `/events` | SSE stream for real-time Talk activity |
| `POST` | `/note` | Inject a context note into a session |
| `GET` | `/files` | List identity files |
| `GET` | `/files/{name}` | Read a file |
| `PUT` | `/files/{name}` | Write a file |
| `POST` | `/distill/short` | Aggregate session logs → MEMORY_SHORT |
| `POST` | `/distill/mid` | Summarize short → MEMORY_MID (LLM) |
| `POST` | `/distill/long` | Integrate mid → MEMORY_LONG (LLM) |
| `POST` | `/distill/all` | Run all three distillation steps |
| `GET` | `/distill/status` | Scheduler status and next run times |
| `POST` | `/orchestrate` | Submit an agent task — returns `{"job_id": "..."}` |
| `GET` | `/orchestrate/{job_id}` | Poll job status and result |
| `GET` | `/settings/models` | Model registry UI |
| `POST` | `/api/models/role` | Set a role assignment (JSON body) |
| `POST` | `/api/models/role-config` | Set per-role tool list and system prompt append |
| `GET` | `/api/push/vapid-key` | VAPID public key (for push subscription) |
| `POST` | `/api/push/subscribe` | Register a push subscription |
| `DELETE` | `/api/push/subscribe` | Remove a push subscription |
| `POST` | `/api/push/test` | Send a test notification via configured channel |
| `POST` | `/api/push/reminders/check` | Run reminder check immediately; returns `{"reminders_found": n}` |
| `GET` | `/api/audit/files` | List available audit log dates (own data) |
| `GET` | `/api/audit/day?date=` | Tool call entries for a specific date (own data) |
| `GET` | `/api/audit/recent` | Recent tool calls across days (admin) |
| `GET` | `/api/audit/stats` | Tool call counts by tool/status/user (admin) |
| `GET` | `/api/usage` | Full daily token usage log (own data) |
| `GET` | `/api/usage/summary` | Per-model token totals, all time (own data) |
| `GET` | `/api/usage/all` | Per-model totals for all users (admin) |
| `GET` | `/setup/model` | Guided OpenRouter setup form (Step 3 / standalone) |
| `POST` | `/setup/model` | Save OpenRouter host + model + assign to chat role |
| `GET` | `/health` | Health check — returns `{"status": "ok"}` |
Chat request body (`POST /chat`):
```json
{
"message": "string",
"session_id": "string | null",
"tier": 2,
"chat_role": "chat",
"slot": "primary | backup_1 | backup_2 | null",
"include_long": true,
"include_mid": true,
"include_short": true,
"off_record": false
}
```
---
*Cortex is a self-hosted personal AI platform. Named after the 'verse-wide communications network in Firefly.*

123
cortex/static/TOOLS.md Normal file
View File

@@ -0,0 +1,123 @@
# Tool Reference
> This reference covers all 45 orchestrator tools available when the ⚡ toggle is on.
> Tools are invoked automatically by the orchestrator — you don't call them directly.
¹ **Admin only** — requires the `admin` role. Invisible to regular users.
² **Confirmation required** — the orchestrator pauses and shows **Confirm / Deny** buttons before executing.
---
## Web
| Tool | What it does |
|---|---|
| `web_search` | DuckDuckGo search — returns titles, URLs, and snippets for the top results |
| `http_fetch` | Fetch a specific URL and return the response body (8 192 char cap) |
## Files ¹
| Tool | What it does |
|---|---|
| `file_read` ¹ | Read any file under the persona home directory |
| `file_list` ¹ | List files and directories with sizes (200 entry cap) |
| `file_write` ¹ ² | Write or append to a file under the persona home directory |
## Shell ¹
| Tool | What it does |
|---|---|
| `shell_exec` ¹ ² | Run any shell command on the Cortex host; timeout 1120 s |
| `claude_allow_dir` ¹ | Add a directory to Claude Code's auto-allowed paths |
## System ¹
| Tool | What it does |
|---|---|
| `cortex_restart` ¹ ² | Restart the Cortex service (5 s delay); connection drops — refresh the page |
| `cortex_logs` ¹ | Recent lines from the systemd journal (default 50, max 200) |
| `cortex_status` ¹ | Current git branch, commit, ahead/behind remote, and service state |
| `cortex_update` ¹ ² | `git pull` + syntax check all `.py` files; reports what changed. Does **not** restart automatically — call `cortex_restart` after reviewing |
## Tasks
| Tool | What it does |
|---|---|
| `task_list` | List personal tasks; pass `include_done=true` to include completed |
| `task_create` | Create a task with title, optional notes and due date |
| `task_update` | Update any fields on an existing task |
| `task_complete` | Mark a task as complete |
## Cron
| Tool | What it does |
|---|---|
| `cron_list` | List all scheduled jobs for this persona |
| `cron_add` | Add a scheduled job — accepts cron syntax or plain-English interval |
| `cron_remove` ² | Remove a scheduled job by ID |
| `cron_toggle` | Enable or disable a job without removing it |
## Reminders
| Tool | What it does |
|---|---|
| `reminders_add` | Add a reminder with optional label; surfaced in context at Tier 2+ |
| `reminders_list` | List all pending reminders, numbered for easy removal |
| `reminders_remove` | Remove a single reminder by number (call `reminders_list` first) |
| `reminders_clear` ² | Clear all reminders at once |
## Scratchpad
| Tool | What it does |
|---|---|
| `scratch_read` | Read the current scratchpad |
| `scratch_write` | Overwrite the scratchpad with new content |
| `scratch_append` | Append a timestamped section to the scratchpad |
| `scratch_clear` | Erase the scratchpad |
## Notifications ¹
| Tool | What it does |
|---|---|
| `web_push` | Send a browser push notification to the active user's registered devices |
| `email_send` ¹ | Send an email via SMTP; recipient must match your `email_allowlist.json` |
| `nc_talk_send` ¹ | Send a message to a Nextcloud Talk conversation |
## Aether Journals
| Tool | What it does |
|---|---|
| `ae_journal_list` | List all journals for the configured AE account (returns names + IDs) |
| `ae_journal_search` | Search entries by keyword, tag, date range, type, status, or priority |
| `ae_journal_entries_list` | Browse all entries in a specific journal, newest first; paginated |
| `ae_journal_entry_read` | Read the full content of a single entry by ID |
| `ae_journal_entry_create` | Create a new entry with title, content, tags, and summary |
| `ae_journal_entry_update` | Patch any fields on an existing entry (title, content, tags, summary, enable) |
| `ae_journal_entry_disable` | Soft-delete an entry (`enable=false`) without permanently removing it |
| `ae_journal_entry_append` | Append a timestamped section to the bottom of an entry's content |
| `ae_journal_entry_prepend` | Prepend a timestamped section to the top of an entry's content |
## Aether Tasks ¹
| Tool | What it does |
|---|---|
| `ae_task_list` ¹ | List tasks from the agents_sync Kanban board |
## Agent Notes
Private, durable notes visible only to the orchestrator — not surfaced to users. Persist across sessions. Only available in orchestrated (tool-enabled) sessions.
| Tool | What it does |
|---|---|
| `agent_notes_read` | Read the current private notes file |
| `agent_notes_write` | Overwrite the notes file completely |
| `agent_notes_append` | Append a timestamped entry (keeps last 3 backups automatically) |
| `agent_notes_clear` | Erase all notes (backs up first) |
## Agents ¹
Spawn sub-agents that run their own tool loop using a specific role's model and tools.
| Tool | What it does |
|---|---|
| `spawn_agent` ¹ | Spawn a sub-agent synchronously — blocks until the task completes or times out. Params: `task`, `role` (default `chat`), `tier` (14, default 1), `timeout` seconds, `max_rounds` override. Only works with `local_openai` and `gemini_api` models. |

2366
cortex/static/app.js Normal file

File diff suppressed because it is too large Load Diff

172
cortex/static/crons.html Normal file
View File

@@ -0,0 +1,172 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Schedules</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Server-generated table + badges ── */
.cron-table {
width: 100%; border-collapse: collapse; font-size: 0.82rem;
margin-bottom: 1.5rem;
}
.cron-table th {
text-align: left; padding: 0.4rem 0.6rem;
border-bottom: 2px solid var(--pg-border);
color: var(--pg-muted); font-weight: 600; font-size: 0.75rem;
text-transform: uppercase; letter-spacing: 0.04em;
}
.cron-table td {
padding: 0.5rem 0.6rem; border-bottom: 1px solid var(--pg-border);
vertical-align: middle;
}
.cron-table tr:last-child td { border-bottom: none; }
.cron-table tr:hover td { background: var(--pg-hover); }
.badge {
display: inline-block; padding: 0.15rem 0.45rem;
border-radius: 4px; font-size: 0.72rem; font-weight: 600;
text-transform: uppercase; letter-spacing: 0.03em;
}
.badge-enabled { background: color-mix(in srgb, var(--pg-accent) 18%, transparent); color: var(--pg-accent); }
.badge-paused { background: color-mix(in srgb, var(--pg-muted) 18%, transparent); color: var(--pg-muted); }
.badge-remind { background: color-mix(in srgb, #a78bfa 15%, transparent); color: #a78bfa; }
.badge-note { background: color-mix(in srgb, #60a5fa 15%, transparent); color: #60a5fa; }
.badge-message { background: color-mix(in srgb, #34d399 15%, transparent); color: #34d399; }
.badge-brief { background: color-mix(in srgb, #fb923c 15%, transparent); color: #fb923c; }
.badge-task { background: color-mix(in srgb, #f472b6 15%, transparent); color: #f472b6; }
.cron-actions { display: flex; gap: 0.35rem; }
.btn-cron {
padding: 0.2rem 0.55rem; border-radius: 4px; border: 1px solid var(--pg-border);
background: transparent; color: var(--pg-muted); font-size: 0.75rem; cursor: pointer;
font-family: inherit;
}
.btn-cron:hover { border-color: var(--pg-accent); color: var(--pg-accent); }
.btn-cron-del { color: var(--pg-dimmer); }
.btn-cron-del:hover { border-color: #ef4444; color: #ef4444; }
.payload-cell {
max-width: 240px; overflow: hidden; text-overflow: ellipsis;
white-space: nowrap; color: var(--pg-dimmer);
}
.persona-group-label {
font-size: 0.72rem; font-weight: 700; text-transform: uppercase;
letter-spacing: 0.06em; color: var(--pg-dimmer); margin: 1.25rem 0 0.5rem;
}
.empty-state {
text-align: center; padding: 2rem 1rem;
color: var(--pg-dimmer); font-size: 0.85rem;
border: 1px dashed var(--pg-border); border-radius: 8px;
margin-bottom: 1.5rem;
}
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link active">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Schedules</h1>
<p class="page-subtitle">Recurring jobs — reminders, notes, briefings, and agentic tasks.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<!-- Edit form (shown only when editing) -->
{{ edit_html }}
<!-- Cron list -->
{{ cron_list_html }}
<!-- Add new schedule -->
<div class="section">
<h2>Add schedule</h2>
<form method="POST" action="/settings/crons/add">
<div class="grid grid-cols-2 gap-x-3">
<div class="field">
<label for="add_persona">Persona</label>
<select id="add_persona" name="persona">
{{ persona_options }}
</select>
</div>
<div class="field">
<label for="add_job_type">Type</label>
<select id="add_job_type" name="job_type">
<option value="remind">remind — append to REMINDERS.md</option>
<option value="note">note — append to SCRATCH.md</option>
<option value="message">message — send payload as-is</option>
<option value="brief">brief — LLM response, no tools</option>
<option value="task">task — full orchestrator tool loop</option>
</select>
</div>
<div class="field">
<label for="add_label">Label</label>
<input type="text" id="add_label" name="label"
placeholder="Monday morning summary"
required autocomplete="off">
</div>
<div class="field">
<label for="add_schedule">Schedule</label>
<input type="text" id="add_schedule" name="schedule"
placeholder="weekly:mon:08:00"
required autocomplete="off" spellcheck="false">
<p class="hint">
hourly · daily · daily:HH:MM · weekly:DOW · weekly:DOW:HH:MM ·
monthly · monthly:DD · monthly:DD:HH:MM · yearly:MM:DD · yearly:MM:DD:HH:MM
</p>
</div>
<div class="field col-span-2">
<label for="add_payload">Payload / prompt</label>
<textarea id="add_payload" name="payload" rows="3"
placeholder="Check my open tasks and send a summary." required></textarea>
</div>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Add schedule</button>
</form>
</div>
</div>
</body>
</html>

239
cortex/static/help.html Normal file
View File

@@ -0,0 +1,239 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Help &amp; Reference</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Tab panels (JS-toggled display) ── */
.tab-panel { display: none; }
.tab-panel.active { display: block; }
/* ── Dynamically-rendered markdown content ── */
.help-body { line-height: 1.7; }
details {
margin-bottom: 0.75rem;
background: var(--pg-surface);
border: 1px solid var(--pg-border);
border-radius: 8px; overflow: hidden;
}
summary {
padding: 0.85rem 1rem; font-weight: 600; font-size: 0.95rem;
color: var(--pg-bright); cursor: pointer; list-style: none;
display: flex; align-items: center; gap: 0.5rem;
}
summary::before {
content: '▶'; font-size: 0.65rem; color: var(--pg-muted);
transition: transform 0.15s;
}
details[open] summary::before { transform: rotate(90deg); }
summary::-webkit-details-marker { display: none; }
details > *:not(summary) { padding: 0 1rem 1rem; }
.help-body p { margin: 0.5rem 0; font-size: 0.9rem; color: var(--pg-bright); }
.help-body ul { margin: 0.5rem 0 0.5rem 1.25rem; }
.help-body li { font-size: 0.9rem; color: var(--pg-bright); margin-bottom: 0.25rem; }
.help-body strong { color: var(--pg-text); }
.help-body code {
background: var(--pg-bg); border: 1px solid var(--pg-border);
border-radius: 4px; padding: 0.1em 0.4em;
font-size: 0.85em; color: var(--pg-accent);
}
.help-body a { color: var(--pg-accent); }
.help-body h1 { font-size: 1.1rem; font-weight: 700; color: var(--pg-text); margin: 0.75rem 0 0.5rem; }
.help-body h3 {
font-size: 0.8rem; font-weight: 600; color: var(--pg-muted);
text-transform: uppercase; letter-spacing: 0.05em; margin: 0.75rem 0 0.25rem;
}
.help-body table { width: 100%; border-collapse: collapse; font-size: 0.88rem; margin: 0.5rem 0 0.75rem; }
.help-body th, .help-body td { padding: 0.45rem 0.7rem; text-align: left; border-bottom: 1px solid var(--pg-border); }
.help-body th { color: var(--pg-muted); font-weight: 600; font-size: 0.8rem; text-transform: uppercase; letter-spacing: 0.04em; }
.help-body td { color: var(--pg-bright); }
.help-body pre { background: var(--pg-bg); border: 1px solid var(--pg-border); border-radius: 6px; padding: 0.75rem 1rem; overflow-x: auto; margin: 0.5rem 0; }
.help-body pre code { background: none; border: none; padding: 0; font-size: 0.85em; color: var(--pg-muted); }
.help-body hr { border: none; border-top: 1px solid var(--pg-border); margin: 0.5rem 0; }
</style>
</head>
<body>
<nav class="page-nav" id="page-nav">
<a id="nav-chat" href="/" class="nav-link">← Chat</a>
<a href="/help" class="nav-link active">Help</a>
<a href="/settings" class="nav-link" id="nav-settings">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="max-w-3xl mx-auto px-6 py-8 pb-16">
<div class="mb-6 pb-4 border-b border-pg-border">
<h1 class="text-xl font-bold text-pg-accent">Help &amp; Reference</h1>
<p id="persona-label" class="text-xs text-pg-muted mt-1"></p>
</div>
<!-- Tab bar -->
<div class="tab-bar">
<button class="tab-btn active" data-tab="ui">UI Guide</button>
<button class="tab-btn" data-tab="tools">Tools</button>
<button class="tab-btn" data-tab="persona" id="tab-btn-persona">Persona</button>
</div>
<div id="tab-ui" class="tab-panel active"><div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
<div id="tab-tools" class="tab-panel"> <div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
<div id="tab-persona" class="tab-panel"> <div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
</div>
<style>
.tab-bar {
display: flex; gap: 0.25rem; margin-bottom: 1.5rem;
border-bottom: 1px solid var(--pg-border);
}
.tab-btn {
padding: 0.45rem 1rem; font-size: 0.85rem; font-weight: 500;
color: var(--pg-dim); background: none; border: none;
border-bottom: 2px solid transparent; margin-bottom: -1px;
cursor: pointer; transition: color 0.15s, border-color 0.15s;
font-family: inherit;
}
.tab-btn:hover { color: var(--pg-bright); }
.tab-btn.active { color: var(--pg-accent); border-bottom-color: var(--pg-accent); }
</style>
<script>
const cfg = window.HELP_CONFIG || {};
const user = cfg.user || 'scott';
const persona = cfg.persona || 'inara';
const params = `user=${encodeURIComponent(user)}&persona=${encodeURIComponent(persona)}`;
document.getElementById('nav-chat').href = cfg.backHref || '/';
if (persona) {
document.getElementById('persona-label').textContent =
`${persona.charAt(0).toUpperCase() + persona.slice(1)} · ${user}`;
}
// Rename Persona tab to the actual persona name
const personaTabBtn = document.getElementById('tab-btn-persona');
personaTabBtn.textContent = persona.charAt(0).toUpperCase() + persona.slice(1);
// ── Tab switching ────────────────────────────────────────────────
const tabBtns = document.querySelectorAll('.tab-btn');
const tabPanels = document.querySelectorAll('.tab-panel');
const TAB_KEY = `cx_help_tab_${user}_${persona}`;
function activateTab(name) {
tabBtns.forEach(b => b.classList.toggle('active', b.dataset.tab === name));
tabPanels.forEach(p => p.classList.toggle('active', p.id === `tab-${name}`));
try { localStorage.setItem(TAB_KEY, name); } catch (_) {}
}
tabBtns.forEach(btn => btn.addEventListener('click', () => activateTab(btn.dataset.tab)));
// Restore last active tab
try {
const saved = localStorage.getItem(TAB_KEY);
if (saved) activateTab(saved);
} catch (_) {}
// ── Collapsible h2 sections ──────────────────────────────────────
function makeCollapsible(container, openAll = false, openSet = null) {
container.querySelectorAll('h2').forEach(h2 => {
const title = h2.textContent.trim();
const details = document.createElement('details');
if (openAll || (openSet && openSet.has(title))) details.open = true;
const summary = document.createElement('summary');
summary.textContent = title;
details.appendChild(summary);
const siblings = [];
let node = h2.nextSibling;
while (node && node.nodeName !== 'H2') { siblings.push(node); node = node.nextSibling; }
siblings.forEach(s => details.appendChild(s));
h2.parentNode.replaceChild(details, h2);
});
}
// ── Render markdown into a panel ────────────────────────────────
function render(panelId, markdown, openAll, openSet) {
const panel = document.querySelector(`#${panelId} .help-body`);
panel.innerHTML = marked.parse(markdown);
panel.querySelectorAll('a').forEach(a => { a.target = '_blank'; a.rel = 'noopener noreferrer'; });
makeCollapsible(panel, openAll, openSet);
}
// ── Load all three tabs in parallel ─────────────────────────────
const UI_OPEN = new Set(['Getting Started', 'Chat', 'Sessions', 'Model Registry']);
async function loadAll() {
// UI Guide
fetch('/static/HELP.md')
.then(r => r.ok ? r.text() : Promise.reject(r.status))
.then(md => render('tab-ui', md, false, UI_OPEN))
.catch(e => { document.querySelector('#tab-ui .help-body').innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">Failed to load: ${e}</p>`; });
// Tools
fetch('/static/TOOLS.md')
.then(r => r.ok ? r.text() : Promise.reject(r.status))
.then(md => render('tab-tools', md, true, null))
.catch(e => { document.querySelector('#tab-tools .help-body').innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">Failed to load: ${e}</p>`; });
// Persona-specific HELP.md
const personaPanel = document.querySelector('#tab-persona .help-body');
try {
const res = await fetch(`/files/HELP.md?${params}`);
if (res.ok) {
const data = await res.json();
const content = (data.content || '').trim();
if (content) {
render('tab-persona', content, true, null);
} else {
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet. Edit <code>HELP.md</code> in the Files panel to add them.</p>`;
}
} else {
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet.</p>`;
}
} catch (_) {
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet.</p>`;
}
}
loadAll();
</script>
</body>
</html>

BIN
cortex/static/icon-192.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.2 KiB

BIN
cortex/static/icon-512.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.7 KiB

4
cortex/static/icon.svg Normal file
View File

@@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
<rect width="512" height="512" rx="96" fill="#1a1228"/>
<text x="256" y="390" font-size="340" text-anchor="middle" font-family="system-ui, -apple-system, sans-serif"></text>
</svg>

After

Width:  |  Height:  |  Size: 251 B

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,134 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Integrations</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
details.channel-block summary::-webkit-details-marker { display: none; }
details.channel-block summary::before {
content: '▶'; font-size: 0.65rem; color: var(--pg-dimmer);
transition: transform 0.15s; flex-shrink: 0;
}
details.channel-block[open] summary::before { transform: rotate(90deg); }
details.channel-block[open] summary { border-bottom: 1px solid var(--pg-border); }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
<a href="/settings/integrations" class="nav-link active">Integrations</a>
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Integrations</h1>
<p class="page-subtitle">External service connections — admin only.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<form method="POST" action="/settings/integrations">
<div class="section">
<h2>Aether Platform Database</h2>
<p class="section-note">
Gives the orchestrator direct read-only access to the Aether MariaDB via the
<code>ae_db_query</code>, <code>ae_db_describe</code>, and <code>ae_db_show_view</code> tools.
Only SELECT, SHOW, DESCRIBE, and EXPLAIN are permitted — no writes possible.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ ae_db_host and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Connection
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer mb-4 -mt-1 leading-relaxed">
Use the same credentials as
<code class="font-mono text-pg-accent bg-pg-bg border border-pg-border rounded px-1 text-xs">agents_sync/mcp/scripts/sql_inspector.py</code>.
Leave the password blank to keep the stored value.
</p>
<div class="grid grid-cols-[1fr_7rem] gap-3 items-start">
<div class="field">
<label for="ae_db_host">Host</label>
<input type="text" id="ae_db_host" name="ae_db_host"
value="{{ ae_db_host }}"
placeholder="192.168.64.5"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ae_db_port">Port</label>
<input type="number" id="ae_db_port" name="ae_db_port"
value="{{ ae_db_port }}"
placeholder="3306" min="1" max="65535"
autocomplete="off">
</div>
</div>
<div class="field">
<label for="ae_db_name">Database name</label>
<input type="text" id="ae_db_name" name="ae_db_name"
value="{{ ae_db_name }}"
placeholder="aether_dev"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ae_db_user">Username</label>
<input type="text" id="ae_db_user" name="ae_db_user"
value="{{ ae_db_user }}"
placeholder="aether_dev"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ae_db_password">Password</label>
<input type="password" id="ae_db_password" name="ae_db_password"
value=""
placeholder="Leave blank to keep existing value"
autocomplete="new-password" spellcheck="false">
</div>
</div>
</details>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save integrations</button>
</form>
</div>
</body>
</html>

1055
cortex/static/local_llm.html Normal file

File diff suppressed because it is too large Load Diff

171
cortex/static/login.html Normal file
View File

@@ -0,0 +1,171 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Sign In</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
body {
min-height: 100vh;
display: flex;
align-items: center;
justify-content: center;
background: #0f1117;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
color: #e2e8f0;
}
.card {
background: #1a1d27;
border: 1px solid #2d3148;
border-radius: 12px;
padding: 2.5rem 2rem;
width: 100%;
max-width: 380px;
}
.logo {
text-align: center;
margin-bottom: 1.75rem;
}
.logo h1 {
font-size: 1.6rem;
font-weight: 700;
letter-spacing: 0.05em;
color: #a78bfa;
}
.logo p {
font-size: 0.8rem;
color: #94a3b8;
margin-top: 0.25rem;
}
label {
display: block;
font-size: 0.8rem;
font-weight: 500;
color: #94a3b8;
margin-bottom: 0.4rem;
}
input {
width: 100%;
padding: 0.65rem 0.85rem;
background: #0f1117;
border: 1px solid #2d3148;
border-radius: 6px;
color: #e2e8f0;
font-size: 0.95rem;
outline: none;
transition: border-color 0.15s;
}
input:focus { border-color: #7c3aed; }
.field { margin-bottom: 1rem; }
button[type="submit"] {
width: 100%;
padding: 0.7rem;
margin-top: 0.5rem;
background: #7c3aed;
border: none;
border-radius: 6px;
color: #fff;
font-size: 1rem;
font-weight: 600;
cursor: pointer;
transition: background 0.15s;
}
button[type="submit"]:hover { background: #6d28d9; }
.divider {
display: flex;
align-items: center;
gap: 0.75rem;
margin: 1.25rem 0;
color: #475569;
font-size: 0.78rem;
}
.divider::before, .divider::after {
content: '';
flex: 1;
border-top: 1px solid #2d3148;
}
.google-btn {
display: flex;
align-items: center;
justify-content: center;
gap: 0.6rem;
width: 100%;
padding: 0.65rem;
background: #fff;
border: 1px solid #dadce0;
border-radius: 6px;
color: #3c4043;
font-size: 0.95rem;
font-weight: 500;
font-family: inherit;
cursor: pointer;
text-decoration: none;
transition: background 0.15s, box-shadow 0.15s;
}
.google-btn:hover { background: #f8f9fa; box-shadow: 0 1px 4px rgba(0,0,0,0.2); }
.error {
color: #f87171;
font-size: 0.85rem;
text-align: center;
margin-bottom: 1rem;
}
</style>
</head>
<body>
<div class="card">
<div class="logo">
<h1>Cortex</h1>
<p>You can't stop the signal.</p>
</div>
<!-- ERROR -->
<a href="/auth/google" class="google-btn">
<svg width="18" height="18" viewBox="0 0 18 18" xmlns="http://www.w3.org/2000/svg">
<path d="M17.64 9.2c0-.637-.057-1.251-.164-1.84H9v3.481h4.844c-.209 1.125-.843 2.078-1.796 2.717v2.258h2.908c1.702-1.567 2.684-3.875 2.684-6.615z" fill="#4285F4"/>
<path d="M9 18c2.43 0 4.467-.806 5.956-2.18l-2.908-2.259c-.806.54-1.837.86-3.048.86-2.344 0-4.328-1.584-5.036-3.711H.957v2.332A8.997 8.997 0 0 0 9 18z" fill="#34A853"/>
<path d="M3.964 10.71A5.41 5.41 0 0 1 3.682 9c0-.593.102-1.17.282-1.71V4.958H.957A8.996 8.996 0 0 0 0 9c0 1.452.348 2.827.957 4.042l3.007-2.332z" fill="#FBBC05"/>
<path d="M9 3.58c1.321 0 2.508.454 3.44 1.345l2.582-2.58C13.463.891 11.426 0 9 0A8.997 8.997 0 0 0 .957 4.958L3.964 7.29C4.672 5.163 6.656 3.58 9 3.58z" fill="#EA4335"/>
</svg>
Sign in with Google
</a>
<div class="divider">or</div>
<form method="POST" action="/login">
<div class="field">
<label for="username">Username</label>
<input type="text" id="username" name="username"
autocomplete="username" autofocus required>
</div>
<div class="field">
<label for="password">Password</label>
<input type="password" id="password" name="password"
autocomplete="current-password" required>
</div>
<button type="submit">Sign In</button>
</form>
</div>
</body>
</html>

View File

@@ -0,0 +1,30 @@
{
"name": "Cortex · Inara",
"short_name": "Cortex",
"description": "Personal AI assistant",
"start_url": "/",
"scope": "/",
"display": "standalone",
"background_color": "#1a1228",
"theme_color": "#1a1228",
"icons": [
{
"src": "/static/icon-192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "any"
},
{
"src": "/static/icon-512.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "any maskable"
},
{
"src": "/static/icon.svg",
"sizes": "any",
"type": "image/svg+xml",
"purpose": "any"
}
]
}

View File

@@ -0,0 +1,348 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Notifications</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Channel collapsible arrow ── */
details.channel-block summary::-webkit-details-marker { display: none; }
details.channel-block summary::before {
content: '▶'; font-size: 0.65rem; color: var(--pg-dimmer);
transition: transform 0.15s; flex-shrink: 0;
}
details.channel-block[open] summary::before { transform: rotate(90deg); }
details.channel-block[open] summary { border-bottom: 1px solid var(--pg-border); }
/* ── Test result feedback (JS-toggled display) ── */
#test-result { display: none; }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link active">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Notifications</h1>
<p class="page-subtitle">How your persona reaches out proactively — reminders, cron jobs, and memory digests.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<form method="POST" action="/settings/notifications">
<!-- Channel selector -->
<div class="section">
<h2>Channel</h2>
<div class="field">
<label for="notification_channel">Default outbound channel</label>
<select id="notification_channel" name="notification_channel"
data-value="{{ notify_channel }}">
<option value="">None (disabled)</option>
<option value="web_push">Browser Push Notification</option>
<option value="email">Email</option>
<option value="nextcloud">Nextcloud Talk</option>
<option value="google_chat">Google Chat</option>
</select>
<p class="hint">Used for reminder alerts, distillation summaries, and cron job notifications.</p>
</div>
<div class="field">
<label for="notification_email">
Email address override
<span class="font-normal text-pg-dim">(optional)</span>
</label>
<input type="email" id="notification_email" name="notification_email"
value="{{ notify_email_override }}"
placeholder="Leave blank to use your login email"
autocomplete="off">
</div>
</div>
<!-- Nextcloud Talk -->
<div class="section">
<h2>Nextcloud Talk</h2>
<p class="section-note">
Configure to send and receive messages via your Nextcloud Talk bot.
<strong>Sending</strong> requires the bot URL, secret, and notification room.
<strong>Reading history</strong> (<code>nc_talk_history</code> tool) additionally
requires a Nextcloud username and app password.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ nc_url and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Bot credentials (sending)
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
Set these up in your Nextcloud Talk room → Bot settings.
See the <a href="/help" class="text-pg-accent">setup guide</a> for step-by-step instructions.
</p>
<div class="field">
<label for="nc_url">Nextcloud URL</label>
<input type="url" id="nc_url" name="nc_url"
value="{{ nc_url }}"
placeholder="https://cloud.example.com"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="nc_bot_secret">Bot secret</label>
<input type="password" id="nc_bot_secret" name="nc_bot_secret"
value="{{ nc_bot_secret }}"
placeholder="Leave blank to keep existing value"
autocomplete="new-password" spellcheck="false">
<p class="hint">Generated when you registered the bot in Nextcloud Talk.</p>
</div>
<div class="field">
<label for="nc_notification_room">Notification room token</label>
<input type="text" id="nc_notification_room" name="nc_notification_room"
value="{{ nc_notify_room }}"
placeholder="Token from the Talk room URL"
autocomplete="off" spellcheck="false">
<p class="hint">The token at the end of the Talk room URL — e.g. <code>abc123def</code>.</p>
</div>
</div>
</details>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ nc_username and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
API credentials (reading history)
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
Required for the <code>nc_talk_history</code> orchestrator tool.
Generate an app password in Nextcloud → Settings → Security → App passwords.
</p>
<div class="field">
<label for="nc_username">Nextcloud username</label>
<input type="text" id="nc_username" name="nc_username"
value="{{ nc_username }}"
placeholder="Your Nextcloud login username"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="nc_app_password">App password</label>
<input type="password" id="nc_app_password" name="nc_app_password"
value="{{ nc_app_password }}"
placeholder="Leave blank to keep existing value"
autocomplete="new-password" spellcheck="false">
</div>
</div>
</details>
</div>
<!-- Home Assistant -->
<div class="section">
<h2>Home Assistant</h2>
<p class="section-note">
Receive events from HA automations and let your persona call the HA REST API
(read states, control devices). Webhook ID is the shared secret used in your
HA <code>rest_command</code> URL.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ ha_url and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Connection
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
HA URL and a Long-Lived Access Token (Profile → scroll to bottom →
Long-Lived Access Tokens → Create Token).
</p>
<div class="field">
<label for="ha_url">Home Assistant URL</label>
<input type="url" id="ha_url" name="ha_url"
value="{{ ha_url }}"
placeholder="https://ha.yourdomain.com"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ha_token">Long-Lived Access Token</label>
<input type="password" id="ha_token" name="ha_token"
value=""
placeholder="Leave blank to keep existing token"
autocomplete="new-password" spellcheck="false">
</div>
</div>
</details>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ ha_webhook_id and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Inbound webhook (HA → Cortex)
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
The webhook ID is the shared secret in your HA <code>rest_command</code> URL.
Your endpoint: <code>https://cortex.dgrzone.com/webhook/ha/{{ ha_username }}/&lt;webhook_id&gt;</code>
</p>
<div class="field">
<label for="ha_webhook_id">Webhook ID</label>
<input type="text" id="ha_webhook_id" name="ha_webhook_id"
value="{{ ha_webhook_id }}"
placeholder="Paste or generate a random secret"
autocomplete="off" spellcheck="false">
<p class="hint">Treat this like a password — use a long, random string.</p>
</div>
<div class="field">
<label class="checkbox-label">
<input type="checkbox" name="ha_tools" value="1" {{ ha_tools_checked }}>
Enable orchestrator tools
</label>
<p class="hint">When checked, HA events trigger the full tool loop (research, home control, tasks). When unchecked, events get a direct LLM response — faster but no tools.</p>
</div>
</div>
</details>
</div>
<!-- Google Chat -->
<div class="section">
<h2>Google Chat</h2>
<p class="section-note">
Outbound webhook for proactive messages to a Google Chat space.
Incoming messages are handled separately via the Google Chat Add-on.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ gc_webhook and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Outbound webhook
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
Create a webhook in your Google Chat space → Manage webhooks. Paste the full URL here.
</p>
<div class="field">
<label for="gc_outbound_webhook">Webhook URL</label>
<input type="url" id="gc_outbound_webhook" name="gc_outbound_webhook"
value="{{ gc_webhook }}"
placeholder="https://chat.googleapis.com/v1/spaces/…"
autocomplete="off" spellcheck="false">
</div>
</div>
</details>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save notification settings</button>
</form>
<!-- Test -->
<div class="section" style="margin-top:2rem;">
<h2>Test</h2>
<p class="section-note">
Fire a notification via your configured channel or run the reminder check
immediately — no need to wait for the daily 09:00 scheduler job.
</p>
<div class="flex gap-3 mt-2">
<button class="flex-1 px-3 py-2.5 text-sm font-medium border border-pg-border rounded-md bg-pg-bg text-pg-text hover:border-pg-action hover:text-pg-accent transition-colors cursor-pointer disabled:opacity-50"
id="btn-test-notify">Send Test Notification</button>
<button class="flex-1 px-3 py-2.5 text-sm font-medium border border-pg-border rounded-md bg-pg-bg text-pg-text hover:border-pg-action hover:text-pg-accent transition-colors cursor-pointer disabled:opacity-50"
id="btn-check-reminders">Check Reminders Now</button>
</div>
<div id="test-result"
class="mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed"></div>
</div>
</div>
<script>
// Set channel select to saved value
const sel = document.getElementById('notification_channel');
if (sel) {
const saved = sel.dataset.value;
if (saved) {
for (const opt of sel.options) {
if (opt.value === saved) { opt.selected = true; break; }
}
}
}
// Test buttons
const resultEl = document.getElementById('test-result');
function showResult(ok, msg) {
resultEl.textContent = msg;
resultEl.className = ok
? 'mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed bg-green-950 text-green-400 border border-green-800'
: 'mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed bg-red-950 text-red-400 border border-red-800';
resultEl.style.display = 'block';
}
async function apiPost(url, btnEl, label) {
btnEl.disabled = true;
btnEl.textContent = label + '…';
resultEl.style.display = 'none';
try {
const r = await fetch(url, { method: 'POST' });
const data = await r.json();
if (r.ok && data.ok) {
if (url.includes('reminders')) {
const n = data.reminders_found ?? 0;
showResult(true, n > 0
? `Found ${n} due reminder${n !== 1 ? 's' : ''} — notification sent.`
: 'No due reminders found — nothing sent.');
} else {
showResult(true, 'Notification sent. Check your configured channel.');
}
} else {
showResult(false, data.detail || 'Request failed.');
}
} catch (e) {
showResult(false, 'Network error: ' + e.message);
} finally {
btnEl.disabled = false;
btnEl.textContent = label;
}
}
document.getElementById('btn-test-notify').addEventListener('click', function() {
apiPost('/api/push/test', this, 'Send Test Notification');
});
document.getElementById('btn-check-reminders').addEventListener('click', function() {
apiPost('/api/push/reminders/check', this, 'Check Reminders Now');
});
</script>
</body>
</html>

189
cortex/static/pg.css Normal file
View File

@@ -0,0 +1,189 @@
/* ─── Cortex settings pages — shared stylesheet ───────────────────────────── */
/* ── Variables ── */
:root {
--pg-bg: #0f1117; --pg-surface: #1a1d27; --pg-border: #2d3148;
--pg-text: #e2e8f0; --pg-muted: #94a3b8;
--pg-dim: #64748b; --pg-dimmer: #475569;
--pg-bright: #cbd5e1; --pg-nav-hover: rgba(255,255,255,0.05);
--pg-accent: #a78bfa; /* heading/highlight purple */
--pg-action: #7c3aed; /* button/focus purple */
}
[data-theme="light"] {
--pg-bg: #f4f2fa; --pg-surface: #ffffff; --pg-border: #d0c8e8;
--pg-text: #1a1228; --pg-muted: #5a5478;
--pg-dim: #7a7290; --pg-dimmer: #9e98b0;
--pg-bright: #1a1228; --pg-nav-hover: rgba(0,0,0,0.05);
--pg-accent: #7c3aed;
--pg-action: #6d28d9;
}
/* ── Reset ── */
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
/* ── Base ── */
body {
min-height: 100vh;
background: var(--pg-bg);
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
color: var(--pg-text);
}
/* ── Top nav ── */
.page-nav {
display: flex; align-items: center; gap: 0.25rem;
padding: 0.5rem 1rem; background: var(--pg-surface);
border-bottom: 1px solid var(--pg-border); flex-wrap: wrap;
}
.nav-link {
display: inline-flex; align-items: center;
padding: 0.3rem 0.6rem; border-radius: 6px;
font-size: 0.8rem; font-weight: 500; color: var(--pg-dim);
text-decoration: none; transition: color 0.15s, background 0.15s;
white-space: nowrap;
}
.nav-link:hover { color: var(--pg-bright); background: var(--pg-nav-hover); }
.nav-link.active { color: var(--pg-accent); }
.nav-spacer { flex: 1; min-width: 0.5rem; }
.nav-link.nav-logout { color: var(--pg-dimmer); }
.nav-link.nav-logout:hover { color: var(--pg-muted); background: none; }
/* ── Page container ── */
.page-wrap {
max-width: 860px; margin: 0 auto;
padding: 2rem 1.5rem 4rem; width: 100%;
}
/* ── Page heading ── */
.page-title {
font-size: 1.4rem; font-weight: 700; color: var(--pg-accent);
}
.page-subtitle {
font-size: 0.8rem; color: var(--pg-muted);
margin-top: 0.2rem; margin-bottom: 1.75rem; line-height: 1.5;
}
/* ── Sections (settings-style, bottom-bordered h2) ── */
.section { margin-bottom: 2rem; }
.section > h2 {
font-size: 0.9rem; font-weight: 600; color: var(--pg-muted);
margin-bottom: 1rem; padding-bottom: 0.4rem;
border-bottom: 1px solid var(--pg-border);
}
/* ── Form elements ── */
.field { margin-bottom: 1rem; }
label {
display: block; font-size: 0.8rem; font-weight: 500;
color: var(--pg-muted); margin-bottom: 0.4rem;
}
input, select, textarea {
width: 100%; padding: 0.65rem 0.85rem;
background: var(--pg-bg); border: 1px solid var(--pg-border);
border-radius: 6px; color: var(--pg-text); font-size: 0.95rem;
font-family: inherit; outline: none; transition: border-color 0.15s;
}
input:focus, select:focus, textarea:focus { border-color: var(--pg-action); }
input[readonly] { color: var(--pg-muted); cursor: default; }
input[type="password"] { font-family: monospace; letter-spacing: 0.05em; }
input[type="checkbox"], input[type="radio"] { width: auto; padding: 0; }
textarea {
font-family: 'SF Mono', 'Fira Mono', 'Menlo', monospace;
font-size: 0.88rem; line-height: 1.55; resize: vertical;
}
/* ── Buttons ── */
/* Primary form submit */
.btn-submit {
padding: 0.6rem 1.5rem; margin-top: 0.25rem;
background: var(--pg-action); border: none; border-radius: 6px;
color: #fff; font-size: 0.9rem; font-weight: 600;
cursor: pointer; transition: opacity 0.15s;
}
.btn-submit:hover { opacity: 0.88; }
/* Compact inline primary (e.g. rename save, inline forms) */
.btn-save {
padding: 0.4rem 0.9rem; background: var(--pg-action); border: none;
border-radius: 6px; color: #fff; font-size: 0.9rem;
font-weight: 600; cursor: pointer; transition: opacity 0.15s;
}
.btn-save:hover { opacity: 0.88; }
/* Outline secondary (e.g. clear cache, cancel, test actions) */
.btn-secondary {
padding: 0.5rem 1rem; background: none;
border: 1px solid var(--pg-border); border-radius: 6px;
color: var(--pg-muted); font-size: 0.88rem; font-weight: 500;
cursor: pointer; transition: border-color 0.15s, color 0.15s;
}
.btn-secondary:hover { border-color: var(--pg-muted); color: var(--pg-text); }
.btn-secondary:disabled { opacity: 0.5; cursor: default; }
/* Inline cancel */
.btn-cancel {
padding: 0.4rem 0.75rem; background: none;
border: 1px solid var(--pg-border); border-radius: 6px;
color: var(--pg-muted); font-size: 0.9rem;
cursor: pointer; transition: border-color 0.15s, color 0.15s;
}
.btn-cancel:hover { border-color: var(--pg-muted); color: var(--pg-text); }
/* Button-styled link (purple, used for "Settings →" style CTAs) */
.action-link {
display: inline-block; padding: 0.5rem 1rem;
background: var(--pg-action); border-radius: 6px;
color: #fff; font-size: 0.88rem; font-weight: 600;
text-decoration: none; transition: opacity 0.15s;
}
.action-link:hover { opacity: 0.88; }
/* Inline button row */
.btn-row { display: flex; gap: 0.5rem; margin-top: 0.5rem; }
/* ── Text utilities ── */
/* Small muted helper text below inputs */
.hint { font-size: 0.78rem; color: var(--pg-dim); margin-top: 0.35rem; line-height: 1.5; }
/* Section-level description paragraph */
.section-note { font-size: 0.8rem; color: var(--pg-muted); margin-bottom: 0.85rem; line-height: 1.55; }
/* Inline code */
code {
font-size: 0.82rem; font-family: 'SF Mono', 'Fira Mono', 'Menlo', monospace;
background: var(--pg-bg); border: 1px solid var(--pg-border);
padding: 0.1rem 0.35rem; border-radius: 4px; color: var(--pg-accent);
}
/* ── Feedback messages ── */
.success { color: #4ade80; font-size: 0.85rem; text-align: center; margin-bottom: 1rem; }
.error { color: #f87171; font-size: 0.85rem; text-align: center; margin-bottom: 1rem; }
/* ── Usage table (JS-rendered in settings) ── */
.usage-table { border-collapse: collapse; width: 100%; min-width: 360px; }
.usage-table th {
padding: 0.35rem 0.5rem; font-size: 0.75rem; color: var(--pg-muted);
font-weight: 600; text-align: right; border-bottom: 1px solid var(--pg-border);
}
.usage-table th:first-child { padding-left: 0; text-align: left; }
.usage-table td {
padding: 0.4rem 0.5rem; font-size: 0.82rem; color: var(--pg-muted); text-align: right;
}
.usage-table td:first-child { padding-left: 0; color: var(--pg-text); text-align: left; white-space: nowrap; }
.usage-table td:last-child { color: var(--pg-text); font-weight: 600; }
/* ── Tool category header row (tools_settings.py generated) ── */
.tool-cat-row td {
padding: 0.75rem 0.9rem 0.3rem;
font-size: 0.72rem; font-weight: 700; letter-spacing: 0.07em;
text-transform: uppercase; color: var(--pg-dimmer);
border-bottom: 1px solid var(--pg-border);
}

397
cortex/static/settings.html Normal file
View File

@@ -0,0 +1,397 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Account Settings</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Server-generated persona list ── */
.persona-list {
list-style: none; display: flex; flex-direction: column;
gap: 0.5rem; margin-top: 0.5rem;
}
.persona-list li { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
.persona-link {
display: inline-block; padding: 0.3rem 0.75rem;
background: var(--pg-bg); border: 1px solid var(--pg-border);
border-radius: 20px; color: var(--pg-accent); font-size: 0.85rem;
text-decoration: none; transition: border-color 0.15s;
}
.persona-link:hover { border-color: var(--pg-action); }
.persona-list li em { color: var(--pg-muted); font-size: 0.85rem; }
.persona-rename-toggle {
background: none; border: 1px solid var(--pg-border);
border-radius: 6px; color: var(--pg-muted); font-size: 0.8rem;
padding: 0.3rem 0.6rem; margin-top: 0.25rem;
cursor: pointer; opacity: 0.7; transition: opacity 0.15s, color 0.15s;
}
.persona-rename-toggle:hover { opacity: 1; color: var(--pg-accent); }
.persona-rename-form { display: flex; align-items: center; gap: 0.4rem; }
.persona-rename-form input[type="text"] {
width: 12rem; padding: 0.3rem 0.6rem;
border-color: var(--pg-action); font-size: 0.9rem;
}
.persona-rename-form .btn-save { padding: 0.3rem 0.75rem; font-size: 0.85rem; }
/* ── Server-generated role badge ── */
.role-badge {
display: inline-block; padding: 0.25rem 0.75rem;
border-radius: 20px; font-size: 0.78rem; font-weight: 600;
text-transform: uppercase; letter-spacing: 0.06em;
}
.role-badge.role-admin {
background: rgba(124,58,237,0.15); color: var(--pg-accent);
border: 1px solid rgba(124,58,237,0.4);
}
.role-badge.role-user {
background: rgba(100,116,139,0.12); color: var(--pg-muted);
border: 1px solid var(--pg-border);
}
/* ── JS-toggled states ── */
#clear-ls-ok { display: none; margin-left: 0.75rem; font-size: 0.8rem; color: #4ade80; }
.usage-wrap { overflow-x: auto; }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link active">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Account Settings</h1>
<p class="page-subtitle">Manage your account and personas.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<!-- OpenRouter quickstart (shown by JS when no model is configured) -->
<div id="openrouter-quickstart"
class="hidden rounded-xl border border-amber-800 bg-amber-950 p-4 mb-5">
<p class="text-xs font-semibold text-amber-400 mb-1">⚡ You're on the server default model</p>
<p class="text-xs text-amber-600 mb-3 leading-relaxed">
You can chat now, but adding your own model gives you more choices, lets you pick
role-specific models, and tracks your usage separately.
OpenRouter is the easiest way to get started — one key, many models.
</p>
<a href="/setup/model"
class="inline-block px-3 py-2 rounded-md bg-amber-900 text-amber-100 text-sm font-medium hover:bg-amber-800 transition-colors">
Set up OpenRouter →
</a>
</div>
<!-- Account info -->
<div class="section">
<h2>Account</h2>
<div class="field">
<label>Username</label>
<input type="text" value="{{ username }}" readonly>
</div>
<div class="field">
<label>Role</label>
<span class="role-badge role-{{ user_role }}">{{ user_role }}</span>
</div>
<button type="button" id="show-rename-user" class="persona-rename-toggle">
✏ Change username
</button>
<form id="rename-user-form" method="POST" action="/settings/username" style="display:none; margin-top:0.75rem;">
<div class="field">
<label for="new_username">New username</label>
<input type="text" id="new_username" name="new_username"
value="{{ username }}"
pattern="[a-z_][a-z0-9_\-]{0,31}" required autofocus
autocomplete="off" data-form-type="other">
<p class="hint">Lowercase letters, digits, _ or - only. You will be logged out after renaming.</p>
</div>
<div class="btn-row">
<button type="submit" class="btn-save">Save</button>
<button type="button" id="cancel-rename-user" class="btn-cancel">Cancel</button>
</div>
</form>
</div>
<!-- Connected accounts -->
<div class="section">
<h2>Connected Accounts</h2>
<div class="field">
<label>Google Account</label>
<input type="text" value="{{ google_email }}" readonly
placeholder="No Google account linked"
style="{{ google_email == '' and 'color:var(--pg-dimmer)' or '' }}">
</div>
<p class="hint" style="margin-top:-0.5rem;">To link or change your Google account, contact Scott.</p>
</div>
<!-- Email Allowlist -->
<div class="section">
<h2>Email Allowlist</h2>
<p class="section-note">
One regex pattern per line. The <code>email_send</code> tool will only send to addresses
that match at least one pattern. Leave blank to block all outbound email.
</p>
<form method="POST" action="/settings/email-allowlist">
<div class="field">
<label for="email_allowlist_ta">Allowed patterns</label>
<textarea id="email_allowlist_ta" name="patterns" rows="6"
placeholder=".*@example\.com&#10;alice@example\.com"
spellcheck="false">{{ email_allowlist }}</textarea>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save allowlist</button>
</form>
</div>
<!-- HTTP POST Allowlist -->
<div class="section">
<h2>HTTP POST Allowlist</h2>
<p class="section-note">
One URL prefix per line. The <code>http_post</code> tool will only POST to URLs that
start with a listed prefix. Leave blank to block all outbound POST requests.
</p>
<form method="POST" action="/settings/http-allowlist">
<div class="field">
<label for="http_allowlist_ta">Allowed URL prefixes</label>
<textarea id="http_allowlist_ta" name="prefixes" rows="5"
placeholder="https://ha.dgrzone.com/api/webhook/&#10;https://n8n.dgrzone.com/webhook/"
spellcheck="false">{{ http_allowlist }}</textarea>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save allowlist</button>
</form>
</div>
<!-- Usage summary -->
<div class="section" id="usage-section">
<h2>Usage</h2>
<p class="section-note">
Token consumption tracked for API-backed models (Gemini API, local OpenAI-compatible).
Claude CLI calls are not metered.
</p>
<div id="usage-table-wrap" class="usage-wrap">
<p class="section-note">Loading…</p>
</div>
</div>
<!-- Browser Cache -->
<div class="section">
<h2>Browser Cache</h2>
<p class="section-note">
Clears UI preferences stored in this browser: active mode, session ID, memory toggles,
theme, font size, and context tier. Does not sign you out.
</p>
<button type="button" id="clear-ls-btn" class="btn-secondary">Clear browser cache</button>
<span id="clear-ls-ok">Cleared.</span>
</div>
<!-- Change Password -->
<div class="section">
<h2>Change Password</h2>
<form method="POST" action="/settings/password" id="password-form">
<div class="field">
<label for="current_password">Current password</label>
<input type="password" id="current_password" name="current_password"
autocomplete="current-password" required>
</div>
<div class="field">
<label for="new_password">New password</label>
<input type="password" id="new_password" name="new_password"
autocomplete="new-password" required minlength="8">
</div>
<div class="field">
<label for="confirm_password">Confirm new password</label>
<input type="password" id="confirm_password" name="confirm_password"
autocomplete="new-password" required>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Update password</button>
</form>
</div>
<!-- Sessions -->
<div class="section">
<h2>Sessions</h2>
<p class="section-note">
Auto-name any sessions that still show a random ID, using their first message as the name.
Only unnamed sessions are affected — existing names are left alone.
</p>
<button type="button" id="backfill-names-btn" class="btn-secondary">Auto-name old sessions</button>
<span id="backfill-names-ok"
class="ml-3 text-xs hidden"
style="color:#4ade80"></span>
</div>
<!-- Personas -->
<div class="section">
<h2>Personas</h2>
<ul class="persona-list">
{{ persona_items }}
</ul>
<a href="/setup/persona"
class="inline-block mt-3 text-xs text-pg-muted hover:text-pg-accent transition-colors">
+ Add new persona
</a>
</div>
</div>
<script>
// Password confirmation check
document.getElementById('password-form').addEventListener('submit', e => {
const np = document.getElementById('new_password').value;
const cfm = document.getElementById('confirm_password').value;
if (np !== cfm) {
e.preventDefault();
alert('New passwords do not match.');
}
});
// Username rename toggle
document.getElementById('show-rename-user').addEventListener('click', () => {
document.getElementById('show-rename-user').style.display = 'none';
document.getElementById('rename-user-form').style.display = 'block';
document.getElementById('new_username').focus();
});
document.getElementById('cancel-rename-user').addEventListener('click', () => {
document.getElementById('rename-user-form').style.display = 'none';
document.getElementById('show-rename-user').style.display = '';
});
// Clear localStorage (keeps JWT cookie — no sign-out)
document.getElementById('clear-ls-btn').addEventListener('click', () => {
localStorage.clear();
document.getElementById('clear-ls-ok').style.display = 'inline';
});
// Show OpenRouter quick-start card if no model is configured
(async () => {
try {
const d = await fetch('/backend').then(r => r.json());
if ((d.available_roles || []).length === 0) {
const el = document.getElementById('openrouter-quickstart');
el.classList.remove('hidden');
el.style.display = 'block';
}
} catch (_) {}
})();
// Usage summary table
(async () => {
const wrap = document.getElementById('usage-table-wrap');
try {
const resp = await fetch('/api/usage/summary');
if (!resp.ok) throw new Error(resp.statusText);
const rows_data = await resp.json();
if (!rows_data.length) {
wrap.innerHTML = '<p class="section-note">No usage recorded yet.</p>';
return;
}
const fmt = n => n >= 1000 ? (n / 1000).toFixed(1) + 'k' : String(n);
const rows = rows_data.map(d => {
const labelCell = d.label !== d.key
? `<span title="${d.key}">${d.label}</span>`
: `<span>${d.key}</span>`;
return `<tr>
<td>${labelCell}</td>
<td>${d.calls}</td>
<td>${fmt(d.prompt_tokens)}</td>
<td>${fmt(d.completion_tokens)}</td>
<td>${fmt(d.total_tokens)}</td>
</tr>`;
}).join('');
wrap.innerHTML = `<table class="usage-table">
<thead><tr>
<th style="text-align:left">Model</th>
<th>Calls</th><th>Prompt</th><th>Output</th><th>Total</th>
</tr></thead>
<tbody>${rows}</tbody>
</table>`;
} catch (e) {
wrap.innerHTML = '<p class="section-note">Could not load usage data.</p>';
}
})();
// Auto-name old sessions backfill
document.getElementById('backfill-names-btn').addEventListener('click', async () => {
const btn = document.getElementById('backfill-names-btn');
const ok = document.getElementById('backfill-names-ok');
btn.disabled = true;
btn.textContent = 'Working…';
try {
const params = new URLSearchParams(window.location.search);
const user = params.get('user') || document.querySelector('input[value]')?.value || '';
const persona = params.get('persona') || '';
const qs = user ? `?user=${encodeURIComponent(user)}&persona=${encodeURIComponent(persona)}` : '';
const res = await fetch(`/api/sessions/backfill-names${qs}`, { method: 'POST' });
const data = await res.json();
if (!res.ok) throw new Error(data.detail || res.statusText);
const n = data.named ?? 0;
ok.textContent = `Named ${n} session${n !== 1 ? 's' : ''}.`;
ok.style.display = 'inline';
ok.classList.remove('hidden');
} catch (e) {
ok.textContent = 'Error — check console.';
ok.style.color = '#f87171';
ok.style.display = 'inline';
ok.classList.remove('hidden');
}
btn.textContent = 'Auto-name old sessions';
btn.disabled = false;
});
// Persona rename toggle
document.querySelectorAll('.persona-rename-toggle').forEach(btn => {
btn.addEventListener('click', () => {
const p = btn.dataset.persona;
const form = document.querySelector(`.persona-rename-form[data-persona="${p}"]`);
btn.style.display = 'none';
form.style.display = 'flex';
form.querySelector('input[type="text"]').focus();
});
});
document.querySelectorAll('.persona-rename-cancel').forEach(btn => {
btn.addEventListener('click', () => {
const form = btn.closest('.persona-rename-form');
const p = form.dataset.persona;
const toggle = document.querySelector(`.persona-rename-toggle[data-persona="${p}"]`);
form.style.display = 'none';
toggle.style.display = '';
});
});
</script>
</body>
</html>

339
cortex/static/setup.html Normal file
View File

@@ -0,0 +1,339 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Setup</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
body {
min-height: 100vh;
display: flex;
align-items: center;
justify-content: center;
background: #0f1117;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
color: #e2e8f0;
padding: 1.5rem;
}
.card {
background: #1a1d27;
border: 1px solid #2d3148;
border-radius: 12px;
padding: 2.5rem 2rem;
width: 100%;
max-width: 440px;
}
.logo {
text-align: center;
margin-bottom: 1.75rem;
}
.logo h1 { font-size: 1.6rem; font-weight: 700; letter-spacing: 0.05em; color: #a78bfa; }
.logo p { font-size: 0.8rem; color: #94a3b8; margin-top: 0.25rem; }
h2 {
font-size: 1rem;
font-weight: 600;
color: #94a3b8;
margin-bottom: 1.25rem;
padding-bottom: 0.5rem;
border-bottom: 1px solid #2d3148;
}
label {
display: block;
font-size: 0.8rem;
font-weight: 500;
color: #94a3b8;
margin-bottom: 0.4rem;
}
label small { font-weight: 400; color: #94a3b8; }
input, select {
width: 100%;
padding: 0.65rem 0.85rem;
background: #0f1117;
border: 1px solid #2d3148;
border-radius: 6px;
color: #e2e8f0;
font-size: 0.95rem;
outline: none;
transition: border-color 0.15s;
}
input:focus, select:focus { border-color: #7c3aed; }
select option { background: #1a1d27; }
.field { margin-bottom: 1rem; }
.hint { font-size: 0.75rem; color: #94a3b8; margin-top: 0.3rem; }
button[type="submit"] {
width: 100%;
padding: 0.7rem;
margin-top: 0.5rem;
background: #7c3aed;
border: none;
border-radius: 6px;
color: #fff;
font-size: 1rem;
font-weight: 600;
cursor: pointer;
transition: background 0.15s;
}
button[type="submit"]:hover { background: #6d28d9; }
.error {
color: #f87171;
font-size: 0.85rem;
text-align: center;
margin-bottom: 1rem;
}
.step-label {
font-size: 0.7rem;
color: #94a3b8;
text-align: right;
margin-bottom: 1rem;
}
.emoji-row {
display: flex;
gap: 0.5rem;
flex-wrap: wrap;
}
.emoji-opt {
font-size: 1.3rem;
cursor: pointer;
padding: 0.2rem 0.35rem;
border-radius: 6px;
border: 2px solid transparent;
transition: border-color 0.1s;
line-height: 1;
}
.emoji-opt.selected { border-color: #7c3aed; background: #2d1f52; }
#emoji-hidden { display: none; }
.provider-badge {
display: inline-flex;
align-items: center;
gap: 0.4rem;
background: #2d1f52;
border: 1px solid #7c3aed;
border-radius: 6px;
padding: 0.3rem 0.6rem;
font-size: 0.78rem;
color: #a78bfa;
margin-bottom: 1rem;
}
.skip-link {
display: block;
text-align: center;
margin-top: 1rem;
font-size: 0.8rem;
color: #64748b;
text-decoration: none;
}
.skip-link:hover { color: #94a3b8; }
.model-hint {
font-size: 0.72rem;
color: #64748b;
margin-top: 0.75rem;
text-align: center;
}
</style>
</head>
<body>
<div class="card">
<div class="logo">
<h1>Cortex</h1>
<p>Let's get you set up.</p>
</div>
<!-- ERROR -->
<!-- ERROR_MODEL -->
<!-- ── Step 1: password ───────────────────────────────────────── -->
<div id="step-password">
<div class="step-label">Step 1 of 3</div>
<h2>Set your password</h2>
<form method="POST" action="" id="password-form">
<input type="hidden" name="step" value="password">
<div class="field">
<label for="password">Password</label>
<input type="password" id="password" name="password"
autocomplete="new-password" autofocus required minlength="8">
<p class="hint">Minimum 8 characters.</p>
</div>
<div class="field">
<label for="confirm">Confirm password</label>
<input type="password" id="confirm" name="confirm"
autocomplete="new-password" required>
</div>
<button type="submit">Continue →</button>
</form>
</div>
<!-- ── Step 2: persona ────────────────────────────────────────── -->
<div id="step-persona" style="display:none">
<div class="step-label">Step 2 of 3</div>
<h2>Create your persona</h2>
<form method="POST" action="" id="persona-form">
<input type="hidden" name="step" value="persona">
<div class="field">
<label for="persona_name">
Persona name <small>(used in the URL)</small>
</label>
<input type="text" id="persona_name" name="persona_name"
pattern="[a-z_][a-z0-9_\-]{0,31}"
placeholder="e.g. tina" required>
<p class="hint">Lowercase, no spaces. This becomes /you/tina in the URL.</p>
</div>
<div class="field">
<label for="display_name">Display name</label>
<input type="text" id="display_name" name="display_name"
placeholder="e.g. Tina" required>
<p class="hint">Shown in the chat header.</p>
</div>
<div class="field">
<label for="user_real_name">Your name</label>
<input type="text" id="user_real_name" name="user_real_name"
placeholder="e.g. Holly" required>
<p class="hint">What your persona should call you.</p>
</div>
<div class="field">
<label>Pick an emoji</label>
<div class="emoji-row" id="emoji-row">
<!-- populated by JS -->
</div>
<input type="hidden" name="emoji" id="emoji-hidden" value="✨">
</div>
<div class="field">
<label for="description">
Short description <small>(optional)</small>
</label>
<input type="text" id="description" name="description"
placeholder="e.g. Friendly, creative, loves music">
</div>
<button type="submit">Create my persona →</button>
</form>
</div>
<!-- ── Step 3: model connect ─────────────────────────────────── -->
<div id="step-model" style="display:none">
<div class="step-label"><!-- SETUP_STEP3_LABEL --></div>
<h2>Connect an AI model</h2>
<div class="provider-badge">⚡ Recommended: OpenRouter</div>
<p style="font-size:0.82rem;color:#94a3b8;margin-bottom:1rem;">
One API key gives you access to Claude, Gemini, Llama, and dozens of other models.
Get a free key at <a href="https://openrouter.ai/keys" target="_blank" style="color:#a78bfa;">openrouter.ai/keys</a>.
</p>
<form method="POST" action="/setup/model" id="model-form">
<div class="field">
<label for="api_key">OpenRouter API key</label>
<input type="password" id="api_key" name="api_key"
autocomplete="off" placeholder="sk-or-v1-..." required>
</div>
<div class="field">
<label for="model_name">Starting model</label>
<select id="model_name" name="model_name">
<option value="anthropic/claude-3-5-haiku-20241022">Claude 3.5 Haiku — Fast &amp; affordable</option>
<option value="anthropic/claude-3-7-sonnet-20250219">Claude 3.7 Sonnet — Smarter Claude</option>
<option value="google/gemini-2.0-flash-001">Gemini 2.0 Flash — Fast Google model</option>
<option value="meta-llama/llama-3.3-70b-instruct">Llama 3.3 70B — Open source</option>
</select>
<p class="hint">You can add more models or switch anytime in Account → Model Registry.</p>
</div>
<button type="submit">Connect &amp; start chatting →</button>
</form>
<p class="model-hint">
Using Ollama, a local model, or something else?
<a href="#" id="skip-model-link" style="color:#64748b;">Skip this step →</a>
</p>
</div>
</div>
<script>
// ── Emoji picker ──────────────────────────────────────────────────
const EMOJIS = ['✨','🌙','🌸','🔮','🦋','🌿','⚡','🎯','🌊','🎨',
'🦊','🐉','🌺','🍀','🎵','💫','🔥','❄️','🌈','🏔️'];
const emojiRow = document.getElementById('emoji-row');
const emojiHidden = document.getElementById('emoji-hidden');
let selected = '✨';
EMOJIS.forEach(e => {
const span = document.createElement('span');
span.className = 'emoji-opt' + (e === selected ? ' selected' : '');
span.textContent = e;
span.addEventListener('click', () => {
document.querySelectorAll('.emoji-opt').forEach(s => s.classList.remove('selected'));
span.classList.add('selected');
selected = e;
emojiHidden.value = e;
});
emojiRow.appendChild(span);
});
// ── Step toggle (server tells us which step via query param) ─────
const params = new URLSearchParams(location.search);
if (params.get('step') === '2') {
document.getElementById('step-password').style.display = 'none';
document.getElementById('step-persona').style.display = 'block';
}
if (params.get('step') === '3') {
document.getElementById('step-password').style.display = 'none';
document.getElementById('step-persona').style.display = 'none';
document.getElementById('step-model').style.display = 'block';
}
// ── Client-side confirm password check ───────────────────────────
document.getElementById('password-form').addEventListener('submit', e => {
const pw = document.getElementById('password').value;
const cfm = document.getElementById('confirm').value;
if (pw !== cfm) {
e.preventDefault();
alert('Passwords do not match.');
}
});
// ── Skip model setup — navigate to user home ─────────────────────
document.getElementById('skip-model-link')?.addEventListener('click', e => {
e.preventDefault();
// Ask server for skip target (the cx_setup_persona cookie has the path)
fetch('/setup/model/skip', { method: 'POST', credentials: 'same-origin' })
.then(r => { if (r.redirected) location.href = r.url; else location.href = '/'; })
.catch(() => { location.href = '/'; });
});
// ── Auto-generate persona slug from display name ─────────────────
document.getElementById('display_name').addEventListener('input', function() {
const slugField = document.getElementById('persona_name');
if (!slugField._touched) {
slugField.value = this.value
.toLowerCase()
.replace(/[^a-z0-9_-]/g, '')
.slice(0, 32);
}
});
document.getElementById('persona_name').addEventListener('input', function() {
this._touched = true;
});
</script>
</body>
</html>

1794
cortex/static/style.css Normal file

File diff suppressed because it is too large Load Diff

106
cortex/static/sw.js Normal file
View File

@@ -0,0 +1,106 @@
const CACHE = 'cortex-v2';
const PRECACHE = [
'/static/style.css',
'/static/app.js',
'/static/marked.min.js',
'/static/icon-192.png',
'/static/icon-512.png',
'/static/icon.svg',
'/static/manifest.json',
];
self.addEventListener('install', evt => {
evt.waitUntil(
caches.open(CACHE)
.then(c => c.addAll(PRECACHE))
.then(() => self.skipWaiting())
);
});
self.addEventListener('activate', evt => {
evt.waitUntil(
caches.keys()
.then(keys => Promise.all(
keys.filter(k => k !== CACHE).map(k => caches.delete(k))
))
.then(() => self.clients.claim())
);
});
self.addEventListener('push', evt => {
let data = { title: 'Cortex', body: '', url: '/' };
if (evt.data) {
try { data = { ...data, ...evt.data.json() }; } catch (_) {}
}
evt.waitUntil(
self.registration.showNotification(data.title, {
body: data.body,
icon: '/static/icon-192.png',
badge: '/static/icon-192.png',
data: { url: data.url },
})
);
});
self.addEventListener('notificationclick', evt => {
evt.notification.close();
const url = evt.notification.data?.url || '/';
evt.waitUntil(
clients.matchAll({ type: 'window', includeUncontrolled: true }).then(list => {
for (const c of list) {
if (c.url.includes(self.location.origin) && 'focus' in c) {
c.navigate(url);
return c.focus();
}
}
if (clients.openWindow) return clients.openWindow(url);
})
);
});
self.addEventListener('fetch', evt => {
const url = new URL(evt.request.url);
// Only handle same-origin GETs
if (evt.request.method !== 'GET' || url.origin !== self.location.origin) return;
// Never intercept streaming or API calls
if (
url.pathname.startsWith('/chat') ||
url.pathname.startsWith('/orchestrate') ||
url.pathname.startsWith('/api/') ||
url.pathname.startsWith('/distill') ||
url.pathname.startsWith('/webhook') ||
url.pathname.startsWith('/auth/')
) return;
// Static assets — cache first, refresh in background (stale-while-revalidate)
if (url.pathname.startsWith('/static/')) {
evt.respondWith(
caches.open(CACHE).then(cache =>
cache.match(evt.request).then(cached => {
const network = fetch(evt.request).then(resp => {
if (resp.ok) cache.put(evt.request, resp.clone());
return resp;
});
return cached || network;
})
)
);
return;
}
// HTML pages — network first, cached shell fallback
evt.respondWith(
fetch(evt.request)
.then(resp => {
if (resp.ok) {
const clone = resp.clone();
caches.open(CACHE).then(c => c.put(evt.request, clone));
}
return resp;
})
.catch(() => caches.match(evt.request))
);
});

View File

@@ -0,0 +1,213 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Tool Settings — Cortex</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Server-generated tool table ── */
.table-section-label {
font-size: 0.7rem; font-weight: 700; letter-spacing: 0.08em;
text-transform: uppercase; color: var(--pg-dimmer);
margin: 1.75rem 0 0.6rem;
}
.tool-table {
width: 100%; border-collapse: collapse;
background: var(--pg-surface); border: 1px solid var(--pg-border);
border-radius: 0.75rem; overflow: hidden; margin-bottom: 0.5rem;
font-size: 0.85rem;
}
.tool-table th {
text-align: left; padding: 0.55rem 0.9rem;
border-bottom: 1px solid var(--pg-border);
color: var(--pg-muted); font-weight: 600; font-size: 0.78rem;
text-transform: uppercase; letter-spacing: 0.04em;
}
.tool-table td { padding: 0.5rem 0.9rem; border-bottom: 1px solid var(--pg-border); vertical-align: middle; }
.tool-table tr:last-child td { border-bottom: none; }
.tool-table tr:hover td { background: rgba(124,58,237,0.04); }
.tool-name { font-family: monospace; font-size: 0.82rem; }
/* Risk badges (server-generated) */
.risk { display: inline-block; font-size: 0.7rem; font-weight: 700;
padding: 0.15rem 0.45rem; border-radius: 9999px; letter-spacing: 0.04em; }
.risk-low { background: rgba(34,197,94,0.12); color: #4ade80; }
.risk-medium { background: rgba(234,179,8,0.12); color: #fbbf24; }
.risk-high { background: rgba(239,68,68,0.12); color: #f87171; }
[data-theme="light"] .risk-low { background: rgba(34,197,94,0.15); color: #16a34a; }
[data-theme="light"] .risk-medium { background: rgba(234,179,8,0.15); color: #ca8a04; }
[data-theme="light"] .risk-high { background: rgba(239,68,68,0.15); color: #dc2626; }
/* Auto-status pill (server-generated, updated by JS) */
.auto-pill {
display: inline-block; font-size: 0.68rem; font-weight: 600;
padding: 0.12rem 0.4rem; border-radius: 9999px;
}
.auto-on { background: rgba(124,58,237,0.12); color: #a78bfa; }
.auto-off { background: rgba(148,163,184,0.12); color: var(--pg-dimmer); }
[data-theme="light"] .auto-on { color: #7c3aed; }
/* Override select (server-generated) */
.override-sel {
font-size: 0.78rem; padding: 0.25rem 0.5rem;
border-radius: 0.3rem; min-width: 7rem; width: auto;
}
.override-sel.forced-on { border-color: #7c3aed; color: #7c3aed; }
.override-sel.forced-off { border-color: #dc2626; color: #dc2626; }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link active">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Tool Settings</h1>
<p class="page-subtitle">
Control which orchestrator tools are available. The risk level sets an automatic threshold;
whitelist and blacklist let you fine-tune individual tools beyond that.
</p>
<!-- SUCCESS -->
<!-- ERROR -->
<form method="POST" action="/settings/tools" id="tools-form">
<!-- Risk policy card -->
<div class="rounded-xl border border-pg-border bg-pg-surface p-5 mb-5">
<h2 class="text-sm font-semibold text-pg-bright mb-4">Risk Policy</h2>
<div class="flex items-center gap-4 flex-wrap mb-3">
<span class="text-sm font-medium text-pg-text min-w-[6rem]">Max risk level</span>
<select name="max_risk" id="max-risk-sel" class="w-auto">
<option value="" {{ sel_none }}>No filter — use all role-permitted tools</option>
<option value="low" {{ sel_low }}>Low — read-only and sandboxed tools only</option>
<option value="medium" {{ sel_medium }}>Medium — low + medium risk (recommended)</option>
<option value="high" {{ sel_high }}>High — all tools including destructive ones</option>
</select>
</div>
<p class="text-xs text-pg-muted leading-relaxed mb-2">
<strong class="text-pg-text">Low</strong> tools are read-only and sandboxed (web search, project file reads, HA status checks).<br>
<strong class="text-pg-text">Medium</strong> tools write to local data or send notifications to you (cron jobs, scratch, task management).<br>
<strong class="text-pg-text">High</strong> tools affect external systems or the host (shell exec, email, device control, service restart).
</p>
<p class="text-xs text-pg-muted leading-relaxed">
The <em>Auto</em> column below shows each tool's status at your current max risk level.
Use the override column to force-include or force-exclude individual tools.
</p>
</div>
<!-- Legend -->
<div class="flex gap-5 flex-wrap mb-4 text-xs text-pg-muted">
<span><span class="inline-block w-2 h-2 rounded-full bg-[#a78bfa] mr-1.5"></span>Auto-included by risk level</span>
<span><span class="inline-block w-2 h-2 rounded-full bg-pg-dimmer mr-1.5"></span>Auto-excluded by risk level</span>
</div>
<!-- Tool table (server-generated) -->
{{ tool_table_html }}
<!-- Confirmation gate card -->
<div class="rounded-xl border border-pg-border bg-pg-surface p-5 mt-5 mb-5">
<h2 class="text-sm font-semibold text-pg-bright mb-2">Confirmation Gate</h2>
<p class="text-xs text-pg-muted leading-relaxed mb-4">
Some tools require explicit confirmation before executing. Override the defaults here.<br>
Tools requiring confirmation by default: <code class="font-mono text-pg-accent bg-pg-bg border border-pg-border rounded px-1">{{ confirm_required_tools }}</code>
</p>
<div class="flex gap-6 flex-wrap items-start">
<div class="flex-1 min-w-[200px]">
<label class="block text-xs font-semibold text-pg-muted mb-1">Allow list — bypass confirmation</label>
<textarea name="allow_list" rows="4"
placeholder="reminders_clear&#10;cron_remove"
autocomplete="off" spellcheck="false">{{ tool_allow }}</textarea>
<p class="hint">One tool name per line. These tools skip the confirmation prompt.</p>
</div>
<div class="flex-1 min-w-[200px]">
<label class="block text-xs font-semibold text-pg-muted mb-1">Deny list — always block</label>
<textarea name="deny_list" rows="4"
placeholder="shell_exec&#10;file_write"
autocomplete="off" spellcheck="false">{{ tool_deny }}</textarea>
<p class="hint">These tools are always blocked regardless of risk policy.</p>
</div>
</div>
</div>
<div class="mt-4">
<button type="submit" class="btn-submit w-full md:w-96">Save tool settings</button>
</div>
</form>
</div>
<script>
const riskRank = { "": 99, "low": 0, "medium": 1, "high": 2 };
const toolRisk = {{ tool_risk_json }};
const sel = document.getElementById('max-risk-sel');
function updateAutoPills() {
const maxRank = riskRank[sel.value] ?? 99;
document.querySelectorAll('[data-tool-risk]').forEach(row => {
const risk = row.dataset.toolRisk;
const pill = row.querySelector('.auto-pill');
const isAuto = riskRank[risk] <= maxRank;
pill.textContent = isAuto ? 'auto ✓' : 'excluded';
pill.className = 'auto-pill ' + (isAuto ? 'auto-on' : 'auto-off');
});
}
sel.addEventListener('change', updateAutoPills);
updateAutoPills();
// Color the override selects
document.querySelectorAll('.override-sel').forEach(s => {
function refresh() {
s.className = 'override-sel';
if (s.value === 'whitelist') s.classList.add('forced-on');
if (s.value === 'blacklist') s.classList.add('forced-off');
}
s.addEventListener('change', refresh);
refresh();
});
</script>
</body>
</html>

119
cortex/tests/conftest.py Normal file
View File

@@ -0,0 +1,119 @@
"""
Shared fixtures for Cortex test suite.
Key design choices:
- All file I/O goes to a tmp_path, never touching home/ or real sessions.
- LLM calls are mocked by default — tests are fast and deterministic.
- The 'client' fixture patches settings before importing main, so all modules
see the temp directory.
Home layout mirrors the two-level structure:
tmp/
scott/
persona/
inara/ ← the default test persona
holly/
persona/
tina/
"""
import json
import pytest
import pytest_asyncio
from pathlib import Path
from unittest.mock import AsyncMock, patch
import httpx
from httpx import ASGITransport
# ---------------------------------------------------------------------------
# Temp home directory
# ---------------------------------------------------------------------------
@pytest.fixture(scope="session")
def home_root(tmp_path_factory) -> Path:
"""A temp home/ dir with minimal user/persona stubs for testing."""
root = tmp_path_factory.mktemp("home")
_make_persona(root, "scott", "inara", "Inara", "Scott")
_make_persona(root, "holly", "tina", "Tina", "Holly")
return root
def _make_persona(root: Path, username: str, persona: str,
agent: str, user: str) -> Path:
p = root / username / "persona" / persona
p.mkdir(parents=True, exist_ok=True)
(p / "IDENTITY.md").write_text(f"# {agent}\nTest identity for {agent}.")
(p / "SOUL.md").write_text(f"# Soul\nTest soul for {agent}.")
(p / "PROTOCOLS.md").write_text("# Protocols\nBe helpful.")
(p / "USER.md").write_text(f"# {user}\nTest user profile.")
(p / "HELP.md").write_text("# Help\nTest help content.")
(p / "MEMORY_LONG.md").write_text("Not yet populated.")
(p / "MEMORY_MID.md").write_text("Not yet populated.")
(p / "MEMORY_SHORT.md").write_text("Not yet populated.")
(p / "TASKS.json").write_text("[]")
(p / "CRONS.json").write_text("[]")
(p / "SCRATCH.md").write_text("")
(p / "REMINDERS.md").write_text("")
(p / "sessions").mkdir()
return p
# ---------------------------------------------------------------------------
# App fixture — patches settings before the ASGI app is started
# ---------------------------------------------------------------------------
@pytest_asyncio.fixture
async def client(home_root, tmp_path):
"""
HTTPX async test client with a valid session cookie for 'scott'.
The auth middleware is active but a JWT cookie is pre-set so API tests
don't need to go through the login flow.
"""
import config
import persona as persona_mod
sessions_dir = tmp_path / "sessions"
sessions_dir.mkdir()
with (
patch.object(config.settings, "home_dir", home_root),
patch.object(config.settings, "sessions_dir", sessions_dir),
patch.object(config.settings, "jwt_secret", "test-secret-key-xxxxxxxxxxxxxxxx"),
patch("scheduler.start"), # don't run APScheduler in tests
patch("scheduler.stop"),
):
persona_mod.set_context("scott", "inara")
from main import app
from auth_utils import create_token
token = create_token("scott")
async with httpx.AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
cookies={"cortex_session": token},
) as c:
yield c
# ---------------------------------------------------------------------------
# LLM mock
# ---------------------------------------------------------------------------
@pytest.fixture
def mock_llm():
"""
Patch complete() at every import site so no real LLM calls are made.
Each router does `from llm_client import complete`, creating a local reference.
Patching llm_client.complete alone won't intercept those — patch each site.
"""
ret = ("Hello, I am a test response.", "claude")
with (
patch("routers.chat.complete", new_callable=AsyncMock, return_value=ret),
patch("routers.nextcloud_talk.complete", new_callable=AsyncMock, return_value=ret),
patch("routers.google_chat.complete", new_callable=AsyncMock, return_value=ret),
patch("llm_client.complete", new_callable=AsyncMock, return_value=ret),
):
yield

View File

@@ -0,0 +1,876 @@
"""
Tests for agent_manager.py and the spawn_agent / aider_run background paths.
Run with:
cd cortex && .venv/bin/python -m pytest tests/test_agent_manager.py -v
No browser, no LLM calls, no Cortex service needed. All LLM interactions are mocked.
The agent_manager tests need no mocks at all — the module is pure asyncio.
"""
import asyncio
import pytest
import pytest_asyncio
from datetime import datetime, timedelta
from unittest.mock import AsyncMock, MagicMock, patch
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_mock_result(response: str = "Agent done."):
"""Build a mock OrchestratorResult returned by openai_orchestrator.run."""
r = MagicMock()
r.checkpoint = None
r.response = response
return r
def _mock_spawn_deps(
model_type: str = "local_openai",
user_role: str = "admin",
tool_policy: dict | None = None,
role_tools: list | None = None,
):
"""Return a context-manager stack that patches all spawn_agent external deps."""
if tool_policy is None:
tool_policy = {"allow": [], "deny": []}
model_cfg = {
"type": model_type,
"api_url": "http://localhost:3000",
"model_name": "test-model",
"api_key": "x",
}
role_cfg = {
"tools": role_tools,
"system_append": "",
"inject_datetime": True,
"inject_mode": True,
}
class _Stack:
def __enter__(self_):
self_._patches = [
patch("model_registry.get_role_config", return_value=role_cfg),
patch("model_registry.get_model_for_role", return_value=model_cfg),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("context_loader.load_context", return_value="Test system prompt"),
patch("auth_utils.get_user_role", return_value=user_role),
patch("auth_utils.get_tool_policy", return_value=tool_policy),
patch("persona.get_user", return_value="scott"),
]
for p in self_._patches:
p.start()
return self_
def __exit__(self_, *args):
for p in self_._patches:
p.stop()
return _Stack()
# ---------------------------------------------------------------------------
# Fixture — reset agent_manager state between tests
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def clear_agent_registry():
"""Wipe the in-process agent registry before each test."""
import agent_manager
agent_manager._agents.clear()
yield
agent_manager._agents.clear()
# ---------------------------------------------------------------------------
# agent_manager — core CRUD
# ---------------------------------------------------------------------------
class TestAgentManagerCore:
@pytest.mark.asyncio
async def test_register_creates_record(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="research", task="Investigate topic X", level=2
)
assert rec.agent_id in agent_manager._agents
assert rec.status == "running"
assert rec.level == 2
assert rec.role == "research"
assert rec.task == "Investigate topic X"
assert rec.user == "scott"
assert rec.finished is None
@pytest.mark.asyncio
async def test_register_truncates_long_task(self):
import agent_manager
long_task = "x" * 500
rec = await agent_manager.register(user="scott", role="chat", task=long_task, level=2)
assert len(rec.task) == 200
@pytest.mark.asyncio
async def test_finish_updates_record(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "All done!", "done")
updated = agent_manager.get(rec.agent_id)
assert updated.status == "done"
assert updated.result == "All done!"
assert updated.finished is not None
@pytest.mark.asyncio
async def test_finish_truncates_result(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "y" * 2000)
updated = agent_manager.get(rec.agent_id)
assert len(updated.result) <= agent_manager._RESULT_PREVIEW_CHARS
@pytest.mark.asyncio
async def test_finish_failed_status(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "Boom", "failed")
assert agent_manager.get(rec.agent_id).status == "failed"
@pytest.mark.asyncio
async def test_cancel_own_agent(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
msg = await agent_manager.cancel_agent(rec.agent_id, "scott")
assert "cancelled" in msg
assert agent_manager.get(rec.agent_id).status == "cancelled"
@pytest.mark.asyncio
async def test_cancel_wrong_user_denied(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
msg = await agent_manager.cancel_agent(rec.agent_id, "holly")
assert "denied" in msg.lower()
assert agent_manager.get(rec.agent_id).status == "running"
@pytest.mark.asyncio
async def test_cancel_nonexistent_agent(self):
import agent_manager
msg = await agent_manager.cancel_agent("does-not-exist", "scott")
assert "No agent found" in msg
@pytest.mark.asyncio
async def test_cancel_already_done(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "done", "done")
msg = await agent_manager.cancel_agent(rec.agent_id, "scott")
assert "already" in msg or "done" in msg
@pytest.mark.asyncio
async def test_cancel_kills_real_task(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
sleep_task = asyncio.create_task(asyncio.sleep(60))
agent_manager.set_task_ref(rec.agent_id, sleep_task)
await agent_manager.cancel_agent(rec.agent_id, "scott")
await asyncio.sleep(0) # let the event loop process the cancellation
assert sleep_task.cancelled() or sleep_task.done()
def test_list_agents_returns_users_agents(self):
import agent_manager
# Manually populate the registry
agent_manager._agents["a1"] = _make_record("a1", "scott", "running")
agent_manager._agents["a2"] = _make_record("a2", "scott", "done")
agent_manager._agents["a3"] = _make_record("a3", "holly", "running")
records = agent_manager.list_agents("scott")
ids = {r.agent_id for r in records}
assert "a1" in ids
assert "a2" in ids
assert "a3" not in ids
def test_list_agents_filters_by_status(self):
import agent_manager
agent_manager._agents["a1"] = _make_record("a1", "scott", "running")
agent_manager._agents["a2"] = _make_record("a2", "scott", "done")
running = agent_manager.list_agents("scott", status="running")
assert len(running) == 1
assert running[0].agent_id == "a1"
def test_list_agents_respects_limit(self):
import agent_manager
for i in range(20):
agent_manager._agents[f"a{i}"] = _make_record(f"a{i}", "scott", "done")
records = agent_manager.list_agents("scott", limit=5)
assert len(records) == 5
@pytest.mark.asyncio
async def test_prune_removes_old_completed(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "done")
# Manually backdate the finished time past the prune threshold
agent_manager._agents[rec.agent_id].finished = (
datetime.now() - agent_manager._PRUNE_AFTER - timedelta(seconds=1)
)
# Trigger pruning via a new registration
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
assert agent_manager.get(rec.agent_id) is None
@pytest.mark.asyncio
async def test_prune_keeps_running_agents(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
# Running agent — finished is None so it should never be pruned
assert rec.agent_id in agent_manager._agents
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
assert agent_manager.get(rec.agent_id) is not None
@pytest.mark.asyncio
async def test_finish_unknown_agent_is_noop(self):
import agent_manager
# Should not raise
await agent_manager.finish("ghost-id", "result", "done")
# ---------------------------------------------------------------------------
# agent_manager — notification hook
# ---------------------------------------------------------------------------
class TestAgentManagerNotify:
@pytest.mark.asyncio
async def test_notify_called_on_done(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=True
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "All good", "done")
mock_notify.assert_called_once()
call_args = mock_notify.call_args
assert call_args[0][0] == "scott" # user
assert "" in call_args[0][1] # success emoji
@pytest.mark.asyncio
async def test_notify_called_on_failed(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=True
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "Oops", "failed")
mock_notify.assert_called_once()
assert "⚠️" in mock_notify.call_args[0][1]
@pytest.mark.asyncio
async def test_no_notify_when_cancelled(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=True
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
mock_notify.assert_not_called()
@pytest.mark.asyncio
async def test_no_notify_when_flag_false(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=False
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "Done", "done")
mock_notify.assert_not_called()
# ---------------------------------------------------------------------------
# spawn_agent — background mode
# ---------------------------------------------------------------------------
class TestSpawnAgentBackground:
@pytest.mark.asyncio
async def test_background_returns_agent_id_immediately(self):
import agent_manager
from tools.agents import spawn_agent
mock_result = _make_mock_result("Research complete.")
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
result = await spawn_agent(
task="Test background research",
role="research",
background=True,
)
assert "Agent started in background" in result
assert "ID:" in result
@pytest.mark.asyncio
async def test_background_registers_agent(self):
import agent_manager
from tools.agents import spawn_agent
mock_result = _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
await spawn_agent(task="Background task", background=True)
agents = agent_manager.list_agents("scott")
assert len(agents) >= 1
@pytest.mark.asyncio
async def test_background_agent_eventually_completes(self):
import agent_manager
from tools.agents import spawn_agent
mock_result = _make_mock_result("Task done!")
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
result = await spawn_agent(task="Quick task", background=True)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
# Poll while patches are still active
for _ in range(40):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
rec = agent_manager.get(agent_id)
assert rec is not None
assert rec.status == "done"
assert "Task done!" in (rec.result or "")
@pytest.mark.asyncio
async def test_background_sync_path_unchanged(self):
"""Verify that background=False still blocks and returns the result string."""
from tools.agents import spawn_agent
mock_result = _make_mock_result("Sync result here.")
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
result = await spawn_agent(task="Sync task", background=False)
assert result == "Sync result here."
@pytest.mark.asyncio
async def test_background_agent_timeout(self):
import agent_manager
from tools.agents import spawn_agent
async def _slow(*args, **kwargs):
await asyncio.sleep(60)
return _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", side_effect=_slow):
result = await spawn_agent(task="Slow task", background=True, timeout=1)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
# Poll while patches are still active (timeout=1s so this completes quickly)
for _ in range(60):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
rec = agent_manager.get(agent_id)
assert rec.status == "timeout"
@pytest.mark.asyncio
async def test_background_agent_failure(self):
import agent_manager
from tools.agents import spawn_agent
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, side_effect=RuntimeError("Boom")):
result = await spawn_agent(task="Failing task", background=True)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
for _ in range(20):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
assert agent_manager.get(agent_id).status == "failed"
# ---------------------------------------------------------------------------
# spawn_agent — level enforcement
# ---------------------------------------------------------------------------
class TestLevelEnforcement:
@pytest.mark.asyncio
async def test_l2_parent_denies_spawn_in_l3_child(self):
"""Level 2 agent spawning a child: spawn_agent and aider_run must be denied."""
from tools.agents import spawn_agent
captured_kwargs = {}
async def _capture_run(**kwargs):
captured_kwargs.update(kwargs)
return _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", side_effect=_capture_run):
await spawn_agent(
task="Test L3 enforcement",
background=False,
_agent_level=2, # this agent is Level 2; its child would be Level 3
)
# The orchestrator should have received spawn_agent and aider_run in confirm_deny
confirm_deny = captured_kwargs.get("confirm_deny", set())
assert "spawn_agent" in confirm_deny, "spawn_agent must be blocked for L3 children"
assert "aider_run" in confirm_deny, "aider_run must be blocked for L3 children"
@pytest.mark.asyncio
async def test_l1_parent_does_not_deny_spawn(self):
"""Level 1 agent (persona) spawning a Level 2 child: no extra denies."""
from tools.agents import spawn_agent
captured_kwargs = {}
async def _capture_run(**kwargs):
captured_kwargs.update(kwargs)
return _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", side_effect=_capture_run):
await spawn_agent(
task="Test L2 spawn",
background=False,
_agent_level=1, # persona is Level 1; child would be Level 2
)
confirm_deny = captured_kwargs.get("confirm_deny", set())
assert "spawn_agent" not in confirm_deny, "L2 agents must be allowed to spawn"
@pytest.mark.asyncio
async def test_l2_deny_intersected_with_tool_list(self):
"""When the role has an explicit tool_list, L3 deny removes from list directly."""
from tools.agents import spawn_agent
captured_kwargs = {}
async def _capture_run(**kwargs):
captured_kwargs.update(kwargs)
return _make_mock_result()
# Role has an explicit tool_list that includes spawn_agent
with _mock_spawn_deps(role_tools=["web_search", "spawn_agent", "aider_run"]):
with patch("openai_orchestrator.run", side_effect=_capture_run):
await spawn_agent(
task="Test",
background=False,
_agent_level=2,
)
# spawn_agent and aider_run must be absent from the tool_list passed to orchestrator
tool_list = captured_kwargs.get("tool_list", [])
assert "spawn_agent" not in tool_list
assert "aider_run" not in tool_list
assert "web_search" in tool_list # unrelated tools must survive
# ---------------------------------------------------------------------------
# Agent lifecycle tools — output formatting
# ---------------------------------------------------------------------------
class TestAgentLifecycleTools:
@pytest.mark.asyncio
async def test_agent_status_running(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="research", task="Do research", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_status
output = await agent_status(rec.agent_id)
assert "running" in output
assert "research" in output
assert rec.agent_id[:8] in output
@pytest.mark.asyncio
async def test_agent_status_done(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="Task", level=2)
await agent_manager.finish(rec.agent_id, "The result text", "done")
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_status
output = await agent_status(rec.agent_id)
assert "done" in output
assert "The result text" in output
@pytest.mark.asyncio
async def test_agent_status_wrong_user(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
with patch("persona.get_user", return_value="holly"):
from tools.agents import agent_status
output = await agent_status(rec.agent_id)
assert "denied" in output.lower()
@pytest.mark.asyncio
async def test_agent_status_not_found(self):
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_status
output = await agent_status("nonexistent-id")
assert "No agent found" in output
@pytest.mark.asyncio
async def test_agent_list_shows_running(self):
import agent_manager
await agent_manager.register(user="scott", role="research", task="Research X", level=2)
await agent_manager.register(user="scott", role="coder", task="Fix bug", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_list
output = await agent_list()
assert "2 agent(s)" in output
assert "research" in output
assert "coder" in output
@pytest.mark.asyncio
async def test_agent_list_status_filter(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "done", "done")
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_list
output = await agent_list(status="running")
assert "1 agent(s)" in output
@pytest.mark.asyncio
async def test_agent_list_empty(self):
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_list
output = await agent_list()
assert "No agents found" in output
@pytest.mark.asyncio
async def test_agent_cancel_tool(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_cancel
output = await agent_cancel(rec.agent_id)
assert "cancelled" in output
assert agent_manager.get(rec.agent_id).status == "cancelled"
# ---------------------------------------------------------------------------
# aider_run — background mode
# ---------------------------------------------------------------------------
class TestAiderRunBackground:
@pytest.mark.asyncio
async def test_background_returns_agent_id(self):
import agent_manager
async def _fake_proc(*args, **kwargs):
mock_proc = MagicMock()
mock_proc.communicate = AsyncMock(return_value=(b"All changes applied.", b""))
mock_proc.returncode = 0
return mock_proc
with (
patch("persona.get_user", return_value="scott"),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
):
from tools.aider import aider_run
result = await aider_run(
project=str(_CORTEX_DIR.parent), # use actual project root (exists)
task="Test background task",
background=True,
)
assert "Aider task started in background" in result
assert "ID:" in result
@pytest.mark.asyncio
async def test_background_agent_completes(self):
import agent_manager
async def _fake_proc(*args, **kwargs):
mock_proc = MagicMock()
mock_proc.communicate = AsyncMock(return_value=(b"Edits applied.", b""))
mock_proc.returncode = 0
return mock_proc
from tools.aider import aider_run
with (
patch("persona.get_user", return_value="scott"),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
):
result = await aider_run(
project=str(_CORTEX_DIR.parent),
task="Test",
background=True,
)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
# Poll while patches are still active
for _ in range(40):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
rec = agent_manager.get(agent_id)
assert rec.status == "done"
assert "Edits applied" in (rec.result or "")
@pytest.mark.asyncio
async def test_invalid_project_directory(self):
from tools.aider import aider_run
result = await aider_run(project="/this/does/not/exist", task="Test")
assert "does not exist" in result
@pytest.mark.asyncio
async def test_sync_path_still_works(self):
async def _fake_proc(*args, **kwargs):
mock_proc = MagicMock()
mock_proc.communicate = AsyncMock(return_value=(b"Done.", b""))
mock_proc.returncode = 0
return mock_proc
with (
patch("persona.get_user", return_value="scott"),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
):
from tools.aider import aider_run
result = await aider_run(
project=str(_CORTEX_DIR.parent),
task="Sync test",
background=False,
)
assert "Done." in result
# ---------------------------------------------------------------------------
# aider_run — credential resolver (_resolve_credentials)
# ---------------------------------------------------------------------------
class TestAiderCredentialResolver:
"""Pure unit tests for _resolve_credentials — no subprocess, no registry I/O."""
def _registry(self, hosts=None, anthropic_key=None):
reg = {"hosts": hosts or [], "providers": {}}
if anthropic_key:
reg["providers"]["anthropic"] = {
"credentials": [{"api_key": anthropic_key}]
}
return reg
def _host(self, label, api_url, api_key="sk-test", host_type="openai"):
return {"id": "x", "label": label, "api_url": api_url,
"api_key": api_key, "host_type": host_type}
# --- Provider detection ---
def test_openrouter_host_gets_api_key_flag(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, model = _resolve_credentials(reg, None, None)
assert "--api-key" in flags
assert "openrouter=or-key" in flags
def test_anthropic_model_hint_uses_provider_key(self):
from tools.aider import _resolve_credentials
reg = self._registry(
hosts=[self._host("OpenRouter", "https://openrouter.ai/api/v1")],
anthropic_key="ant-key",
)
flags, model = _resolve_credentials(reg, "claude-3-5-sonnet-20241022", None)
assert "anthropic=ant-key" in flags
assert model == "claude-3-5-sonnet-20241022"
def test_anthropic_slash_prefix_hint(self):
from tools.aider import _resolve_credentials
reg = self._registry(anthropic_key="ant-key")
flags, _ = _resolve_credentials(reg, "anthropic/claude-opus-4", None)
assert "anthropic=ant-key" in flags
def test_local_openwebui_host_gets_base_url(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://192.168.32.19:3000", "localkey", host_type="openwebui"),
])
flags, model = _resolve_credentials(reg, None, None)
assert "--openai-api-base" in flags
base = flags[flags.index("--openai-api-base") + 1]
assert base == "http://192.168.32.19:3000/api"
assert "--openai-api-key" in flags
def test_local_host_appends_api_suffix_for_openwebui(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenWebUI", "http://localhost:3000", host_type="openwebui"),
])
flags, _ = _resolve_credentials(reg, None, None)
base = flags[flags.index("--openai-api-base") + 1]
assert base.endswith("/api")
def test_generic_openai_host_no_api_suffix(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Custom", "http://localhost:8080/v1", host_type="openai"),
])
flags, _ = _resolve_credentials(reg, None, None)
base = flags[flags.index("--openai-api-base") + 1]
assert not base.endswith("/api")
assert base == "http://localhost:8080/v1"
# --- Model name adjustment ---
def test_local_host_prefixes_model_without_slash(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
])
_, model = _resolve_credentials(reg, "gemma-4-27b-it", None)
assert model == "openai/gemma-4-27b-it"
def test_local_host_leaves_model_with_slash(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
])
_, model = _resolve_credentials(reg, "ollama/gemma4", None)
assert model == "ollama/gemma4" # already prefixed, don't touch
def test_cloud_provider_does_not_prefix_model(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1"),
])
_, model = _resolve_credentials(reg, "google/gemma-3-27b-it", None)
assert model == "google/gemma-3-27b-it"
# --- Host label override ---
def test_host_label_selects_local_over_openrouter(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
self._host("Local RTX", "http://192.168.32.19:3000", "local-key", host_type="openwebui"),
])
flags, _ = _resolve_credentials(reg, None, "Local")
assert "--openai-api-base" in flags
assert "--api-key" not in flags
def test_host_label_case_insensitive(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, _ = _resolve_credentials(reg, None, "openrouter")
assert "openrouter=or-key" in flags
# --- Model prefix routing ---
def test_model_openrouter_prefix_routes_to_openrouter(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, model = _resolve_credentials(reg, "openrouter/google/gemma-3-27b-it", None)
assert "openrouter=or-key" in flags
assert model == "openrouter/google/gemma-3-27b-it"
def test_model_groq_prefix_routes_to_groq_host(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Groq", "https://api.groq.com/openai/v1", "groq-key"),
])
flags, _ = _resolve_credentials(reg, "groq/llama-3.3-70b", None)
assert "groq=groq-key" in flags
# --- Default fallback priority ---
def test_prefers_openrouter_over_local_when_no_hint(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, _ = _resolve_credentials(reg, None, None)
assert "openrouter=or-key" in flags
def test_prefers_anthropic_over_local_when_no_openrouter(self):
from tools.aider import _resolve_credentials
reg = self._registry(
hosts=[self._host("Local", "http://localhost:3000", host_type="openwebui")],
anthropic_key="ant-key",
)
flags, _ = _resolve_credentials(reg, None, None)
assert "anthropic=ant-key" in flags
def test_empty_registry_returns_no_flags(self):
from tools.aider import _resolve_credentials
flags, model = _resolve_credentials({}, "gemma-4", None)
assert flags == []
assert model == "gemma-4"
# ---------------------------------------------------------------------------
# Helpers for manual test record creation (used in list tests above)
# ---------------------------------------------------------------------------
import agent_manager as _am
_CORTEX_DIR = _am.__file__ and _am and __import__("pathlib").Path(_am.__file__).parent
def _make_record(agent_id: str, user: str, status: str) -> "_am.AgentRecord":
from datetime import datetime
import agent_manager
rec = agent_manager.AgentRecord(
agent_id=agent_id,
level=2,
role="chat",
task="test task",
status=status,
started=datetime.now(),
user=user,
finished=datetime.now() if status != "running" else None,
)
return rec

View File

@@ -0,0 +1,122 @@
"""
Tests for POST /chat — persona routing, session handling, LLM mocking.
"""
import json
import pytest
def _parse_sse(text: str) -> list[dict]:
"""Extract JSON payloads from an SSE response body."""
events = []
for line in text.splitlines():
if line.startswith("data: "):
try:
events.append(json.loads(line[6:]))
except json.JSONDecodeError:
pass
return events
@pytest.mark.anyio
async def test_chat_basic(client, mock_llm):
r = await client.post("/chat", json={"message": "Hello", "persona": "inara"})
assert r.status_code == 200
events = _parse_sse(r.text)
responses = [e for e in events if e.get("type") == "response"]
assert len(responses) == 1
assert responses[0]["response"] == "Hello, I am a test response."
assert responses[0]["backend"] == "claude"
assert "session_id" in responses[0]
@pytest.mark.anyio
async def test_chat_default_persona(client, mock_llm):
"""persona defaults to 'inara' when not specified."""
r = await client.post("/chat", json={"message": "Hi"})
assert r.status_code == 200
events = _parse_sse(r.text)
assert any(e.get("type") == "response" for e in events)
@pytest.mark.anyio
async def test_chat_unknown_persona(client, mock_llm):
r = await client.post("/chat", json={"message": "Hi", "persona": "nobody"})
assert r.status_code == 200
events = _parse_sse(r.text)
errors = [e for e in events if e.get("type") == "error"]
assert len(errors) == 1
assert "nobody" in errors[0]["message"]
@pytest.mark.anyio
async def test_chat_path_traversal_persona(client, mock_llm):
r = await client.post("/chat", json={"message": "Hi", "persona": "../../etc"})
assert r.status_code == 200
events = _parse_sse(r.text)
errors = [e for e in events if e.get("type") == "error"]
assert len(errors) == 1
@pytest.mark.anyio
async def test_chat_session_persists(client, mock_llm):
"""Same session_id reuses history."""
r1 = await client.post("/chat", json={"message": "First message", "session_id": "test-sess-1"})
r2 = await client.post("/chat", json={"message": "Second message", "session_id": "test-sess-1"})
assert r1.status_code == 200
assert r2.status_code == 200
@pytest.mark.anyio
async def test_sessions_list(client, mock_llm):
"""After a chat, the session appears in /sessions."""
await client.post("/chat", json={"message": "Hello", "session_id": "test-list-sess"})
r = await client.get("/sessions")
assert r.status_code == 200
sessions = r.json()["sessions"]
ids = [s["session_id"] for s in sessions]
assert "test-list-sess" in ids
@pytest.mark.anyio
async def test_session_history(client, mock_llm):
await client.post("/chat", json={"message": "Hello", "session_id": "test-hist-sess"})
r = await client.get("/history/test-hist-sess")
assert r.status_code == 200
msgs = r.json()["messages"]
assert any(m["role"] == "user" and "Hello" in m["content"] for m in msgs)
@pytest.mark.anyio
async def test_session_delete(client, mock_llm):
await client.post("/chat", json={"message": "Hello", "session_id": "test-del-sess"})
r = await client.delete("/sessions/test-del-sess")
assert r.status_code == 200
assert r.json()["ok"] is True
@pytest.mark.anyio
async def test_session_delete_unknown(client):
r = await client.delete("/sessions/does-not-exist")
assert r.status_code == 404
@pytest.mark.anyio
async def test_backend_get(client):
r = await client.get("/backend")
assert r.status_code == 200
assert r.json()["primary"] in ("claude", "gemini")
@pytest.mark.anyio
async def test_backend_set(client):
r = await client.post("/backend", json={"primary": "gemini"})
assert r.status_code == 200
assert r.json()["primary"] == "gemini"
# Reset
await client.post("/backend", json={"primary": "claude"})
@pytest.mark.anyio
async def test_backend_set_invalid(client):
r = await client.post("/backend", json={"primary": "gpt-4"})
assert r.status_code == 400

View File

@@ -0,0 +1,71 @@
"""
Tests for GET/PUT /files/* — allowed set enforcement, read/write, IDENTITY.md.
"""
import pytest
@pytest.mark.anyio
async def test_files_list(client):
r = await client.get("/files")
assert r.status_code == 200
files = r.json()["files"]
names = [f["name"] for f in files]
assert "SOUL.md" in names
assert "IDENTITY.md" in names
assert "USER.md" in names
@pytest.mark.anyio
async def test_files_get_allowed(client):
r = await client.get("/files/IDENTITY.md")
assert r.status_code == 200
assert "content" in r.json()
@pytest.mark.anyio
async def test_files_get_not_in_allowed(client):
"""Files outside the ALLOWED set should return 404, not the file content."""
# Note: paths with '..' are normalized at the ASGI layer (e.g. /files/../config.py
# becomes /config.py which hits the /{username} UI catch-all, not the files router).
# Only test paths that stay within the files router's scope.
for name in ("TASKS.json", "CRONS.json", "SCRATCH.md", ".env"):
r = await client.get(f"/files/{name}")
assert r.status_code == 404, f"Expected 404 for {name}, got {r.status_code}"
@pytest.mark.anyio
async def test_files_put_and_get(client):
"""Write a new value and read it back."""
content = "# Updated PROTOCOLS\nTest content."
r = await client.put("/files/PROTOCOLS.md", json={"content": content})
assert r.status_code == 200
assert r.json()["ok"] is True
r2 = await client.get("/files/PROTOCOLS.md")
assert r2.status_code == 200
assert r2.json()["content"] == content
@pytest.mark.anyio
async def test_files_put_not_allowed(client):
# '../../etc/passwd' normalizes to '/etc/passwd' at the ASGI layer —
# no route handles PUT there, so 404 or 405 are both acceptable safe responses.
r = await client.put("/files/../../etc/passwd", json={"content": "pwned"})
assert r.status_code in (404, 405)
@pytest.mark.anyio
async def test_files_get_missing_but_allowed(client, home_root):
"""An allowed file that doesn't exist yet returns 404."""
# Temporarily remove MEMORY_MID.md
f = home_root / "scott" / "persona" / "inara" / "MEMORY_MID.md"
existed = f.exists()
if existed:
backup = f.read_text()
f.unlink()
try:
r = await client.get("/files/MEMORY_MID.md")
assert r.status_code == 404
finally:
if existed:
f.write_text(backup)

View File

@@ -0,0 +1,36 @@
"""
Basic smoke tests — if these fail, nothing else matters.
"""
import pytest
@pytest.mark.anyio
async def test_health(client):
r = await client.get("/health")
assert r.status_code == 200
assert r.json()["status"] == "ok"
@pytest.mark.anyio
async def test_auth_status(client):
r = await client.get("/auth/status")
assert r.status_code == 200
data = r.json()
assert "claude" in data
assert "gemini" in data
assert "ok" in data["claude"]
@pytest.mark.anyio
async def test_distill_status(client):
r = await client.get("/distill/status")
assert r.status_code == 200
assert "enabled" in r.json()
@pytest.mark.anyio
async def test_unknown_route_404(client):
# Single-segment paths hit the /{username} persona-picker catch-all (302 redirect).
# Three-segment paths don't match any route pattern → genuine 404.
r = await client.get("/totally/unknown/deep-path")
assert r.status_code == 404

View File

@@ -0,0 +1,805 @@
"""
Unit tests for model_registry.py — no HTTP, no LLM calls, no running service.
All file I/O is redirected to tmp_path via patch.object(config.settings, "home_dir", ...).
Coverage:
- Empty registry (no files)
- Save/load round-trip
- Migration from local_llm.json (v0 flat and v1 hosts/models)
- Host CRUD
- Model CRUD (including role reference cleanup on remove)
- Role assignment (set_role, validation)
- Model resolution (_resolve_model: built-ins, local_openai, missing host/model)
- get_model_for_role: registry chain → .env fallback → hardcoded fallback
- get_best_local_model: role chain, first-local fallback, no-local case
- Backup chain: skips missing models, returns next valid
"""
import json
import pytest
from pathlib import Path
from unittest.mock import patch
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _home(tmp_path: Path) -> Path:
"""Create a minimal home directory and return the root."""
root = tmp_path / "home"
root.mkdir()
return root
def _user_dir(home: Path, username: str = "scott") -> Path:
d = home / username
d.mkdir(exist_ok=True)
return d
def _write_registry(home: Path, data: dict, username: str = "scott") -> Path:
_user_dir(home, username)
path = home / username / "model_registry.json"
path.write_text(json.dumps(data))
return path
def _write_local_llm(home: Path, data: dict, username: str = "scott") -> Path:
_user_dir(home, username)
path = home / username / "local_llm.json"
path.write_text(json.dumps(data))
return path
def _read_registry(home: Path, username: str = "scott") -> dict:
path = home / username / "model_registry.json"
return json.loads(path.read_text())
# ---------------------------------------------------------------------------
# Empty / fresh state
# ---------------------------------------------------------------------------
def test_empty_registry_no_files(tmp_path):
"""With no files, _load returns an empty structure."""
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert data["version"] == 2
assert data["hosts"] == []
assert data["models"] == []
assert data["roles"] == {}
def test_empty_registry_missing_user_dir(tmp_path):
"""Even with no user dir, _load returns an empty structure gracefully."""
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("nobody")
assert data["hosts"] == []
# ---------------------------------------------------------------------------
# Save / load round-trip
# ---------------------------------------------------------------------------
def test_save_and_load(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
registry = {
"version": 1,
"hosts": [{"id": "h1", "label": "ML Box", "api_url": "http://10.0.0.1:3000", "api_key": "sk-test"}],
"models": [{"id": "m1", "type": "local_openai", "label": "Gemma Small",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": ["fast"]}],
"roles": {"chat": {"primary": "m1"}},
}
with patch.object(config.settings, "home_dir", home):
reg._save("scott", registry)
loaded = reg._load("scott")
assert loaded["hosts"][0]["label"] == "ML Box"
assert loaded["models"][0]["model_name"] == "gemma4:e4b"
assert loaded["roles"]["chat"]["primary"] == "m1"
def test_corrupt_registry_falls_back_to_empty(tmp_path):
home = _home(tmp_path)
path = _user_dir(home) / "model_registry.json"
path.write_text("{bad json{{")
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert data["hosts"] == []
# ---------------------------------------------------------------------------
# Migration from local_llm.json
# ---------------------------------------------------------------------------
def test_migrate_v1_hosts_models(tmp_path):
"""v1 local_llm.json (hosts/models/active_model_id) migrates correctly."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Home", "api_url": "http://10.0.0.1:3000", "api_key": "sk-1"}],
"models": [
{"id": "m1", "host_id": "h1", "label": "Gemma Small", "model_name": "gemma4:e4b"},
{"id": "m2", "host_id": "h1", "label": "Gemma Med", "model_name": "gemma4:26b"},
],
"active_model_id": "m1",
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["api_url"] == "http://10.0.0.1:3000"
assert len(data["models"]) == 2
assert all(m["type"] == "local_openai" for m in data["models"])
# active_model_id → roles.chat.primary
assert data["roles"].get("chat", {}).get("primary") == "m1"
def test_migrate_v1_no_active_model(tmp_path):
"""Migration with active_model_id=null: chat role stays unset."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "host_id": "h1", "label": "Model", "model_name": "llama3"}],
"active_model_id": None,
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert "chat" not in data["roles"] or data["roles"]["chat"].get("primary") is None
def test_migrate_v0_flat_format(tmp_path):
"""v0 flat local_llm.json is wrapped into hosts/models structure."""
home = _home(tmp_path)
_write_local_llm(home, {
"api_url": "http://10.0.0.2:3000",
"api_key": "sk-flat",
"model": "qwen3:8b",
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["api_url"] == "http://10.0.0.2:3000"
assert len(data["models"]) == 1
assert data["models"][0]["model_name"] == "qwen3:8b"
def test_migrate_v0_empty_url_returns_empty(tmp_path):
"""v0 with no api_url and no .env fallback → nothing to migrate, empty registry."""
home = _home(tmp_path)
_write_local_llm(home, {"api_url": "", "api_key": "", "model": ""})
import config
import model_registry as reg
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "local_api_url", ""), # ensure no .env fallback
patch.object(config.settings, "local_model", ""),
):
data = reg._load("scott")
assert data["hosts"] == []
assert data["models"] == []
def test_migrate_v1_distill_local_sets_role(tmp_path):
"""When DISTILL_BACKEND_MID=local and active model exists, distill role is set."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "host_id": "h1", "label": "G", "model_name": "gemma4:e4b"}],
"active_model_id": "m1",
})
import config
import model_registry as reg
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "distill_backend_mid", "local"),
):
data = reg._load("scott")
assert data["roles"].get("distill", {}).get("primary") == "m1"
def test_migration_saves_registry_file(tmp_path):
"""After migration, model_registry.json is written so next load skips migration."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [],
"active_model_id": None,
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg._load("scott") # triggers migration + save
# Second load should read model_registry.json, not re-run migration
data2 = reg._load("scott")
assert (home / "scott" / "model_registry.json").exists()
assert data2["version"] == 2
# ---------------------------------------------------------------------------
# Built-in model resolution
# ---------------------------------------------------------------------------
def test_builtin_claude_cli(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "claude_cli")
assert result is not None
assert result["type"] == "claude_cli"
assert result["id"] == "claude_cli"
def test_builtin_gemini_api(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "gemini_api")
assert result["type"] == "gemini_api"
def test_builtin_gemini_cli(tmp_path):
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "gemini_cli")
assert result["type"] == "gemini_cli"
def test_builtin_unknown_returns_none(tmp_path):
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "does_not_exist")
assert result is None
# ---------------------------------------------------------------------------
# User model resolution
# ---------------------------------------------------------------------------
def test_resolve_local_openai_merges_host(tmp_path):
"""local_openai model gets api_url and api_key merged from its host."""
home = _home(tmp_path)
registry = {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": "sk-test"}],
"models": [{"id": "m1", "type": "local_openai", "label": "G", "model_name": "gemma4:e4b",
"host_id": "h1", "context_k": 72, "tags": []}],
"roles": {},
}
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(registry, "m1")
assert result["api_url"] == "http://10.0.0.1:3000"
assert result["api_key"] == "sk-test"
assert result["model_name"] == "gemma4:e4b"
def test_resolve_local_openai_missing_host_returns_none(tmp_path):
"""A model pointing to a non-existent host_id returns None."""
home = _home(tmp_path)
registry = {
"version": 1, "hosts": [], "roles": {},
"models": [{"id": "m1", "type": "local_openai", "host_id": "missing",
"label": "X", "model_name": "x", "context_k": 0, "tags": []}],
}
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(registry, "m1")
assert result is None
def test_resolve_unknown_model_id_returns_none(tmp_path):
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "no_such_model")
assert result is None
# ---------------------------------------------------------------------------
# get_model_for_role
# ---------------------------------------------------------------------------
def test_get_model_for_role_uses_registry(tmp_path):
"""Registry primary assignment is returned first."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
"roles": {"chat": {"primary": "m1"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["model_name"] == "gemma4:e4b"
assert result["api_url"] == "http://10.0.0.1:3000"
def test_get_model_for_role_uses_builtin_from_registry(tmp_path):
"""Registry can assign built-in IDs (claude_cli, gemini_api, etc.)."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["type"] == "claude_cli"
def test_get_model_for_role_skips_missing_primary(tmp_path):
"""If primary model_id is not found, falls through to backup_1."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m2", "type": "local_openai", "label": "Backup",
"model_name": "llama3:8b", "host_id": "h1", "context_k": 8, "tags": []}],
"roles": {"chat": {"primary": "gone", "backup_1": "m2"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["model_name"] == "llama3:8b"
def test_get_model_for_role_env_fallback(tmp_path):
"""No registry entry for role → falls back to .env setting."""
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "role_chat", "gemini_cli"),
):
result = reg.get_model_for_role("scott", "chat")
assert result["type"] == "gemini_cli"
def test_get_model_for_role_hardcoded_fallback(tmp_path):
"""No registry + no .env for role → hardcoded last resort."""
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
# Clear the .env default for 'chat' to simulate unset
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "role_chat", ""),
):
result = reg.get_model_for_role("scott", "chat")
# claude_cli is the hardcoded last resort for 'chat'
assert result["type"] == "claude_cli"
def test_get_model_for_role_custom_role(tmp_path):
"""Custom roles not in DEFINED_ROLES can still be assigned and resolved."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"therapy": {"primary": "gemini_api"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "therapy")
assert result["type"] == "gemini_api"
def test_get_model_for_role_full_backup_chain(tmp_path):
"""Walks the entire priority chain before falling back."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m4", "type": "local_openai", "label": "Last",
"model_name": "tiny:1b", "host_id": "h1", "context_k": 4, "tags": []}],
"roles": {"chat": {
"primary": "gone1",
"backup_1": "gone2",
"backup_2": "gone3",
"backup_3": "gone4",
"backup_4": "m4",
}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["model_name"] == "tiny:1b"
# ---------------------------------------------------------------------------
# get_best_local_model
# ---------------------------------------------------------------------------
def test_get_best_local_prefers_role_chain(tmp_path):
"""Returns the first local_openai model in the chat role chain."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [
{"id": "m1", "type": "local_openai", "label": "Preferred",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []},
],
"roles": {"chat": {"primary": "claude_cli", "backup_1": "m1"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
# primary is claude_cli (not local), backup_1 is m1 (local)
result = reg.get_best_local_model("scott", "chat")
assert result["model_name"] == "gemma4:e4b"
def test_get_best_local_falls_back_to_first_model(tmp_path):
"""No local model in role chain → returns first configured local model."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [
{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []},
],
"roles": {}, # no chat role assigned
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_best_local_model("scott", "chat")
assert result["model_name"] == "gemma4:e4b"
def test_get_best_local_returns_none_when_no_local_models(tmp_path):
"""No local_openai models configured → returns None."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_best_local_model("scott", "chat")
assert result is None
# ---------------------------------------------------------------------------
# Host CRUD
# ---------------------------------------------------------------------------
def test_save_host_creates_new(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
host_id = reg.save_host("scott", None, "ML Box", "http://10.0.0.1:3000", "sk-abc")
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["id"] == host_id
assert data["hosts"][0]["label"] == "ML Box"
assert data["hosts"][0]["api_key"] == "sk-abc"
def test_save_host_updates_existing(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Old Label", "api_url": "http://10.0.0.1:3000", "api_key": "sk-old"}],
"models": [], "roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_host("scott", "h1", "New Label", "http://10.0.0.2:3000", "")
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["label"] == "New Label"
assert data["hosts"][0]["api_url"] == "http://10.0.0.2:3000"
# Empty api_key → existing key preserved
assert data["hosts"][0]["api_key"] == "sk-old"
def test_save_host_unknown_id_creates_new(tmp_path):
"""Passing a host_id that doesn't exist creates a new host instead of crashing."""
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_host("scott", "ghost-id", "New", "http://10.0.0.3:3000", "")
data = reg._load("scott")
assert len(data["hosts"]) == 1
def test_remove_host_also_removes_models(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "host_id": "h1",
"label": "G", "model_name": "gemma4:e4b", "context_k": 72, "tags": []}],
"roles": {"chat": {"primary": "m1"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_host("scott", "h1")
data = reg._load("scott")
assert found is True
assert data["hosts"] == []
assert data["models"] == []
def test_remove_host_not_found_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_host("scott", "nope")
assert found is False
# ---------------------------------------------------------------------------
# Model CRUD
# ---------------------------------------------------------------------------
def test_save_model_creates(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [], "roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
model_id = reg.save_model("scott", None, "h1", "Gemma Small", "gemma4:e4b", 72, ["fast", "distill"])
data = reg._load("scott")
assert len(data["models"]) == 1
assert data["models"][0]["id"] == model_id
assert data["models"][0]["context_k"] == 72
assert data["models"][0]["tags"] == ["fast", "distill"]
assert data["models"][0]["type"] == "local_openai"
def test_save_model_updates_existing(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "Old",
"model_name": "llama3", "host_id": "h1", "context_k": 8, "tags": []}],
"roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_model("scott", "m1", "h1", "New Label", "llama3:latest", 128, ["updated"])
data = reg._load("scott")
assert len(data["models"]) == 1
assert data["models"][0]["label"] == "New Label"
assert data["models"][0]["context_k"] == 128
def test_remove_model_clears_role_refs(tmp_path):
"""Removing a model clears it from any role assignments."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
"roles": {
"chat": {"primary": "m1", "backup_1": "m1"},
"distill": {"primary": "claude_cli", "backup_1": "m1"},
},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_model("scott", "m1")
data = reg._load("scott")
assert found is True
assert data["models"] == []
assert data["roles"]["chat"].get("primary") is None
assert data["roles"]["chat"].get("backup_1") is None
assert data["roles"]["distill"].get("backup_1") is None
# claude_cli assignment preserved
assert data["roles"]["distill"]["primary"] == "claude_cli"
def test_remove_model_not_found_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_model("scott", "ghost")
assert found is False
# ---------------------------------------------------------------------------
# set_role
# ---------------------------------------------------------------------------
def test_set_role_assigns_model(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
"roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "primary", "m1")
data = reg._load("scott")
assert ok is True
assert data["roles"]["chat"]["primary"] == "m1"
def test_set_role_assigns_builtin(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "orchestrator", "primary", "gemini_api")
data = reg._load("scott")
assert ok is True
assert data["roles"]["orchestrator"]["primary"] == "gemini_api"
def test_set_role_clears_with_none(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "primary", None)
data = reg._load("scott")
assert ok is True
assert data["roles"]["chat"]["primary"] is None
def test_set_role_invalid_slot_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "backup_99", "claude_cli")
assert ok is False
def test_set_role_unknown_model_id_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "primary", "nonexistent_model")
assert ok is False
def test_set_role_creates_role_key_if_missing(tmp_path):
"""set_role on a role that isn't in roles{} yet creates it."""
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.set_role("scott", "medical", "primary", "claude_cli")
data = reg._load("scott")
assert data["roles"]["medical"]["primary"] == "claude_cli"
# ---------------------------------------------------------------------------
# get_defined_roles
# ---------------------------------------------------------------------------
def test_get_defined_roles_returns_registry_roles(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}, "distill": {}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
roles = reg.get_defined_roles("scott")
# Should include all settings.defined_roles, filling gaps with {}
for role in config.settings.get_defined_roles():
assert role in roles
def test_get_defined_roles_fills_gaps(tmp_path):
"""Roles in settings.defined_roles that aren't in registry get empty dicts."""
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
roles = reg.get_defined_roles("scott")
assert "chat" in roles
assert roles["chat"] == {}
# ---------------------------------------------------------------------------
# Multi-user isolation
# ---------------------------------------------------------------------------
def test_registries_are_isolated_per_user(tmp_path):
"""Each user has their own registry file — changes don't bleed across users."""
home = _home(tmp_path)
(home / "scott").mkdir()
(home / "holly").mkdir()
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_host("scott", None, "Scott Host", "http://10.0.0.1:3000", "")
scott_data = reg._load("scott")
holly_data = reg._load("holly")
assert len(scott_data["hosts"]) == 1
assert holly_data["hosts"] == []

View File

@@ -0,0 +1,125 @@
"""
Unit tests for persona.py — validation, routing, path traversal.
Tests the two-level home/{username}/persona/{name}/ structure.
No HTTP involved.
"""
import pytest
from pathlib import Path
from unittest.mock import patch
def _make_home(tmp_path: Path) -> Path:
"""Create a minimal home/ tree with two users and some personas."""
root = tmp_path / "home"
for username, persona in [("scott", "inara"), ("holly", "tina"), ("scott", "alt")]:
p = root / username / "persona" / persona
p.mkdir(parents=True)
(p / "IDENTITY.md").write_text(f"# {persona}")
# A persona dir WITHOUT IDENTITY.md — should be invisible to list_user_personas()
incomplete = root / "scott" / "persona" / "broken"
incomplete.mkdir(parents=True)
return root
def test_validate_good(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
assert persona.validate("scott", "inara") == ("scott", "inara")
assert persona.validate("holly", "tina") == ("holly", "tina")
def test_validate_unknown_user(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
with pytest.raises(ValueError, match="Unknown user"):
persona.validate("charlie", "inara")
def test_validate_unknown_persona(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
with pytest.raises(ValueError, match="Unknown persona"):
persona.validate("scott", "ghost")
def test_validate_path_traversal(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
for bad in ("../../etc/passwd", "../scott", "scott/../../etc"):
with pytest.raises(ValueError, match="Invalid"):
persona.validate(bad, "inara")
with pytest.raises(ValueError, match="Invalid"):
persona.validate("scott", "../../etc/passwd")
def test_validate_special_chars(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
for bad in ("alice bob", "alice;bob", "alice\x00bob", "A" * 33, ""):
with pytest.raises(ValueError):
persona.validate(bad, "inara")
def test_validate_allows_hyphen_underscore(tmp_path):
root = _make_home(tmp_path)
p = root / "my_user" / "persona" / "my-agent"
p.mkdir(parents=True)
(p / "IDENTITY.md").write_text("# My Agent")
import config, persona
with patch.object(config.settings, "home_dir", root):
assert persona.validate("my_user", "my-agent") == ("my_user", "my-agent")
def test_list_users(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
users = persona.list_users()
assert "scott" in users
assert "holly" in users
def test_list_user_personas(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
names = persona.list_user_personas("scott")
assert "inara" in names
assert "alt" in names
assert "broken" not in names # no IDENTITY.md
def test_persona_path_uses_contextvars(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
persona.set_context("scott", "inara")
assert persona.persona_path() == root / "scott" / "persona" / "inara"
persona.set_context("holly", "tina")
assert persona.persona_path() == root / "holly" / "persona" / "tina"
def test_persona_path_explicit_args(tmp_path):
root = _make_home(tmp_path)
import config, persona
with patch.object(config.settings, "home_dir", root):
persona.set_context("scott", "inara")
# Explicit args override the ContextVar
assert persona.persona_path("holly", "tina") == root / "holly" / "persona" / "tina"
# ContextVar unchanged
assert persona.persona_path() == root / "scott" / "persona" / "inara"
def test_get_user_and_persona(tmp_path):
import persona
persona.set_context("scott", "inara")
assert persona.get_user() == "scott"
assert persona.get_persona() == "inara"
persona.set_context("holly", "tina")
assert persona.get_user() == "holly"
assert persona.get_persona() == "tina"

View File

@@ -0,0 +1,136 @@
"""
Security-focused tests — what should be blocked or rejected.
These document the current security posture and will catch regressions.
Tests marked 'known_gap' document real issues that are not yet fixed;
they assert the current (insecure) behaviour so we notice when it changes.
"""
import pytest
# ---------------------------------------------------------------------------
# Path traversal
# ---------------------------------------------------------------------------
@pytest.mark.anyio
async def test_files_no_path_traversal_in_filename(client):
"""
File endpoint must not serve files outside the ALLOWED set.
Note: paths containing '..' are URL-normalized before reaching FastAPI.
'/files/../../etc/passwd' becomes '/etc/passwd' at the ASGI layer — it
never hits the files router. We verify no file content is returned (any
non-200 code is safe); 302 redirects to login are fine.
"""
dangerous = [
"../config.py",
"../../etc/passwd",
"SOUL.md/../config.py",
".env",
"TASKS.json",
"CRONS.json",
]
for name in dangerous:
r = await client.get(f"/files/{name}")
assert r.status_code != 200 or "content" not in r.json(), \
f"Got 200 with file content for {name!r} — path traversal may be possible"
@pytest.mark.anyio
async def test_persona_traversal_blocked_in_chat(client, mock_llm):
"""Path traversal in persona name must be rejected before any file access."""
for bad in ("../inara", "../../etc", "inara/../inara", "inara\x00extra"):
r = await client.post("/chat", json={"message": "hi", "persona": bad})
assert r.status_code == 200 # SSE stream, not HTTP error
import json
for line in r.text.splitlines():
if line.startswith("data: "):
event = json.loads(line[6:])
if event.get("type") == "error":
break
else:
pytest.fail(f"Expected error event for persona={bad!r}, got: {r.text[:200]}")
@pytest.mark.anyio
async def test_orchestrate_path_traversal(client, mock_llm):
r = await client.post("/orchestrate", json={"task": "hi", "persona": "../../etc"})
assert r.status_code == 400
# ---------------------------------------------------------------------------
# Signature verification
# ---------------------------------------------------------------------------
@pytest.mark.anyio
async def test_nct_replayed_request_rejected(client):
"""A request with correct format but wrong HMAC should always be rejected."""
import json, hashlib, hmac as hmac_lib
payload = json.dumps({"type": "Create", "actor": {}, "object": {}, "target": {}}).encode()
# Use wrong secret to generate sig
wrong_sig = hmac_lib.new(b"wrong-secret", b"abc123" + payload, hashlib.sha256).hexdigest()
_channels = {"nextcloud": {"bot_secret": "correct-secret", "url": "https://nc.example.com"}}
from unittest.mock import patch
with patch("routers.nextcloud_talk.get_user_channels", return_value=_channels):
r = await client.post(
"/webhook/nextcloud/scott",
content=payload,
headers={
"Content-Type": "application/json",
"X-Nextcloud-Talk-Random": "abc123",
"X-Nextcloud-Talk-Signature": wrong_sig,
},
)
assert r.status_code == 401
# ---------------------------------------------------------------------------
# Known gaps — document current behaviour, alert when it changes
# ---------------------------------------------------------------------------
@pytest.mark.anyio
async def test_known_gap__distill_no_app_auth(client):
"""
KNOWN GAP: /distill/* has no app-layer auth.
Anyone reaching port 8000 directly can trigger LLM calls and overwrite memory.
Protection is currently nginx-only.
This test documents the current state — update when app-layer auth is added.
"""
r = await client.get("/distill/status")
assert r.status_code == 200 # currently open
@pytest.mark.anyio
async def test_known_gap__files_put_no_app_auth(client):
"""
KNOWN GAP: PUT /files/{filename} has no app-layer auth.
Overwriting SOUL.md or IDENTITY.md changes agent behavior.
Protection is currently nginx-only.
"""
r = await client.put("/files/PROTOCOLS.md", json={"content": "# Modified"})
assert r.status_code == 200 # currently open
@pytest.mark.anyio
async def test_known_gap__gchat_no_audience_bypass(client, mock_llm):
"""
KNOWN GAP: Google Chat JWT verification is silently skipped when
GOOGLE_CHAT_AUDIENCE is empty (the default). Anyone can POST and get
LLM responses without a valid token.
Fix: make audience required; fail loudly if not set.
"""
# Channel config with no audience — JWT check is skipped (the known gap).
_channels = {"google_chat": {"persona": "inara"}}
from unittest.mock import patch
with patch("routers.google_chat.get_user_channels", return_value=_channels):
r = await client.post("/channels/google-chat/scott", json={
"chat": {
"messagePayload": {
"message": {"text": "Exploit"},
"space": {"name": "spaces/x", "type": "DM"},
},
"user": {"displayName": "Attacker"},
}
})
# This currently succeeds — it should not when audience is unconfigured
assert r.status_code == 200 # documents the gap

272
cortex/tests/test_tools.py Normal file
View File

@@ -0,0 +1,272 @@
"""
Unit tests for Inara's internal tools — scratch, tasks, cron.
These test the sync implementation functions directly, using a temp directory.
No HTTP, no LLM calls.
"""
import json
import pytest
from pathlib import Path
from unittest.mock import patch
@pytest.fixture
def persona_dir(tmp_path) -> Path:
"""A temp persona directory pre-populated with empty tool files."""
d = tmp_path / "inara"
d.mkdir()
(d / "SCRATCH.md").write_text("")
(d / "TASKS.json").write_text("[]")
(d / "CRONS.json").write_text("[]")
(d / "REMINDERS.md").write_text("")
return d
@pytest.fixture(autouse=True)
def patch_persona_path(persona_dir):
"""
Route all persona_path() calls to the temp dir.
Each tool does `from persona import persona_path`, so we must patch
the name in each module's namespace, not just in persona itself.
"""
with (
patch("tools.scratch.persona_path", return_value=persona_dir),
patch("tools.tasks.persona_path", return_value=persona_dir),
patch("tools.cron.persona_path", return_value=persona_dir),
patch("cron_runner._persona_path", return_value=persona_dir),
):
yield
# ---------------------------------------------------------------------------
# Scratch
# ---------------------------------------------------------------------------
class TestScratch:
def test_read_empty(self):
from tools.scratch import _scratch_read
result = _scratch_read()
assert "empty" in result.lower()
def test_write_and_read(self, persona_dir):
from tools.scratch import _scratch_write, _scratch_read
_scratch_write("# My notes\n\nSome content here.")
content = _scratch_read()
assert "My notes" in content
assert "Some content here" in content
def test_append_adds_section(self, persona_dir):
from tools.scratch import _scratch_write, _scratch_append, _scratch_read
_scratch_write("# Existing")
_scratch_append("New section content", heading="Section A")
content = _scratch_read()
assert "Existing" in content
assert "Section A" in content
assert "New section content" in content
def test_append_auto_heading(self, persona_dir):
from tools.scratch import _scratch_append, _scratch_read
_scratch_append("Content without explicit heading")
content = _scratch_read()
assert "UTC" in content # auto heading includes timestamp
def test_clear(self, persona_dir):
from tools.scratch import _scratch_write, _scratch_clear, _scratch_read
_scratch_write("Some content")
_scratch_clear()
assert "empty" in _scratch_read().lower()
def test_write_strips_trailing_whitespace(self, persona_dir):
from tools.scratch import _scratch_write, _scratch_read
_scratch_write("Content \n\n\n")
content = (persona_dir / "SCRATCH.md").read_text()
assert content.endswith("\n")
assert not content.endswith("\n\n")
# ---------------------------------------------------------------------------
# Tasks
# ---------------------------------------------------------------------------
class TestTasks:
def _mk(self, title, description=None, priority="normal"):
from tools.tasks import _task_create
return _task_create(title, description, priority)
def _id(self, result: str) -> str:
import re
m = re.search(r't_\w+', result)
assert m, f"No task ID in: {result}"
return m.group()
def test_list_empty(self):
from tools.tasks import _task_list
assert "No tasks" in _task_list(status=None, priority=None)
def test_create_and_list(self):
from tools.tasks import _task_list
self._mk("Buy coffee", description="Dark roast", priority="high")
result = _task_list(status=None, priority=None)
assert "Buy coffee" in result
assert "[high]" in result
def test_create_bad_priority_defaults_to_normal(self):
from tools.tasks import _task_list
self._mk("Test task", priority="urgent") # invalid — becomes "normal"
result = _task_list(status=None, priority=None)
assert "Test task" in result
assert "[normal]" not in result # normal priority not shown in brackets
def test_update_status(self):
from tools.tasks import _task_update, _task_list
tid = self._id(self._mk("Work item"))
_task_update(tid, status="in_progress", title=None, description=None, priority=None)
assert "Work item" in _task_list(status="in_progress", priority=None)
def test_complete(self):
from tools.tasks import _task_complete, _task_list
tid = self._id(self._mk("Finish this"))
_task_complete(tid)
assert "Finish this" in _task_list(status="done", priority=None)
assert "Finish this" not in _task_list(status="todo", priority=None)
def test_filter_by_status(self):
from tools.tasks import _task_list
self._mk("A task")
assert "A task" in _task_list(status="todo", priority=None)
assert "A task" not in _task_list(status="done", priority=None)
def test_update_unknown_id(self):
from tools.tasks import _task_update
result = _task_update("t_doesnotexist", status="done",
title=None, description=None, priority=None)
assert "not found" in result.lower()
def test_persistence(self, persona_dir):
"""Tasks survive between _load() calls (written to TASKS.json)."""
self._mk("Persistent task")
data = json.loads((persona_dir / "TASKS.json").read_text())
assert any(t["title"] == "Persistent task" for t in data)
# ---------------------------------------------------------------------------
# Cron
# ---------------------------------------------------------------------------
class TestCronRunner:
def test_parse_hourly(self):
from cron_runner import parse_schedule
assert parse_schedule("hourly") == {"minute": 0}
def test_parse_daily_default(self):
from cron_runner import parse_schedule
r = parse_schedule("daily")
assert r["hour"] == 9
assert r["minute"] == 0
def test_parse_daily_custom_time(self):
from cron_runner import parse_schedule
r = parse_schedule("daily:14:30")
assert r["hour"] == 14
assert r["minute"] == 30
def test_parse_weekly(self):
from cron_runner import parse_schedule
r = parse_schedule("weekly:mon")
assert r["day_of_week"] == "mon"
assert r["hour"] == 9
def test_parse_weekly_with_time(self):
from cron_runner import parse_schedule
r = parse_schedule("weekly:fri:17:00")
assert r["day_of_week"] == "fri"
assert r["hour"] == 17
assert r["minute"] == 0
def test_parse_full_day_names(self):
from cron_runner import parse_schedule
assert parse_schedule("weekly:monday")["day_of_week"] == "mon"
assert parse_schedule("weekly:friday")["day_of_week"] == "fri"
def test_parse_unknown_schedule(self):
from cron_runner import parse_schedule
with pytest.raises(ValueError, match="Unrecognised"):
parse_schedule("every-tuesday")
def test_parse_bad_dow(self):
from cron_runner import parse_schedule
with pytest.raises(ValueError, match="day of week"):
parse_schedule("weekly:funday")
def test_parse_bad_time_non_integer(self):
from cron_runner import parse_schedule
with pytest.raises((ValueError, Exception)):
parse_schedule("daily:noon") # non-integer — parse error
class TestCronTools:
def test_list_empty(self):
from tools.cron import _cron_list
result = _cron_list()
assert "No crons" in result
def test_add_and_list(self):
from tools.cron import _cron_add, _cron_list
with patch("tools.cron._scheduler_add"):
r = _cron_add("Morning reminder", "daily:09:00", "remind", "Check in.")
assert "c_" in r
listing = _cron_list()
assert "Morning reminder" in listing
assert "daily:09:00" in listing
def test_add_bad_schedule(self):
from tools.cron import _cron_add
r = _cron_add("Bad job", "every-day", "remind", "Hello")
assert "Bad schedule" in r
def test_add_bad_type(self):
from tools.cron import _cron_add
r = _cron_add("Bad job", "daily", "email", "Hello")
assert "Bad type" in r
def _extract_id(self, result: str) -> str:
import re
# token_urlsafe can include '-'; use [\w-]+ to capture the full ID
m = re.search(r'c_[\w-]+', result)
assert m, f"No cron ID in: {result}"
return m.group()
def test_remove(self):
from tools.cron import _cron_add, _cron_remove, _cron_list
with patch("tools.cron._scheduler_add"), patch("tools.cron._scheduler_remove"):
r = _cron_add("To remove", "hourly", "note", "Tick")
cron_id = self._extract_id(r)
result = _cron_remove(cron_id)
assert "Removed" in result
assert "To remove" not in _cron_list()
def test_remove_unknown(self):
from tools.cron import _cron_remove
result = _cron_remove("c_doesnotexist")
assert "Not found" in result
def test_toggle_pause_resume(self):
from tools.cron import _cron_add, _cron_toggle, _cron_list
with patch("tools.cron._scheduler_add"), patch("tools.cron._scheduler_pause"), patch("tools.cron._scheduler_resume"):
r = _cron_add("Toggleable", "daily", "note", "Content")
cron_id = self._extract_id(r)
result = _cron_toggle(cron_id)
assert "Paused" in result
assert "PAUSED" in _cron_list()
result = _cron_toggle(cron_id)
assert "Resumed" in result
assert "enabled" in _cron_list()
def test_reminders_clear(self, persona_dir):
from tools.cron import _reminders_clear
(persona_dir / "REMINDERS.md").write_text("## Some reminder\n\nContent")
result = _reminders_clear()
assert "cleared" in result.lower()
assert (persona_dir / "REMINDERS.md").read_text() == ""

View File

@@ -0,0 +1,186 @@
"""
Webhook auth tests — NC Talk HMAC, Google Chat JWT.
These tests verify that auth is enforced, not that full LLM responses work.
Architecture note: channel config (secrets, audience) lives in per-user channels.json,
not in settings. Tests mock get_user_channels() rather than patching settings fields.
Endpoints are per-user: /webhook/nextcloud/{username} and /channels/google-chat/{username}.
"""
import hashlib
import hmac
import json
import pytest
from unittest.mock import AsyncMock, patch
# ---------------------------------------------------------------------------
# Nextcloud Talk
# ---------------------------------------------------------------------------
_NC_SECRET = "test-bot-secret-12345"
_VALID_NC_PAYLOAD = {
"type": "Create",
"actor": {"type": "users", "id": "testuser", "name": "Test User"},
"object": {
"type": "Note",
"content": json.dumps({"message": "Hello Inara"}),
},
"target": {"id": "abc123token"},
}
_NCT_CHANNELS = {
"nextcloud": {
"bot_secret": _NC_SECRET,
"notification_room": "abc123token",
"url": "https://nc.example.com",
}
}
def _nc_headers(body: bytes, secret: str) -> dict:
random_str = "abc123"
sig = hmac.new(
secret.encode(),
(random_str + body.decode("utf-8")).encode(),
hashlib.sha256,
).hexdigest()
return {
"X-Nextcloud-Talk-Random": random_str,
"X-Nextcloud-Talk-Signature": sig,
}
@pytest.mark.anyio
async def test_nct_valid_signature(client, mock_llm):
body = json.dumps(_VALID_NC_PAYLOAD).encode()
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
with patch("routers.nextcloud_talk._send_reply", new_callable=AsyncMock):
headers = _nc_headers(body, _NC_SECRET)
r = await client.post(
"/webhook/nextcloud/scott",
content=body,
headers={**headers, "Content-Type": "application/json"},
)
assert r.status_code == 200
@pytest.mark.anyio
async def test_nct_wrong_signature(client):
body = json.dumps(_VALID_NC_PAYLOAD).encode()
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
r = await client.post(
"/webhook/nextcloud/scott",
content=body,
headers={
"Content-Type": "application/json",
"X-Nextcloud-Talk-Random": "abc123",
"X-Nextcloud-Talk-Signature": "badsignature",
},
)
assert r.status_code == 401
@pytest.mark.anyio
async def test_nct_missing_signature(client):
body = json.dumps(_VALID_NC_PAYLOAD).encode()
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
r = await client.post(
"/webhook/nextcloud/scott",
content=body,
headers={"Content-Type": "application/json"},
)
assert r.status_code == 401
@pytest.mark.anyio
async def test_nct_no_secret_configured(client):
"""Service should return 500 if bot_secret is missing, not process the message."""
body = json.dumps(_VALID_NC_PAYLOAD).encode()
# cfg must be non-empty (truthy) to get past the 404 guard; missing bot_secret → 500
empty_cfg = {"nextcloud": {"url": "https://nc.example.com"}}
with patch("routers.nextcloud_talk.get_user_channels", return_value=empty_cfg):
r = await client.post(
"/webhook/nextcloud/scott",
content=body,
headers={"Content-Type": "application/json"},
)
assert r.status_code == 500
@pytest.mark.anyio
async def test_nct_bot_message_ignored(client):
"""Messages from other bots should be silently ignored (not processed)."""
payload = {**_VALID_NC_PAYLOAD, "actor": {"type": "bots", "id": "otherbot", "name": "Bot"}}
body = json.dumps(payload).encode()
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
headers = _nc_headers(body, _NC_SECRET)
r = await client.post(
"/webhook/nextcloud/scott",
content=body,
headers={**headers, "Content-Type": "application/json"},
)
assert r.status_code == 200
# ---------------------------------------------------------------------------
# Google Chat
# ---------------------------------------------------------------------------
_GCHAT_PAYLOAD = {
"chat": {
"messagePayload": {
"message": {"text": "Hello", "argumentText": "Hello"},
"space": {"name": "spaces/test123", "type": "ROOM"},
},
"user": {"displayName": "Test User"},
}
}
_GCHAT_CHANNELS_NO_AUDIENCE = {
# cfg must be non-empty (truthy) to pass the 404 guard; no audience → JWT skipped
"google_chat": {"persona": "inara"}
}
_GCHAT_CHANNELS_WITH_AUDIENCE = {
"google_chat": {"audience": "123456789"}
}
@pytest.mark.anyio
async def test_gchat_no_audience_configured(client, mock_llm):
"""When audience is not set, JWT check is skipped (current behaviour — documented bypass)."""
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_NO_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=_GCHAT_PAYLOAD)
assert r.status_code == 200
@pytest.mark.anyio
async def test_gchat_missing_token_with_audience(client):
"""When audience IS configured, requests without a token must be rejected."""
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_WITH_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=_GCHAT_PAYLOAD)
assert r.status_code == 401
@pytest.mark.anyio
async def test_gchat_invalid_token_with_audience(client):
"""A fake token should fail JWT verification."""
payload_with_token = {
**_GCHAT_PAYLOAD,
"authorizationEventObject": {"systemIdToken": "not.a.valid.jwt"},
}
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_WITH_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=payload_with_token)
assert r.status_code == 401
@pytest.mark.anyio
async def test_gchat_added_to_space(client, mock_llm):
"""Bot added to a space — should return a greeting, no auth when audience empty."""
payload = {"chat": {"addedToSpacePayload": {"space": {"type": "ROOM"}}}}
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_NO_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=payload)
assert r.status_code == 200
assert "hostAppDataAction" in r.json()

156
cortex/tool_audit.py Normal file
View File

@@ -0,0 +1,156 @@
"""
Tool call audit log.
One JSONL file per user per day:
home/{user}/tool_audit/YYYY-MM-DD.jsonl
Each line is a JSON object:
ts ISO timestamp (seconds)
user username
tool tool name
args call arguments (string values truncated at ARG_MAX chars)
status "ok" | "error" | "denied"
result_chars length of full result string
result_snippet first SNIPPET_MAX chars of result
"""
import asyncio
import json
import logging
from contextvars import ContextVar
from datetime import datetime, date
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
_ARG_MAX = 500 # truncate individual arg string values longer than this
_SNIPPET_MAX = 300 # chars of result to keep as snippet
# Per-file write locks — prevents interleaved lines under concurrent tool calls
_locks: dict[str, asyncio.Lock] = {}
# ContextVars set by orchestrators before their tool loop runs
_audit_engine: ContextVar[str] = ContextVar("_audit_engine", default="")
_audit_model: ContextVar[str] = ContextVar("_audit_model", default="")
def set_context(engine: str, model: str) -> None:
"""Call at the start of each orchestrator run to tag subsequent tool calls."""
_audit_engine.set(engine)
_audit_model.set(model)
def _truncate_args(args: dict) -> dict:
out = {}
for k, v in args.items():
if isinstance(v, str) and len(v) > _ARG_MAX:
out[k] = v[:_ARG_MAX] + f" …[{len(v)} chars total]"
else:
out[k] = v
return out
def _audit_path(user: str, day: date | None = None) -> Path:
d = day or date.today()
audit_dir = settings.home_root() / user / "tool_audit"
audit_dir.mkdir(parents=True, exist_ok=True)
return audit_dir / f"{d.isoformat()}.jsonl"
async def record(
user: str,
tool: str,
args: dict,
status: str, # "ok" | "error" | "denied"
result: str = "",
) -> None:
"""Append one audit entry. Fire with asyncio.create_task — never awaited directly."""
path = _audit_path(user)
key = str(path)
if key not in _locks:
_locks[key] = asyncio.Lock()
entry = {
"ts": datetime.now().isoformat(timespec="seconds"),
"user": user,
"engine": _audit_engine.get(),
"model": _audit_model.get(),
"tool": tool,
"args": _truncate_args(args),
"status": status,
"result_chars": len(result),
"result_snippet": result[:_SNIPPET_MAX],
}
async with _locks[key]:
try:
with path.open("a", encoding="utf-8") as f:
f.write(json.dumps(entry) + "\n")
except Exception as e:
logger.warning("audit log write failed for %s: %s", user, e)
def read_recent(user: str, days: int = 7, limit: int = 200) -> list[dict]:
"""Read the most recent `limit` entries across the last `days` days.
Returns entries sorted newest-first (by ts field, file order within a day).
"""
from datetime import timedelta
today = date.today()
entries: list[dict] = []
for offset in range(days):
day = today - timedelta(days=offset)
path = settings.home_root() / user / "tool_audit" / f"{day.isoformat()}.jsonl"
if not path.exists():
continue
try:
lines = path.read_text(encoding="utf-8").splitlines()
except Exception:
continue
day_entries = []
for line in lines:
line = line.strip()
if not line:
continue
try:
day_entries.append(json.loads(line))
except json.JSONDecodeError:
pass
# Newest within the day first
entries.extend(reversed(day_entries))
if len(entries) >= limit:
break
return entries[:limit]
def read_day(user: str, day_str: str) -> list[dict]:
"""Read all entries for a specific date string (YYYY-MM-DD), chronological order."""
path = settings.home_root() / user / "tool_audit" / f"{day_str}.jsonl"
if not path.exists():
return []
entries = []
try:
for line in path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line:
continue
try:
entries.append(json.loads(line))
except json.JSONDecodeError:
pass
except Exception:
pass
return entries
def read_recent_all_users(days: int = 7, limit: int = 500) -> list[dict]:
"""Read recent entries across all users, sorted newest-first."""
from persona import list_users
all_entries: list[dict] = []
for user in list_users():
all_entries.extend(read_recent(user, days=days, limit=limit))
all_entries.sort(key=lambda e: e.get("ts", ""), reverse=True)
return all_entries[:limit]

689
cortex/tools/__init__.py Normal file
View File

@@ -0,0 +1,689 @@
"""
Orchestrator tool registry.
Declarations live in each domain module alongside their callables.
This file assembles them into the unified registry used by both engines.
To add a new tool:
1. Implement it in tools/<domain>.py — add the async callable + append to DECLARATIONS
2. Import the callable here and add it to _CALLABLES
3. If admin-only, add it to TOOL_ROLES; if confirmation needed, add to CONFIRM_REQUIRED
IMPORTANT: These tools are separate from the ae_* MCP tools used by the fleet agents.
Do not modify the ae_* MCP server to support orchestrator needs.
"""
from google.genai import types
# ── Callable imports ──────────────────────────────────────────────────────────
from tools.web import search as _web_search, http_fetch as _http_fetch, web_read as _web_read, http_post as _http_post
from tools.ae_knowledge import (
journal_list as _ae_journal_list,
journal_search as _ae_journal_search,
journal_entry_read as _ae_journal_entry_read,
journal_entries_list as _ae_journal_entries_list,
journal_entry_create as _ae_journal_entry_create,
journal_entry_update as _ae_journal_entry_update,
journal_entry_disable as _ae_journal_entry_disable,
journal_entry_append as _ae_journal_entry_append,
journal_entry_prepend as _ae_journal_entry_prepend,
)
from tools.ae_tasks import task_list as _ae_task_list
from tools.files import (
project_file_read as _project_file_read,
project_file_list as _project_file_list,
file_stat as _file_stat,
file_grep as _file_grep,
file_diff as _file_diff,
file_syntax_check as _file_syntax_check,
file_read as _file_read,
file_list as _file_list,
file_write as _file_write,
session_read as _session_read,
session_search as _session_search,
)
from tools.system import (
shell_exec as _shell_exec,
claude_allow_dir as _claude_allow_dir,
cortex_restart as _cortex_restart,
cortex_logs as _cortex_logs,
cortex_status as _cortex_status,
cortex_update as _cortex_update,
)
from tools.tasks import (
task_list as _task_list,
task_create as _task_create,
task_update as _task_update,
task_complete as _task_complete,
)
from tools.cron import (
cron_list as _cron_list,
cron_add as _cron_add,
cron_remove as _cron_remove,
cron_toggle as _cron_toggle,
)
from tools.reminders import (
reminders_add as _reminders_add,
reminders_list as _reminders_list,
reminders_remove as _reminders_remove,
reminders_clear as _reminders_clear,
)
from tools.scratch import (
scratch_read as _scratch_read,
scratch_write as _scratch_write,
scratch_append as _scratch_append,
scratch_clear as _scratch_clear,
)
from tools.notify import nc_talk_send as _nc_talk_send, email_send as _email_send, web_push as _web_push, nc_talk_history as _nc_talk_history
from tools.agent_notes import (
agent_notes_read as _agent_notes_read,
agent_notes_write as _agent_notes_write,
agent_notes_append as _agent_notes_append,
agent_notes_clear as _agent_notes_clear,
)
from tools.git import (
git_status as _git_status,
git_log as _git_log,
git_diff as _git_diff,
)
from tools.agents import (
spawn_agent as _spawn_agent,
agent_status as _agent_status,
agent_list as _agent_list,
agent_cancel as _agent_cancel,
)
from tools.aider import aider_run as _aider_run
from tools.homeassistant import (
ha_get_state as _ha_get_state,
ha_get_states as _ha_get_states,
ha_call_service as _ha_call_service,
)
from tools.ae_database import (
ae_db_query as _ae_db_query,
ae_db_describe as _ae_db_describe,
ae_db_show_view as _ae_db_show_view,
)
# ── Declaration imports ───────────────────────────────────────────────────────
import tools.web as _mod_web
import tools.ae_knowledge as _mod_ae_knowledge
import tools.ae_tasks as _mod_ae_tasks
import tools.files as _mod_files
import tools.system as _mod_system
import tools.tasks as _mod_tasks
import tools.cron as _mod_cron
import tools.reminders as _mod_reminders
import tools.scratch as _mod_scratch
import tools.notify as _mod_notify
import tools.agent_notes as _mod_agent_notes
import tools.git as _mod_git
import tools.agents as _mod_agents
import tools.aider as _mod_aider
import tools.homeassistant as _mod_homeassistant
import tools.ae_database as _mod_ae_database
# ── Tool categories — used by the Model Registry UI for grouped checkboxes ───
TOOL_CATEGORIES: dict[str, list[str]] = {
"Web": ["web_search", "http_fetch", "web_read", "http_post"],
"Project Files": ["project_file_read", "project_file_list", "file_stat", "file_grep", "file_diff", "file_syntax_check"],
"Git": ["git_status", "git_log", "git_diff"],
"System Files": ["file_read", "file_list", "file_write", "session_read", "session_search"],
"Shell": ["shell_exec", "claude_allow_dir"],
"System": ["cortex_restart", "cortex_logs", "cortex_status", "cortex_update"],
"Tasks": ["task_list", "task_create", "task_update", "task_complete"],
"Cron": ["cron_list", "cron_add", "cron_remove", "cron_toggle"],
"Reminders": ["reminders_add", "reminders_list", "reminders_remove", "reminders_clear"],
"Scratchpad": ["scratch_read", "scratch_write", "scratch_append", "scratch_clear"],
"Notifications": ["web_push", "email_send", "nc_talk_send", "nc_talk_history"],
"Aether Journals": [
"ae_journal_list", "ae_journal_search",
"ae_journal_entries_list", "ae_journal_entry_read",
"ae_journal_entry_create", "ae_journal_entry_update",
"ae_journal_entry_disable", "ae_journal_entry_append",
"ae_journal_entry_prepend",
],
"Aether Tasks": ["ae_task_list"],
"Agent Notes": ["agent_notes_read", "agent_notes_write", "agent_notes_append", "agent_notes_clear"],
"Agents": ["spawn_agent", "agent_status", "agent_list", "agent_cancel", "aider_run"],
"Home Assistant": ["ha_get_state", "ha_get_states", "ha_call_service"],
"Aether Database": ["ae_db_query", "ae_db_describe", "ae_db_show_view"],
}
# ── Callable registry ─────────────────────────────────────────────────────────
_CALLABLES: dict[str, callable] = {
"web_search": _web_search,
"http_fetch": _http_fetch,
"web_read": _web_read,
"http_post": _http_post,
"ae_journal_list": _ae_journal_list,
"ae_journal_search": _ae_journal_search,
"ae_journal_entry_read": _ae_journal_entry_read,
"ae_journal_entries_list": _ae_journal_entries_list,
"ae_journal_entry_create": _ae_journal_entry_create,
"ae_journal_entry_update": _ae_journal_entry_update,
"ae_journal_entry_disable": _ae_journal_entry_disable,
"ae_journal_entry_append": _ae_journal_entry_append,
"ae_journal_entry_prepend": _ae_journal_entry_prepend,
"ae_task_list": _ae_task_list,
"project_file_read": _project_file_read,
"project_file_list": _project_file_list,
"file_stat": _file_stat,
"file_grep": _file_grep,
"file_diff": _file_diff,
"file_syntax_check": _file_syntax_check,
"file_read": _file_read,
"file_list": _file_list,
"file_write": _file_write,
"session_read": _session_read,
"session_search": _session_search,
"shell_exec": _shell_exec,
"claude_allow_dir": _claude_allow_dir,
"cortex_restart": _cortex_restart,
"cortex_logs": _cortex_logs,
"cortex_status": _cortex_status,
"cortex_update": _cortex_update,
"task_list": _task_list,
"task_create": _task_create,
"task_update": _task_update,
"task_complete": _task_complete,
"cron_list": _cron_list,
"cron_add": _cron_add,
"cron_remove": _cron_remove,
"cron_toggle": _cron_toggle,
"reminders_add": _reminders_add,
"reminders_list": _reminders_list,
"reminders_remove": _reminders_remove,
"reminders_clear": _reminders_clear,
"scratch_read": _scratch_read,
"scratch_write": _scratch_write,
"scratch_append": _scratch_append,
"scratch_clear": _scratch_clear,
"email_send": _email_send,
"nc_talk_send": _nc_talk_send,
"web_push": _web_push,
"nc_talk_history": _nc_talk_history,
"agent_notes_read": _agent_notes_read,
"agent_notes_write": _agent_notes_write,
"agent_notes_append": _agent_notes_append,
"agent_notes_clear": _agent_notes_clear,
"git_status": _git_status,
"git_log": _git_log,
"git_diff": _git_diff,
"spawn_agent": _spawn_agent,
"agent_status": _agent_status,
"agent_list": _agent_list,
"agent_cancel": _agent_cancel,
"aider_run": _aider_run,
"ha_get_state": _ha_get_state,
"ha_get_states": _ha_get_states,
"ha_call_service": _ha_call_service,
"ae_db_query": _ae_db_query,
"ae_db_describe": _ae_db_describe,
"ae_db_show_view": _ae_db_show_view,
}
# ── Role-based access control ─────────────────────────────────────────────────
# Minimum role required to use each tool. Unlisted tools default to "user".
TOOL_ROLES: dict[str, str] = {
"shell_exec": "admin",
"claude_allow_dir": "admin",
"cortex_restart": "admin",
"cortex_logs": "admin",
"cortex_status": "admin",
"cortex_update": "admin",
"file_read": "admin",
"file_list": "admin",
"file_write": "admin",
"ae_task_list": "admin",
"spawn_agent": "admin",
"agent_status": "user",
"agent_list": "user",
"agent_cancel": "admin",
"aider_run": "admin",
"email_send": "admin",
"nc_talk_send": "admin",
"http_post": "admin",
"nc_talk_history": "admin",
"ha_call_service": "admin",
"ae_db_query": "admin",
"ae_db_describe": "admin",
"ae_db_show_view": "admin",
}
# Tools that require explicit user confirmation before executing.
CONFIRM_REQUIRED: set[str] = {
"cortex_restart",
"cortex_update",
"file_write",
"shell_exec",
"cron_remove",
"reminders_clear",
"http_post",
"ha_call_service",
"ae_journal_entry_disable", # disables a journal entry — not easily reversed
"agent_cancel", # kills a running background task
"aider_run", # edits files and commits — irreversible without git revert
}
# Security risk ratings — informational for now; will drive auto-allow tiers later.
# Unlisted tools default to "medium".
#
# low — read-only, sandboxed, no external side effects
# medium — writes to local/controlled data, or reads beyond project scope,
# or sends notifications to the same user
# high — affects external systems, physical devices, other users,
# or the host process/filesystem in ways that are hard to reverse
TOOL_RISK: dict[str, str] = {
# Web — read-only fetches are low; posting to external services is high
"web_search": "low",
"http_fetch": "low",
"web_read": "low",
"http_post": "high",
# Project Files — all read-only and project-sandboxed
"project_file_read": "low",
"project_file_list": "low",
"file_stat": "low",
"file_grep": "low",
"file_diff": "low",
"file_syntax_check": "low",
# System Files — reads beyond project scope are medium; writes are high
"file_read": "medium",
"file_list": "medium",
"file_write": "high",
"session_read": "low",
"session_search": "low",
# Shell — arbitrary execution and permission changes are high
"shell_exec": "high",
"claude_allow_dir": "high",
# System — read-only status is low; restart/update affect the live service
"cortex_logs": "low",
"cortex_status": "low",
"cortex_restart": "high",
"cortex_update": "high",
# Tasks — local persona data, all reversible
"task_list": "low",
"task_create": "low",
"task_update": "low",
"task_complete": "low",
# Cron — list is low; add/remove/toggle affect scheduled behavior
"cron_list": "low",
"cron_add": "medium",
"cron_remove": "medium",
"cron_toggle": "medium",
# Reminders — single-item ops are low; clear-all is medium
"reminders_add": "low",
"reminders_list": "low",
"reminders_remove": "low",
"reminders_clear": "medium",
# Scratchpad — local persona file, ephemeral by design
"scratch_read": "low",
"scratch_write": "low",
"scratch_append": "low",
"scratch_clear": "low",
# Notifications — push to same user is medium; external messages are high
"web_push": "medium",
"nc_talk_send": "high",
"nc_talk_history": "low",
"email_send": "high",
# Aether Journals — reads are low; writes to external DB are medium
"ae_journal_list": "low",
"ae_journal_search": "low",
"ae_journal_entries_list": "low",
"ae_journal_entry_read": "low",
"ae_journal_entry_create": "medium",
"ae_journal_entry_update": "medium",
"ae_journal_entry_disable": "medium",
"ae_journal_entry_append": "medium",
"ae_journal_entry_prepend": "medium",
# Aether Tasks
"ae_task_list": "low",
# Agent Notes — local persona file
"agent_notes_read": "low",
"agent_notes_write": "low",
"agent_notes_append": "low",
"agent_notes_clear": "low",
# Git — all read-only inspections
"git_status": "low",
"git_log": "low",
"git_diff": "low",
# Agents — spawning is high; lifecycle reads are low; cancel is medium (kills a task)
"spawn_agent": "high",
"agent_status": "low",
"agent_list": "low",
"agent_cancel": "medium",
"aider_run": "high",
# Home Assistant — reads are low; controlling physical devices is high
"ha_get_state": "low",
"ha_get_states": "low",
"ha_call_service": "high",
# Aether Database — all read-only; query reads data, describe/show_view read schema only
"ae_db_query": "medium",
"ae_db_describe": "low",
"ae_db_show_view": "low",
}
_RISK_RANK: dict[str, int] = {"low": 0, "medium": 1, "high": 2}
_ROLE_RANK: dict[str, int] = {"user": 0, "admin": 1}
def _role_allowed(tool_name: str, role: str) -> bool:
required = TOOL_ROLES.get(tool_name, "user")
return _ROLE_RANK.get(role, 0) >= _ROLE_RANK.get(required, 0)
# ── Declaration assembly ──────────────────────────────────────────────────────
_ALL_DECLARATIONS: list[types.FunctionDeclaration] = (
_mod_web.DECLARATIONS
+ _mod_files.DECLARATIONS
+ _mod_git.DECLARATIONS
+ _mod_system.DECLARATIONS
+ _mod_tasks.DECLARATIONS
+ _mod_cron.DECLARATIONS
+ _mod_reminders.DECLARATIONS
+ _mod_scratch.DECLARATIONS
+ _mod_notify.DECLARATIONS
+ _mod_ae_knowledge.DECLARATIONS
+ _mod_ae_tasks.DECLARATIONS
+ _mod_agent_notes.DECLARATIONS
+ _mod_agents.DECLARATIONS
+ _mod_aider.DECLARATIONS
+ _mod_homeassistant.DECLARATIONS
+ _mod_ae_database.DECLARATIONS
)
# Full Gemini Tool object (all tools — use get_tools_for_role() in production)
TOOL_DECLARATIONS = [types.Tool(function_declarations=_ALL_DECLARATIONS)]
# ── Tool dispatch ─────────────────────────────────────────────────────────────
async def call_tool(name: str, args: dict, callables: dict | None = None) -> str:
"""Dispatch a tool call by name. Returns result as a string.
Pass `callables` (from get_tools_for_role) to enforce role restrictions.
Falls back to the full _CALLABLES dict if omitted.
Every call is recorded to the tool audit log (tool_audit.py).
"""
import asyncio
import tool_audit
from persona import get_user
user = get_user() or "unknown"
dispatch = callables if callables is not None else _CALLABLES
fn = dispatch.get(name)
if fn is None:
asyncio.create_task(tool_audit.record(user, name, args, "denied"))
return f"Tool not available or access denied: {name}"
try:
result = await fn(**args)
asyncio.create_task(tool_audit.record(user, name, args, "ok", result))
return result
except Exception as e:
asyncio.create_task(tool_audit.record(user, name, args, "error", str(e)))
raise
# ── OpenAI JSON Schema conversion ────────────────────────────────────────────
_GEMINI_TYPE_TO_JSON = {
"OBJECT": "object",
"STRING": "string",
"INTEGER": "integer",
"NUMBER": "number",
"BOOLEAN": "boolean",
"ARRAY": "array",
}
def _schema_to_json(schema) -> dict:
"""Recursively convert a Gemini types.Schema to a JSON Schema dict."""
type_name = getattr(getattr(schema, "type", None), "name", "STRING")
result: dict = {"type": _GEMINI_TYPE_TO_JSON.get(type_name, "string")}
if getattr(schema, "description", None):
result["description"] = schema.description
props = getattr(schema, "properties", None) or {}
if result["type"] == "object":
result["properties"] = {k: _schema_to_json(v) for k, v in props.items()}
req = getattr(schema, "required", None)
if req:
result["required"] = list(req)
return result
def _build_openai_tools() -> list[dict]:
out = []
for decl in _ALL_DECLARATIONS:
params = (
_schema_to_json(decl.parameters)
if decl.parameters
else {"type": "object", "properties": {}}
)
out.append({
"type": "function",
"function": {
"name": decl.name,
"description": decl.description or "",
"parameters": params,
},
})
return out
# OpenAI-format tool list — all tools (use get_openai_tools_for_role() in production)
OPENAI_TOOL_SCHEMAS: list[dict] = _build_openai_tools()
# ── Role-filtered tool access ─────────────────────────────────────────────────
def _apply_risk_policy(
allowed: set[str],
max_risk: str | None,
whitelist: list[str] | None,
blacklist: list[str] | None,
) -> set[str]:
"""Apply risk-level filtering on top of an already role-gated allowed set.
Filtering order (each step can only restrict or restore within what the
role already permits — risk policy can never elevate above role):
1. max_risk auto-include: keep tools whose risk ≤ max_risk
2. whitelist union: force-add specific tools (still role-gated)
3. blacklist subtract: force-remove specific tools
When max_risk is None, all role-allowed tools remain (no risk filter).
"""
if max_risk is not None:
max_rank = _RISK_RANK.get(max_risk, 2)
auto = {n for n in allowed if _RISK_RANK.get(TOOL_RISK.get(n, "medium"), 1) <= max_rank}
extra = {n for n in (whitelist or []) if n in allowed}
allowed = (auto | extra)
if blacklist:
allowed -= set(blacklist)
return allowed
def get_tools_for_role(
role: str,
tool_list: list[str] | None = None,
max_risk: str | None = None,
whitelist: list[str] | None = None,
blacklist: list[str] | None = None,
) -> tuple[list, dict]:
"""Return (gemini_tool_declarations, callables_dict) filtered to what the role can use.
role — user access level ("user" | "admin"); gates admin-only tools
tool_list — optional model-level allow-list; intersected so it can only restrict
max_risk — auto-include tools at/below this risk level ("low"|"medium"|"high")
whitelist — force-include specific tools above max_risk (still role-gated)
blacklist — force-exclude specific tools regardless of max_risk
"""
allowed = {name for name in _CALLABLES if _role_allowed(name, role)}
allowed = _apply_risk_policy(allowed, max_risk, whitelist, blacklist)
if tool_list is not None:
allowed &= set(tool_list)
decls = [d for d in _ALL_DECLARATIONS if d.name in allowed]
callables = {k: v for k, v in _CALLABLES.items() if k in allowed}
return [types.Tool(function_declarations=decls)], callables
def get_openai_tools_for_role(
role: str,
tool_list: list[str] | None = None,
max_risk: str | None = None,
whitelist: list[str] | None = None,
blacklist: list[str] | None = None,
) -> list[dict]:
"""Return OpenAI tool schemas filtered to what the role can use.
role — user access level ("user" | "admin")
tool_list — optional model-level allow-list
max_risk — auto-include tools at/below this risk level
whitelist — force-include specific tools above max_risk
blacklist — force-exclude specific tools
"""
allowed = {name for name in _CALLABLES if _role_allowed(name, role)}
allowed = _apply_risk_policy(allowed, max_risk, whitelist, blacklist)
if tool_list is not None:
allowed &= set(tool_list)
return [t for t in OPENAI_TOOL_SCHEMAS if t["function"]["name"] in allowed]
# ── Keyword-based tool routing ─────────────────────────────────────────────────
# Maps classifier category names → tool names in that category
CATEGORY_TOOL_MAP: dict[str, list[str]] = {
"web": ["web_search", "web_read", "http_fetch"],
"web_post": ["http_post"],
"file": ["project_file_read", "project_file_list", "file_stat", "file_grep",
"file_diff", "file_syntax_check", "file_read", "file_list", "file_write"],
"git": ["git_status", "git_log", "git_diff"],
"system": ["cortex_restart", "cortex_logs", "cortex_status", "cortex_update", "shell_exec"],
"tasks": ["task_list", "task_create", "task_update", "task_complete"],
"cron": ["cron_list", "cron_add", "cron_remove", "cron_toggle"],
"reminders": ["reminders_add", "reminders_list", "reminders_remove", "reminders_clear"],
"scratchpad": ["scratch_read", "scratch_write", "scratch_append", "scratch_clear"],
"ha": ["ha_get_state", "ha_get_states", "ha_call_service"],
"aether": ["ae_journal_list", "ae_journal_search", "ae_journal_entries_list",
"ae_journal_entry_read", "ae_journal_entry_create", "ae_journal_entry_update",
"ae_journal_entry_disable", "ae_journal_entry_append", "ae_journal_entry_prepend"],
"aether_db": ["ae_db_query", "ae_db_describe", "ae_db_show_view"],
"notifications":["web_push", "email_send", "nc_talk_send", "nc_talk_history"],
"agents": ["spawn_agent", "agent_status", "agent_list", "agent_cancel", "aider_run"],
"notes": ["agent_notes_read", "agent_notes_write", "agent_notes_append", "agent_notes_clear"],
"session": ["session_read", "session_search"],
"ae_tasks": ["ae_task_list"],
"claude": ["claude_allow_dir"],
}
_KEYWORD_CATEGORY_MAP: dict[str, list[str]] = {
"web": ["search", "look up", "what is", "who is", "weather", "forecast",
"news", "find on", "google", "website", "article", "research",
"temperature"],
"web_post": ["post to", "send to", "webhook", "trigger webhook"],
"file": ["read file", "show file", "list file", "directory", "grep",
"search in", "find in", "diff", "compare", "syntax check", "open file"],
"git": ["git", "commit", "branch", "pulled", "merged", "repository", "repo"],
"system": ["restart", "update", "status", "logs", "log", "deploy", "run command",
"shell", "is it running", "health"],
"tasks": ["task", "todo", "to-do", "to do", "add task", "create task",
"pending", "what's on my list"],
"cron": ["schedule", "cron", "every day", "every week", "recurring",
"automate", "job"],
"reminders": ["remind", "reminder", "don't forget"],
"scratchpad": ["scratch", "scratchpad", "working note", "jot down", "notepad"],
"ha": ["home assistant", "light", "thermostat", "turn on", "turn off",
"switch", "sensor", "temperature in", "kitchen", "bedroom", "garage"],
"aether": ["journal", "aether journal", "note entry", "log entry",
"search journal", "ae_journal"],
"aether_db": ["database", "query", "sql", "select", "db", "table",
"schema", "maria", "run query"],
"notifications":["notify", "push notification", "send email", "email",
"talk message", "nextcloud"],
"agents": ["spawn", "sub-agent", "delegate", "spawn agent",
"agent status", "agent list", "cancel agent", "background agent",
"aider", "code change", "edit code", "make a change to", "fix the code"],
"notes": ["agent notes", "private notes", "my notes", "agent_notes"],
"session": ["session", "history", "last time", "what did we", "earlier",
"yesterday", "last week", "previously"],
"ae_tasks": ["ae task", "kanban", "board", "ae_task"],
"claude": ["claude allow", "claude directory"],
}
def classify_tool_categories(message: str) -> list[str]:
"""Return category names whose keywords appear in message (case-insensitive).
Empty return means no tool category matched — route as pure chat with zero tool overhead.
"""
low = message.lower()
return [cat for cat, kws in _KEYWORD_CATEGORY_MAP.items() if any(kw in low for kw in kws)]
def narrow_tools_by_keywords(
message: str,
role_tools: list[str] | None,
context_messages: list[dict] | None = None,
) -> list[str]:
"""Narrow the active tool list to categories relevant to this message.
Also scans the last assistant message in context_messages — this catches follow-up
patterns like "yes, please do that" where the tool intent was expressed by the assistant
in the prior turn and the user is simply confirming.
Returns [] if no keywords matched (zero tool overhead).
Returns keyword-matched tools, intersected with role_tools if role_tools is set.
"""
scan_text = message
if context_messages:
for m in reversed(context_messages):
if m.get("role") == "assistant":
scan_text = scan_text + " " + (m.get("content") or "")
break
matched = classify_tool_categories(scan_text)
if not matched:
return []
seen: set[str] = set()
dynamic: list[str] = []
for cat in matched:
for t in CATEGORY_TOOL_MAP.get(cat, []):
if t not in seen:
seen.add(t)
dynamic.append(t)
if role_tools is not None:
role_set = set(role_tools)
dynamic = [t for t in dynamic if t in role_set]
return dynamic

253
cortex/tools/ae_database.py Normal file
View File

@@ -0,0 +1,253 @@
"""
Aether MariaDB tools — SELECT-only access to the Aether Platform database.
Credentials are read from the current user's channels.json:
"aether_db": {
"host": "192.168.64.5",
"port": 3306,
"name": "aether_dev",
"user": "aether_dev",
"password": "..."
}
Configure per-user in Settings → Notifications (or edit channels.json directly).
Only SELECT, SHOW, DESCRIBE, and EXPLAIN statements are permitted — no writes possible.
"""
import asyncio
import logging
import re
from google.genai import types
from auth_utils import get_user_channels
from persona import get_user
logger = logging.getLogger(__name__)
_MAX_ROWS = 200
_MAX_CELL = 120
_ALLOWED = {"select", "show", "describe", "desc", "explain"}
_SAFE_ID = re.compile(r'^[a-zA-Z0-9_]+$')
def _get_db_cfg() -> tuple[dict, str | None]:
"""Return (cfg_dict, error_string). cfg is empty dict on error."""
channels = get_user_channels(get_user())
cfg = channels.get("aether_db") or {}
if not cfg.get("host") or not cfg.get("user"):
return {}, (
"Aether DB not configured for this user. "
"Add an 'aether_db' block to channels.json: "
'{"host": "...", "port": 3306, "name": "aether_dev", "user": "...", "password": "..."}'
)
return cfg, None
def _is_read_only(sql: str) -> bool:
stripped = sql.strip()
if not stripped:
return False
first = stripped.split()[0].lower().rstrip(";")
return first in _ALLOWED
def _fmt(columns: list[str], rows: list[tuple]) -> str:
if not rows:
return f"({len(columns)} column{'s' if len(columns) != 1 else ''}, 0 rows)"
str_rows = [
[("NULL" if v is None else str(v))[:_MAX_CELL] for v in row]
for row in rows
]
widths = [
max([len(col)] + [len(r[i]) for r in str_rows])
for i, col in enumerate(columns)
]
sep = "+" + "+".join("-" * (w + 2) for w in widths) + "+"
header = "|" + "|".join(f" {c:<{w}} " for c, w in zip(columns, widths)) + "|"
lines = [sep, header, sep]
for row in str_rows:
lines.append("|" + "|".join(f" {v:<{w}} " for v, w in zip(row, widths)) + "|")
lines.append(sep)
note = " — results truncated at limit" if len(rows) == _MAX_ROWS else ""
lines.append(f"({len(rows)} row{'s' if len(rows) != 1 else ''}{note})")
return "\n".join(lines)
def _connect(cfg: dict):
import pymysql
import pymysql.cursors
return pymysql.connect(
host=cfg["host"],
port=int(cfg.get("port", 3306)),
user=cfg["user"],
password=cfg.get("password", ""),
database=cfg.get("name", "aether_dev"),
cursorclass=pymysql.cursors.Cursor,
connect_timeout=10,
)
async def ae_db_query(sql: str) -> str:
"""Run a read-only SQL query against the Aether MariaDB and return formatted results."""
cfg, err = _get_db_cfg()
if err:
return err
if not _is_read_only(sql):
first = sql.strip().split()[0] if sql.strip() else "(empty)"
return f"Only SELECT, SHOW, DESCRIBE, and EXPLAIN are permitted. Got: {first!r}"
def _run() -> tuple[list[str], list[tuple]]:
conn = _connect(cfg)
try:
with conn.cursor() as cur:
cur.execute(sql)
columns = [d[0] for d in cur.description] if cur.description else []
rows = list(cur.fetchmany(_MAX_ROWS))
return columns, rows
finally:
conn.close()
try:
columns, rows = await asyncio.to_thread(_run)
return _fmt(columns, rows)
except Exception as e:
logger.warning("ae_db_query error: %s", e)
return f"Query error: {e}"
async def ae_db_describe(table: str, detailed: bool = False) -> str:
"""Describe the columns of an Aether DB table or view."""
cfg, err = _get_db_cfg()
if err:
return err
if not _SAFE_ID.match(table):
return f"Invalid table name: {table!r}. Only letters, digits, and underscores allowed."
def _run():
conn = _connect(cfg)
try:
with conn.cursor() as cur:
cur.execute(f"DESCRIBE `{table}`")
columns = [d[0] for d in cur.description] if cur.description else []
rows = list(cur.fetchall())
return columns, rows
finally:
conn.close()
try:
columns, rows = await asyncio.to_thread(_run)
if not detailed:
fields = [row[0] for row in rows]
return f"{table}: " + ", ".join(fields)
return _fmt(columns, rows)
except Exception as e:
logger.warning("ae_db_describe error: %s", e)
return f"Describe error: {e}"
async def ae_db_show_view(view_name: str) -> str:
"""Return the CREATE VIEW SQL for an Aether DB view."""
cfg, err = _get_db_cfg()
if err:
return err
if not _SAFE_ID.match(view_name):
return f"Invalid view name: {view_name!r}. Only letters, digits, and underscores allowed."
def _run():
conn = _connect(cfg)
try:
with conn.cursor() as cur:
cur.execute(f"SHOW CREATE VIEW `{view_name}`")
return cur.fetchone()
finally:
conn.close()
try:
row = await asyncio.to_thread(_run)
if not row:
return f"View not found: {view_name}"
return str(row[1]) if len(row) > 1 else str(row[0])
except Exception as e:
logger.warning("ae_db_show_view error: %s", e)
return f"Show view error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="ae_db_describe",
description=(
"Describe the columns of an Aether Platform table or view. "
"Returns a compact field list by default; pass detailed=true for full schema "
"(type, nullability, default, key). Use to understand data structure before "
"writing a SELECT query, or to answer 'what fields does X have?'. "
"Examples: table='ae_journals'; table='clients'; table='time_entries'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"table": types.Schema(
type=types.Type.STRING,
description="Table or view name (letters, digits, underscores only)",
),
"detailed": types.Schema(
type=types.Type.BOOLEAN,
description="Return full schema (type, nullability, key, default) instead of just field names",
),
},
required=["table"],
),
),
types.FunctionDeclaration(
name="ae_db_show_view",
description=(
"Return the CREATE VIEW SQL for an Aether Platform database view. "
"Use to understand how a view is constructed before querying it, "
"or to debug unexpected results from a view. "
"Example: view_name='v_active_journals'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"view_name": types.Schema(
type=types.Type.STRING,
description="View name (letters, digits, underscores only)",
),
},
required=["view_name"],
),
),
types.FunctionDeclaration(
name="ae_db_query",
description=(
"Run a read-only SQL query against the Aether Platform MariaDB. "
"Permitted statements: SELECT, SHOW, DESCRIBE, EXPLAIN. No writes are possible. "
"Use for debugging: bad data, missing records, broken foreign keys, schema questions. "
"Results capped at 200 rows; cells truncated at 120 chars. "
"Examples: SELECT * FROM clients WHERE email = 'x@y.com'; "
"SELECT COUNT(*) FROM time_entries WHERE billed = 0 AND deleted_at IS NULL; "
"SHOW TABLES; DESCRIBE ae_journals; "
"SELECT id_random, enable, deleted_at FROM ae_journals WHERE id_random = 'abc123'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"sql": types.Schema(
type=types.Type.STRING,
description=(
"SQL query to run — SELECT, SHOW, DESCRIBE, or EXPLAIN only. "
"No semicolons required but harmless if present."
),
),
},
required=["sql"],
),
),
]

View File

@@ -0,0 +1,747 @@
"""
Aether Platform knowledge tools — journal search, listing, and entry management.
These tools give the orchestrator read/write access to the AE Journals module,
which serves as the primary long-term knowledge base.
Auth: x-aether-api-key + x-account-id headers (same pattern as agents_sync scripts).
API: V3 CRUD — POST /v3/crud/journal_entry/search, POST /v3/crud/journal/{id}/journal_entry/
PATCH /v3/crud/journal_entry/{entry_id}, GET /v3/crud/journal_entry/{entry_id}
"""
import asyncio
import logging
from google.genai import types
from config import settings
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Shared helpers
# ---------------------------------------------------------------------------
def _headers() -> dict:
return {
"x-aether-api-key": settings.ae_api_key,
"x-account-id": settings.ae_account_id,
"Content-Type": "application/json",
}
def _check_config() -> str | None:
"""Return an error string if AE API is not configured, else None."""
if not settings.ae_api_key or not settings.ae_account_id:
return (
"AE API not configured. Set AE_API_KEY and AE_ACCOUNT_ID in .env. "
"Values are the same as agents_sync/mcp/.env."
)
return None
# ---------------------------------------------------------------------------
# Tool: ae_journal_search
# ---------------------------------------------------------------------------
async def journal_search(
query: str = "",
journal_id: str = "",
tags: str = "",
type_code: str = "",
topic_code: str = "",
date_from: str = "",
date_to: str = "",
sort_by: str = "updated",
sort_order: str = "desc",
status: int | None = None,
priority: int | None = None,
max_results: int = 10,
page: int = 1,
) -> str:
"""Search AE Journal entries.
At least one of query, tags, type_code, topic_code, date_from, or journal_id
should be provided. All filters combine with AND.
"""
err = _check_config()
if err:
return err
return await asyncio.to_thread(
_sync_journal_search,
query, journal_id, tags, type_code, topic_code,
date_from, date_to, sort_by, sort_order,
status, priority, max_results, page,
)
def _sync_journal_search(
query: str,
journal_id: str,
tags: str,
type_code: str,
topic_code: str,
date_from: str,
date_to: str,
sort_by: str,
sort_order: str,
status: int | None,
priority: int | None,
max_results: int,
page: int,
) -> str:
import requests
# Build sort field
sort_field_map = {
"updated": "updated_on",
"created": "created_on",
"name": "name",
"priority": "priority",
}
sort_field = sort_field_map.get(sort_by, "updated_on")
order_by = f"{'-' if sort_order == 'desc' else ''}{sort_field}"
search_body: dict = {"page_size": max_results, "page": page, "order_by": order_by}
# Fulltext keyword — uses MATCH/AGAINST index
if query:
search_body["query_string"] = query
# Additional AND filters
and_filters: list[dict] = []
if tags:
and_filters.append({"field": "tags", "op": "icontains", "value": tags})
if type_code:
and_filters.append({"field": "type_code", "op": "eq", "value": type_code})
if topic_code:
and_filters.append({"field": "topic_code", "op": "eq", "value": topic_code})
if date_from:
and_filters.append({"field": "created_on", "op": "gte", "value": date_from})
if date_to:
and_filters.append({"field": "created_on", "op": "lte", "value": date_to})
if status is not None:
and_filters.append({"field": "status", "op": "eq", "value": status})
if priority is not None:
and_filters.append({"field": "priority", "op": "eq", "value": priority})
if and_filters:
search_body["and"] = and_filters
# query_string must be present for `and` filters to apply
if "query_string" not in search_body:
search_body["query_string"] = "%"
params: dict = {}
if journal_id:
params["for_obj_type"] = "journal"
params["for_obj_id"] = journal_id
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
try:
resp = requests.post(
url,
headers=_headers(),
params=params,
json=search_body,
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
data = resp.json()
except Exception as e:
logger.warning("ae_journal_search failed: %s", e)
return f"Journal search error: {e}"
entries = data.get("data", [])
total = (data.get("meta") or {}).get("data_list_count") or len(entries)
if not entries:
desc = query or tags or type_code or topic_code or f"journal {journal_id}"
return f"No journal entries found for: {desc}"
label = query or tags or f"{len(entries)} entries"
lines = [f"Journal entries — **{label}** ({total} total, page {page}):\n"]
for entry in entries:
title = entry.get("name") or "(untitled)"
entry_id = entry.get("journal_entry_id") or entry.get("id") or ""
journal_name = entry.get("journal_name") or entry.get("parent_name") or ""
summary = entry.get("summary") or ""
entry_tags = entry.get("tags") or []
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:10]
content_preview = (entry.get("content") or "")[:400].replace("\n", " ")
header = f"**{title}**"
if journal_name:
header += f" ({journal_name})"
header += f" — id: `{entry_id}`"
if updated:
header += f" [{updated}]"
lines.append(header)
if entry_tags:
tag_list = entry_tags if isinstance(entry_tags, list) else [t.strip() for t in str(entry_tags).split(",")]
lines.append(f" Tags: {', '.join(tag_list)}")
if summary:
lines.append(f" {summary}")
elif content_preview:
lines.append(f" {content_preview}{'' if len(entry.get('content', '')) > 400 else ''}")
lines.append("")
if total > page * max_results:
lines.append(f"(More results — call again with page={page + 1})")
return "\n".join(lines).strip()
# ---------------------------------------------------------------------------
# Tool: ae_journal_list
# ---------------------------------------------------------------------------
async def journal_list() -> str:
"""List all journals accessible to the configured AE account."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_list)
def _sync_journal_list() -> str:
import requests
url = f"{settings.ae_api_url}/v3/crud/journal/search"
try:
resp = requests.post(
url,
headers=_headers(),
json={"page_size": 100},
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
data = resp.json()
except Exception as e:
logger.warning("ae_journal_list failed: %s", e)
return f"Journal list error: {e}"
journals = data.get("data", [])
if not journals:
return "No journals found for this account."
lines = [f"Journals ({len(journals)}):\n"]
for j in journals:
jid = j.get("journal_id") or j.get("id_random") or j.get("id") or "?"
name = j.get("name") or "(untitled)"
desc = j.get("description") or ""
line = f"- **{name}** — id: `{jid}`"
if desc:
line += f"\n {desc}"
lines.append(line)
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_create
# ---------------------------------------------------------------------------
async def journal_entry_create(
journal_id: str,
title: str,
content: str,
summary: str = "",
tags: str = "",
) -> str:
"""Create a new entry in an AE Journal.
Args:
journal_id: The id_random of the target journal (use ae_journal_search to find it,
or ask the user which journal to write to).
title: Entry title (name field).
content: Full entry content (markdown supported).
summary: Optional short summary (1-2 sentences).
tags: Optional comma-separated tags.
Returns a confirmation with the new entry's id_random, or an error message.
"""
err = _check_config()
if err:
return err
return await asyncio.to_thread(
_sync_journal_entry_create, journal_id, title, content, summary, tags
)
def _sync_journal_entry_create(
journal_id: str, title: str, content: str, summary: str, tags: str
) -> str:
import requests
url = f"{settings.ae_api_url}/v3/crud/journal/{journal_id}/journal_entry/"
data: dict = {"name": title, "content": content}
if summary:
data["summary"] = summary
if tags:
data["tags"] = [t.strip() for t in tags.split(",") if t.strip()]
try:
resp = requests.post(
url,
headers=_headers(),
json=data,
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
result = resp.json()
except Exception as e:
logger.warning("ae_journal_entry_create failed: %s", e)
return f"Journal entry creation error: {e}"
entry_id = (
result.get("data", {}).get("journal_entry_id")
or result.get("data", {}).get("id_random")
or result.get("id_random")
or "unknown"
)
return f"Journal entry created. id: `{entry_id}`, title: \"{title}\", journal: `{journal_id}`"
# ---------------------------------------------------------------------------
# Shared helper: fetch a single journal entry by id
# ---------------------------------------------------------------------------
def _get_entry(entry_id: str) -> dict | str:
"""Return the entry dict, or an error string on failure."""
import requests
url = f"{settings.ae_api_url}/v3/crud/journal_entry/{entry_id}"
try:
resp = requests.get(url, headers=_headers(), timeout=settings.ae_api_timeout)
resp.raise_for_status()
data = resp.json()
entry = data.get("data") or data
if not isinstance(entry, dict):
return f"Unexpected response shape for entry {entry_id}"
return entry
except Exception as e:
logger.warning("_get_entry %s failed: %s", entry_id, e)
return f"Error fetching entry {entry_id}: {e}"
def _patch_entry(entry_id: str, payload: dict) -> str:
"""PATCH a journal entry. Returns a success/error string."""
import requests
url = f"{settings.ae_api_url}/v3/crud/journal_entry/{entry_id}"
try:
resp = requests.patch(
url,
headers=_headers(),
json=payload,
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
return "ok"
except Exception as e:
logger.warning("_patch_entry %s failed: %s", entry_id, e)
return f"Error updating entry {entry_id}: {e}"
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_read
# ---------------------------------------------------------------------------
async def journal_entry_read(entry_id: str, max_content_chars: int = 4000) -> str:
"""Return the full content of a single journal entry by its id_random."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_read, entry_id, max_content_chars)
def _sync_journal_entry_read(entry_id: str, max_content_chars: int) -> str:
entry = _get_entry(entry_id)
if isinstance(entry, str):
return entry
title = entry.get("name") or "(untitled)"
journal = entry.get("journal_name") or entry.get("parent_name") or ""
summary = entry.get("summary") or ""
raw_tags = entry.get("tags") or []
tags = raw_tags if isinstance(raw_tags, list) else [t.strip() for t in str(raw_tags).split(",") if t.strip()]
content = entry.get("content") or ""
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:19].replace("T", " ")
enabled = entry.get("enable", True)
lines = [f"# {title}"]
meta: list[str] = [f"id: `{entry_id}`"]
if journal:
meta.append(f"journal: {journal}")
if updated:
meta.append(f"updated: {updated}")
if not enabled:
meta.append("**DISABLED**")
lines.append(" ".join(meta))
if tags:
lines.append(f"Tags: {', '.join(tags)}")
if summary:
lines.append(f"\nSummary: {summary}")
lines.append("\n---\n")
truncated = len(content) > max_content_chars
lines.append(content[:max_content_chars])
if truncated:
lines.append(
f"\n\n[Content truncated at {max_content_chars} chars — "
f"{len(content)} total. Call again with a higher max_content_chars to read more.]"
)
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Tool: ae_journal_entries_list
# ---------------------------------------------------------------------------
async def journal_entries_list(journal_id: str, max_results: int = 20, page: int = 1) -> str:
"""List entries in a specific journal, newest first."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entries_list, journal_id, max_results, page)
def _sync_journal_entries_list(journal_id: str, max_results: int, page: int) -> str:
import requests
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
search_body: dict = {
"page_size": max_results,
"page": page,
"order_by": "-updated_on",
}
params = {"for_obj_type": "journal", "for_obj_id": journal_id}
try:
resp = requests.post(
url,
headers=_headers(),
params=params,
json=search_body,
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
data = resp.json()
except Exception as e:
logger.warning("ae_journal_entries_list failed: %s", e)
return f"Journal entries list error: {e}"
entries = data.get("data", [])
total = (data.get("meta") or {}).get("data_list_count") or len(entries)
if not entries:
return f"No entries found in journal `{journal_id}`."
offset = (page - 1) * max_results + 1
lines = [f"Entries in journal `{journal_id}` — showing {offset}{offset + len(entries) - 1} of {total}:\n"]
for i, entry in enumerate(entries, offset):
title = entry.get("name") or "(untitled)"
entry_id = entry.get("journal_entry_id") or entry.get("id") or ""
raw_tags = entry.get("tags") or []
tags = raw_tags if isinstance(raw_tags, list) else [t.strip() for t in str(raw_tags).split(",") if t.strip()]
summary = entry.get("summary") or ""
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:10]
enabled = entry.get("enable", True)
status = "" if enabled else " [disabled]"
date_str = f" [{updated}]" if updated else ""
lines.append(f"{i}. **{title}**{status} — id: `{entry_id}`{date_str}")
if tags:
lines.append(f" Tags: {', '.join(tags)}")
if summary:
lines.append(f" {summary[:150]}{'' if len(summary) > 150 else ''}")
lines.append("")
if total > offset + len(entries) - 1:
lines.append(f"(More entries available — call again with page={page + 1})")
return "\n".join(lines).rstrip()
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_update
# ---------------------------------------------------------------------------
async def journal_entry_update(
entry_id: str,
title: str = "",
content: str = "",
summary: str = "",
tags: str = "",
enable: bool | None = None,
) -> str:
"""Update fields on an existing journal entry. Only provided fields are changed."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_update, entry_id, title, content, summary, tags, enable)
def _sync_journal_entry_update(
entry_id: str,
title: str,
content: str,
summary: str,
tags: str,
enable: bool | None,
) -> str:
payload: dict = {}
if title:
payload["name"] = title
if content:
payload["content"] = content
if summary:
payload["summary"] = summary
if tags:
payload["tags"] = [t.strip() for t in tags.split(",") if t.strip()]
if enable is not None:
payload["enable"] = enable
if not payload:
return "Nothing to update — no fields provided."
result = _patch_entry(entry_id, payload)
if result != "ok":
return result
updated = ", ".join(payload.keys())
return f"Journal entry `{entry_id}` updated. Fields changed: {updated}"
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_disable
# ---------------------------------------------------------------------------
async def journal_entry_disable(entry_id: str) -> str:
"""Soft-delete a journal entry by setting enable=false."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_patch_entry, entry_id, {"enable": False})
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_append
# ---------------------------------------------------------------------------
async def journal_entry_append(entry_id: str, content: str, heading: str = "") -> str:
"""Append a timestamped section to the bottom of a journal entry's content."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_append, entry_id, content, heading)
def _sync_journal_entry_append(entry_id: str, content: str, heading: str) -> str:
from datetime import datetime, timezone
entry = _get_entry(entry_id)
if isinstance(entry, str):
return entry
existing = (entry.get("content") or "").rstrip()
ts = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
section_heading = heading or ts
new_content = f"{existing}\n\n### {section_heading}\n{content.strip()}"
result = _patch_entry(entry_id, {"content": new_content})
if result != "ok":
return result
return f"Appended to journal entry `{entry_id}` under heading \"{section_heading}\"."
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_prepend
# ---------------------------------------------------------------------------
async def journal_entry_prepend(entry_id: str, content: str, heading: str = "") -> str:
"""Prepend a timestamped section to the top of a journal entry's content."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_prepend, entry_id, content, heading)
def _sync_journal_entry_prepend(entry_id: str, content: str, heading: str) -> str:
from datetime import datetime, timezone
entry = _get_entry(entry_id)
if isinstance(entry, str):
return entry
existing = (entry.get("content") or "").lstrip()
ts = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
section_heading = heading or ts
new_content = f"### {section_heading}\n{content.strip()}\n\n{existing}"
result = _patch_entry(entry_id, {"content": new_content})
if result != "ok":
return result
return f"Prepended to journal entry `{entry_id}` under heading \"{section_heading}\"."
DECLARATIONS = [
types.FunctionDeclaration(
name="ae_journal_list",
description=(
"List all Aether Journals available for this account. "
"Returns each journal's name and id_random. "
"Call this first when you need to write a new entry or scope a search to a specific journal "
"and don't already know the journal's id."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="ae_journal_search",
description=(
"Search Aether Journal entries. All parameters are optional — combine freely. "
"Use 'query' for fulltext keyword search (supports boolean: +required -excluded \"phrase\"). "
"Use 'tags' to filter by tag substring. Use 'date_from'/'date_to' for date ranges (YYYY-MM-DD). "
"Always search before creating a new entry to avoid duplicates."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"query": types.Schema(type=types.Type.STRING, description="Fulltext keyword search. Supports boolean mode: +required -excluded \"exact phrase\"."),
"journal_id": types.Schema(type=types.Type.STRING, description="Scope results to a specific journal by its id_random. Omit to search all journals."),
"tags": types.Schema(type=types.Type.STRING, description="Filter by tag substring (e.g. 'networking' matches entries tagged 'networking' or 'home-networking')."),
"type_code": types.Schema(type=types.Type.STRING, description="Filter by exact type_code (e.g. 'note', 'meeting', 'log')."),
"topic_code": types.Schema(type=types.Type.STRING, description="Filter by exact topic_code."),
"date_from": types.Schema(type=types.Type.STRING, description="Return entries created on or after this date (YYYY-MM-DD)."),
"date_to": types.Schema(type=types.Type.STRING, description="Return entries created on or before this date (YYYY-MM-DD)."),
"sort_by": types.Schema(type=types.Type.STRING, description="Sort field: 'updated' (default), 'created', 'name', or 'priority'."),
"sort_order": types.Schema(type=types.Type.STRING, description="Sort direction: 'desc' (default, newest first) or 'asc'."),
"status": types.Schema(type=types.Type.INTEGER, description="Filter by exact status code."),
"priority": types.Schema(type=types.Type.INTEGER, description="Filter by exact priority (1=low, 5=high)."),
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of results per page (default 10)."),
"page": types.Schema(type=types.Type.INTEGER, description="Page number for pagination (default 1)."),
},
required=[],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_read",
description=(
"Fetch the full content of a single journal entry by its id_random. "
"Use this when you need to read an entry before editing it, or when search results "
"don't show enough content. Returns title, journal, tags, summary, and full content."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="The id_random of the journal entry to read."),
"max_content_chars": types.Schema(type=types.Type.INTEGER, description="Maximum characters of content to return (default 4000). Increase for long entries."),
},
required=["entry_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entries_list",
description=(
"List entries in a specific journal, newest first. "
"Use this to browse what's in a journal when you don't have a search keyword, "
"or to find entries by browsing rather than searching. "
"Returns numbered entries with id, title, tags, summary, and date."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"journal_id": types.Schema(type=types.Type.STRING, description="The id_random of the journal to list entries from."),
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of entries to return (default 20, max 50)."),
"page": types.Schema(type=types.Type.INTEGER, description="Page number for pagination (default 1)."),
},
required=["journal_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_create",
description=(
"Create a new entry in an Aether Journal. "
"Use this to save notes, summaries, or any content the user wants to store. "
"Always call ae_journal_search first to check for existing entries on the same topic."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"journal_id": types.Schema(type=types.Type.STRING, description="The id_random of the target journal. Ask the user which journal to write to if not specified."),
"title": types.Schema(type=types.Type.STRING, description="Entry title"),
"content": types.Schema(type=types.Type.STRING, description="Full entry content (markdown supported)"),
"summary": types.Schema(type=types.Type.STRING, description="Optional short summary (1-2 sentences)"),
"tags": types.Schema(type=types.Type.STRING, description="Optional comma-separated tags (e.g. 'wireguard, networking, homelab')"),
},
required=["journal_id", "title", "content"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_update",
description=(
"Update fields on an existing journal entry. Only the fields you provide are changed — "
"omitted fields are left as-is. Use ae_journal_search to find the entry_id first. "
"To soft-delete, use ae_journal_entry_disable instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
"title": types.Schema(type=types.Type.STRING, description="New title"),
"content": types.Schema(type=types.Type.STRING, description="Replacement content (full, markdown supported)"),
"summary": types.Schema(type=types.Type.STRING, description="New summary"),
"tags": types.Schema(type=types.Type.STRING, description="Replacement comma-separated tags"),
"enable": types.Schema(type=types.Type.BOOLEAN, description="Set false to hide/disable the entry"),
},
required=["entry_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_disable",
description=(
"Soft-delete a journal entry by setting enable=false. "
"The entry is hidden but not permanently removed. "
"Use ae_journal_search to find the entry_id first."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
},
required=["entry_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_append",
description=(
"Append a new section to the bottom of a journal entry's content. "
"Each section gets a UTC timestamp heading unless you provide one. "
"Ideal for timestamped logs, running notes, or data logs."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
"content": types.Schema(type=types.Type.STRING, description="The text to append (markdown supported)"),
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading (defaults to current UTC timestamp)"),
},
required=["entry_id", "content"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_prepend",
description=(
"Prepend a new section to the top of a journal entry's content. "
"Each section gets a UTC timestamp heading unless you provide one. "
"Useful for most-recent-first logs."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
"content": types.Schema(type=types.Type.STRING, description="The text to prepend (markdown supported)"),
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading (defaults to current UTC timestamp)"),
},
required=["entry_id", "content"],
),
),
]

119
cortex/tools/ae_tasks.py Normal file
View File

@@ -0,0 +1,119 @@
"""
Aether task list tool — reads the agents_sync Kanban board.
Reads task JSON files directly from the agents_sync filesystem rather than
making an HTTP call, since the tasks directory is always locally available
(synced via Syncthing). This avoids needing a separate API endpoint for tasks.
Structure:
agents_sync/tasks/01_todo/ — pending tasks
agents_sync/tasks/02_in_progress/ — active tasks
agents_sync/tasks/03_done/ — completed tasks (not included by default)
"""
import asyncio
import json
import logging
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
# Resolved at import time — agents_sync is always at ~/agents_sync on this machine.
# If the path doesn't exist the tool returns a helpful error rather than crashing.
_AGENTS_SYNC = Path.home() / "agents_sync"
_TASKS_ROOT = _AGENTS_SYNC / "tasks"
async def task_list(include_done: bool = False) -> str:
"""List tasks from the agents_sync Kanban board.
Reads the todo and in_progress buckets (and optionally done).
Returns a markdown summary grouped by status.
Args:
include_done: If True, also include completed tasks (can be noisy).
"""
return await asyncio.to_thread(_sync_task_list, include_done)
def _sync_task_list(include_done: bool) -> str:
if not _TASKS_ROOT.exists():
return f"Task directory not found: {_TASKS_ROOT}"
buckets = [
("01_todo", "Todo"),
("02_in_progress", "In Progress"),
]
if include_done:
buckets.append(("03_done", "Done"))
sections: list[str] = []
total = 0
for dir_name, label in buckets:
bucket_dir = _TASKS_ROOT / dir_name
if not bucket_dir.exists():
continue
tasks = _read_bucket(bucket_dir)
total += len(tasks)
if not tasks:
continue
lines = [f"## {label} ({len(tasks)})\n"]
for task in tasks:
title = task.get("title") or task.get("name") or "(untitled)"
assigned = task.get("assigned_to") or ""
task_id = task.get("id") or ""
desc = task.get("description") or ""
header = f"- **{title}**"
if assigned:
header += f" (assigned: {assigned})"
if task_id:
header += f" — `{task_id}`"
lines.append(header)
if desc:
# First sentence / 120 chars of description
short = desc.split(".")[0][:120]
lines.append(f" {short}")
sections.append("\n".join(lines))
if not sections:
return "No tasks found on the Kanban board."
header_line = f"# Kanban Board — {total} task(s)\n"
return header_line + "\n\n".join(sections)
def _read_bucket(bucket_dir: Path) -> list[dict]:
"""Read and parse all JSON task files in a bucket directory."""
tasks = []
for path in sorted(bucket_dir.glob("*.json")):
try:
data = json.loads(path.read_text())
tasks.append(data)
except Exception as e:
logger.warning("Failed to read task file %s: %s", path, e)
return tasks
DECLARATIONS = [
types.FunctionDeclaration(
name="ae_task_list",
description=(
"List tasks from the agents_sync Kanban board (todo and in-progress). "
"Use this when asked about current work, pending tasks, or project status."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"include_done": types.Schema(type=types.Type.BOOLEAN, description="If true, also include completed tasks (default false)"),
},
),
),
]

155
cortex/tools/agent_notes.py Normal file
View File

@@ -0,0 +1,155 @@
"""
Agent private notes — AGENT_NOTES.md.
A persistent notepad only the orchestrator can write to. The file itself is
never exposed in the Files panel or loaded into user-facing context tiers.
Up to 3 rolling backups are kept automatically before each write so past
versions can be reviewed.
Use for: observations about the user's patterns, working hypotheses,
long-running goals, things to remember across sessions that shouldn't
be part of the distilled memory visible to the user.
"""
import asyncio
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path
_FILENAME = "AGENT_NOTES.md"
_N_BACKUPS = 3
def _notes_path() -> Path:
return persona_path() / _FILENAME
def _now_label() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
def _rotate(path: Path) -> None:
"""Rotate up to _N_BACKUPS rolling backups before a write."""
if not path.exists():
return
for i in range(_N_BACKUPS, 1, -1):
older = path.parent / f"{path.stem}.bak{i}.md"
newer = path.parent / f"{path.stem}.bak{i - 1}.md"
if newer.exists():
older.write_text(newer.read_text())
bak1 = path.parent / f"{path.stem}.bak1.md"
bak1.write_text(path.read_text())
# ── Sync implementations ────────────────────────────────────────────────────
def _agent_notes_read() -> str:
p = _notes_path()
if not p.exists() or not p.read_text().strip():
return "Agent notes are empty."
return p.read_text()
def _agent_notes_write(content: str) -> str:
p = _notes_path()
_rotate(p)
p.write_text(content.rstrip() + "\n")
return "Agent notes updated."
def _agent_notes_append(content: str, heading: str | None = None) -> str:
p = _notes_path()
_rotate(p)
existing = p.read_text() if p.exists() else ""
label = heading or _now_label()
section = f"\n## {label}\n\n{content.strip()}\n"
p.write_text(existing.rstrip() + "\n" + section)
return f"Appended to agent notes: {label}"
def _agent_notes_clear() -> str:
p = _notes_path()
_rotate(p)
p.write_text("")
return "Agent notes cleared."
# ── Async wrappers ───────────────────────────────────────────────────────────
async def agent_notes_read() -> str:
return await asyncio.to_thread(_agent_notes_read)
async def agent_notes_write(content: str) -> str:
return await asyncio.to_thread(_agent_notes_write, content)
async def agent_notes_append(content: str, heading: str | None = None) -> str:
return await asyncio.to_thread(_agent_notes_append, content, heading)
async def agent_notes_clear() -> str:
return await asyncio.to_thread(_agent_notes_clear)
# ── Gemini FunctionDeclarations ──────────────────────────────────────────────
DECLARATIONS = [
types.FunctionDeclaration(
name="agent_notes_read",
description=(
"Read your private agent notes — a persistent notepad only you can write to. "
"Use this to recall observations, working hypotheses, long-running goals, or "
"anything you want to remember across sessions without surfacing it to the user. "
"This file is never shown in the user's Files panel."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="agent_notes_write",
description=(
"Replace your private agent notes with new content. "
"A backup is saved automatically before writing. "
"Use agent_notes_append to add without replacing."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(
type=types.Type.STRING,
description="The new notes content (markdown supported).",
),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="agent_notes_append",
description=(
"Add a new section to your private agent notes without replacing existing content. "
"A backup is saved automatically before writing. "
"Each section gets a UTC timestamp heading unless you supply one."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(
type=types.Type.STRING,
description="The content to append (markdown supported).",
),
"heading": types.Schema(
type=types.Type.STRING,
description="Optional section heading. Defaults to current UTC timestamp.",
),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="agent_notes_clear",
description=(
"Erase all private agent notes. A backup is saved automatically before clearing."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

446
cortex/tools/agents.py Normal file
View File

@@ -0,0 +1,446 @@
"""
Agent spawning and lifecycle tools.
spawn_agent — synchronous or background sub-agent via any configured role model.
agent_status / agent_list / agent_cancel — lifecycle management for background agents.
Sub-agents run using the model and tools assigned to the given role. The three-level
hierarchy (Persona → Specialized → Support) is enforced by denying spawn_agent and
aider_run at the L2→L3 boundary — Level 3 agents cannot delegate further.
Supported model types for sub-agents: local_openai, gemini_api.
claude_cli / gemini_cli are chat-only and do not support tool-enabled sub-agents.
"""
import asyncio
import logging
from datetime import datetime
from google.genai import types
import agent_manager
logger = logging.getLogger(__name__)
# Per-host semaphores — keyed by "host:<host_id>" or "type:<model_type>"
# Created lazily on first use; never deleted (module-level singletons)
_semaphores: dict[str, asyncio.Semaphore] = {}
_sem_lock = asyncio.Lock()
# Tools denied at the L2→L3 boundary so Level 3 agents cannot delegate further.
_L3_DENY_TOOLS = ["spawn_agent", "aider_run"]
async def _get_semaphore(key: str, max_concurrent: int) -> asyncio.Semaphore:
"""Return (or create) the semaphore for a given host/type key."""
async with _sem_lock:
if key not in _semaphores:
_semaphores[key] = asyncio.Semaphore(max_concurrent)
return _semaphores[key]
async def spawn_agent(
task: str,
role: str = "chat",
tier: int = 1,
timeout: int = 120,
max_rounds: int | None = None,
allow_tools: list[str] | None = None,
deny_tools: list[str] | None = None,
background: bool = False,
notify: bool = False,
_agent_level: int = 2,
) -> str:
"""
Spawn a sub-agent to complete a task.
In synchronous mode (background=False, the default): blocks until done and returns
the result string.
In background mode (background=True): registers the agent, fires it as an asyncio
background task, and returns an agent_id string immediately. Use agent_status() to
poll, or set notify=True to receive a push notification on completion.
Level enforcement: this agent (level _agent_level) spawns children at level+1.
Children at level 3 automatically have spawn_agent and aider_run denied so they
cannot delegate further.
"""
import model_registry
from context_loader import load_context
from auth_utils import get_user_role, get_tool_policy
from persona import get_user
user = get_user() or "scott"
role_cfg = model_registry.get_role_config(user, role)
model_cfg = model_registry.get_model_for_role(user, role)
if not model_cfg:
return f"spawn_agent: no model configured for role '{role}'"
model_type = model_cfg.get("type", "unknown")
if model_type not in ("local_openai", "gemini_api"):
return (
f"spawn_agent: model type '{model_type}' does not support tool-enabled sub-agents. "
f"Assign a local_openai or gemini_api model to role '{role}'."
)
# Determine concurrency key and semaphore limit
host_id = model_cfg.get("host_id")
if host_id:
registry = model_registry.get_registry(user)
host = next((h for h in registry.get("hosts", []) if h["id"] == host_id), None)
max_concurrent = (host or {}).get("max_concurrent", 3)
sem_key = f"host:{host_id}"
else:
max_concurrent = 5 if model_type == "gemini_api" else 3
sem_key = f"type:{model_type}"
sem = await _get_semaphore(sem_key, max_concurrent)
system_prompt = load_context(
tier=tier,
include_long=(tier >= 2),
include_mid=(tier >= 2),
include_short=(tier >= 2),
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
)
user_role = get_user_role(user)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(user)
confirm_allow = set(policy.get("allow", []))
confirm_deny = set(policy.get("deny", []))
# Per-call tool restrictions — role config remains the authoritative ceiling
if allow_tools is not None:
if tool_list is not None:
tool_list = [t for t in tool_list if t in allow_tools]
else:
tool_list = list(allow_tools)
if deny_tools is not None:
deny_set = set(deny_tools)
if tool_list is not None:
tool_list = [t for t in tool_list if t not in deny_set]
else:
confirm_deny = confirm_deny | deny_set
# Level enforcement: children of this agent are at level _agent_level + 1.
# Level 3 children cannot delegate — auto-deny the spawning tools.
child_level = _agent_level + 1
if child_level >= 3:
l3_deny = set(_L3_DENY_TOOLS)
if tool_list is not None:
tool_list = [t for t in tool_list if t not in l3_deny]
else:
confirm_deny = confirm_deny | l3_deny
if max_rounds is not None:
model_cfg = dict(model_cfg)
model_cfg["max_rounds"] = max_rounds
async def _run() -> str:
if model_type == "local_openai":
import openai_orchestrator
result = await openai_orchestrator.run(
task=task,
system_prompt=system_prompt,
model_cfg=model_cfg,
respond_with_final=True,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
)
if result.checkpoint:
return (
"Sub-agent requires user confirmation — "
"confirmation gates are not supported inside spawn_agent. "
"Pre-allow the tool in the user's tool policy or use a different role."
)
return result.response or "(sub-agent returned no output)"
# gemini_api
import orchestrator_engine
from auth_utils import get_user_gemini_key
gemini_key = model_cfg.get("api_key") or get_user_gemini_key(user)
result = await orchestrator_engine.run(
task=task,
system_prompt=system_prompt,
session_messages=None,
respond_with_claude=True,
gemini_api_key=gemini_key,
model_name=model_cfg.get("model_name"),
response_role=role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
max_rounds=model_cfg.get("max_rounds"),
)
if result.checkpoint:
return (
"Sub-agent requires user confirmation — "
"confirmation gates are not supported inside spawn_agent."
)
return result.response or "(sub-agent returned no output)"
if background:
rec = await agent_manager.register(
user=user,
role=role,
task=task,
level=_agent_level,
notify=notify,
)
async def _bg_task() -> None:
async with sem:
try:
logger.info(
"spawn_agent [bg]: %s role=%s level=%d timeout=%ds",
rec.agent_id[:8], role, _agent_level, timeout,
)
result = await asyncio.wait_for(_run(), timeout=float(timeout))
await agent_manager.finish(rec.agent_id, result, "done")
logger.info("spawn_agent [bg]: done %s", rec.agent_id[:8])
except asyncio.CancelledError:
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
raise
except asyncio.TimeoutError:
msg = f"Sub-agent timed out after {timeout}s (role={role})"
logger.warning("spawn_agent [bg]: timeout %s", rec.agent_id[:8])
await agent_manager.finish(rec.agent_id, msg, "timeout")
except Exception as e:
logger.exception("spawn_agent [bg]: failed %s", rec.agent_id[:8])
await agent_manager.finish(rec.agent_id, str(e), "failed")
bg = asyncio.create_task(_bg_task())
agent_manager.set_task_ref(rec.agent_id, bg)
return f"Agent started in background. ID: {rec.agent_id}\nUse agent_status('{rec.agent_id}') to check progress."
# Synchronous path — unchanged behaviour
async with sem:
try:
logger.info(
"spawn_agent: role=%s tier=%d timeout=%ds task=%.80s",
role, tier, timeout, task,
)
response = await asyncio.wait_for(_run(), timeout=float(timeout))
logger.info("spawn_agent: done role=%s response=%d chars", role, len(response))
return response
except asyncio.TimeoutError:
logger.warning("spawn_agent: timed out after %ds role=%s", timeout, role)
return f"Sub-agent timed out after {timeout}s (role={role})"
except Exception as e:
logger.exception("spawn_agent: failed role=%s", role)
return f"Sub-agent error ({role}): {e}"
# ── Agent lifecycle tools ─────────────────────────────────────────────────────
async def agent_status(agent_id: str) -> str:
"""Return the status and result preview of a background agent."""
from persona import get_user
user = get_user() or "unknown"
rec = agent_manager.get(agent_id)
if not rec:
return f"No agent found with ID: {agent_id}"
if rec.user != user:
return "Access denied."
now = datetime.now()
end = rec.finished or now
elapsed = int((end - rec.started).total_seconds())
lines = [
f"Agent {rec.agent_id[:8]}",
f" Status: {rec.status}",
f" Role: {rec.role} (Level {rec.level})",
f" Elapsed: {elapsed}s",
f" Started: {rec.started.strftime('%Y-%m-%d %H:%M:%S')}",
f" Task: {rec.task}",
]
if rec.parent_id:
lines.append(f" Parent: {rec.parent_id[:8]}")
if rec.result is not None:
lines.append(f" Result: {rec.result[:300]}")
return "\n".join(lines)
async def agent_list(status: str | None = None, limit: int = 10) -> str:
"""List background agents for the current user."""
from persona import get_user
user = get_user() or "unknown"
limit = min(max(int(limit), 1), 50)
records = agent_manager.list_agents(user, status=status, limit=limit)
if not records:
suffix = f" (filter: status={status})" if status else ""
return f"No agents found.{suffix}"
now = datetime.now()
lines = []
for rec in records:
end = rec.finished or now
elapsed = int((end - rec.started).total_seconds())
preview = rec.task[:60].replace("\n", " ")
result_hint = f"{rec.result[:50]}" if rec.result else ""
lines.append(
f"[{rec.agent_id[:8]}] {rec.status:<10s} L{rec.level} "
f"{rec.role:<12s} {elapsed:>5}s {preview}{result_hint}"
)
header = f"{len(records)} agent(s)" + (f" (status={status})" if status else "") + ":"
return header + "\n" + "\n".join(lines)
async def agent_cancel(agent_id: str) -> str:
"""Cancel a running background agent."""
from persona import get_user
user = get_user() or "unknown"
return await agent_manager.cancel_agent(agent_id, user)
# ── Declarations ──────────────────────────────────────────────────────────────
DECLARATIONS = [
types.FunctionDeclaration(
name="spawn_agent",
description=(
"Spawn a sub-agent to complete a task. "
"In synchronous mode (default): blocks until the sub-agent finishes and returns its response. "
"In background mode (background=True): fires the agent asynchronously and returns an agent_id "
"immediately — use agent_status() to check progress or set notify=True for a completion alert. "
"The sub-agent uses the model and tool set assigned to the given role. "
"Use for processing pipelines, parallel analysis, or delegating specialized work "
"(research, coding, data migration, etc.)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"task": types.Schema(
type=types.Type.STRING,
description="The complete task description for the sub-agent.",
),
"role": types.Schema(
type=types.Type.STRING,
description=(
"Role determining the model and tools. "
"E.g. 'research' for web lookups, 'coder' for code tasks, "
"'distill' for summarization. Defaults to 'chat'."
),
),
"tier": types.Schema(
type=types.Type.INTEGER,
description=(
"Context tier: 1 = minimal (fast, identity only), "
"2 = standard (+ memory), 3 = + last 2 session logs. "
"Use 1 for pure processing tasks."
),
),
"timeout": types.Schema(
type=types.Type.INTEGER,
description="Max seconds to wait (default 120). Applies in both sync and background mode.",
),
"max_rounds": types.Schema(
type=types.Type.INTEGER,
description="Override max tool-loop iterations for this call.",
),
"allow_tools": types.Schema(
type=types.Type.ARRAY,
items=types.Schema(type=types.Type.STRING),
description=(
"Restrict the sub-agent to only these tools. "
"Intersected with the role's tool set — cannot grant more than the role allows. "
"Example: ['web_search', 'web_read'] for a pure research agent."
),
),
"deny_tools": types.Schema(
type=types.Type.ARRAY,
items=types.Schema(type=types.Type.STRING),
description=(
"Block these tools from the sub-agent regardless of role config. "
"Example: ['shell_exec', 'file_write', 'cortex_restart']."
),
),
"background": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Run asynchronously in the background (default: false). "
"When true, returns an agent_id immediately instead of blocking for the result. "
"Use agent_status(agent_id) to check progress. "
"Best for tasks that take more than ~30 seconds."
),
),
"notify": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Send a push/Talk notification when the background agent completes (default: false). "
"Only meaningful when background=true."
),
),
},
required=["task"],
),
),
types.FunctionDeclaration(
name="agent_status",
description=(
"Get the current status of a background agent by ID. "
"Returns status (running/done/failed/cancelled/timeout), role, elapsed time, "
"task description, and result preview."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"agent_id": types.Schema(
type=types.Type.STRING,
description="The agent ID returned by spawn_agent(background=True) or aider_run(background=True).",
),
},
required=["agent_id"],
),
),
types.FunctionDeclaration(
name="agent_list",
description=(
"List background agents for the current user. "
"Returns recent agents with ID, status, role, level, elapsed time, and task preview. "
"Use to survey what's running or recently completed."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"status": types.Schema(
type=types.Type.STRING,
description="Filter by status: 'running', 'done', 'failed', 'cancelled', 'timeout'. Omit for all.",
),
"limit": types.Schema(
type=types.Type.INTEGER,
description="Max agents to return (default 10, max 50).",
),
},
),
),
types.FunctionDeclaration(
name="agent_cancel",
description=(
"Cancel a running background agent. ADMIN ONLY. Requires confirmation. "
"Use agent_list() to find the agent ID first."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"agent_id": types.Schema(
type=types.Type.STRING,
description="The agent ID to cancel.",
),
},
required=["agent_id"],
),
),
]

406
cortex/tools/aider.py Normal file
View File

@@ -0,0 +1,406 @@
"""
Aider coding agent tool — invokes Aider AI pair programming as a subprocess.
Aider handles repo-map generation, file editing, git commits, and linting automatically.
It works with any OpenAI-compatible model — point it at DeepSeek, Ollama, OpenRouter, etc.
via AIDER_MODEL / AIDER_OPENAI_API_BASE env vars or the project's .aider.conf.yml.
Credentials are pulled automatically from the Cortex model registry:
- Named cloud providers (OpenRouter, OpenAI, Groq, Anthropic, …) → --api-key slug=key
- Generic OpenAI-compatible hosts (Open WebUI, Ollama, local) → --openai-api-base + key
- Anthropic from providers.anthropic.credentials → --api-key anthropic=key
background=True runs the subprocess asynchronously and returns an agent_id immediately.
"""
import asyncio
import logging
import os
from pathlib import Path
from google.genai import types
import agent_manager
logger = logging.getLogger(__name__)
_CORTEX_DIR = Path(__file__).parent # .../Cortex_and_Inara_dev/cortex/
_PROJECT_ROOT = _CORTEX_DIR.parent # .../Cortex_and_Inara_dev/
# Known project aliases — expand before passing to subprocess
_PROJECT_ALIASES: dict[str, str] = {
"cortex": str(_PROJECT_ROOT),
"aether_api": "~/OSIT_dev/aether_api_fastapi",
"aether_frontend": "~/OSIT_dev/aether_app_sveltekit",
"aether_container": "~/OSIT_dev/aether_container_env",
}
_MAX_OUTPUT_CHARS = 12_000
# Maps URL fragments → Aider --api-key provider slug.
# Order matters: more specific patterns first.
_CLOUD_PROVIDER_URL_MAP: list[tuple[str, str]] = [
("openrouter.ai", "openrouter"),
("api.openai.com", "openai"),
("groq.com", "groq"),
("api.together.xyz", "togetherai"),
("fireworks.ai", "fireworks"),
("api.x.ai", "xai"),
("api.deepseek.com", "deepseek"),
("api.mistral.ai", "mistral"),
]
def _provider_slug(api_url: str) -> str | None:
"""Return the Aider --api-key provider slug for a known cloud URL, None for generic."""
url_lower = api_url.lower()
for fragment, slug in _CLOUD_PROVIDER_URL_MAP:
if fragment in url_lower:
return slug
return None
def _host_flags(host: dict, model: str | None) -> tuple[list[str], str | None]:
"""Build Aider credential flags for a specific host entry.
Returns (extra_args, adjusted_model). For generic (local) endpoints the model
name may be prefixed with 'openai/' so Aider routes through the OpenAI client.
"""
api_url = (host.get("api_url") or "").rstrip("/")
api_key = host.get("api_key") or "none"
host_type = host.get("host_type", "openai")
slug = _provider_slug(api_url)
if slug:
# Named cloud provider — Aider maps --api-key slug=key → SLUG_API_KEY env var
flags = ["--api-key", f"{slug}={api_key}"] if api_key and api_key != "none" else []
return flags, model
# Generic OpenAI-compatible (local Open WebUI, Ollama, custom)
base_url = api_url
if host_type == "openwebui":
# Open WebUI serves the chat endpoint at /api/chat/completions
base_url = base_url + "/api"
flags = ["--openai-api-base", base_url, "--openai-api-key", api_key]
# Prefix model with 'openai/' for generic endpoints when no provider prefix is set
adj_model = model
if model and "/" not in model:
adj_model = f"openai/{model}"
return flags, adj_model
def _resolve_credentials(
registry: dict,
model: str | None,
host_label: str | None,
) -> tuple[list[str], str | None]:
"""Determine Aider credential flags and (possibly adjusted) model name.
Resolution order:
1. Anthropic model hint (claude-* / anthropic/*) → Anthropic API key
2. Explicit host_label → that host's credentials
3. Model prefix hint (openrouter/*, groq/*, …) → matching host
4. Default priority: OpenRouter → Anthropic → any keyed cloud host → local host
Returns (extra_args, adjusted_model).
"""
hosts = registry.get("hosts", [])
# Extract Anthropic key from providers.anthropic.credentials (not a host entry)
anthropic_key = None
for cred in registry.get("providers", {}).get("anthropic", {}).get("credentials", []):
if cred.get("api_key"):
anthropic_key = cred["api_key"]
break
# ── 1. Anthropic model hint ────────────────────────────────────────────────
if model and any(h in model.lower() for h in ("claude-", "anthropic/")):
if anthropic_key:
logger.debug("aider: Anthropic model detected — using Anthropic API key")
return ["--api-key", f"anthropic={anthropic_key}"], model
# ── 2. Explicit host_label override ───────────────────────────────────────
if host_label:
ll = host_label.lower()
host = next((h for h in hosts if ll in h.get("label", "").lower()), None)
if host:
logger.debug("aider: using explicitly requested host '%s'", host.get("label"))
return _host_flags(host, model)
# ── 3. Model prefix hints ─────────────────────────────────────────────────
if model:
ml = model.lower()
for fragment, slug in _CLOUD_PROVIDER_URL_MAP:
if ml.startswith(slug + "/") or ml.startswith(fragment):
host = next(
(h for h in hosts if fragment in h.get("api_url", "").lower()), None
)
if host:
logger.debug("aider: model prefix '%s' → host '%s'", slug, host.get("label"))
return _host_flags(host, model)
# ── 4. Default priority ───────────────────────────────────────────────────
# OpenRouter first (most model coverage)
or_host = next((h for h in hosts if "openrouter.ai" in h.get("api_url", "")), None)
if or_host and or_host.get("api_key"):
logger.debug("aider: defaulting to OpenRouter")
return _host_flags(or_host, model)
# Anthropic API key (no model hint but it's configured)
if anthropic_key:
logger.debug("aider: defaulting to Anthropic API key")
return ["--api-key", f"anthropic={anthropic_key}"], model
# Any other keyed cloud host
for host in hosts:
slug = _provider_slug(host.get("api_url", ""))
if slug and host.get("api_key"):
logger.debug("aider: using keyed cloud host '%s'", host.get("label"))
return _host_flags(host, model)
# Generic / local host (no key or unknown provider)
for host in hosts:
flags, adj_model = _host_flags(host, model)
if flags:
logger.debug("aider: using local host '%s'", host.get("label"))
return flags, adj_model
logger.debug("aider: no credentials found in registry — relying on env vars / .aider.conf.yml")
return [], model
async def aider_run(
project: str,
task: str,
files: list[str] | None = None,
model: str | None = None,
host_label: str | None = None,
auto_commit: bool = True,
timeout: int = 300,
background: bool = False,
notify: bool = False,
) -> str:
"""Run Aider with a single task in a project directory, then exit.
Credentials are resolved automatically from the Cortex model registry. Use
host_label to pick a specific configured host (e.g. 'OpenRouter', 'Local').
When background=True, fires the subprocess asynchronously and returns an agent_id
immediately. Use agent_status(agent_id) to check progress; set notify=True to
receive a push/Talk notification on completion.
"""
resolved = _PROJECT_ALIASES.get(project, project)
cwd = Path(os.path.expanduser(resolved))
if not cwd.is_dir():
return f"Error: project directory '{resolved}' does not exist."
timeout = min(max(int(timeout), 10), 600)
# Resolve credentials before building the command (model name may be adjusted)
user = "scott"
extra_cred_flags: list[str] = []
try:
import model_registry
from persona import get_user
user = get_user() or "scott"
registry = model_registry.get_registry(user)
extra_cred_flags, model = _resolve_credentials(registry, model, host_label)
except Exception as e:
logger.debug("aider: credential resolution failed (%s) — relying on env", e)
cmd: list[str] = [
"aider",
"--message", task,
"--yes-always",
"--no-pretty",
"--no-stream",
"--no-check-update",
"--no-detect-urls",
"--auto-commits" if auto_commit else "--no-auto-commits",
]
cmd += extra_cred_flags
if model:
cmd += ["--model", model]
for f in (files or []):
cmd += ["--file", f]
logger.info(
"aider_run: project=%s model=%s host_label=%s auto_commit=%s background=%s task=%.120s",
project, model, host_label, auto_commit, background, task,
)
async def _run() -> str:
proc = await asyncio.create_subprocess_exec(
*cmd,
cwd=str(cwd),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=float(timeout))
out = stdout.decode(errors="replace").strip()
err = stderr.decode(errors="replace").strip()
parts = []
if out:
parts.append(out)
if err:
parts.append(f"[stderr]\n{err}")
combined = "\n".join(parts) if parts else "(no output)"
if len(combined) > _MAX_OUTPUT_CHARS:
half = _MAX_OUTPUT_CHARS // 2
combined = (
combined[:half]
+ f"\n\n[... {len(combined) - _MAX_OUTPUT_CHARS} chars trimmed ...]\n\n"
+ combined[-half:]
)
if proc.returncode not in (0, 1):
return f"[exit {proc.returncode}]\n{combined}"
return combined
if background:
rec = await agent_manager.register(
user=user,
role="aider",
task=task,
level=2,
notify=notify,
)
async def _bg_task() -> None:
try:
result = await _run()
await agent_manager.finish(rec.agent_id, result, "done")
logger.info("aider_run [bg]: done %s", rec.agent_id[:8])
except asyncio.CancelledError:
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
raise
except asyncio.TimeoutError:
msg = f"Aider timed out after {timeout}s"
logger.warning("aider_run [bg]: timeout %s", rec.agent_id[:8])
await agent_manager.finish(rec.agent_id, msg, "timeout")
except FileNotFoundError:
msg = "Error: 'aider' not found in PATH — run: pip install aider-chat"
await agent_manager.finish(rec.agent_id, msg, "failed")
except Exception as e:
logger.error("aider_run [bg]: failed %s: %s", rec.agent_id[:8], e)
await agent_manager.finish(rec.agent_id, str(e), "failed")
bg = asyncio.create_task(_bg_task())
agent_manager.set_task_ref(rec.agent_id, bg)
return (
f"Aider task started in background. ID: {rec.agent_id}\n"
f"Use agent_status('{rec.agent_id}') to monitor progress."
)
# Synchronous path
try:
return await _run()
except asyncio.TimeoutError:
return f"Error: aider timed out after {timeout}s"
except FileNotFoundError:
return "Error: 'aider' not found in PATH — run: pip install aider-chat"
except Exception as e:
logger.error("aider_run error: %s", e)
return f"Error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="aider_run",
description=(
"Run the Aider AI coding agent on a project with a single task, then exit. "
"Aider maps the repo, edits files, runs lint checks, and optionally commits. "
"Credentials are resolved automatically from the Cortex model registry — "
"OpenRouter, local Open WebUI/Ollama, Anthropic API, and other configured hosts "
"are all supported. Use host_label to pick a specific host. "
"Set background=True for long tasks — returns an agent_id immediately and sends "
"a notification when done. ADMIN ONLY. Requires confirmation."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"project": types.Schema(
type=types.Type.STRING,
description=(
"Project alias or absolute path. Known aliases: "
"'cortex' (this project), 'aether_api', 'aether_frontend', "
"'aether_container'. Or provide an absolute path."
),
),
"task": types.Schema(
type=types.Type.STRING,
description=(
"Full task description sent to Aider as --message. "
"Be specific — include file names, what to change, and why."
),
),
"files": types.Schema(
type=types.Type.ARRAY,
items=types.Schema(type=types.Type.STRING),
description=(
"Optional files to add explicitly to the editing context "
"(paths relative to project root). Aider builds a repo map "
"automatically — these get priority."
),
),
"model": types.Schema(
type=types.Type.STRING,
description=(
"Optional model override. Format depends on the provider: "
"'openrouter/anthropic/claude-3-5-haiku-20241022' (OpenRouter), "
"'claude-3-5-sonnet-20241022' (Anthropic direct), "
"'gemma-4-27b-it' or 'openai/gemma-4-27b-it' (local Open WebUI), "
"'deepseek/deepseek-chat' (DeepSeek via OpenRouter). "
"Defaults to the project's .aider.conf.yml model or AIDER_MODEL env var."
),
),
"host_label": types.Schema(
type=types.Type.STRING,
description=(
"Pick a specific configured host by label (partial match, case-insensitive). "
"Examples: 'OpenRouter', 'Local', 'scott-lt-i7-rtx'. "
"Overrides automatic credential resolution. "
"Omit to let credentials be chosen automatically."
),
),
"auto_commit": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Auto-commit changes after edits (default: true). "
"Set to false to review diffs before committing manually."
),
),
"timeout": types.Schema(
type=types.Type.INTEGER,
description="Max seconds to wait for Aider to finish (default 300, max 600).",
),
"background": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Run asynchronously in the background (default: false). "
"Returns an agent_id immediately; use agent_status(agent_id) to monitor. "
"Recommended for tasks expected to take more than ~60 seconds."
),
),
"notify": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Send a push/Talk notification when the background task completes "
"(default: false). Only applies when background=true."
),
),
},
required=["project", "task"],
),
)
]

268
cortex/tools/cron.py Normal file
View File

@@ -0,0 +1,268 @@
"""
Cron job management tools for Inara.
Jobs are stored in inara/CRONS.json and registered with the live APScheduler
instance so they survive restarts and take effect immediately without a restart.
Tools:
cron_list — show all scheduled jobs
cron_add — create a job and register it immediately
cron_remove — delete a job and unschedule it
cron_toggle — pause or resume a job without deleting it
reminders_clear — erase inara/REMINDERS.md (dismiss all pending reminders)
"""
import asyncio
import secrets
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path, get_user, get_persona
from cron_runner import load_crons, save_crons, parse_schedule
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _short_id() -> str:
return "c_" + secrets.token_urlsafe(6)
# ---------------------------------------------------------------------------
# Sync implementations
# ---------------------------------------------------------------------------
def _cron_list() -> str:
crons = load_crons(get_user(), get_persona())
if not crons:
return "No crons scheduled."
lines = [f"Crons ({len(crons)}):\n"]
for c in crons:
status = "enabled" if c.get("enabled", True) else "PAUSED "
last = c.get("last_run")
last_str = last[:10] if last else "never"
lines.append(
f" {c['id']} [{status}] {c['schedule']:<18} "
f"{c['type']:<7} {c['label']} (last: {last_str})"
)
return "\n".join(lines)
def _cron_add(label: str, schedule: str, job_type: str, payload: str) -> str:
# Validate schedule first — raises ValueError with a clear message on bad input
try:
sched_kwargs = parse_schedule(schedule)
except ValueError as e:
return f"Bad schedule: {e}"
_VALID_TYPES = ("remind", "note", "message", "brief", "task")
if job_type not in _VALID_TYPES:
return f"Bad type: must be one of {', '.join(_VALID_TYPES)}."
current_user = get_user()
current_persona = get_persona()
crons = load_crons(current_user, current_persona)
job = {
"id": _short_id(),
"user": current_user,
"persona": current_persona,
"label": label,
"schedule": schedule,
"type": job_type,
"payload": payload,
"enabled": True,
"created_at": _now(),
"last_run": None,
}
crons.append(job)
save_crons(crons, current_user, current_persona)
# Register with the live scheduler
_scheduler_add(job, sched_kwargs)
return f"Created and scheduled: {job['id']} {schedule} [{job_type}] {label} (user: {current_user}, persona: {current_persona})"
def _cron_remove(cron_id: str) -> str:
user = get_user()
persona = get_persona()
crons = load_crons(user, persona)
before = len(crons)
crons = [c for c in crons if c["id"] != cron_id]
if len(crons) == before:
return f"Not found: {cron_id}"
save_crons(crons, user, persona)
_scheduler_remove(f"{user}:{persona}:{cron_id}")
return f"Removed: {cron_id}"
def _cron_toggle(cron_id: str) -> str:
user = get_user()
persona = get_persona()
crons = load_crons(user, persona)
for c in crons:
if c["id"] == cron_id:
c["enabled"] = not c.get("enabled", True)
save_crons(crons, user, persona)
action = "resumed" if c["enabled"] else "paused"
sched_id = f"{user}:{persona}:{cron_id}"
_scheduler_resume(sched_id) if c["enabled"] else _scheduler_pause(sched_id)
return f"{action.capitalize()}: {cron_id} {c['label']}"
return f"Not found: {cron_id}"
def _reminders_clear() -> str:
p = persona_path() / "REMINDERS.md"
p.write_text("")
return "Reminders cleared."
# ---------------------------------------------------------------------------
# Scheduler bridge — thin wrappers so the tool layer never touches APScheduler
# directly, keeping it swappable
# ---------------------------------------------------------------------------
def _scheduler_add(job: dict, sched_kwargs: dict) -> None:
try:
import scheduler as sched_module
from cron_runner import run_job
s = sched_module.get_scheduler()
if s and s.running:
sched_id = f"{job.get('user', 'scott')}:{job.get('persona', 'inara')}:{job['id']}"
s.add_job(
lambda j=job: asyncio.ensure_future(run_job(j)),
"cron",
id=sched_id,
replace_existing=True,
**sched_kwargs,
)
except Exception as e:
import logging
logging.getLogger(__name__).warning("scheduler_add failed: %s", e)
def _scheduler_remove(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.remove_job(job_id)
except Exception:
pass
def _scheduler_pause(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.pause_job(job_id)
except Exception:
pass
def _scheduler_resume(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.resume_job(job_id)
except Exception:
pass
# ---------------------------------------------------------------------------
# Async wrappers
# ---------------------------------------------------------------------------
async def cron_list() -> str:
return await asyncio.to_thread(_cron_list)
async def cron_add(label: str, schedule: str, job_type: str, payload: str) -> str:
return await asyncio.to_thread(_cron_add, label, schedule, job_type, payload)
async def cron_remove(cron_id: str) -> str:
return await asyncio.to_thread(_cron_remove, cron_id)
async def cron_toggle(cron_id: str) -> str:
return await asyncio.to_thread(_cron_toggle, cron_id)
async def reminders_clear() -> str:
return await asyncio.to_thread(_reminders_clear)
DECLARATIONS = [
types.FunctionDeclaration(
name="cron_list",
description=(
"List all scheduled cron jobs — their ID, label, schedule, type, and last run time. "
"Use this to see what's scheduled before adding or removing jobs."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="cron_add",
description=(
"Create a new scheduled cron job and register it immediately (no restart needed). "
"Job types: "
"'remind' — appends to REMINDERS.md, auto-surfaced in chat context at tier 2+; "
"'note' — appends to SCRATCH.md, read on demand; "
"'message' — sends payload text directly to the user's notification channel; "
"'brief' — calls the LLM (no tools) with payload as the prompt, sends the response; "
"'task' — runs the full orchestrator tool loop with payload as the request, sends "
"Claude's response to the notification channel (use for agentic scheduled work: "
"research, checks, file updates, summaries that need tool access). "
"Schedule formats: 'hourly' | 'daily' | 'daily:HH:MM' | 'weekly:DOW' | 'weekly:DOW:HH:MM' | "
"'monthly' | 'monthly:DD' | 'monthly:DD:HH:MM' | 'yearly:MM:DD' | 'yearly:MM:DD:HH:MM'. "
"Examples: schedule='weekly:mon:08:00' for Monday briefings; "
"schedule='monthly:1:09:00' for a first-of-month review; "
"schedule='yearly:03:15' for a March 15 birthday reminder."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"label": types.Schema(type=types.Type.STRING, description="Short human-readable name for this job (e.g. 'Monday task summary')"),
"schedule": types.Schema(type=types.Type.STRING, description="When to run: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM | monthly | monthly:DD | monthly:DD:HH:MM | yearly:MM:DD | yearly:MM:DD:HH:MM"),
"job_type": types.Schema(type=types.Type.STRING, description="remind | note | message | brief | task"),
"payload": types.Schema(type=types.Type.STRING, description="The text/prompt to use when the job fires"),
},
required=["label", "schedule", "job_type", "payload"],
),
),
types.FunctionDeclaration(
name="cron_remove",
description=(
"Permanently delete a scheduled cron job. Use cron_list first to get the ID. "
"To temporarily disable without deleting, use cron_toggle instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"cron_id": types.Schema(type=types.Type.STRING, description="Job ID (e.g. c_abc123) — get from cron_list"),
},
required=["cron_id"],
),
),
types.FunctionDeclaration(
name="cron_toggle",
description=(
"Pause a running cron job, or resume a paused one. "
"The job stays in the list and can be re-enabled later. "
"Use cron_list to see current enabled/paused state."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"cron_id": types.Schema(type=types.Type.STRING, description="Job ID (e.g. c_abc123) — get from cron_list"),
},
required=["cron_id"],
),
),
]

804
cortex/tools/files.py Normal file
View File

@@ -0,0 +1,804 @@
"""
File read/write/search tools — two access scopes.
Project scope (no admin required):
project_file_read — read a file with optional line-range (offset)
project_file_list — list a directory with sizes + timestamps
file_stat — size, modified time, line count for a path
file_grep — regex search with context lines; up to 50 matches
file_syntax_check — py_compile (.py) or json.loads (.json) check
System scope (admin-only):
file_read — read a file from ~/agents_sync/, ~/OSIT_dev/, etc.
file_list — list a directory (same roots)
file_write — write/append (~/agents_sync/ + Cortex home/)
Session tools (user-level, persona-isolated):
session_read — read a session log by date
session_search — keyword search across session logs
All project-scope tools are restricted to the Cortex project root:
~/agents_sync/projects/Cortex_and_Inara_dev/
"""
import asyncio
import json
import logging
import re
import subprocess
from datetime import datetime
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
# ── Access roots ──────────────────────────────────────────────────────────────
# Project root: two levels up from cortex/tools/files.py → Cortex_and_Inara_dev/
_PROJECT_ROOT: Path = Path(__file__).parent.parent.parent.resolve()
# System-wide read roots
def _build_allowed_roots() -> list[Path]:
roots = [
Path.home() / "agents_sync",
Path.home() / "OSIT_dev",
Path.home() / "DgrZone_Nextcloud",
Path.home() / "OSIT_Nextcloud",
]
try:
from config import settings
roots.append(settings.home_root())
except Exception:
pass
return roots
_ALLOWED_ROOTS: list[Path] = _build_allowed_roots()
# Write is tighter
_WRITE_ROOTS: list[Path] = [Path.home() / "agents_sync"]
# Size limits
_MAX_BYTES = 50_000
_MAX_LINES = 500
_MAX_GREP_MATCHES = 50
def _is_project_allowed(resolved: Path) -> bool:
try:
resolved.relative_to(_PROJECT_ROOT)
return True
except ValueError:
return False
def _is_allowed(resolved: Path) -> bool:
for root in _ALLOWED_ROOTS:
try:
resolved.relative_to(root)
return True
except ValueError:
continue
return False
def _is_write_allowed(resolved: Path) -> bool:
for root in _WRITE_ROOTS:
try:
resolved.relative_to(root)
return True
except ValueError:
continue
try:
from config import settings
resolved.relative_to(settings.home_root())
return True
except (ValueError, Exception):
pass
return False
# ── Shared implementations ────────────────────────────────────────────────────
def _read_impl(path_str: str, offset: int | None, max_lines: int | None, is_allowed_fn) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not is_allowed_fn(resolved):
return f"Access denied: {resolved}"
if not resolved.exists():
return f"File not found: {resolved}"
if not resolved.is_file():
try:
entries = sorted(resolved.iterdir())
names = [e.name + ("/" if e.is_dir() else "") for e in entries[:100]]
return f"Directory listing for {resolved}:\n" + "\n".join(names)
except Exception as e:
return f"Cannot list directory: {e}"
try:
raw = resolved.read_bytes()
except Exception as e:
return f"Read error: {e}"
try:
text = raw.decode("utf-8")
except UnicodeDecodeError:
return f"Binary file (not readable as text): {resolved} [{len(raw)} bytes]"
all_lines = text.splitlines()
total = len(all_lines)
# offset is 1-based; default = start of file
start = max(0, (offset or 1) - 1)
working = all_lines[start:]
limit = min(max_lines or _MAX_LINES, _MAX_LINES)
truncated = False
if len(working) > limit:
working = working[:limit]
truncated = True
result = "\n".join(working)
if len(result) > _MAX_BYTES:
result = result[:_MAX_BYTES]
truncated = True
end_line = start + len(working)
header = f"[Lines {start + 1}{end_line} of {total}]\n" if (start > 0 or truncated) else ""
trailer = f"\n\n… [truncated — file has {total} lines; use offset={end_line + 1} to read more]" if truncated else ""
return header + result + trailer
def _list_impl(path_str: str, is_allowed_fn) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not is_allowed_fn(resolved):
return f"Access denied: {resolved}"
if not resolved.exists():
return f"Path not found: {resolved}"
if resolved.is_file():
return f"{resolved} is a file. Use file_read / project_file_read to read it."
try:
entries = sorted(resolved.iterdir(), key=lambda e: (e.is_file(), e.name.lower()))
lines = []
for e in entries[:200]:
if e.is_dir():
suffix = "/"
else:
try:
st = e.stat()
mtime = datetime.fromtimestamp(st.st_mtime).strftime("%Y-%m-%d %H:%M")
suffix = f" ({st.st_size:,} B, {mtime})"
except Exception:
suffix = ""
lines.append(f"{e.name}{suffix}")
result = "\n".join(lines)
if len(entries) > 200:
result += f"\n… ({len(entries) - 200} more not shown)"
return f"Contents of {resolved}:\n\n{result}"
except Exception as e:
return f"Cannot list directory: {e}"
# ── Project-scoped tools ──────────────────────────────────────────────────────
async def project_file_read(path: str, offset: int | None = None, max_lines: int | None = None) -> str:
"""Read a file within the Cortex project directory, with optional line range."""
return await asyncio.to_thread(_read_impl, path, offset, max_lines, _is_project_allowed)
async def project_file_list(path: str) -> str:
"""List directory contents within the Cortex project directory, with sizes and timestamps."""
return await asyncio.to_thread(_list_impl, path, _is_project_allowed)
async def file_stat(path: str) -> str:
"""Return metadata for a file or directory: type, size, modified time, line count."""
return await asyncio.to_thread(_sync_file_stat, path)
def _sync_file_stat(path_str: str) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
if not resolved.exists():
return f"Path not found: {resolved}"
try:
st = resolved.stat()
except Exception as e:
return f"Cannot stat: {e}"
modified = datetime.fromtimestamp(st.st_mtime).strftime("%Y-%m-%d %H:%M:%S")
lines = [
f"Path: {resolved}",
f"Type: {'directory' if resolved.is_dir() else 'file'}",
f"Size: {st.st_size:,} bytes",
f"Modified: {modified}",
]
if resolved.is_file():
try:
raw = resolved.read_bytes()
if b'\x00' not in raw[:1024]:
lines.append(f"Lines: {len(raw.decode('utf-8', errors='replace').splitlines())}")
except Exception:
pass
elif resolved.is_dir():
try:
entries = list(resolved.iterdir())
n_files = sum(1 for e in entries if e.is_file())
n_dirs = sum(1 for e in entries if e.is_dir())
lines.append(f"Contents: {n_files} file(s), {n_dirs} subdirector{'y' if n_dirs == 1 else 'ies'}")
except Exception:
pass
return "\n".join(lines)
async def file_grep(path: str, pattern: str, context_lines: int = 2, recursive: bool = True) -> str:
"""Search for a regex pattern in a file or directory, returning matching lines with context."""
return await asyncio.to_thread(_sync_file_grep, path, pattern, context_lines, recursive)
def _sync_file_grep(path_str: str, pattern: str, context_lines: int, recursive: bool) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
if not resolved.exists():
return f"Path not found: {resolved}"
try:
regex = re.compile(pattern, re.IGNORECASE)
except re.error as e:
return f"Invalid regex pattern: {e}"
ctx = max(0, min(context_lines, 5))
if resolved.is_file():
files_to_search = [resolved]
elif recursive:
files_to_search = sorted(f for f in resolved.rglob("*") if f.is_file())
else:
files_to_search = sorted(f for f in resolved.iterdir() if f.is_file())
total_matches = 0
sections: list[str] = []
capped = False
for fp in files_to_search:
if total_matches >= _MAX_GREP_MATCHES:
capped = True
break
try:
raw = fp.read_bytes()
except OSError:
continue
if b'\x00' in raw[:1024]:
continue # skip binary
try:
text = raw.decode("utf-8", errors="replace")
except Exception:
continue
file_lines = text.splitlines()
match_indices = [i for i, line in enumerate(file_lines) if regex.search(line)]
if not match_indices:
continue
total_matches += len(match_indices)
try:
label = str(fp.relative_to(_PROJECT_ROOT))
except ValueError:
label = str(fp)
file_output = [f"── {label} ──"]
printed: set[int] = set()
for mi in match_indices:
start = max(0, mi - ctx)
end = min(len(file_lines), mi + ctx + 1)
if printed and start > max(printed) + 1:
file_output.append(" ···")
for j in range(start, end):
if j not in printed:
marker = "" if j == mi else " "
file_output.append(f" {j + 1:4d}{marker} {file_lines[j]}")
printed.add(j)
sections.append("\n".join(file_output))
if not sections:
return f"No matches for '{pattern}' in {resolved}"
cap_note = f" (capped at {_MAX_GREP_MATCHES})" if capped else ""
header = f"grep '{pattern}'{total_matches} match(es){cap_note}:"
return header + "\n\n" + "\n\n".join(sections)
async def file_diff(path_a: str, path_b: str) -> str:
"""Compare two files and return a unified diff."""
return await asyncio.to_thread(_sync_file_diff, path_a, path_b)
def _sync_file_diff(path_a: str, path_b: str) -> str:
try:
resolved_a = Path(path_a).expanduser().resolve()
resolved_b = Path(path_b).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
for resolved in (resolved_a, resolved_b):
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}"
if not resolved.exists():
return f"File not found: {resolved}"
if not resolved.is_file():
return f"Not a file: {resolved}"
try:
result = subprocess.run(
["diff", "-u", str(resolved_a), str(resolved_b)],
capture_output=True, text=True, timeout=15,
)
if result.returncode == 0:
return f"Files are identical: {resolved_a.name} vs {resolved_b.name}"
output = result.stdout
if not output:
return f"diff returned no output (exit {result.returncode}): {result.stderr}"
if len(output) > _MAX_BYTES:
output = output[:_MAX_BYTES] + "\n… [truncated]"
return output
except subprocess.TimeoutExpired:
return "Timeout running diff"
except Exception as e:
return f"Error: {e}"
async def file_syntax_check(path: str) -> str:
"""Check syntax of a Python (.py) or JSON (.json) file."""
return await asyncio.to_thread(_sync_file_syntax_check, path)
def _sync_file_syntax_check(path_str: str) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
if not resolved.exists():
return f"File not found: {resolved}"
if not resolved.is_file():
return f"Not a file: {resolved}"
suffix = resolved.suffix.lower()
if suffix == ".py":
try:
result = subprocess.run(
["python3", "-m", "py_compile", str(resolved)],
capture_output=True, text=True, timeout=15,
)
if result.returncode == 0:
return f"OK — {resolved.name}: syntax valid"
err = (result.stderr or result.stdout).strip()
return f"Syntax error in {resolved.name}:\n{err}"
except subprocess.TimeoutExpired:
return f"Timeout running py_compile on {resolved.name}"
except Exception as e:
return f"Error: {e}"
elif suffix == ".json":
try:
text = resolved.read_text(encoding="utf-8")
json.loads(text)
return f"OK — {resolved.name}: valid JSON"
except json.JSONDecodeError as e:
return f"JSON error in {resolved.name}: {e}"
except Exception as e:
return f"Error reading {resolved.name}: {e}"
else:
return f"Syntax check not supported for '{suffix}' files. Supported: .py, .json"
# ── System-scoped tools ───────────────────────────────────────────────────────
async def file_read(path: str, offset: int | None = None, max_lines: int | None = None) -> str:
"""Read a local file from the broader system. Allowed: ~/agents_sync/, ~/OSIT_dev/, etc. ADMIN ONLY."""
return await asyncio.to_thread(_read_impl, path, offset, max_lines, _is_allowed)
async def file_list(path: str) -> str:
"""List directory contents from the broader system. ADMIN ONLY."""
return await asyncio.to_thread(_list_impl, path, _is_allowed)
async def file_write(path: str, content: str, mode: str = "overwrite") -> str:
"""Write or append content to a file. Write roots: ~/agents_sync/ and Cortex home/. ADMIN ONLY."""
return await asyncio.to_thread(_sync_file_write, path, content, mode)
def _sync_file_write(path: str, content: str, mode: str) -> str:
try:
resolved = Path(path).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_write_allowed(resolved):
return (
f"Write access denied: {resolved}\n"
f"Allowed write roots: ~/agents_sync/ and the Cortex home/ directory."
)
if mode not in ("overwrite", "append"):
return f"Invalid mode '{mode}' — use 'overwrite' or 'append'."
try:
resolved.parent.mkdir(parents=True, exist_ok=True)
if mode == "append":
with resolved.open("a", encoding="utf-8") as f:
f.write(content)
return f"Appended {len(content)} chars to {resolved}"
else:
resolved.write_text(content, encoding="utf-8")
return f"Wrote {len(content)} chars to {resolved}"
except Exception as e:
logger.error("file_write error for %s: %s", resolved, e)
return f"Write error: {e}"
# ── Session tools ─────────────────────────────────────────────────────────────
_SEARCH_EXCERPT_CHARS = 150
async def session_read(date: str) -> str:
"""Read a full session log by date (YYYY-MM-DD)."""
return await asyncio.to_thread(_sync_session_read, date.strip())
def _sync_session_read(date: str) -> str:
from persona import persona_path
sessions_dir = persona_path() / "sessions"
if not sessions_dir.exists():
return "No session logs found."
target = sessions_dir / f"{date}.md"
if target.exists():
content = target.read_text()
return f"Session log for {date} ({len(content)} chars):\n\n{content}"
available = sorted([f.stem for f in sessions_dir.glob("*.md")], reverse=True)
if not available:
return "No session logs found."
recent = "\n".join(f" {d}" for d in available[:15])
return f"No session log found for '{date}'. Available dates (most recent first):\n{recent}"
async def session_search(query: str, limit: int = 5) -> str:
"""Search past session logs for a keyword or phrase."""
return await asyncio.to_thread(_sync_session_search, query, limit)
def _sync_session_search(query: str, limit: int) -> str:
from persona import persona_path
sessions_dir = persona_path() / "sessions"
if not sessions_dir.exists():
return "No session logs found."
limit = max(1, min(limit, 20))
pattern = re.compile(re.escape(query), re.IGNORECASE)
session_files = sorted(sessions_dir.glob("*.md"), reverse=True)
matches = []
for sf in session_files:
if len(matches) >= limit:
break
try:
text = sf.read_text()
except OSError:
continue
for m in pattern.finditer(text):
if len(matches) >= limit:
break
start = max(0, m.start() - _SEARCH_EXCERPT_CHARS)
end = min(len(text), m.end() + _SEARCH_EXCERPT_CHARS)
excerpt = text[start:end].strip()
if start > 0:
excerpt = "" + excerpt
if end < len(text):
excerpt = excerpt + ""
matches.append(f"[{sf.stem}] {excerpt}")
if not matches:
return f"No matches for '{query}' across {len(session_files)} session logs."
header = f"Session search: '{query}'{len(matches)} match(es) across {len(session_files)} logs\n"
return header + "\n\n".join(matches)
# ── Declarations ──────────────────────────────────────────────────────────────
DECLARATIONS = [
# Project-scoped
types.FunctionDeclaration(
name="project_file_read",
description=(
"Read a file within the Cortex project directory (source code, docs, config, persona files). "
"Supports reading a specific line range via offset — use to page through large files "
"without re-reading from the top. If given a directory path, returns a listing instead. "
"Project root: ~/agents_sync/projects/Cortex_and_Inara_dev/"
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the file",
),
"offset": types.Schema(
type=types.Type.INTEGER,
description="Start reading from this line number (1-based). Omit to read from the top.",
),
"max_lines": types.Schema(
type=types.Type.INTEGER,
description="Maximum lines to return (default 500)",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="project_file_list",
description=(
"List files and subdirectories within the Cortex project directory. "
"Shows file sizes and modified timestamps. "
"Project root: ~/agents_sync/projects/Cortex_and_Inara_dev/"
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the directory",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_stat",
description=(
"Get metadata for a file or directory: type, size, modified timestamp, line count (for text files) "
"or entry counts (for directories). Use before reading to check recency or size. "
"Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the file or directory",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_grep",
description=(
"Search for a regex pattern in a file or directory, returning matching lines with surrounding "
"context. Much more efficient than reading an entire source file — use this to find function "
"definitions, variable names, TODO comments, imports, error strings, etc. "
"Searches recursively by default. Capped at 50 matches. Skips binary files. "
"Case-insensitive. Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="File or directory to search (e.g. ~/agents_sync/projects/Cortex_and_Inara_dev/cortex/)",
),
"pattern": types.Schema(
type=types.Type.STRING,
description="Regex pattern to search for (case-insensitive). Examples: 'def ha_', 'import httpx', 'TODO'",
),
"context_lines": types.Schema(
type=types.Type.INTEGER,
description="Lines of context before/after each match (default 2, max 5)",
),
"recursive": types.Schema(
type=types.Type.BOOLEAN,
description="Search subdirectories recursively (default true)",
),
},
required=["path", "pattern"],
),
),
types.FunctionDeclaration(
name="file_diff",
description=(
"Compare two files and return a unified diff (diff -u). "
"Use for code review, verifying what changed between two versions of a file, "
"or comparing config files side-by-side. "
"Returns 'Files are identical' if there are no differences. "
"Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path_a": types.Schema(
type=types.Type.STRING,
description="Path to the first file (the 'before' or reference file)",
),
"path_b": types.Schema(
type=types.Type.STRING,
description="Path to the second file (the 'after' or comparison file)",
),
},
required=["path_a", "path_b"],
),
),
types.FunctionDeclaration(
name="file_syntax_check",
description=(
"Check the syntax of a Python (.py) or JSON (.json) file without executing it. "
"Returns OK or the error with line number. "
"Use after editing a file before restarting Cortex. "
"Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Path to the .py or .json file to check",
),
},
required=["path"],
),
),
# System-scoped
types.FunctionDeclaration(
name="file_read",
description=(
"Read a local file from the broader system (~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, "
"~/OSIT_Nextcloud/, Cortex home/). Supports offset for reading specific line ranges. "
"For files within the Cortex project, prefer project_file_read instead. "
"ADMIN ONLY."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the file",
),
"offset": types.Schema(
type=types.Type.INTEGER,
description="Start reading from this line number (1-based)",
),
"max_lines": types.Schema(
type=types.Type.INTEGER,
description="Maximum lines to return (default 500)",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_list",
description=(
"List files and subdirectories from the broader system. "
"Shows sizes and modified timestamps. "
"Allowed: ~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, ~/OSIT_Nextcloud/. "
"ADMIN ONLY."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the directory",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_write",
description=(
"Write or append content to a file. "
"Write-allowed paths: ~/agents_sync/ and the Cortex home/ directory. "
"Creates parent directories if needed. "
"ADMIN ONLY. Requires user confirmation before executing."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to write to",
),
"content": types.Schema(
type=types.Type.STRING,
description="Content to write",
),
"mode": types.Schema(
type=types.Type.STRING,
description="'overwrite' (default, replaces file) or 'append' (adds to end)",
),
},
required=["path", "content"],
),
),
types.FunctionDeclaration(
name="session_read",
description=(
"Read a full conversation session log by date (YYYY-MM-DD). "
"Useful for continuity and recalling past decisions. "
"If the date is not found, lists available dates. "
"Only reads this user's own sessions."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"date": types.Schema(
type=types.Type.STRING,
description="Date in YYYY-MM-DD format (e.g. '2026-05-08')",
),
},
required=["date"],
),
),
types.FunctionDeclaration(
name="session_search",
description=(
"Search past conversation session logs for a keyword or phrase. "
"Returns matching excerpts with session dates, newest first. "
"Only searches this user's own sessions."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"query": types.Schema(
type=types.Type.STRING,
description="Keyword or phrase to search for",
),
"limit": types.Schema(
type=types.Type.INTEGER,
description="Max results to return (default 5, max 20)",
),
},
required=["query"],
),
),
]

158
cortex/tools/git.py Normal file
View File

@@ -0,0 +1,158 @@
"""
Git inspection tools — project-scoped, read-only.
git_status — working tree status (staged, unstaged, untracked changes)
git_log — recent commit history with optional path filter
git_diff — diff between commits, branches, or working tree vs HEAD
"""
import asyncio
import logging
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
_PROJECT_ROOT: Path = Path(__file__).parent.parent.parent.resolve()
_MAX_OUTPUT = 50_000
async def _git(*args: str, timeout: int = 15) -> tuple[int, str]:
"""Run a git command in the project root. Returns (returncode, output)."""
proc = await asyncio.create_subprocess_exec(
"git", "-C", str(_PROJECT_ROOT), *args,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
except asyncio.TimeoutError:
proc.kill()
return 1, "git command timed out"
out = (stdout or b"").decode(errors="replace").strip()
err = (stderr or b"").decode(errors="replace").strip()
combined = out if out else err
return proc.returncode, combined
def _cap(text: str) -> str:
if len(text) > _MAX_OUTPUT:
return text[:_MAX_OUTPUT] + "\n… [truncated]"
return text
async def git_status() -> str:
"""Return the current git working tree status."""
rc, out = await _git("status")
if rc != 0:
return f"git status failed: {out}"
return out or "Working tree clean — nothing to report."
async def git_log(n: int = 20, path: str = "", oneline: bool = True) -> str:
"""Return recent git commit history."""
args = ["log"]
if oneline:
args += ["--oneline"]
else:
args += ["--format=%H %as %an%n %s", "--date=short"]
args += [f"-{max(1, min(n, 200))}"]
if path:
args += ["--", path]
rc, out = await _git(*args)
if rc != 0:
return f"git log failed: {out}"
return _cap(out) or "No commits found."
async def git_diff(ref_a: str = "", ref_b: str = "", path: str = "", stat_only: bool = False) -> str:
"""Show a git diff. Defaults to working tree vs HEAD (unstaged changes)."""
args = ["diff"]
if stat_only:
args += ["--stat"]
if ref_a and ref_b:
args += [f"{ref_a}..{ref_b}"]
elif ref_a:
args += [ref_a]
if path:
args += ["--", path]
rc, out = await _git(*args)
# diff exits 1 when there are differences — that's normal
if rc not in (0, 1):
return f"git diff failed: {out}"
return _cap(out) or "No differences found."
# ── Declarations ──────────────────────────────────────────────────────────────
DECLARATIONS = [
types.FunctionDeclaration(
name="git_status",
description=(
"Show the current git working tree status for the Cortex project: "
"staged changes, unstaged modifications, and untracked files. "
"Use to check whether there are uncommitted changes before restarting or deploying."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={},
),
),
types.FunctionDeclaration(
name="git_log",
description=(
"Show recent git commit history for the Cortex project. "
"Returns commit hashes, dates, and messages. "
"Optionally filter to a specific file or directory path."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"n": types.Schema(
type=types.Type.INTEGER,
description="Number of commits to return (default 20, max 200)",
),
"path": types.Schema(
type=types.Type.STRING,
description="Optional file or directory path to filter commits by",
),
"oneline": types.Schema(
type=types.Type.BOOLEAN,
description="Use compact one-line format (default true). Set false for more detail.",
),
},
),
),
types.FunctionDeclaration(
name="git_diff",
description=(
"Show a git diff for the Cortex project. "
"With no arguments: shows unstaged working tree changes vs HEAD. "
"With ref_a only: shows changes between that ref and HEAD. "
"With ref_a and ref_b: shows changes between the two refs (commits, branches, or tags). "
"Use stat_only to get a summary of changed files instead of full patch output."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"ref_a": types.Schema(
type=types.Type.STRING,
description="First ref (commit hash, branch name, or tag). Omit for working tree diff.",
),
"ref_b": types.Schema(
type=types.Type.STRING,
description="Second ref. When provided with ref_a, shows diff between the two.",
),
"path": types.Schema(
type=types.Type.STRING,
description="Optional file or directory path to restrict the diff to",
),
"stat_only": types.Schema(
type=types.Type.BOOLEAN,
description="Return only a file-change summary (--stat) instead of the full diff",
),
},
),
),
]

View File

@@ -0,0 +1,277 @@
"""
Home Assistant tools — read device states and call services.
Credentials are read automatically from the current user's channels.json:
"homeassistant": {"url": "https://ha.example.com", "token": "<long-lived-token>"}
Configure in Settings → Notifications → Home Assistant.
"""
import json
import logging
import httpx
from google.genai import types
from auth_utils import get_user_channels
from persona import get_user
logger = logging.getLogger(__name__)
_TIMEOUT = 10
# Attributes that are internal/noisy and not useful to show
_SKIP_ATTRS = {
"friendly_name", "icon", "entity_picture", "supported_features",
"supported_color_modes", "color_mode", "min_color_temp_kelvin",
"max_color_temp_kelvin", "min_mireds", "max_mireds",
"assumed_state", "attribution",
}
def _get_ha_cfg() -> tuple[str, str]:
"""Return (base_url, token) from the current user's channels.json."""
channels = get_user_channels(get_user())
ha = channels.get("homeassistant") or {}
url = (ha.get("url") or "").rstrip("/")
token = ha.get("token") or ""
if not url or not token:
raise ValueError(
"Home Assistant not configured — add URL and token in Settings → Notifications."
)
return url, token
def _auth(token: str) -> dict:
return {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
def _fmt_state(s: dict) -> str:
"""Format a single HA state dict as a compact readable line."""
entity_id = s.get("entity_id", "")
state = s.get("state", "unknown")
attrs = s.get("attributes", {})
name = attrs.get("friendly_name", entity_id)
label = f"{name} ({entity_id})" if name != entity_id else entity_id
useful = {k: v for k, v in attrs.items() if k not in _SKIP_ATTRS}
extra = ""
if useful:
parts = []
for k, v in list(useful.items())[:6]: # cap at 6 attrs per entity
parts.append(f"{k}: {v}")
extra = " [" + ", ".join(parts) + "]"
return f"{label}: {state}{extra}"
async def ha_get_state(entity_id: str) -> str:
"""Return the current state and attributes of a single Home Assistant entity."""
try:
url, token = _get_ha_cfg()
except ValueError as e:
return str(e)
try:
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
resp = await client.get(f"{url}/api/states/{entity_id}", headers=_auth(token))
if resp.status_code == 404:
return f"Entity not found: {entity_id}"
if resp.status_code != 200:
return f"HA API error {resp.status_code}: {resp.text[:400]}"
s = resp.json()
attrs = s.get("attributes", {})
lines = [
f"**{attrs.get('friendly_name', entity_id)}** (`{entity_id}`)",
f"State: **{s.get('state', 'unknown')}**",
]
changed = (s.get("last_changed") or "")[:19].replace("T", " ")
if changed:
lines.append(f"Last changed: {changed} UTC")
useful = {k: v for k, v in attrs.items() if k not in _SKIP_ATTRS}
if useful:
lines.append("Attributes:")
for k, v in useful.items():
lines.append(f" {k}: {v}")
return "\n".join(lines)
except httpx.HTTPError as e:
return f"Connection error: {e}"
except Exception as e:
logger.warning("ha_get_state error: %s", e)
return f"Error: {e}"
async def ha_get_states(domain: str = "", area: str = "") -> str:
"""List HA entity states, optionally filtered by domain (e.g. 'light') or area name."""
try:
url, token = _get_ha_cfg()
except ValueError as e:
return str(e)
try:
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
resp = await client.get(f"{url}/api/states", headers=_auth(token))
if resp.status_code != 200:
return f"HA API error {resp.status_code}: {resp.text[:400]}"
states = resp.json()
if domain:
states = [s for s in states if s.get("entity_id", "").startswith(f"{domain}.")]
if area:
al = area.lower()
states = [s for s in states
if al in (s.get("attributes", {}).get("friendly_name") or "").lower()]
if not states:
filters = [f"domain={domain}"] * bool(domain) + [f"area={area}"] * bool(area)
return "No entities found" + (f" ({', '.join(filters)})" if filters else "")
lines = [f"{len(states)} entit{'y' if len(states) == 1 else 'ies'}:"]
for s in sorted(states, key=lambda x: x.get("entity_id", "")):
lines.append(_fmt_state(s))
return "\n".join(lines)
except httpx.HTTPError as e:
return f"Connection error: {e}"
except Exception as e:
logger.warning("ha_get_states error: %s", e)
return f"Error: {e}"
async def ha_call_service(
domain: str,
service: str,
entity_id: str = "",
data: str = "",
) -> str:
"""Call a Home Assistant service (turn on/off lights, set thermostat, lock doors, etc.)."""
try:
url, token = _get_ha_cfg()
except ValueError as e:
return str(e)
payload: dict = {}
if entity_id:
payload["entity_id"] = entity_id
if data:
try:
extra = json.loads(data)
if isinstance(extra, dict):
payload.update(extra)
except json.JSONDecodeError:
return f"Invalid JSON in data: {data}"
try:
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
resp = await client.post(
f"{url}/api/services/{domain}/{service}",
headers=_auth(token),
json=payload,
)
if resp.status_code not in (200, 201):
return f"HA API error {resp.status_code}: {resp.text[:400]}"
changed = resp.json()
if not changed:
return f"{domain}.{service} called (no state changes reported)."
lines = [f"{domain}.{service}{len(changed)} entity state(s) updated:"]
for s in changed:
lines.append(f" {s.get('entity_id', '')}: {s.get('state', '')}")
return "\n".join(lines)
except httpx.HTTPError as e:
return f"Connection error: {e}"
except Exception as e:
logger.warning("ha_call_service error: %s", e)
return f"Error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="ha_get_state",
description=(
"Get the current state and attributes of a single Home Assistant entity. "
"Use to check if a light is on, read a thermostat temperature, check a "
"door/window sensor, battery level, HVAC mode, etc. "
"entity_id format: domain.name — e.g. light.living_room, switch.garage, "
"climate.ecobee, binary_sensor.front_door, sensor.outdoor_temp."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entity_id": types.Schema(
type=types.Type.STRING,
description="Full entity ID, e.g. light.living_room or climate.ecobee_main",
),
},
required=["entity_id"],
),
),
types.FunctionDeclaration(
name="ha_get_states",
description=(
"List Home Assistant entity states, optionally filtered by domain or area. "
"Use to survey what devices exist or check multiple entities at once. "
"Domain examples: light, switch, sensor, climate, binary_sensor, lock, cover, "
"media_player, input_boolean. Leave both blank to list everything (can be large)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"domain": types.Schema(
type=types.Type.STRING,
description="Filter to this domain, e.g. 'light' or 'switch' (optional)",
),
"area": types.Schema(
type=types.Type.STRING,
description="Filter by area name substring match on friendly name (optional)",
),
},
),
),
types.FunctionDeclaration(
name="ha_call_service",
description=(
"Call a Home Assistant service to control a device or trigger an automation. "
"Requires user confirmation before executing. Common examples: "
"domain=light service=turn_on entity_id=light.living_room; "
"domain=light service=turn_off entity_id=light.all; "
"domain=switch service=toggle entity_id=switch.garage; "
"domain=climate service=set_temperature data={\"temperature\":72}; "
"domain=lock service=lock entity_id=lock.front_door; "
"domain=script service=turn_on entity_id=script.goodnight."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"domain": types.Schema(
type=types.Type.STRING,
description="Service domain: light, switch, climate, lock, cover, script, automation, etc.",
),
"service": types.Schema(
type=types.Type.STRING,
description="Service name: turn_on, turn_off, toggle, set_temperature, lock, unlock, open, close, etc.",
),
"entity_id": types.Schema(
type=types.Type.STRING,
description="Target entity ID — omit for services that don't target a specific entity",
),
"data": types.Schema(
type=types.Type.STRING,
description='Extra service data as JSON string, e.g. {"temperature": 72, "hvac_mode": "heat"}',
),
},
required=["domain", "service"],
),
),
]

234
cortex/tools/notify.py Normal file
View File

@@ -0,0 +1,234 @@
"""
Notification tools — proactively send messages to user channels.
nc_talk_send routes through notification.py → channels.json.
email_send uses the server SMTP config from .env (smtp_server, smtp_from_*).
"""
import asyncio
import json
import logging
import re
import httpx
from google.genai import types
from config import settings
from persona import get_user
logger = logging.getLogger(__name__)
def _load_allowlist(username: str) -> list[str]:
"""Load the per-user email allowlist. Returns empty list if not configured."""
path = settings.home_root() / username / "email_allowlist.json"
try:
return [str(p).strip() for p in json.loads(path.read_text()) if str(p).strip()]
except FileNotFoundError:
return []
except Exception as e:
logger.warning("failed to read email_allowlist.json for %s: %s", username, e)
return []
def _email_allowed(address: str, patterns: list[str]) -> bool:
"""Return True if address matches any pattern (regex, case-insensitive full match)."""
addr = address.strip()
for pattern in patterns:
try:
if re.fullmatch(pattern, addr, re.IGNORECASE):
return True
except re.error:
logger.warning("invalid regex in email allowlist: %r", pattern)
return False
async def email_send(to: str, subject: str, body: str) -> str:
"""Send an email via the server's configured SMTP account."""
username = get_user()
allowlist = _load_allowlist(username)
if not allowlist:
return (
"Email blocked — no allowlist configured. "
f"Add allowed patterns to home/{username}/email_allowlist.json as a JSON array."
)
if not _email_allowed(to, allowlist):
return f"Email blocked — {to} does not match any allowed pattern for {username}."
from email_utils import send_email
ok = await asyncio.to_thread(
send_email,
to_email=to,
subject=subject,
body_text=body,
body_html=body.replace("\n", "<br>"),
)
if ok:
return f"Email sent to {to}."
return "Failed to send email — check SMTP configuration in .env."
async def web_push(title: str, body: str, url: str = "") -> str:
"""Send a browser push notification to the current user's registered devices."""
import push_utils
username = get_user()
result = await push_utils.send_push(username, title, body, url)
if "error" in result:
return f"Push failed: {result['error']}"
return f"Push sent to {result['sent']} device(s) for {username} (pruned {result['pruned']} stale)."
async def nc_talk_history(conversation_token: str = "", limit: int = 20) -> str:
"""Read recent messages from a Nextcloud Talk conversation.
Requires nc_username and nc_app_password in channels.json under 'nextcloud'.
conversation_token defaults to notification_room if not specified.
"""
from auth_utils import get_user_channels
username = get_user()
channels = get_user_channels(username)
nct = channels.get("nextcloud", {})
url = nct.get("url", "").rstrip("/")
nc_username = nct.get("nc_username", "").strip()
nc_app_password = nct.get("nc_app_password", "").strip()
token = conversation_token.strip() or nct.get("notification_room", "").strip()
if not url or not nc_username or not nc_app_password:
return (
"nc_talk_history requires nc_username and nc_app_password in channels.json "
f"(under 'nextcloud'). Add these to home/{username}/channels.json to enable message reading."
)
if not token:
return "No conversation token provided and no notification_room set in channels.json."
limit = min(max(int(limit), 1), 200)
return await asyncio.to_thread(_sync_nc_talk_history, url, nc_username, nc_app_password, token, limit)
def _sync_nc_talk_history(url: str, nc_user: str, nc_pass: str, token: str, limit: int) -> str:
from datetime import datetime, timezone
endpoint = f"{url}/ocs/v2.php/apps/spreed/api/v4/chat/{token}"
try:
resp = httpx.get(
endpoint,
params={"limit": limit, "lookIntoFuture": 0, "setReadMarker": 0, "noStatusUpdate": 1},
auth=(nc_user, nc_pass),
headers={"OCS-APIRequest": "true", "Accept": "application/json"},
timeout=15,
)
except Exception as e:
return f"NC Talk API error: {e}"
if resp.status_code != 200:
return f"NC Talk API returned HTTP {resp.status_code}: {resp.text[:200]}"
try:
messages = resp.json().get("ocs", {}).get("data", [])
except Exception as e:
return f"Failed to parse NC Talk response: {e}"
if not messages:
return "No messages found in this conversation."
# NC Talk returns newest-first; reverse to chronological order
lines = [f"Last {len(messages)} messages from {token}:\n"]
for msg in reversed(messages):
sender = msg.get("actorDisplayName") or msg.get("actorId") or "Unknown"
ts = msg.get("timestamp", 0)
time_str = datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
text = msg.get("message", "")
if msg.get("messageType") == "system":
lines.append(f"[system {time_str}] {text}")
else:
lines.append(f"{sender} ({time_str}): {text}")
return "\n".join(lines)
async def nc_talk_send(message: str) -> str:
"""Send a message to the user via their configured notification channel.
Channel is resolved from the user's channels.json (notification_channel key).
Falls back to Nextcloud Talk if configured. No-op if no channel is set.
"""
from notification import notify
username = get_user()
try:
await notify(username, message)
return f"Message sent to {username}'s notification channel."
except Exception as e:
logger.warning("nc_talk_send error for %s: %s", username, e)
return f"Failed to send notification: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="web_push",
description=(
"Send a browser push notification to the current user. Works even when the "
"Cortex tab is not open. Use for completing long tasks, reminders that fire "
"in the background, or anything the user should see immediately. "
"url is optional — if set, clicking the notification opens that URL."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"title": types.Schema(type=types.Type.STRING, description="Notification title (short)"),
"body": types.Schema(type=types.Type.STRING, description="Notification body text"),
"url": types.Schema(type=types.Type.STRING, description="Optional URL to open on click"),
},
required=["title", "body"],
),
),
types.FunctionDeclaration(
name="email_send",
description=(
"Send an email from the server's configured SMTP account. Use for delivering "
"summaries, reports, reminders, or any content the user wants emailed. "
"body is plain text; newlines are preserved."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"to": types.Schema(type=types.Type.STRING, description="Recipient email address"),
"subject": types.Schema(type=types.Type.STRING, description="Email subject line"),
"body": types.Schema(type=types.Type.STRING, description="Plain-text email body"),
},
required=["to", "subject", "body"],
),
),
types.FunctionDeclaration(
name="nc_talk_send",
description=(
"Send a proactive message to the user via their configured notification channel "
"(Nextcloud Talk by default). Use this to notify the user of completed background "
"tasks, important events, or anything they should know between sessions. "
"Requires notification_channel and notification_room set in channels.json."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"message": types.Schema(type=types.Type.STRING, description="The message to send to the user"),
},
required=["message"],
),
),
types.FunctionDeclaration(
name="nc_talk_history",
description=(
"Read recent messages from a Nextcloud Talk conversation. Useful for checking "
"what was said in a room before composing a reply, or reviewing recent context. "
"Requires nc_username and nc_app_password in channels.json under 'nextcloud'. "
"conversation_token defaults to notification_room if not provided."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"conversation_token": types.Schema(type=types.Type.STRING, description="NC Talk room token (defaults to notification_room from channels.json)"),
"limit": types.Schema(type=types.Type.INTEGER, description="Number of messages to return (default 20, max 200)"),
},
required=[],
),
),
]

255
cortex/tools/reminders.py Normal file
View File

@@ -0,0 +1,255 @@
"""
Reminders tools.
Reminders are stored in persona/REMINDERS.md and automatically surfaced
in the system prompt at Tier 2+. Each reminder can have an optional due date —
only due or undated reminders surface in context; future-dated ones are stored
but invisible until their date arrives.
Operations:
reminders_add — append a new reminder, optional due date (YYYY-MM-DD)
reminders_list — return all reminders with due status (including future)
reminders_remove — remove a single reminder by number
reminders_clear — erase all reminders
"""
import asyncio
import re
from datetime import datetime, timezone, date as _date
from pathlib import Path
from google.genai import types
from persona import persona_path
def _reminders_path() -> Path:
return persona_path() / "REMINDERS.md"
def _now_label() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
def _parse_sections(text: str) -> list[tuple[str, str]]:
"""Split REMINDERS.md into (heading, body) tuples, one per ## section."""
sections: list[tuple[str, str]] = []
heading: str | None = None
body_lines: list[str] = []
for line in text.splitlines():
if line.startswith("## "):
if heading is not None:
sections.append((heading, "\n".join(body_lines).strip()))
heading = line[3:].strip()
body_lines = []
elif heading is not None:
body_lines.append(line)
if heading is not None:
sections.append((heading, "\n".join(body_lines).strip()))
return sections
def _sections_to_text(sections: list[tuple[str, str]]) -> str:
return "".join(f"\n## {h}\n\n{b}\n" for h, b in sections)
def _parse_due(body: str) -> _date | None:
"""Extract due date from a 'due: YYYY-MM-DD' line in the body, if present."""
m = re.search(r'^due:\s*(\d{4}-\d{2}-\d{2})', body, re.MULTILINE | re.IGNORECASE)
if not m:
return None
try:
return _date.fromisoformat(m.group(1))
except ValueError:
return None
def _today() -> _date:
return datetime.now().astimezone().date()
def _is_due_or_undated(body: str) -> bool:
"""Return True if this reminder has no due date or its due date is today or past."""
due = _parse_due(body)
return due is None or due <= _today()
def _due_label(body: str) -> str:
"""Return a human-readable due status string for reminders_list output."""
due = _parse_due(body)
if due is None:
return ""
today = _today()
if due < today:
days = (today - due).days
return f" [OVERDUE by {days} day{'s' if days != 1 else ''} — due {due}]"
if due == today:
return " [due TODAY]"
return f" [due: {due}]"
def _body_without_due(body: str) -> str:
"""Strip the due: line from body for display (due status shown in heading line)."""
return re.sub(r'^due:\s*\S+\s*\n?', '', body, count=1, flags=re.MULTILINE | re.IGNORECASE).strip()
# ---------------------------------------------------------------------------
# Sync implementations
# ---------------------------------------------------------------------------
def _reminders_list() -> str:
p = _reminders_path()
if not p.exists() or not p.read_text().strip():
return "No pending reminders."
sections = _parse_sections(p.read_text())
if not sections:
return "No pending reminders."
lines = []
for i, (heading, body) in enumerate(sections, 1):
status = _due_label(body)
lines.append(f"{i}. {heading}{status}")
display_body = _body_without_due(body)
if display_body:
for bline in display_body.splitlines()[:4]:
lines.append(f" {bline}")
lines.append("")
return "\n".join(lines).rstrip()
def _reminders_add(text: str, label: str | None = None, due: str | None = None) -> str:
p = _reminders_path()
existing = p.read_text() if p.exists() else ""
heading = label or _now_label()
body = text.strip()
if due:
body = f"due: {due}\n{body}"
section = f"\n## {heading}\n\n{body}\n"
p.write_text(existing.rstrip() + "\n" + section)
msg = f"Reminder added: {heading}"
if due:
msg += f" (due: {due})"
return msg
def _reminders_remove(index: int) -> str:
p = _reminders_path()
if not p.exists() or not p.read_text().strip():
return "No reminders to remove."
sections = _parse_sections(p.read_text())
if not sections:
return "No reminders to remove."
if index < 1 or index > len(sections):
return (
f"Index {index} is out of range. "
f"There {'is' if len(sections) == 1 else 'are'} {len(sections)} "
f"reminder{'s' if len(sections) != 1 else ''} (1{len(sections)}). "
"Call reminders_list to see them."
)
removed_heading = sections[index - 1][0]
sections.pop(index - 1)
p.write_text(_sections_to_text(sections))
return f"Removed reminder {index}: {removed_heading}"
def _reminders_clear() -> str:
p = _reminders_path()
p.write_text("")
return "All reminders cleared."
# ---------------------------------------------------------------------------
# Public helper for context_loader
# ---------------------------------------------------------------------------
def load_due_reminders() -> str:
"""Return REMINDERS.md content filtered to only due and undated sections.
Called by context_loader at Tier 2+. Future-dated reminders are excluded
from the system prompt until their due date arrives.
"""
p = _reminders_path()
if not p.exists():
return ""
text = p.read_text()
if not text.strip():
return ""
sections = _parse_sections(text)
due_sections = [(h, b) for h, b in sections if _is_due_or_undated(b)]
if not due_sections:
return ""
# Strip the raw due: line from body — the date is already part of the heading context
cleaned = [(h, _body_without_due(b)) for h, b in due_sections]
return _sections_to_text(cleaned).strip()
# ---------------------------------------------------------------------------
# Async wrappers
# ---------------------------------------------------------------------------
async def reminders_list() -> str:
return await asyncio.to_thread(_reminders_list)
async def reminders_add(text: str, label: str | None = None, due: str | None = None) -> str:
return await asyncio.to_thread(_reminders_add, text, label, due)
async def reminders_remove(index: int) -> str:
return await asyncio.to_thread(_reminders_remove, index)
async def reminders_clear() -> str:
return await asyncio.to_thread(_reminders_clear)
DECLARATIONS = [
types.FunctionDeclaration(
name="reminders_add",
description=(
"Add a new reminder to REMINDERS.md. Reminders are automatically surfaced "
"in context at the start of each session (Tier 2+). "
"Use this when the user asks you to remember something or follow up on something. "
"Set a due date to suppress the reminder until that date — useful for future tasks "
"that would be noise today."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"text": types.Schema(type=types.Type.STRING, description="The reminder text"),
"label": types.Schema(type=types.Type.STRING, description="Optional heading (e.g. 'Follow up on NC Talk'). Defaults to current timestamp."),
"due": types.Schema(type=types.Type.STRING, description="Optional due date in YYYY-MM-DD format. Reminder is hidden from context until this date arrives. Omit for an always-visible reminder."),
},
required=["text"],
),
),
types.FunctionDeclaration(
name="reminders_list",
description=(
"Read all pending reminders, including future-dated ones not yet in context. "
"Shows due status for each (due today, overdue, or future date). "
"Use this before adding to avoid duplicates, or to show the user what's queued."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="reminders_remove",
description=(
"Remove a single reminder by its number. "
"Call reminders_list first to get the numbered list, then pass the number to remove."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"index": types.Schema(type=types.Type.INTEGER, description="The number of the reminder to remove (1 = first in reminders_list output)."),
},
required=["index"],
),
),
types.FunctionDeclaration(
name="reminders_clear",
description=(
"Erase all pending reminders from REMINDERS.md. "
"Use this after you have acknowledged and acted on the reminders shown in your context."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

128
cortex/tools/scratch.py Normal file
View File

@@ -0,0 +1,128 @@
"""
Scratchpad tools for Inara.
A lightweight, persistent notepad stored at inara/SCRATCH.md.
Nothing here is ever distilled or archived — it is intentionally transient.
Good for: working notes mid-task, half-formed ideas, things too long for
a chat response but not worth saving to memory or a journal entry.
Operations:
scratch_read — return the full contents (or a message if empty)
scratch_write — replace the entire scratchpad
scratch_append — add a new timestamped section at the bottom
scratch_clear — erase everything
"""
import asyncio
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path
def _scratch_path() -> Path:
return persona_path() / "SCRATCH.md"
def _now_label() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
# ---------------------------------------------------------------------------
# Sync implementations
# ---------------------------------------------------------------------------
def _scratch_read() -> str:
p = _scratch_path()
if not p.exists() or not p.read_text().strip():
return "Scratchpad is empty."
return p.read_text()
def _scratch_write(content: str) -> str:
_scratch_path().write_text(content.rstrip() + "\n")
return "Scratchpad updated."
def _scratch_append(content: str, heading: str | None = None) -> str:
p = _scratch_path()
existing = p.read_text() if p.exists() else ""
label = heading or _now_label()
section = f"\n## {label}\n\n{content.strip()}\n"
p.write_text(existing.rstrip() + "\n" + section)
return f"Appended section: {label}"
def _scratch_clear() -> str:
p = _scratch_path()
p.write_text("")
return "Scratchpad cleared."
# ---------------------------------------------------------------------------
# Async wrappers
# ---------------------------------------------------------------------------
async def scratch_read() -> str:
return await asyncio.to_thread(_scratch_read)
async def scratch_write(content: str) -> str:
return await asyncio.to_thread(_scratch_write, content)
async def scratch_append(content: str, heading: str | None = None) -> str:
return await asyncio.to_thread(_scratch_append, content, heading)
async def scratch_clear() -> str:
return await asyncio.to_thread(_scratch_clear)
DECLARATIONS = [
types.FunctionDeclaration(
name="scratch_read",
description=(
"Read the full contents of the scratchpad. "
"Use this to recall working notes, mid-task context, or anything previously jotted down. "
"The scratchpad is transient — nothing here is distilled or archived."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="scratch_write",
description=(
"Replace the entire scratchpad with new content. "
"Use this to set a clean working note, replacing whatever was there before. "
"For adding without replacing, use scratch_append instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(type=types.Type.STRING, description="The new scratchpad content (markdown supported)"),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="scratch_append",
description=(
"Add a new section to the bottom of the scratchpad without replacing existing content. "
"Each section gets a timestamp heading unless you supply one."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(type=types.Type.STRING, description="The content to append (markdown supported)"),
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading. Defaults to current UTC timestamp."),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="scratch_clear",
description="Erase everything in the scratchpad. Use when the working notes are no longer needed.",
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

334
cortex/tools/system.py Normal file
View File

@@ -0,0 +1,334 @@
"""
System tools — local machine operations.
These tools affect the host system directly. Use with care.
All tools in this module require the admin role.
"""
import asyncio
import logging
import os
import subprocess
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
# Absolute paths — resolved relative to this file so they work regardless of cwd
_CORTEX_DIR = Path(__file__).parent # .../Cortex_and_Inara_dev/cortex/
_PROJECT_ROOT = _CORTEX_DIR.parent # .../Cortex_and_Inara_dev/
ALLOW_SCRIPT = "/home/scott/.local/bin/claude-allow-dir"
async def claude_allow_dir(path: str, mode: str = "rw") -> str:
"""Add Read/Edit allow rules to ~/.claude/settings.json for a directory.
Calls the claude-allow-dir script, which edits settings.json directly.
Changes take effect in the next Claude Code session (or after /hooks reload).
"""
if mode not in ("r", "w", "rw"):
return f"Error: mode must be r, w, or rw (got '{mode}')"
try:
proc = await asyncio.create_subprocess_exec(
"python3", ALLOW_SCRIPT, path, mode,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=10)
output = stdout.decode().strip()
err = stderr.decode().strip()
if proc.returncode != 0:
logger.warning("claude-allow-dir failed (rc=%d): %s", proc.returncode, err)
return f"Failed (exit {proc.returncode}): {err or output}"
return output or "Done."
except asyncio.TimeoutError:
return "Error: script timed out"
except Exception as e:
logger.error("claude_allow_dir error: %s", e)
return f"Error: {e}"
async def shell_exec(command: str, working_dir: str | None = None, timeout: int = 30) -> str:
"""Execute a shell command on the Cortex host and return combined stdout/stderr."""
timeout = min(max(timeout, 1), 120)
cwd = None
if working_dir:
cwd = os.path.expanduser(working_dir)
if not os.path.isdir(cwd):
return f"Error: working_dir '{working_dir}' does not exist or is not a directory"
try:
proc = await asyncio.create_subprocess_shell(
command,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
out = stdout.decode(errors="replace").strip()
err = stderr.decode(errors="replace").strip()
parts = []
if out:
parts.append(out)
if err:
parts.append(f"[stderr]\n{err}")
combined = "\n".join(parts) if parts else "(no output)"
if proc.returncode != 0:
return f"Exit {proc.returncode}:\n{combined}"
return combined
except asyncio.TimeoutError:
return f"Error: command timed out after {timeout}s"
except Exception as e:
logger.error("shell_exec error: %s", e)
return f"Error: {e}"
async def cortex_restart() -> str:
"""Schedule a Cortex service restart 5 seconds from now.
Uses a detached subprocess so the restart survives the current process being
terminated by systemd. The calling session will drop — user should refresh.
"""
subprocess.Popen(
["bash", "-c", "sleep 5 && systemctl --user restart cortex"],
start_new_session=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
close_fds=True,
)
logger.info("cortex_restart: restart scheduled in 5 seconds")
return (
"Cortex restart scheduled in 5 seconds. "
"The current connection will drop — please refresh the page after a moment."
)
async def cortex_logs(lines: int = 50) -> str:
"""Return recent lines from the Cortex systemd journal."""
n = min(max(int(lines), 1), 200)
try:
proc = await asyncio.create_subprocess_exec(
"journalctl", "--user", "-u", "cortex", f"-n{n}", "--no-pager",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=15)
out = stdout.decode(errors="replace").strip()
return out or stderr.decode(errors="replace").strip() or "No log output."
except asyncio.TimeoutError:
return "Error: journalctl timed out"
except Exception as e:
logger.error("cortex_logs error: %s", e)
return f"Error: {e}"
async def cortex_status() -> str:
"""Return Cortex service status: git branch/commit, ahead/behind remote, and systemctl state."""
lines = []
async def _git(*args: str) -> str:
proc = await asyncio.create_subprocess_exec(
"git", "-C", str(_PROJECT_ROOT), *args,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.DEVNULL,
)
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=10)
return stdout.decode(errors="replace").strip()
try:
branch = await _git("rev-parse", "--abbrev-ref", "HEAD")
commit = await _git("log", "--oneline", "-1")
# fetch quietly so ahead/behind is current
await asyncio.create_subprocess_exec(
"git", "-C", str(_PROJECT_ROOT), "fetch", "--quiet",
stdout=asyncio.subprocess.DEVNULL, stderr=asyncio.subprocess.DEVNULL,
)
ahead_behind = await _git("rev-list", "--left-right", "--count", f"HEAD...origin/{branch}")
ahead, behind = (ahead_behind.split() + ["?", "?"])[:2]
lines.append(f"**Branch:** {branch} | ahead {ahead} / behind {behind}")
lines.append(f"**Commit:** {commit}")
except Exception as e:
lines.append(f"Git info unavailable: {e}")
lines.append("")
try:
proc = await asyncio.create_subprocess_exec(
"systemctl", "--user", "status", "cortex", "--no-pager", "-l",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
)
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=10)
# First 15 lines of systemctl output is enough — avoids log flood
status_lines = stdout.decode(errors="replace").splitlines()[:15]
lines.extend(status_lines)
except Exception as e:
lines.append(f"systemctl status unavailable: {e}")
return "\n".join(lines)
async def cortex_update() -> str:
"""Pull the latest code from git, syntax-check all Python files, and report.
Does NOT restart automatically — call cortex_restart separately after reviewing
the output if you want to apply changes.
"""
lines = []
async def _run(*cmd: str, cwd: Path = _PROJECT_ROOT, timeout: int = 30) -> tuple[int, str]:
proc = await asyncio.create_subprocess_exec(
*cmd, cwd=str(cwd),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
)
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout)
return proc.returncode, stdout.decode(errors="replace").strip()
# 1. Check for incoming commits before pulling
try:
await _run("git", "fetch", "--quiet")
rc, incoming = await _run("git", "log", "--oneline", "HEAD..origin/HEAD")
if rc == 0 and not incoming:
# Double-check with branch name in case origin/HEAD isn't set
branch_rc, branch = await _run("git", "rev-parse", "--abbrev-ref", "HEAD")
_, incoming = await _run("git", "log", "--oneline", f"HEAD..origin/{branch.strip()}")
except asyncio.TimeoutError:
return "Error: git fetch timed out — check network connectivity."
except Exception as e:
return f"Error during git fetch: {e}"
if not incoming:
rc2, current = await _run("git", "log", "--oneline", "-1")
return f"Already up to date.\n\nCurrent commit: {current}"
lines.append(f"**Incoming commits:**\n{incoming}\n")
# 2. Pull
try:
rc, pull_out = await _run("git", "pull", "--ff-only")
except asyncio.TimeoutError:
return "Error: git pull timed out."
except Exception as e:
return f"Error during git pull: {e}"
if rc != 0:
return f"git pull failed (exit {rc}):\n{pull_out}"
lines.append(f"**git pull:**\n{pull_out}\n")
# 3. Syntax check all Python files under cortex/
py_files = sorted(_CORTEX_DIR.rglob("*.py"))
errors = []
for f in py_files:
result = subprocess.run(
["python3", "-m", "py_compile", str(f)],
capture_output=True, text=True,
)
if result.returncode != 0:
errors.append(f" {f.relative_to(_PROJECT_ROOT)}: {result.stderr.strip()}")
if errors:
lines.append(f"**Syntax errors — do NOT restart until fixed:**")
lines.extend(errors)
else:
lines.append(f"**Syntax check:** {len(py_files)} files — all OK.")
lines.append("Call `cortex_restart` to apply the update.")
return "\n".join(lines)
DECLARATIONS = [
types.FunctionDeclaration(
name="shell_exec",
description=(
"Execute a shell command on the Cortex host machine and return its output. "
"Use for system diagnostics: disk usage (df -h), process status (ps aux), "
"directory listings (ls), memory (free -h), uptime, network info, log tails, etc. "
"Commands run as the Cortex service user. Timeout enforced (default 30s, max 120s). "
"Avoid destructive commands — prefer read-only system queries."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"command": types.Schema(type=types.Type.STRING, description="Shell command to run (e.g. 'df -h', 'ls ~/agents_sync/', 'journalctl --user -u cortex -n 50')"),
"working_dir": types.Schema(type=types.Type.STRING, description="Optional working directory (e.g. '~/agents_sync/projects'). Defaults to home directory."),
"timeout": types.Schema(type=types.Type.INTEGER, description="Timeout in seconds (default 30, max 120)"),
},
required=["command"],
),
),
types.FunctionDeclaration(
name="claude_allow_dir",
description=(
"Add a directory to Claude Code's auto-allow list so Claude can read or write "
"files there without prompting. Edits ~/.claude/settings.json on the local machine. "
"Use this when Claude is silently hanging or being blocked from accessing a directory. "
"Changes take effect in the next Claude Code session."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(type=types.Type.STRING, description="Absolute or home-relative path to the directory (e.g. ~/OSIT_dev/aether_api_fastapi or /home/scott/agents_sync)"),
"mode": types.Schema(type=types.Type.STRING, description="Permission mode: 'r' (read-only), 'w' (write-only), or 'rw' (both). Default: rw"),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="cortex_restart",
description=(
"Restart the Cortex service via systemd. Schedules a restart 5 seconds from now. "
"The current connection will drop — inform the user to refresh the page. "
"Use after config changes, memory edits, or when the service needs a fresh start. "
"ADMIN ONLY."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="cortex_logs",
description=(
"Fetch recent lines from the Cortex systemd service journal. "
"Use for debugging errors, checking startup status, or reviewing recent activity. "
"ADMIN ONLY."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"lines": types.Schema(type=types.Type.INTEGER, description="Number of log lines to return (default 50, max 200)"),
},
),
),
types.FunctionDeclaration(
name="cortex_status",
description=(
"Return Cortex service status: current git branch and commit, how many commits "
"ahead/behind the remote, and the systemctl service state. "
"Use to check what version is running or whether the service is healthy. "
"ADMIN ONLY."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="cortex_update",
description=(
"Pull the latest code from git, run a syntax check on all Python files, and report "
"what changed. Does NOT restart automatically — call cortex_restart separately after "
"reviewing the output. Will report syntax errors if the pull introduces broken code. "
"ADMIN ONLY. Requires confirmation."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

206
cortex/tools/tasks.py Normal file
View File

@@ -0,0 +1,206 @@
"""
Personal task management tools for Inara.
Tasks are stored in inara/TASKS.json — private to each agent instance.
Schema per task:
{
"id": short random string,
"title": str,
"description": str | None,
"status": "todo" | "in_progress" | "done",
"priority": "low" | "normal" | "high",
"created_at": ISO 8601,
"updated_at": ISO 8601
}
"""
import json
import secrets
import asyncio
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path
def _tasks_path() -> Path:
return persona_path() / "TASKS.json"
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _load() -> list[dict]:
p = _tasks_path()
if not p.exists():
return []
try:
return json.loads(p.read_text())
except Exception:
return []
def _save(tasks: list[dict]) -> None:
_tasks_path().write_text(json.dumps(tasks, indent=2) + "\n")
def _short_id() -> str:
return "t_" + secrets.token_urlsafe(6)
def _format_task(t: dict) -> str:
pri = f"[{t['priority']}]" if t.get("priority") != "normal" else ""
desc = f"\n {t['description']}" if t.get("description") else ""
return f"• [{t['status']}] {t['id']} {pri} {t['title']}{desc}".strip()
# ---------------------------------------------------------------------------
# Sync implementations — called via asyncio.to_thread
# ---------------------------------------------------------------------------
def _task_list(status: str | None, priority: str | None) -> str:
tasks = _load()
if status:
tasks = [t for t in tasks if t["status"] == status]
if priority:
tasks = [t for t in tasks if t.get("priority") == priority]
if not tasks:
filters = " ".join(f for f in [status, priority] if f)
return f"No {filters} tasks." if filters else "No tasks yet."
lines = [f"Tasks ({len(tasks)}):\n"]
for t in tasks:
lines.append(_format_task(t))
return "\n".join(lines)
def _task_create(title: str, description: str | None, priority: str) -> str:
if priority not in ("low", "normal", "high"):
priority = "normal"
tasks = _load()
task = {
"id": _short_id(),
"title": title,
"description": description,
"status": "todo",
"priority": priority,
"created_at": _now(),
"updated_at": _now(),
}
tasks.append(task)
_save(tasks)
return f"Created: {_format_task(task)}"
def _task_update(task_id: str, status: str | None, title: str | None,
description: str | None, priority: str | None) -> str:
tasks = _load()
for t in tasks:
if t["id"] == task_id:
if status and status in ("todo", "in_progress", "done"):
t["status"] = status
if title:
t["title"] = title
if description is not None:
t["description"] = description
if priority and priority in ("low", "normal", "high"):
t["priority"] = priority
t["updated_at"] = _now()
_save(tasks)
return f"Updated: {_format_task(t)}"
return f"Task not found: {task_id}"
def _task_complete(task_id: str) -> str:
return _task_update(task_id, status="done", title=None, description=None, priority=None)
# ---------------------------------------------------------------------------
# Async wrappers
# ---------------------------------------------------------------------------
async def task_list(status: str | None = None, priority: str | None = None) -> str:
return await asyncio.to_thread(_task_list, status, priority)
async def task_create(title: str, description: str | None = None,
priority: str = "normal") -> str:
return await asyncio.to_thread(_task_create, title, description, priority)
async def task_update(task_id: str, status: str | None = None, title: str | None = None,
description: str | None = None, priority: str | None = None) -> str:
return await asyncio.to_thread(_task_update, task_id, status, title, description, priority)
async def task_complete(task_id: str) -> str:
return await asyncio.to_thread(_task_complete, task_id)
DECLARATIONS = [
types.FunctionDeclaration(
name="task_list",
description=(
"List personal tasks from Inara's task list. "
"Use this to check what's on the list, review pending work, or find a task ID. "
"Optionally filter by status: 'todo', 'in_progress', or 'done'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"status": types.Schema(type=types.Type.STRING, description="Filter by status: 'todo', 'in_progress', or 'done'. Omit to list all."),
"priority": types.Schema(type=types.Type.STRING, description="Filter by priority: 'low', 'normal', or 'high'. Omit to list all priorities."),
},
),
),
types.FunctionDeclaration(
name="task_create",
description=(
"Add a new task to Inara's personal task list. "
"Use this when the user asks to remember something, add a to-do, or track a follow-up."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"title": types.Schema(type=types.Type.STRING, description="Short task title"),
"description": types.Schema(type=types.Type.STRING, description="Optional longer description or context"),
"priority": types.Schema(type=types.Type.STRING, description="Priority: 'low', 'normal', or 'high'. Default: normal."),
},
required=["title"],
),
),
types.FunctionDeclaration(
name="task_update",
description=(
"Update an existing task. Use task_list first to get the task ID. "
"Can update status, title, description, or priority. "
"To just mark complete, use task_complete instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"task_id": types.Schema(type=types.Type.STRING, description="Task ID (e.g. t_abc123) — get from task_list"),
"status": types.Schema(type=types.Type.STRING, description="New status: 'todo', 'in_progress', or 'done'"),
"title": types.Schema(type=types.Type.STRING, description="Updated title"),
"description": types.Schema(type=types.Type.STRING, description="Updated description"),
"priority": types.Schema(type=types.Type.STRING, description="Updated priority: 'low', 'normal', or 'high'"),
},
required=["task_id"],
),
),
types.FunctionDeclaration(
name="task_complete",
description=(
"Mark a task as done. Use task_list first to get the task ID. "
"Shorthand for task_update with status='done'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"task_id": types.Schema(type=types.Type.STRING, description="Task ID (e.g. t_abc123) — get from task_list"),
},
required=["task_id"],
),
),
]

267
cortex/tools/web.py Normal file
View File

@@ -0,0 +1,267 @@
"""
Web tools — search (DuckDuckGo), direct HTTP fetch, clean content extraction, and HTTP POST.
"""
import asyncio
import json
import logging
from urllib.parse import urlparse
import httpx
from google.genai import types
from config import settings
from persona import get_user
logger = logging.getLogger(__name__)
async def search(query: str, max_results: int | None = None) -> str:
"""Search DuckDuckGo and return results as a formatted string.
Returns a markdown-formatted list of results: title, URL, and snippet.
The orchestrator includes this in the context it passes to Claude.
"""
n = min(max_results or settings.ddg_max_results, 10)
results = await asyncio.to_thread(_sync_search, query, n)
if not results:
return f"No results found for: {query}"
lines = [f"Search results for: **{query}**\n"]
for i, r in enumerate(results, 1):
lines.append(f"{i}. [{r['title']}]({r['href']})")
if r.get("body"):
lines.append(f" {r['body']}")
lines.append("")
return "\n".join(lines).strip()
def _sync_search(query: str, max_results: int) -> list[dict]:
"""Synchronous DuckDuckGo search — run via asyncio.to_thread."""
from ddgs import DDGS
kwargs = {}
if settings.ddg_api_key:
# Paid account — pass token for higher rate limits
kwargs["headers"] = {"Authorization": f"Bearer {settings.ddg_api_key}"}
try:
with DDGS(**kwargs) as ddgs:
return list(ddgs.text(query, max_results=max_results))
except Exception as e:
logger.warning("DuckDuckGo search error: %s", e)
return []
async def http_fetch(
url: str,
method: str = "GET",
body: str | None = None,
timeout: int = 15,
max_chars: int = 8192,
) -> str:
"""Fetch a URL directly and return the raw response body.
Unlike web_search, this hits a specific URL — useful for health checks,
API probing, JSON endpoints, webhook testing, or reading raw page source.
For readable article content, use web_read instead.
Response body is capped at max_chars (default 8192, max 32768).
"""
method = method.upper()
timeout = min(max(int(timeout), 1), 60)
max_chars = min(max(int(max_chars), 100), 131072)
try:
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
resp = await client.request(method, url, content=body)
body_text = resp.text[:max_chars]
truncated = len(resp.text) > max_chars
suffix = f"\n\n[… truncated at {max_chars} chars]" if truncated else ""
return f"HTTP {resp.status_code} {resp.url}\n\n{body_text}{suffix}"
except httpx.HTTPError as e:
return f"HTTP error: {e}"
except Exception as e:
logger.warning("http_fetch error for %s: %s", url, e)
return f"Error: {e}"
async def web_read(url: str, max_chars: int = 16000) -> str:
"""Fetch a URL and extract clean readable text, stripping ads, navigation, and boilerplate.
Uses trafilatura to extract the main article content — ideal for blog posts,
documentation, news articles, and any page where you want the text without
surrounding noise. Returns markdown-formatted output.
For raw responses (JSON APIs, health checks), use http_fetch instead.
"""
max_chars = min(max(int(max_chars), 1000), 131072)
return await asyncio.to_thread(_sync_web_read, url, max_chars)
def _sync_web_read(url: str, max_chars: int) -> str:
try:
import trafilatura
except ImportError:
return "web_read requires trafilatura — run: pip install trafilatura"
downloaded = trafilatura.fetch_url(url)
if downloaded is None:
return f"Failed to download content from: {url}"
text = trafilatura.extract(downloaded, output_format="markdown", include_links=True, url=url)
if not text:
text = trafilatura.extract(downloaded, url=url)
if not text:
return f"Could not extract readable content from: {url}"
if len(text) > max_chars:
text = text[:max_chars] + f"\n\n[… truncated at {max_chars} chars — pass a larger max_chars (up to 131072) to see more]"
return f"Content from {url}:\n\n{text}"
def _load_http_allowlist(username: str) -> list[str]:
"""Load per-user HTTP POST allowlist (URL prefixes). Empty list = all blocked."""
path = settings.home_root() / username / "http_allowlist.json"
try:
return [str(p).strip() for p in json.loads(path.read_text()) if str(p).strip()]
except FileNotFoundError:
return []
except Exception as e:
logger.warning("failed to read http_allowlist.json for %s: %s", username, e)
return []
def _http_post_allowed(url: str, allowlist: list[str]) -> bool:
"""Return True if url starts with any allowlist entry (prefix match)."""
for prefix in allowlist:
if url.startswith(prefix):
return True
return False
async def http_post(
url: str,
body: str = "",
headers: dict | None = None,
max_chars: int = 4096,
) -> str:
"""POST to an external URL. Requires the URL to match home/{user}/http_allowlist.json.
body may be a JSON string or plain text. If body is valid JSON, Content-Type is set
to application/json; otherwise text/plain. Override via the headers param.
Response is capped at max_chars (default 4096, max 131072).
"""
username = get_user()
allowlist = _load_http_allowlist(username)
if not allowlist:
return (
f"http_post blocked — no allowlist configured. "
f"Add allowed URL prefixes to home/{username}/http_allowlist.json as a JSON array. "
f"Example: [\"https://api.example.com\"]"
)
if not _http_post_allowed(url, allowlist):
return (
f"http_post blocked — {url} does not match any allowlist entry for {username}. "
f"Add the URL prefix to home/{username}/http_allowlist.json."
)
max_chars = min(max(int(max_chars), 100), 131072)
# Auto-detect content type from body
body_str = body if isinstance(body, str) else json.dumps(body)
try:
json.loads(body_str)
content_type = "application/json"
except (json.JSONDecodeError, ValueError):
content_type = "text/plain"
req_headers = {"Content-Type": content_type}
if headers:
req_headers.update(headers)
try:
async with httpx.AsyncClient(timeout=30, follow_redirects=True) as client:
resp = await client.post(url, content=body_str.encode(), headers=req_headers)
body_text = resp.text[:max_chars]
truncated = len(resp.text) > max_chars
suffix = f"\n\n[… truncated at {max_chars} chars]" if truncated else ""
return f"HTTP {resp.status_code} {resp.url}\n\n{body_text}{suffix}"
except httpx.HTTPError as e:
return f"HTTP error: {e}"
except Exception as e:
logger.warning("http_post error for %s: %s", url, e)
return f"Error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="web_search",
description=(
"Search the web for current information. Use this when you need up-to-date "
"facts, news, documentation, or anything not in your training data."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"query": types.Schema(type=types.Type.STRING, description="The search query string"),
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of results to return (default 5, max 10)"),
},
required=["query"],
),
),
types.FunctionDeclaration(
name="http_fetch",
description=(
"Fetch a specific URL and return the raw response body. Unlike web_search, this hits "
"a direct URL — useful for health checks, JSON API endpoints, webhook testing, "
"or inspecting raw page source. For readable article/doc content, use web_read instead. "
"Response body is capped at max_chars (default 8192, max 32768)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"url": types.Schema(type=types.Type.STRING, description="Full URL to fetch"),
"method": types.Schema(type=types.Type.STRING, description="HTTP method: GET (default), POST, HEAD"),
"body": types.Schema(type=types.Type.STRING, description="Optional request body (for POST requests)"),
"timeout": types.Schema(type=types.Type.INTEGER, description="Request timeout in seconds (default 15, max 60)"),
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max characters to return (default 8192, max 131072)"),
},
required=["url"],
),
),
types.FunctionDeclaration(
name="web_read",
description=(
"Fetch a URL and extract clean readable text, stripping ads, navigation, sidebars, "
"and other boilerplate. Returns the main article/document content as markdown. "
"Use this for blog posts, documentation, news articles, GitHub READMEs, or any page "
"where you want the content without surrounding noise. "
"For raw HTTP responses (JSON APIs, health checks, source inspection), use http_fetch."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"url": types.Schema(type=types.Type.STRING, description="Full URL to fetch and extract"),
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max characters to return (default 16000, max 131072)"),
},
required=["url"],
),
),
types.FunctionDeclaration(
name="http_post",
description=(
"POST to an external URL. Requires the URL to match the user's http_allowlist.json. "
"Use for calling webhooks, triggering automations, posting to APIs, or any HTTP action. "
"body is a string — JSON or plain text are both accepted (Content-Type auto-detected). "
"Override headers as needed. Response capped at max_chars (default 4096, max 131072)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"url": types.Schema(type=types.Type.STRING, description="Full URL to POST to"),
"body": types.Schema(type=types.Type.STRING, description="Request body — JSON string or plain text"),
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max response chars (default 4096, max 131072)"),
},
required=["url"],
),
),
]

75
cortex/usage_tracker.py Normal file
View File

@@ -0,0 +1,75 @@
"""
API usage and token tracking.
Writes daily buckets to home/{username}/usage.json:
{
"2026-05-01": {
"gemini_api/gemini-2.0-flash": {"calls": 3, "prompt_tokens": 8400, "completion_tokens": 520},
"local/llama3.2:latest": {"calls": 2, "prompt_tokens": 1200, "completion_tokens": 310}
}
}
Claude CLI and Gemini CLI backends produce no structured token data and are not tracked.
"""
import asyncio
import json
import logging
from datetime import date
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
_LOCK = asyncio.Lock()
def _usage_path(username: str) -> Path:
return settings.home_root() / username / "usage.json"
async def record(
username: str,
backend: str,
model_name: str,
prompt_tokens: int,
completion_tokens: int,
) -> None:
"""Append one call's token counts to the daily usage log for this user.
backend — "gemini_api" | "local"
model_name — the exact model string (e.g. "gemini-2.0-flash", "llama3.2:latest")
"""
path = _usage_path(username)
today = date.today().isoformat()
key = f"{backend}/{model_name}"
async with _LOCK:
try:
data: dict = json.loads(path.read_text()) if path.exists() else {}
except Exception:
data = {}
entry = data.setdefault(today, {}).setdefault(
key, {"calls": 0, "prompt_tokens": 0, "completion_tokens": 0}
)
entry["calls"] += 1
entry["prompt_tokens"] += prompt_tokens
entry["completion_tokens"] += completion_tokens
try:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, indent=2))
except Exception as e:
logger.warning("Failed to write usage data to %s: %s", path, e)
def read_usage(username: str) -> dict:
"""Return the full usage dict for this user. Empty dict if no file yet."""
path = _usage_path(username)
try:
return json.loads(path.read_text()) if path.exists() else {}
except Exception:
return {}

Some files were not shown because too many files have changed in this diff Show More