Compare commits

..

163 Commits

Author SHA1 Message Date
Scott Idem
e8819773ee docs: update TODO — mark completed items from 2026-06-03 session
- aider_run async/notify: done
- L2→L3 boundary enforcement: done (default _agent_level=2)
- aider_run multi-provider credentials: done
- Added remaining item: pass _agent_level=1 from main orchestrators

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 23:06:47 -04:00
Scott Idem
0c1cf3989a feat: aider multi-provider credentials + test suite green (182/182)
aider_run multi-provider credentials (tools/aider.py):
- _resolve_credentials() — general credential resolver; replaces the previous
  OpenRouter-only injection; resolution priority: Anthropic model hint → explicit
  host_label → model prefix (openrouter/*, groq/*, deepseek/*, …) → OpenRouter
  default → Anthropic API key → any keyed cloud host → local/generic host
- _host_flags() — generates --api-key slug=key for known cloud providers (OpenRouter,
  OpenAI, Groq, Together, Fireworks, X.ai, DeepSeek, Mistral); generates
  --openai-api-base + --openai-api-key for generic/local hosts (Open WebUI, Ollama);
  appends /api suffix for openwebui host_type; auto-prefixes model with 'openai/'
  for generic endpoints when model has no / prefix
- Anthropic API keys from providers.anthropic.credentials (not a host entry)
- host_label param added to aider_run and FunctionDeclaration — pick a configured
  host by partial label match (e.g. 'OpenRouter', 'Local', 'scott-lt-i7-rtx')
- 16 unit tests for _resolve_credentials covering all resolution paths

main.py: move @app.get("/health") before app.include_router(ui.router) — the
/{username} catch-all in ui.router was swallowing the /health path

Test suite: 37 pre-existing failures → 182/182 passing
- test_tools.py: _task_list() missing priority arg (6 callsites); cron ID regex
  c_\w+ → c_[\w-]+ (token_urlsafe includes '-', causing intermittent truncation)
- test_webhooks.py: rewritten for per-user channel config architecture —
  patch routers.nextcloud_talk/google_chat.get_user_channels instead of removed
  settings fields; corrected endpoints /webhook/nextcloud/scott and
  /channels/google-chat/scott; non-empty cfg dicts so falsy-guard passes
- test_health.py: test_unknown_route_404 now uses 3-segment path (/{u}/{p}/x)
  since single-segment paths hit the /{username} UI catch-all
- test_api_files.py: removed '../config.py' from not-in-allowed test (ASGI
  normalizes it to /config.py which hits /{username} catch-all, not files router)
- test_security.py: same webhook patch target fix; per-user endpoint URLs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 23:00:45 -04:00
Scott Idem
658c508925 feat: multi-level agent management — background agents, lifecycle tools, 3-level hierarchy
agent_manager.py (new):
- AgentRecord dataclass: agent_id, level (1/2/3), role, task, status, started,
  parent_id (lineage), finished, result, notify, _task_ref
- register() / finish() / cancel_agent() / list_agents() / get() / set_task_ref()
- Calls notification.notify() on completion when notify=True (same channel as
  reminders and cron completions)
- 24-hour pruning of completed records on each new registration

spawn_agent (tools/agents.py):
- background=True: fires asyncio.create_task(), registers in agent_manager, returns
  agent_id string immediately — sync path unchanged (no regression)
- notify=True: push/Talk notification when the background task completes
- Level enforcement: _agent_level param tracks hierarchy depth; when spawning from
  Level 2, child automatically gets spawn_agent + aider_run denied so Level 3 agents
  cannot delegate further

New lifecycle tools (tools/agents.py + __init__.py):
- agent_status(agent_id) — status, role, level, elapsed, task, result preview; user-level
- agent_list(status, limit) — all agents for current user, newest first; user-level
- agent_cancel(agent_id) — kills background task; admin-only, confirm-required

tests/test_agent_manager.py (new, 41 tests):
- agent_manager CRUD, pruning, notification hook
- spawn_agent background: returns immediately, completes async, timeout, failure
- Level enforcement: L1→L2 permits spawn, L2→L3 auto-denies; explicit tool_list path
- agent_status / agent_list / agent_cancel output formatting
- aider_run background: returns agent_id, completes async, sync path unchanged
- All tests run without browser or Cortex service (~2.5s total)
  Run: cd cortex && .venv/bin/python -m pytest tests/test_agent_manager.py -v

Docs: ARCH__FUTURE.md §13 (full design), ROADMAP.md, TODO__Agents.md, MASTER.md,
HELP.md (orchestrator description corrected, tool schema line updated to reflect
keyword routing), CLAUDE.md tool count 66→69.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 22:40:20 -04:00
Scott Idem
29d8aa4aae feat: tool schema optimization, keyword routing, aider_run coding agent
Tool schema optimization (PLAN__Tool_Schema_Optimization.md Phases 1-3):
- model_registry.py: ROLE_DEFAULT_TOOLS — distill gets [], research/coder get
  narrow tool lists by default; applied in get_role_config() when user hasn't
  configured a custom list
- openai_orchestrator.py: keyword routing via narrow_tools_by_keywords() — scans
  user message + last assistant turn; narrows active schemas to matched categories
  only (e.g. "weather" → 3 web tools instead of 69); zero tools sent for pure chat
- openai_orchestrator.py: _get_cached_tools() — module-level schema cache keyed by
  (role, sorted_tool_list, risk_params); eliminates redundant schema rebuilds
- openai_orchestrator.py: _TOOL_SCHEMA_OVERHEAD 3000 → 500 tokens (schemas now
  excluded from the per-call fixed estimate since they're cached separately)
- tools/__init__.py: CATEGORY_TOOL_MAP + _KEYWORD_CATEGORY_MAP + classify_tool_categories()
  + narrow_tools_by_keywords() — the classifier logic lives here so both orchestrators
  can share it

aider_run tool (cortex/tools/aider.py):
- Invokes Aider as a subprocess with --message --yes-always --no-pretty --no-stream
- Project aliases: cortex / aether_api / aether_frontend / aether_container
- Auto-injects OpenRouter API key from Cortex model registry (no ~/.env needed)
- background=True fires async + registers in agent_manager; notify=True sends push
  notification on completion
- admin-only, confirm-required, TOOL_RISK=high
- .gitignore: added .aider.chat.history.md / .aider.input.history / .aider.llm.history

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-03 22:39:44 -04:00
Scott Idem
29940c299b docs: fix Chat and Tools section accuracy in HELP.md
- Copy button appears on all messages (user + assistant), not just assistant
- Delete requires a confirm step — clarified in description
- Model tag position is below-left for assistant messages, not bottom-right
- Tool calls render as expandable per-call cards, not a single  N summary note
- help.html: default-open sections updated to Getting Started / Chat / Sessions / Model Registry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:36:24 -04:00
Scott Idem
105ff8507f fix: restore proper tab button CSS in help.html (browser-reset issue with Tailwind preflight: false)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:19:08 -04:00
Scott Idem
c2a12a895a docs: rename Backends → Switching Models in HELP.md; fix stale Backup 2 reference
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:14:55 -04:00
Scott Idem
df1f358912 docs: fix role slots description in HELP.md (Primary + Backup 1, not Backup 2)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:11:03 -04:00
Scott Idem
7a27190ffe feat: custom roles, Tailwind settings pages, pg.css fixes, doc cleanup
Model Registry:
- Add per-user custom roles (add/remove via UI); required roles chat/orchestrator/distill
  are always present and cannot be removed
- Auto-migrate legacy .env-defined roles to custom_roles on first access
- Role config panel (gear): Remove role button moved inside panel; required badge below name
- Role select: Primary + Backup slots only (was three)

Settings pages — Tailwind CSS migration (CDN, preflight: false):
- local_llm.html, settings.html, help.html, notifications.html, tools_settings.html,
  crons.html, integrations.html all migrated; pg-* color tokens; dark/light via data-theme

pg.css fixes:
- input[type=checkbox/radio]: width: auto — prevents pg.css width:100% from stretching checkboxes
- btn-submit: responsive sizing via Tailwind w-full md:w-96 on each button (no longer full-width on desktop)

Documentation:
- MASTER.md, TODO__Agents.md: remove "/ Inara" from titles; description updated to "per-user AI personas"
- HELP.md: persona-agnostic language throughout (NC Talk, Google Chat, push, schedules, HA sections);
  roles section restructured to show required vs. custom roles with examples
- notifications.html: subtitle and HA description use "your persona" not "Inara"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 21:03:11 -04:00
Scott Idem
070f1ce156 fix: align settings/models nav and heading with all other settings pages
local_llm.html had a stale nav (hardcoded / and /help, missing Notifications /
Tools / Schedules / Integrations links) and used a different heading markup
(.page-header) than the rest of the settings pages (.page-title/.page-subtitle).

- Add pg.css link; strip duplicate base CSS (vars, reset, body, nav) from inline
  <style> — page-specific styles (provider blocks, model rows, roles, badges) kept
- Nav: use {{ back_href }} / {{ help_href }} / {{ integrations_nav }} placeholders;
  add Notifications, Tools, Schedules, Integrations (admin) links
- Heading: .page-header → <h1 class="page-title"> + <p class="page-subtitle">
- Body structure: move <nav> outside the .page wrapper; rename .page → .page-wrap
- local_llm.py: _render() accepts request, injects back_href / help_href /
  integrations_nav; adds _preferred_persona(), _integrations_nav() helpers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 21:51:06 -04:00
Scott Idem
a92fd90f0d feat: Anthropic SDK backend — API key alternative to Claude CLI OAuth
Adds `anthropic_api` model type so users can authenticate with a direct
Anthropic API key instead of (or alongside) the CLI OAuth session.

- model_registry.py: `anthropic_api` type; `save/get/remove_anthropic_api_key()`
  mirroring the Google account pattern; `save_cloud_model()` now picks type
  based on credential type (cli → claude_cli, api_key → anthropic_api);
  `_resolve_model()` merges api_key from the credential entry
- llm_client.py: `_anthropic_api()` backend (AsyncAnthropic SDK); dispatch
  and fallback wiring; usage tracking
- routers/local_llm.py: Anthropic API key management routes
  (POST /settings/local/anthropic-key, /anthropic-key/{id}/remove);
  `anthropic_api` badge and edit-form credential selector
- static/local_llm.html: Anthropic Cloud Provider block now shows API key
  management (add/remove); Add Model → Anthropic tab has credential selector
  (CLI vs API key)
- requirements.txt: enable anthropic>=0.40.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 21:30:56 -04:00
Scott Idem
70665fadff feat: schedules UI, task cron type, monthly/yearly schedules, AE DB tools, integrations page
- Schedules web UI (/settings/crons): list, add, edit, pause/resume, delete jobs
- cron task type: full orchestrator tool loop on a schedule, result → notification channel
- parse_schedule: monthly/yearly formats (monthly:DD:HH:MM, yearly:MM:DD:HH:MM)
- HA inbound webhook tools toggle: orchestrator loop vs. direct LLM, configurable in UI
- ae_db_query/describe/show_view: SELECT-only Aether MariaDB access (admin, per-user creds)
- /settings/integrations: admin-only page for Aether DB credentials
- Schedules nav link added to all settings pages
- pymysql added to requirements
- Docs updated: HELP.md, MASTER.md, CLAUDE.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 21:06:43 -04:00
Scott Idem
96b3c796c5 feat: file attachment support in chat (images + text/code files)
Text files (.md, .py, .js, .json, etc.): read client-side and injected
into the message body as a fenced code block — works with all backends
with zero model capability requirements.

Images (PNG/JPG/WebP/GIF, max 5 MB): encoded as base64 data URL on the
client and sent as a separate attachment field. Backend formats them as
OpenAI multimodal content (text + image_url) for local_openai backends.
Claude CLI and Gemini CLI see the text message with a "📎 filename.png"
note; image data is never written to session history.

- index.html: 📎 button + hidden file input in mode-select row;
  attachment-row preview area with thumbnail (images) or filename chip
- app.js: _resolveAttachment(), file reader, clearAttachment();
  sendMessage/sendOrchestrate updated to allow no-text sends when a
  file is pending; attachment spread into chat payload for images
- chat.py: Attachment model; attachment field on ChatRequest;
  llm_attachment extracted in _stream_chat and passed to complete()
- llm_client.py: attachment param through complete()/_dispatch()/_local();
  _local() builds multimodal content array for vision calls
- style.css: #attach-btn, #attachment-row, #attachment-preview, thumb

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:46:50 -04:00
Scott Idem
50c1997e91 docs: mark Phase 2/3 done; add file_diff, git tools, spawn_agent restrictions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:34:26 -04:00
Scott Idem
3716e5974f feat: Phase 3 model toggle — cycle chat-role slot models in UI
Replaces the role-cycle toggle with a slot model toggle in the Context &
Memory panel. The active model label is shown on the button; clicking cycles
through Primary → Backup 1 → Backup 2 slots configured for the Chat role.

- app.js: remove activeRole()/availableRoles role-cycling; add
  activeChatModel()/chatModels slot cycling; update send/orchestrate
  payloads to send slot + chat_role:"chat"; fix updateSendBtnTitle and
  startRunTimer to use activeChatModel()
- chat.py: add slot field to ChatRequest; pass slot= to complete();
  resolve backend_label from slot config; add _chat_slot_models() helper;
  include chat_models in GET /backend response
- HELP.md: update Model toggle description, tool count (62/16),
  Backends section, API chat payload example

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:32:43 -04:00
Scott Idem
85e13314a2 feat: add confirm step to message-level delete
Clicking del now shows 'confirm delete / cancel' inline in the action
bar. Cancel rebuilds the original buttons; confirm proceeds as before.
Matches the session delete pattern added in the prior commit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:20:13 -04:00
Scott Idem
20f3fe4f71 docs: mark completed items in TODO__Agents.md
task_list priority filter, session delete confirm, spawn_agent tool
restrictions, HA API tools/config/confirm-required, file_diff,
git_status/log/diff — all completed 2026-05-12.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:17:42 -04:00
Scott Idem
f336ae9687 feat: task_list priority filter, session delete confirm, spawn_agent tool restrictions
- task_list: add priority param ('low'/'normal'/'high') alongside existing status filter
- Session delete: inline confirm row (Delete / Cancel) instead of immediate delete
- spawn_agent: allow_tools and deny_tools per-call params; role config remains ceiling;
  deny_tools falls back to confirm_deny gate when no explicit tool_list is set

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 21:09:50 -04:00
Scott Idem
76fef827c5 docs: update tool count to 62 and current state to 2026-05-12
Reflects file_diff and git_status/log/diff additions, pg.css refactor,
and reasoning level controls added this session.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 00:14:12 -04:00
Scott Idem
b7144d5903 feat: add git_status, git_log, git_diff orchestrator tools
Read-only wrappers around git commands, project-scoped. Covers working
tree status, commit history browsing (with optional path filter), and
diffs between refs or the working tree — cleaner than shell_exec for
code review and change verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 00:12:00 -04:00
Scott Idem
3c9b8f5909 feat: add file_diff orchestrator tool
Runs diff -u on two project-scoped files. Low risk, no admin required.
Covers code review, config comparison, and before/after verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 00:08:36 -04:00
Scott Idem
54eef73b74 docs: add spawn_agent per-invocation tool restriction design (ARCH__FUTURE §12)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:23:17 -04:00
Scott Idem
4c3d9a7a65 refactor: extract shared settings stylesheet (pg.css) and clean up inline styles
- Create cortex/static/pg.css with shared variables, nav, containers, form
  elements, button classes, text utilities, and feedback messages
- All four settings pages (settings, notifications, tools, help) now link to
  pg.css and have page-specific-only <style> blocks
- Style block line counts reduced: settings 244→70, notifications 189→42,
  tools 126→88, help 122→75
- Inline style= attributes reduced: settings 45→4, notifications 7→3,
  tools 12→4, help 0
- Introduced shared CSS classes: .btn-submit, .btn-save, .btn-cancel,
  .btn-secondary, .action-link, .hint, .section-note, .page-title,
  .page-subtitle, .page-wrap, .usage-table, .tool-cat-row
- Fix tools_settings.py: replace server-generated inline styles on category
  header rows with .tool-cat-row CSS class
- Fix settings.py: add btn-save/btn-cancel/persona-rename-cancel classes to
  server-rendered persona rename form buttons

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:22:59 -04:00
Scott Idem
8ab1942514 refactor: migrate Tool Permissions from Settings to /settings/tools
- Remove Tool Permissions form from settings.html; replace with a
  "Tool Settings →" link that redirects to /settings/tools
- Add Confirmation Gate section to tools_settings.html (allow/deny
  textareas) inside the same form as risk policy — one save covers all
- tools_settings.py save handler now writes allow/deny alongside
  max_risk/whitelist/blacklist into tool_policy.json
- Remove /settings/tool-policy POST route from settings.py (no longer needed)
- Remove get_tool_policy, save_tool_policy, CONFIRM_REQUIRED imports
  from settings.py (now owned by tools_settings.py)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:50:48 -04:00
Scott Idem
69ec2f667d feat: tool risk policy UI + wiring through all orchestrators
- New /settings/tools page: max_risk selector (low/medium/high) + per-tool
  override dropdowns (Default / Force include / Force exclude) for all 58 tools
  grouped by category with color-coded risk badges; JS updates Auto status live
- get_tools_for_role() + get_openai_tools_for_role() now accept max_risk,
  whitelist, blacklist; _apply_risk_policy() handles the filtering logic
- get_risk_policy() helper in auth_utils reads from tool_policy.json
- Risk policy wired through orchestrator.py, openai_orchestrator.py,
  orchestrator_engine.py, nextcloud_talk.py, homeassistant.py
- Tools nav link added to settings.html and notifications.html
- CLAUDE.md and ARCH__SYSTEM.md updated: tool count 50→58, risk system docs,
  tool access control three-layer model documented

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:45:04 -04:00
Scott Idem
c9c1ca7de6 feat: TOOL_RISK ratings for all 58 orchestrator tools
Add informational security risk ratings (low/medium/high) to every tool.
Groundwork for future auto-allow tiers (max_risk + whitelist + blacklist).
Breakdown: 36 low, 12 medium, 10 high.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:32:22 -04:00
Scott Idem
ac06b3bc7b feat: project-scoped file tools — grep, stat, syntax_check, offset reads
Add five project-scoped tools (user-level, no admin required):
  project_file_read — read with 1-based offset for paging large files
  project_file_list — list with sizes + timestamps
  file_stat         — size, modified time, line count / entry count
  file_grep         — regex search with context lines, up to 50 matches
  file_syntax_check — py_compile (.py) or json.loads (.json)

Also add offset support to existing file_read (system scope).
Rename "Files" tool category to "System Files"; add "Project Files" category.
Project scope restricted to Cortex_and_Inara_dev/ project root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:23:50 -04:00
Scott Idem
fc6600c33e feat: Home Assistant API tools (ha_get_state, ha_get_states, ha_call_service)
Register three HA orchestrator tools so Inara can read device states and
control devices via the HA REST API. ha_call_service requires admin role
and user confirmation. Also includes accumulated UI fixes (setProcessing
helper, wasNewSession flag cleanup).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:39:35 -04:00
Scott Idem
ba91de37c5 feat: Home Assistant settings UI + fix channels.json
notifications.html: add Home Assistant section with two collapsible
blocks — Connection (HA URL + Long-Lived Access Token) and Inbound
webhook (webhook ID with endpoint URL hint showing the username).
Token field uses keep-existing pattern (blank = no change).

settings.py: wire ha_url, ha_token, ha_webhook_id through
_notifications_page() template substitution and save_notifications()
POST handler. Preserves existing HA config fields (persona, tier,
role, tools) on save.

TODO__Agents.md: add Home Assistant integration planning section
(event design, richer payload template, HA API tools).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:18:45 -04:00
Scott Idem
1d361fe809 feat: NCT orchestrator support + Home Assistant webhook
nextcloud_talk.py:
- Fix missing import hmac / import hashlib (NameError bug in _verify_signature)
- Add orchestrator routing when channels.json "tools": true — sends
  " Working on it…" immediately, then runs the full tool loop and
  replies with the result; checkpoint case gets a web UI confirmation note
- Read tier and role from channel config (defaults: default_tier / "chat")
- Pass cfg through to _process_message

homeassistant.py (new):
- POST /webhook/ha/{username}/{webhook_id}
- Auth: webhook_id path segment matched against channels.json
- Accepts JSON or form-encoded body from HA automations
- Builds natural-language task from payload (uses "message" key if present,
  otherwise serialises full body as context)
- Same orchestrator/direct dispatch as NCT
- Delivers response via notify() — NC Talk, web push, or configured channel
- Session key: ha_{username} for continuity across HA events
- Registered in main.py; /webhook/ prefix already public in auth_middleware

channels.json schema addition:
  "homeassistant": {
    "webhook_id": "your-secret-id",
    "persona": "inara",
    "tier": 2,
    "role": "chat",
    "tools": false
  }

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 19:45:59 -04:00
Scott Idem
19d6f004ed feat: reasoning level select (Off/Light/Moderate/High/Max)
Replace free-form reasoning_budget_tokens number input with a 5-level
select in both the edit form (local_llm.py) and add-model form
(local_llm.html). Values: 0 / 1024 / 4096 / 8192 / 32768 tokens.
Edit form pre-selects the stored value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:44:20 -04:00
Scott Idem
a66c5a7f84 feat: reasoning token budget + session name in header
- reasoning_budget_tokens: optional int field on local_openai models;
  when set, injects {"reasoning": {"budget_tokens": N}} via extra_body
  into every OpenRouter API call (both tool-loop and confirmation-gate
  rounds). Field exposed in the model edit form in Settings.

- session name moved from standalone full-row div between #messages
  and #input-area into the persona-switcher block in the header, as a
  third dim line under "Cortex · Local". Collapses when empty via
  :empty CSS. No JS changes required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:35:23 -04:00
Scott Idem
85792a7bcf feat: per-role inject_mode, OTR fixes, hover metadata, send/stop tooltip
- inject_mode: per-role toggle (parallel to inject_datetime) gates the
  "Current mode: Off The Record" line in the system prompt; wired through
  model_registry, context_loader, chat router, orchestrator router, and
  local_llm settings UI

- OTR orchestrator fix: OrchestrateRequest now carries off_record;
  _finalize_job stores it per message and gates log_turn on it; JS
  orchestrate payload sends off_record correctly

- Per-message hover metadata: removed always-visible .model-tag; replaced
  with .msg-meta strip in the action bar (hover-only); shows model label,
  host, fallback indicator, and OTR badge; stored in session JSON

- Send/stop button tooltip: shows role + model and (when tools on)
  separate orchestrator model + engine label; live elapsed timer on stop
  button via startRunTimer/stopRunTimer

- OrchestratorResult.backend_label: new field; openai_orchestrator fills
  it; finalize_job propagates it to job dict and session messages

- GET /backend: exposes orchestrator_model label so the frontend tooltip
  can show both models separately

- TODO: session delete confirmation added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 16:12:03 -04:00
Scott Idem
0afa135ce9 docs: document System block and OTR mode in ARCH__PERSONA.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:40:21 -04:00
Scott Idem
128d8a7c1e feat: inject session mode into persona system prompt
context_loader.load_context() now accepts a mode param ("chat"|"otr").
In OTR mode, the --- System --- block gains a second line:

  Current mode: Off The Record — this conversation is private
  and will not be logged or included in memory distillation

routers/chat.py passes mode="otr" when req.off_record is True.
Normal chat and all orchestrator calls stay at mode="chat" (no change
to the System block). The System block consolidates date/time and mode
in one place, matching the existing timestamp pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:35:09 -04:00
Scott Idem
3a4f518300 docs: add SELF_UPDATE.md — agent self-maintenance bootstrap
Short reference covering: git repo location, Syncthing fleet sync,
ignore files (.gitignore / .stignore), helper scripts (install.py,
dev-restart.sh, backup.sh), standard change workflow, doc update
checklist, pip dependency process, and key paths on the service host.
Linked from MASTER.md document map.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:00:28 -04:00
Scott Idem
348ca120c1 feat: full channels.json UI + http_allowlist settings
Notifications page:
- NC Talk section expanded: url, bot_secret, notification_room,
  nc_username, nc_app_password — all fields from channels.json now editable
- Per-channel sections use <details>/<summary> collapsibles; auto-open
  when values are present
- Secrets use type=password with "leave blank to keep" semantics
- Google Chat outbound webhook in its own collapsible section

Account settings:
- HTTP POST Allowlist section added (same textarea pattern as email allowlist)
- POST /settings/http-allowlist route saves home/{user}/http_allowlist.json
- Example placeholder shows ha.dgrzone.com and n8n patterns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:57:18 -04:00
Scott Idem
7b443b40a4 feat: http_post tool, nc_talk_history tool, local orchestrator retry
- http_post: POST to external URLs with per-user URL prefix allowlist
  (home/{user}/http_allowlist.json); admin-only, confirm-required
- nc_talk_history: read recent NC Talk messages via Basic Auth (requires
  nc_username + nc_app_password in channels.json under nextcloud)
- openai_orchestrator: _chat_with_retry() wraps both API calls with
  exponential backoff (3 attempts, 1s/2s) on connection errors and
  transient status codes (429, 500, 502, 503, 504)
- Docs updated: CLAUDE.md, HELP.md, TODO, MASTER, ROADMAP (50 tools)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:38:38 -04:00
Scott Idem
b9a78819ac docs: add LLM wiki concept (Karpathy pattern) to ARCH__FUTURE.md
Inara's exploration of a living-wiki knowledge compilation architecture
as an alternative to RAG — three-layer model, ingest/query/lint ops,
and a mapping to existing Cortex concepts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:22:55 -04:00
Scott Idem
3672fa1506 docs: comprehensive doc audit — sync all docs to current state
- MASTER.md: tool count 40→47, add proactive notifications + spawn_agent rows, date bump
- ROADMAP.md: mark local orchestrator/web push/proactive notifs/spawn_agent/web_read/session_read as done, date bump
- ARCH__CHANNELS.md: rewrite notification channel config section — all 4 channels, all triggers, on-demand endpoints
- ARCH__SYSTEM.md: update tools/ module list to include files, agents
- README.md: update LLM backends in architecture diagram, add browser push to channels table
- CLAUDE.md: add doc update checklist to Documentation Philosophy section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:13:45 -04:00
Scott Idem
52c19afbcc fix: raise web_read and http_fetch max_chars cap to 128K
Both tools now accept max_chars up to 131072 to accommodate long
documentation pages and large API responses.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:08:17 -04:00
Scott Idem
17e8869d12 docs: update tool count (45→47), HELP.md, and TODO for new web/file tools
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:05:04 -04:00
Scott Idem
7c3291960a feat: web_read (trafilatura), session_read, http_fetch max_chars
web_read(url, max_chars=16000) — fetches a URL and extracts clean article
text via trafilatura, stripping ads/nav/boilerplate. Returns markdown.

session_read(date) — reads a full session log by YYYY-MM-DD date; lists
available dates if the requested one is not found.

http_fetch gains a max_chars param (default 8192, max 32768) so the cap
is configurable instead of hardcoded.

Tool count: 45 → 47.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:04:24 -04:00
Scott Idem
a99ebb8c30 feat: retry button for orchestrator errors + explicit client timeout
Extract orchestrator inner loop into _doOrchestrate() so the retry button
can re-run without re-adding the user message to DOM or history — same
pattern as the existing chat retry.

Also set AsyncOpenAI(timeout=settings.timeout_local) so slow remote models
(OpenRouter/DeepSeek) get the same 300s budget as local chat calls instead
of the SDK default which varies by connection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 12:39:34 -04:00
Scott Idem
ff154b1ec0 docs: update CLAUDE.md, HELP.md, and TODO for notifications page + push fix
- CLAUDE.md: date → 2026-05-08, add Proactive notifications row to channel table
- HELP.md: update Notifications settings entry, expand Push Notifications section
  with channel config link, add test API endpoints to reference table
- TODO__Agents.md: mark notifications dedicated page and pywebpush fix as done

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:58:47 -04:00
Scott Idem
c21f9a23ec fix: use Vapid.from_pem() instead of passing PEM string to webpush()
pywebpush 2.x routes string keys through Vapid.from_string() which only
handles raw/DER base64 — not PEM. Pre-build the Vapid object so the key
deserializes correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:56:17 -04:00
Scott Idem
19475610be feat: move Notifications to its own settings sub-page
Adds GET /settings/notifications (dedicated page with channel form + two
test buttons) and updates POST /settings/notifications to render that page.
Settings page now shows a compact link card instead of the full form.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:43:52 -04:00
Scott Idem
3c7ecf4e4f feat: notification test endpoints — POST /api/push/test and /api/push/reminders/check
- POST /api/push/test: sends "Test notification from Cortex" via the
  user's configured notification channel (web_push / NCT / email / etc.)
- POST /api/push/reminders/check: runs the daily reminder check immediately
  for the current user, returns reminders_found count

Both require an active session cookie. Useful for verifying channel setup
without waiting for the 09:00 scheduler job.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:34:58 -04:00
Scott Idem
64020ad982 feat: proactive notifications — web_push channel + daily reminder check
Routes web_push through notification.py alongside NCT/email/Google Chat,
and fires daily reminder summaries via the scheduler.

- notification.py: _notify_web_push() + "web_push" case in notify();
  all four channels (web_push/email/nextcloud/google_chat) now routable
- scheduler.py: _run_reminder_check() daily at 09:00 — reads due reminders
  per persona via set_context(), formats up to 3 entries, calls notify()
- routers/settings.py: "web_push" added to valid notification_channel values
- static/settings.html: "Browser Push Notification" option in channel selector
- TODO__Agents.md: proactive notifications section marked complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:28:49 -04:00
Scott Idem
47d23a7b2f feat: per-model max_rounds for Gemini orchestrator engine
Mirrors the pattern already in openai_orchestrator.py. The Gemini engine
was still hardcoded to the global orchestrator_max_rounds setting.

- orchestrator_engine.py: max_rounds param on run() and _run_from_contents();
  effective_limit = min(per_model_limit, global_limit); stored in checkpoint
  so resume() respects it across confirmation gates
- routers/orchestrator.py: passes orch_model.get("max_rounds") to run()
- tools/agents.py: passes model_cfg.get("max_rounds") for gemini_api spawns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:54:37 -04:00
Scott Idem
09d775b47b feat: spawn_agent tool + host max_concurrent + docs
Adds a synchronous sub-agent spawning tool that lets the orchestrator
delegate tasks to a specific role's model and tool set.

- cortex/tools/agents.py: spawn_agent(task, role, tier, timeout, max_rounds)
  - Supports local_openai and gemini_api model types
  - Per-host asyncio semaphore (keyed by host_id or model type)
  - asyncio.wait_for() enforces timeout; admin-only tool
- cortex/model_registry.py: max_concurrent field in host schema (default 3,
  clamped 1-20); backfilled on _normalize() for existing hosts
- cortex/routers/local_llm.py + local_llm.html: "Max parallel" number input
  in host add/edit forms
- cortex/tools/__init__.py: spawn_agent registered in TOOL_CATEGORIES["Agents"],
  _CALLABLES, TOOL_ROLES (admin), and _ALL_DECLARATIONS
- Docs: TOOLS.md count 44→45, spawn_agent section; HELP.md tool table updated;
  ARCH__FUTURE.md Round 2 completed items; TODO__Agents.md spawn_agent checked;
  CLAUDE.md tool count and list updated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:48:21 -04:00
Scott Idem
6ad7597db8 feat: per-role inject_datetime toggle for system prompt
Each role can now disable the current date/time header injected into the
system prompt. Default is true (all existing roles unchanged). Useful for
pure processing roles (summarizer, classifier, translator) where temporal
context is irrelevant or could cause unexpected model behavior.

Changes:
- model_registry: set_role_config/get_role_config gain inject_datetime field
- context_loader: load_context gains inject_datetime param (default True)
- orchestrator router: passes inject_datetime from role_cfg to load_context
- local_llm router: reads inject_datetime from POST body, passes to registry;
  role_config_data_js includes the field
- local_llm.html: checkbox in role config panel; populate on open, save on submit

Session logs still timestamp every turn (HH:MM header in YYYY-MM-DD.md files)
regardless of this setting — the toggle only affects the system prompt header.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:53:35 -04:00
Scott Idem
8e512d4e11 feat: reminders due-date support + context filtering
reminders_add now accepts optional due: YYYY-MM-DD parameter.
Due date stored as first line of section body in REMINDERS.md.

context_loader.py calls load_due_reminders() instead of loading REMINDERS.md
wholesale — future-dated reminders are suppressed in the system prompt until
their date arrives. Undated reminders always surface (backward compatible).

reminders_list shows due status per entry: [OVERDUE by N days], [due TODAY],
or [due: YYYY-MM-DD] for future items. All reminders visible via the tool
regardless of date; only context surfacing is filtered.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:46:45 -04:00
Scott Idem
750cde489d feat: session_search tool + tool expansion docs update
session_search (tools/files.py):
- Full-text search across past session logs, exposed to the orchestrator
- Params: query (required), limit (default 5, max 20)
- Returns dated excerpts, newest first; own sessions only via ContextVars
- User-level — no TOOL_ROLES gating needed
- Registered in __init__.py callables + TOOL_CATEGORIES["Files"]

ARCH__FUTURE.md §2: updated tool count to 44, marked prior tools complete,
added Round 2 planned tools table (session_search now done, reminders due dates,
http_post, nc_talk_history, task_list priority filter, http_fetch max_chars),
noted datetime_now is not needed (already in system prompt via context_loader)

TODO__Agents.md: session_search checked off, Round 2 task list added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:41:26 -04:00
Scott Idem
f8f7cd75da feat: audit log, usage tracking UI, OpenAI orchestrator compaction, onboarding + docs
Tool audit log:
- Every orchestrator tool call logged to home/{user}/tool_audit/YYYY-MM-DD.jsonl
- Files panel sidebar: audit log group (collapsed), date-linked read-only table
- Admin endpoints: /api/audit/files, /api/audit/day, /api/audit/recent, /api/audit/stats
- Engine and model name recorded per entry

OpenAI orchestrator improvements:
- Context budget enforcement: 75% of model context_k (min 16k)
- Message compaction: truncates old tool results when approaching budget
- max_rounds respected per model config (intersected with server cap)

OpenRouter onboarding (setup.html, onboarding.py, app.js, settings.html):
- Step 3 of 3: /setup/model with curated model picker
- Chat banner for users on server-default model (informational, not alarmist)
- Settings quick-link card; /setup/model works standalone for existing users

Model registry + session store:
- set_role_config / get_role_config for per-role tool lists and system_append
- session_store: session rename, session name backfill endpoint

UI updates (app.js, index.html, style.css, local_llm.html):
- Role toggle in context panel
- Off-the-record mode
- Agent notes read-only viewer
- OPERATIONS.md loaded at T2+ in context

Documentation:
- HELP.md: full tool table, per-role tool sets, Agent Notes, usage tracking
- TOOLS.md: Agent Notes section, count corrected to 44
- ARCH__SYSTEM.md, ARCH__BACKENDS.md, MASTER.md updated to match reality
- CLAUDE.md: onboarding flow, documentation philosophy sections
- README.md: stack in practice, DeepSeek TUI mention, architecture diagram updated
- TODO__Agents.md: onboarding task completed with deviation notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:26:43 -04:00
Scott Idem
c02d2462b0 feat: agent notes, OpenRouter onboarding, usage tracking, per-role tools docs
Agent notes tool (cortex/tools/agent_notes.py):
- Private durable notepad for the orchestrator — not user-visible
- agent_notes_read/write/append/clear with 3 rolling backups
- Per-persona isolation via ContextVars; no TOOL_ROLES gating needed
- PROTOCOLS.md updated to make this a core proactive tool

OpenRouter guided onboarding:
- Setup Step 3 (/setup/model) — OpenRouter quick-connect with curated model list
- Amber banner in chat for users on server-default model
- Settings quick-link card (/settings/models OpenRouter section)
- POST /setup/model/skip for users who want to bypass Step 3
- Holly pre-configured: DeepSeek V4 Flash (OpenRouter) → Gemma Medium (local) → claude_cli

Usage tracking:
- cortex/routers/usage.py — GET /api/usage, /api/usage/summary, /api/usage/all (admin)

Documentation:
- HELP.md: Tools section rewritten — full tool table by category, per-role tool sets explained
- TOOLS.md: Agent Notes section added; count corrected to 44
- ARCH__SYSTEM.md, ARCH__BACKENDS.md, MASTER.md, CLAUDE.md, README.md updated
- TODO__Agents.md: onboarding task checked off with deviation notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:25:31 -04:00
Scott Idem
5d4f5ee598 feat: OPERATIONS.md bootstrap doc + load at T2+; patch stale persona files
- New home/scott/persona/inara/OPERATIONS.md: self-maintenance workflow
  (cortex_update → review → cortex_restart), access control table, key
  paths, memory file map, distillation cadence, channel/architecture notes
- context_loader.py: load OPERATIONS.md at Tier 2+ after PROTOCOLS.md
- TOOLS.md: count 39→40, add web_push to Notifications section
- PROTOCOLS.md: replace stale 10-tool list with reference to TOOLS.md
- CONTEXT_TIERS.md: fix memory file names (MEMORY.md → LONG/MID/SHORT),
  update Tier 2 load list, fix Hard Rules credentials note

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 22:21:03 -04:00
Scott Idem
a75546485b feat: context budget enforcement + compaction in OpenAI orchestrator
Protects all models in the Primary/Backup chain regardless of context window:
- _context_budget(): 75% of model_cfg["context_k"] * 1000 (default 32k if unset)
- _estimate_tokens(): char count / 4 + 3k overhead for tool schemas
- _compact_messages(): truncates old tool results to 400 chars, keeps last 6
  intact (~2 recent rounds), logs chars saved per compaction pass
- Compaction runs before every API call; log line now shows estimated token count
- Malformed tool call args logged with model/args detail instead of silent {}
- finish_reason check accepts "stop" and None alongside "tool_calls" (some
  models return wrong reason even when tool_calls are present)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 22:01:54 -04:00
Scott Idem
7d221863dc feat: engine/model in audit log + docs update
- tool_audit: ContextVars (engine, model) set at orchestrator run start; fields added to every entry
- orchestrator_engine: tool_audit.set_context("gemini", model_name) at run() start
- openai_orchestrator: tool_audit.set_context("openai", model label) at run() start
- audit table: Model column between Status and Args
- HELP.md: push notifications section, audit log in Files section, tool count 30→40, new API endpoints
- TODO__Agents.md: web_push and audit log marked complete with full detail

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:42:32 -04:00
Scott Idem
02accefe8f feat: audit log in Files panel sidebar
Adds an "Audit Log" section (collapsed by default) at the bottom of the Files
panel showing tool_audit/YYYY-MM-DD.jsonl files for the current user.

- GET /api/audit/files  — lists available dates (newest first, any auth user)
- GET /api/audit/day    — returns entries for one date as JSON (any auth user)
- tool_audit.read_day() — reads a single day's JSONL file chronologically
- Clicking a date renders a read-only table: time / tool / status / args / result
- Status cells are colour-coded (green ok, red error, amber denied)
- Edit/Raw/Preview/Save buttons are hidden in audit view, restored on file switch
- Audit group starts collapsed; expands on click like other file groups

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:36:08 -04:00
Scott Idem
584ae679a6 feat: tool call audit log
Every orchestrator tool invocation is recorded to home/{user}/tool_audit/YYYY-MM-DD.jsonl.
Each entry captures: timestamp, user, tool, args (truncated), status (ok/error/denied),
result length, and a 300-char result snippet.

- tool_audit.py: JSONL writer with per-file asyncio locks; read_recent / read_recent_all_users helpers
- tools/__init__.py: hook in call_tool() — fire-and-forget record on every dispatch
- routers/audit.py: GET /api/audit/recent and /api/audit/stats (admin-only)
- tools/files.py: add home_root() to file_read allowed roots so agents can read audit JSONL

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:55:59 -04:00
Scott Idem
ddf44a2aee feat: web push notifications (VAPID)
- push_utils.py: subscription storage + send helper (auto-prunes 410 endpoints)
- routers/push.py: GET /api/push/vapid-key (public), POST/DELETE /api/push/subscribe
- sw.js: push event listener shows notification; notificationclick focuses/opens tab
- app.js: subscribe/unsubscribe flow + "Enable notifications" toggle in settings dropdown
- tools/notify.py: web_push orchestrator tool (user-level, no admin required)
- VAPID keys in .env; pywebpush added to requirements.txt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:38:58 -04:00
Scott Idem
0b96772fa6 fix: show session friendly name in resume message and status bar
/history/{session_id} now returns a 'name' field alongside messages.
resumeSession() uses data.name first, then the sessionNames map, then
raw ID as fallback — so named sessions display correctly even on page
load before the sessions panel has been opened.

'Resumed session X' message also now shows the friendly name.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:14:59 -04:00
Scott Idem
5d23d04e7e fix: session panel wider + two-line layout for session names
Root cause: 300px panel minus edit btn (28px) + meta (~130px) + delete
btn (28px) + gaps/padding left only ~70px (~7 chars) for the session name.

- Panel: 300px → 420px desktop, 300px → 380px mobile drawer
- Max-height: 340px → 400px
- Session item: name and meta now in a .session-body flex column, so the
  name gets full body width (panel minus two buttons) — meta lives below
- Edit mode: hides .session-body + delete, input takes the full body slot

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:07:33 -04:00
Scott Idem
7a0fbdb659 feat: session rename UX overhaul
- Edit button (✎) moved to left of row, separated from delete (×)
- Clicking ✎ hides name/meta/delete and expands input to full row width
- Button changes to ✓ (accent color) while editing
- Enter or ✓ click = save; Escape = cancel without saving
- Removed accidental-save-on-blur behavior
- Edit button: 30% opacity at rest, 75% on row hover, 100% on direct hover
- Touch devices: edit button always at 60% opacity (no hover to reveal it)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:00:39 -04:00
Scott Idem
508fb638ad feat: distill safeguards — rolling backups + sanity checks
Before any memory file is overwritten, _rotate_backup() keeps 2 rolling
backups: MEMORY_*.bak1.md (most recent) and MEMORY_*.bak2.md (older).

_sanity_check() now also guards against size anomalies: the new content
must be between 40% and 250% of the old file size — anything outside that
range looks like truncation or runaway output and aborts the write.
Existing checks (min length, refusal phrases) still apply.

Backup files exposed in the Files panel (ALLOWED set) so they can be
reviewed and manually restored if needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:54:27 -04:00
Scott Idem
0ffcd57c95 fix: multi-user distillation + datetime in context + session log labels
Distillation was silently operating on scott/inara for all users due to
ContextVar defaults. All three distill endpoints now require ?user=&persona=
query params and validate them via persona.validate(). Memory distiller
signatures changed from Optional to required positional args — no more
global settings fallback. Scheduler now iterates all users/personas instead
of hardcoding the primary user.

- context_loader: inject current date/time as first system prompt section
- session_logger: use get_user()/get_persona() from context instead of
  settings globals so Holly/Brian sessions show correct speaker labels
- memory_distiller: system prompts now reference u.title()/p.title()
  instead of settings.user_name/settings.agent_name
- distill router: Query(...) enforces params; _resolve() validates persona
- scheduler: _all_personas() helper iterates every user/persona for distill
- app.js: runDistill() now appends ?user=&persona= via _fileParams

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:44:51 -04:00
Scott Idem
8d4aa4094c feat: usage tracking + knowledge import script
- usage_tracker.py: daily token/call buckets per user (home/{user}/usage.json)
- Hook into local backend (OpenAI usage field) and Gemini API (usage_metadata)
- Claude/Gemini CLI backends produce no structured token data and are not tracked
- Fix CLAUDE.md stale tool count (27 → 39) and refresh tool list
- scripts/import_knowledge.py: walk markdown dirs, chunk by H2, call local LLM
  for summaries, create AE journal entries with path-derived tags; resumable via
  state file; --dry-run and --limit flags for safe testing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 20:38:31 -04:00
Scott Idem
eab92d876d refactor: split tool declarations into domain files + role config UI
tools/__init__.py shrinks from 1,137 → 250 lines. Each domain file now
owns both its callables and its FunctionDeclarations (DECLARATIONS list),
so adding a new tool only touches one file.

New TOOL_CATEGORIES dict exported from __init__ — used by the UI for
grouped tool checkboxes.

Role config UI (Settings → Model Registry → Role Assignments):
- ⚙ button per role expands an inline configure panel
- Textarea for system_append (injected into system prompt for this role)
- Grouped checkboxes for tool allow-list (all checked = no restriction)
- POST /api/models/role-config saves both fields; updates ROLE_CONFIG_DATA
  in-page so re-open reflects current state without a page reload

Backend:
- model_registry.set_role_config() writes system_append + tools to registry
- TOOL_CATEGORIES exported from tools/__init__ for UI rendering
- TOOLS.md header updated: 30 → 39 tools (ae_journal_* and cortex_* additions)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 20:40:50 -04:00
Scott Idem
49123cdd5c feat: per-role tool lists and system prompt overlays
Each role in model_registry.json can now carry two optional keys:
  system_append — injected into the system prompt at position 7 (after
                  memory, closest to the turn) for the active chat_role
  tools         — explicit tool allow-list; intersected with the user's
                  access-level filter so it can only restrict, never elevate

No changes needed for existing users — missing keys fall back to current
behavior. Add keys to a role to give it a specialty focus:

  "coder": {
    "primary": "claude_cli",
    "system_append": "You are in code-specialist mode...",
    "tools": ["web_search", "file_read", "shell_exec", "scratch_write"]
  }

Changes:
- model_registry.py: get_role_config() returns system_append + tools
- context_loader.py: role_append param appended as "--- Role Context ---"
- tools/__init__.py: get_tools_for_role/get_openai_tools_for_role accept
  optional tool_list and intersect with access-level filter
- orchestrator_engine.py: tool_list threaded through run/resume/checkpoint
- openai_orchestrator.py: tool_list threaded through run/resume/checkpoint;
  _build_client now calls get_openai_tools_for_role instead of returning
  unfiltered OPENAI_TOOL_SCHEMAS
- routers/orchestrator.py: pulls role_cfg for chat_role, passes both
  role_append and tool_list to context loader and engine

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 20:00:38 -04:00
Scott Idem
5ad2e50d69 feat: split help into tabbed UI Guide / Tools / Persona pages
- cortex/static/TOOLS.md — tool reference extracted from HELP.md; uses ##
  headers so each category is collapsible. All 30 tools with descriptions.
- cortex/static/HELP.md — UI guide only; tools section replaced with a
  one-line pointer to the Tools tab.
- help.html — three tabs (UI Guide / Tools / Persona); tab choice persists
  in localStorage. Tools tab defaults all sections open. Persona tab shows
  home/{user}/persona/{name}/HELP.md with an empty-state message if unset.
- context_loader.py — loads cortex/static/TOOLS.md into context at tier 2+
  (replaces the previously empty persona HELP.md load). Persona HELP.md
  still loaded if non-empty, as persona-specific additions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:25:47 -04:00
Scott Idem
552fd56abb docs: expand tools section in HELP.md with per-tool descriptions
Replaces single flat table with category sections, each with a description
column. Footnotes moved to the top of the section for clarity. Covers all
30 tools including the new cortex_status, cortex_update, reminders_remove,
ae_journal_entry_read, ae_journal_entries_list, and email_send.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:11:22 -04:00
Scott Idem
77997bc4ae feat: add cortex_status and cortex_update tools
cortex_status: git branch/commit/ahead-behind + systemctl state — read-only
cortex_update: git pull + syntax check all .py files + report; does NOT auto-restart.
  If syntax errors are found after pull, warns and blocks restart suggestion.
  Call cortex_restart separately to apply a clean update.

Both are admin-only. cortex_update is confirm-required (modifies files on disk).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:01:42 -04:00
Scott Idem
1ffa846edd docs: sync HELP.md tools table and files list with current implementation
- Add reminders_remove (targeted single-reminder removal, no confirm needed)
- Add ae_journal_entry_read, ae_journal_entries_list to AE Journals row
- Add email_send (admin-only) to Notifications row
- Remove TASKS.json from Files table (not in the Files panel)
- Add email_allowlist.json to Files table (Settings group in Files panel)
- Update last-updated date

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:54:50 -04:00
Scott Idem
98546abe21 docs: update ARCH__AE_INTEGRATION with verified API behavior
- query_string required for and/or filters to apply; use "%" as wildcard
- Total count is in meta.data_list_count, not top-level
- id_random is None in responses; Vision ID convention uses {obj_type}_id
- tags comes back as string on read, not list — normalize before joining
- Replace stale "Planned: Search Improvements" with current signature + notes
- Clarify date_to boundary (lte midnight, use next day to include full day)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:17:19 -04:00
Scott Idem
1fa5151d8a fix: correct V3 search filter key and response field names in ae_knowledge
- Filter key is "and" not "and_filters" (V3 API format)
- Entry IDs use journal_entry_id/id, not id_random (id_random is None)
- Dates use updated_on/created_on, not updated_at/created_at
- Total count lives in meta.data_list_count, not top-level total/count
- Inject query_string="%" when and filters present but no query, since
  the V3 search engine requires query_string for filters to apply
- Normalize tags from string to list in both entry_read and entries_list
- Fix order_by to use updated_on (not updated_at) in entries_list
- Correct ARCH__AE_INTEGRATION.md: and_filters → and, or_filters → or

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:12:44 -04:00
Scott Idem
71e472bebe feat: improved ae_journal_search + AE integration docs
Search improvements:
- Switched from LIKE on default_qry_str to query_string path (fulltext
  MATCH/AGAINST IN BOOLEAN MODE — uses the index, supports +/- boolean ops)
- Added tag filter (icontains on tags field)
- Added date_from / date_to filters (created_on gte/lte)
- Added type_code / topic_code exact-match filters
- Added sort_by / sort_order control (updated, created, name, priority)
- Added status / priority filters
- Added page parameter for pagination
- Richer output: updated date, tags, pagination hint
- Updated Gemini tool declaration with all new params

Docs:
- documentation/ARCH__AE_INTEGRATION.md — journal_entry full schema,
  search operator reference, current tool inventory, planned phases
  (broader AE integration: tasks, people, calendar, knowledge import)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 20:10:04 -04:00
Scott Idem
77327d97ad feat: improve AE Journal read toolset
- ae_journal_entry_read: expose full entry content by id_random (title,
  journal, tags, summary, full content with configurable truncation)
- ae_journal_entries_list: browse all entries in a journal newest-first,
  numbered with id/title/tags/summary/date and pagination support
- ae_journal_search: richer output — tags, updated date, 400-char preview
  (was 200), show summary OR preview (not both when summary exists)

_get_entry() was already implemented; read tool just exposes it properly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:47:59 -04:00
Scott Idem
36fdda6728 feat: add reminders_remove tool for single-reminder removal
- reminders_remove(index) removes one reminder by 1-based index
- reminders_list now returns numbered output (1. heading / body)
  so any model can easily identify which index to pass
- _parse_sections() / _sections_to_text() helpers for clean round-trip
- Not in CONFIRM_REQUIRED — targeted removal is safe without a gate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:27:53 -04:00
Scott Idem
6405dd338d feat: proper confirmation-resume flow + per-user tool policy
Fixes the broken confirmation gate where users had no way to approve
or deny a blocked tool call in the web UI.

Changes:
- orchestrator_engine.py: add OrchestrateCheckpoint dataclass, extract
  loop into _run_from_contents(), add resume() function
- openai_orchestrator.py: same treatment — _run_from_messages(), resume()
- routers/orchestrator.py: POST /{job_id}/confirm and /deny endpoints,
  separate _checkpoints store, _resume_job() + _finalize_job() helpers,
  "awaiting_confirmation" job status with pending_confirmation payload
- auth_utils.py: get_tool_policy() and save_tool_policy() helpers reading
  home/{user}/tool_policy.json (allow/deny lists)
- routers/orchestrator.py: loads tool_policy per user and passes
  confirm_allow/confirm_deny to both engines
- app.js: poll loop handles awaiting_confirmation — shows Confirm/Deny
  buttons inline, resumes polling after user action
- settings.html + settings.py: Tool Permissions section with allow/deny
  textareas, POST /settings/tool-policy route
- style.css: .confirm-gate, .confirm-btn, .deny-btn styles

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 19:14:53 -04:00
Scott Idem
bce7de647c feat: proactive notifications — email, NC Talk, Google Chat per user
notification.py now handles all three outbound channels. Email defaults
to the user's login address (google_email from auth.json); an optional
override can be set in channels.json. Google Chat uses an incoming
webhook URL. NC Talk was already wired, just needs notification_room set.

Settings page gains a Notifications section: channel dropdown, optional
email override, NC room token, and Google Chat webhook URL. All stored
in per-user channels.json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 22:32:22 -04:00
Scott Idem
165cf3552d docs: update TODO and ROADMAP for 2026-04-29 session
Mark completed: email_send + allowlist UI, model inline edit, cross-session
search. Remove completed items from active lists (distill review, model edit,
session search). Update ROADMAP Phase 3 tool suite line and last-updated date.
Remove resolved Unsloth entry from Deferred.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 22:03:50 -04:00
Scott Idem
db3dd465b2 feat: email allowlist management in Settings + Files panel
Settings page gets an editable textarea (POST /settings/email-allowlist)
so users can view and update their per-user regex allowlist without
touching the raw JSON file.

Files panel gains a "Settings" group containing email_allowlist.json as
a raw JSON editor backup — served from home/{user}/ via files.py USER_FILES.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:56:45 -04:00
Scott Idem
e0e3170de3 feat: regex support in email allowlist
Each entry in email_allowlist.json is treated as a re.fullmatch pattern
(case-insensitive). Allows domain wildcards, plus-addressing, and any
variation expressible as a regex. Invalid patterns are logged and skipped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:43:38 -04:00
Scott Idem
b8bc4ea21f feat: email_send allowlist — block sends to non-whitelisted addresses
Reads home/{username}/email_allowlist.json (JSON array of addresses).
Fails safe: if file is missing or address not listed, send is blocked with
an informative message. home/ is gitignored; create the file manually per user.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:40:10 -04:00
Scott Idem
fd0fb76c08 feat: add email_send orchestrator tool
Wraps the existing email_utils.send_email helper as an admin-only tool.
Accepts to, subject, body (plain text); newlines converted to <br> for HTML part.
Registered in _CALLABLES, _ALL_DECLARATIONS, and TOOL_ROLES (admin).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:35:29 -04:00
Scott Idem
a5658eb3c4 feat: edit existing model entries in the Model Registry
- Inline edit form per model row (label, model name/ID, host/account, context, tags)
- Fetch models button in edit form for local models — same live-picker UX as Add Model
- POST /settings/local/models/{id}/edit route in local_llm.py
- Admin role badge (ADMIN/USER pill) in Account Settings page
- HELP.md updated: new tools table with admin/confirm markers, PWA install section
- TODO updated: tool expansions marked done, distill review and Unsloth resolved,
  role-based access and admin badge added to completed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:08:09 -04:00
Scott Idem
334e7f0dea feat: role-based tool access, confirmation gates, and new orchestrator tools
- auth_utils: get_user_role() reads role from auth.json (admin|user, default user)
- manage_passwords: new `role` command to promote/demote users (admin-only by convention)
- tools/__init__: TOOL_ROLES map, CONFIRM_REQUIRED set, get_tools_for_role(),
  get_openai_tools_for_role() — both orchestrators now filter tools by caller's role
- tools/system: cortex_restart (detached subprocess, 5s delay), cortex_logs (admin-only)
- tools/web: http_fetch — direct URL fetch, distinct from web_search
- tools/files: file_list (directory listing), file_write (restricted paths, admin-only)
- tools/notify: nc_talk_send — proactive outbound via notification.py
- orchestrator_engine + openai_orchestrator: user_role param; CONFIRM_REQUIRED tools
  return a confirmation-request result instead of executing — loop breaks after Claude
  asks user to confirm in a follow-up message
- home/scott/auth.json: role set to admin

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:23:53 -04:00
Scott Idem
1603ad5124 docs: sync TODO and ARCH__FUTURE — local orchestrator status, new tools, fleet/mesh plans
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:05:11 -04:00
Scott Idem
25182a1765 feat: PWA support — manifest, service worker, icons, public auth exemption
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:46:33 -04:00
Scott Idem
f726d78979 docs: add five new feature items to TODO (PWA, proactive notifications, attachments, search, usage)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:23:59 -04:00
Scott Idem
217c7c3d6a feat: CodeMirror markdown editor for identity/memory file editor
Replace plain textarea with CodeMirror 5 + markdown mode loaded from
jsDelivr CDN. Editor fills the modal body via flex layout, theme-aware
via CSS vars (cursor, selection, headings, bold/em/links/code all mapped
to Cortex dark/light palette). Lazy init on first file open; history
cleared per-file so undo doesn't bleed across files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 23:05:57 -04:00
Scott Idem
66cb197de0 feat: last-used persona cookie, emoji dropdown, theme support, auth status move
- cx_last_persona cookie set on serve_ui; root/login/help/settings
  redirects use preferred persona from cookie instead of alphabetically first
- /api/personas returns [{name, emoji}] objects; persona switcher dropdown
  renders emoji + name with flex layout and .pd-emoji span
- Help, Settings, Model Registry pages apply localStorage theme on load
  (no flash); CSS variables for dark/light replacing all hardcoded hex values
- Claude CLI auth status moved from prominent chat banner to Anthropic
  provider block in Model Registry — live dot indicator (ok/warn/err)
- Auth banner removed from main chat UI (index.html, app.js, style.css)
- Add Model collapsed into Models section as <details> to shorten page
- Light-mode overrides for provider icons, model badges, ctx-badge, tags
  (Anthropic/Google/local colors now readable in both themes)
- Help page gains table, pre/code, hr styles for HELP.md rendered content

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 22:52:34 -04:00
Scott Idem
1222f806ce docs: sync all documentation to current state (2026-04-28)
HELP.md:
- Replace stale "Agent Mode" section with accurate "Tools ()" section
  including full tool table (27 tools across 9 categories)
- Fix header controls table: distinguish Context & Memory panel (sliders)
  from Settings dropdown (☰)
- Update Settings panel section: "Backend" → "Role", add S/M/L and ⌃↵
  to Display controls list
- Update Backends section to match Role toggle terminology

TODO__Agents.md:
- Mark 5 new journal tools as complete (ae_journal_list, entry_update,
  entry_disable, entry_append, entry_prepend)
- Add completed sections for: shell_exec tool, Tools toggle decoupling,
  UI input area polish (2026-04-28)

MASTER.md:
- Date: 2026-04-27 → 2026-04-28
- Orchestrator row: "Agent mode in UI" → " Tools toggle in UI (27 tools)"

CLAUDE.md:
- Date: 2026-04-03 → 2026-04-28
- Add orchestrator row to Current State table
- Add full 27-tool list for quick reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 22:07:48 -04:00
Scott Idem
ed191cf0b4 feat: add journal entry update, disable, append, prepend tools
Four new tools for full journal entry lifecycle management:

- ae_journal_entry_update  — PATCH any combination of fields (title,
  content, summary, tags, enable); only provided fields are changed
- ae_journal_entry_disable — soft-delete via enable=false
- ae_journal_entry_append  — fetch entry, append timestamped section
  to the bottom (ideal for running logs / data logs)
- ae_journal_entry_prepend — fetch entry, prepend timestamped section
  to the top (most-recent-first pattern)

Shared _get_entry / _patch_entry helpers keep the read-modify-write
logic DRY. Also fixed journal_entry_create to prefer the canonical
journal_entry_id field over the legacy id_random alias.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 22:02:22 -04:00
Scott Idem
44f215c764 feat: add ae_journal_list tool
Lists all Aether Journals for the configured account via
POST /v3/crud/journal/search with no filters (account scoped by header).
Returns name + id_random for each journal so the agent can discover
available journals before searching or writing entries.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:50:02 -04:00
Scott Idem
d61e39d614 feat: S/M/L height drives mode-select row vs column layout
When height is set to S, mode-select collapses to a row (mode button +
compact tools toggle side by side). M and L keep the vertical column
layout where each control gets its own full-width row. Driven by
data-size attribute set in JS so the switch is instant on click, not
reliant on a viewport media query. Removed the redundant max-height
landscape query.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:20:28 -04:00
Scott Idem
93a692f3f0 feat: vertical mode-select column on desktop
Stack Chat/Note/OTR button and tools toggle vertically (flex-direction:
column, align-items: stretch) on desktop so they share a tidy left column.
Mobile (≤520px) restores row layout; landscape phone (≤400px height) also
reverts to row to avoid crowding a short viewport.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:14:16 -04:00
Scott Idem
af4d78136a fix: textarea height setting now visibly changes empty-state size
The previous approach used a 600ms preview animation + syncHeight() which
collapsed the textarea back to 1 line (empty scrollHeight). Now syncHeight
enforces minHeight = maxHeight/3, so each setting (S/M/L) has a visibly
distinct resting height even when the input is empty.

  S (120px): min ~40px  ≈ 1-2 lines at rest
  M (240px): min ~80px  ≈ 3 lines at rest
  L (480px): min ~160px ≈ 5-6 lines at rest

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:08:46 -04:00
Scott Idem
af7d8b40e2 feat: single cycling height button, panel mutual exclusion, consistent shadows
- Replace 3 S/M/L height buttons with one cycling button (like font size)
- Fix closeAllPanels() to include ctx-panel so Context and Settings menus
  cannot be open simultaneously
- Fix ctxOpenBtn handler to use the same toggle-via-closeAllPanels pattern
  as the settings button
- Align .hdr-dropdown shadow to var(--shadow) instead of hardcoded rgba
- Align #ctx-panel z-index to 200 (match .hdr-dropdown)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 21:03:56 -04:00
Scott Idem
4159f470d6 fix: context panel polish — height buttons, amber theme vars, label cleanup
- Replace height <select> with S/M/L buttons (data-height); active class shows
  current setting; clicking an empty textarea briefly expands it as a preview
  so the effect is immediately visible, then auto-shrinks back
- Add --amber/--amber-border/--amber-glow CSS vars to all 4 theme blocks:
  dark=#f59e0b (bright), light=#b45309 (deep, 4:1 contrast on light bg)
  Fixes local-on/tools-toggle/backend-hint being nearly invisible in light mode
- Rename "Backend" ctx-section to "Role" (matches the role-cycle toggle)
- Update backend-toggle title from stale "primary backend" to "Active role"
- Capitalize distill buttons (Short/Mid/Long/All) to match Memory layer style
- Improve all ctx-panel button titles for clarity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:53:16 -04:00
Scott Idem
e2a61bb78d fix: mode-select row layout so tools toggle doesn't push textarea down
- #mode-select changed from flex column to flex row (desktop + mobile unified)
- Chat/ buttons now sit side-by-side at the same height as the textarea
- Removed stale mode-agent CSS rules (mode removed in prior commit)
- Mobile: simplified override — flex:1 only, direction/align already desktop default

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:44:46 -04:00
Scott Idem
80702a21e2 fix: add distinct off/on styles for tools toggle button
OFF: very dim (nearly invisible) — makes it clear tools are inactive
ON: amber with glow — matches local-on pattern, clearly active

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:38:38 -04:00
Scott Idem
2b9dd53566 feat: replace Agent mode with independent Tools toggle
- Remove 'agent' from mode dropdown; Chat/Note/OTR remain
- Add  tools toggle button in input bar (persisted in localStorage)
  When on: routes to POST /orchestrate (Gemini tool loop); send btn → "Run"
  When off: routes to POST /chat (direct to active role); no change
- Role selector and tools toggle are now fully independent:
  active chat_role sent in orchestrate payload → used for final response
- orchestrator_engine.run() accepts response_role param; passes it to
  complete(role=...) instead of hardcoded model="claude"
- OrchestrateRequest gains chat_role field (default "chat")
- Migrate stored 'agent' mode/MRU entries to 'chat' on load

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:36:15 -04:00
Scott Idem
1cc7988953 feat: add shell_exec tool and fix orchestrator model name resolution
- Add shell_exec to orchestrator tool suite (system.py + __init__.py)
  Runs arbitrary shell commands on the Cortex host with timeout (1–120s),
  combined stdout/stderr output, optional working_dir, and exit code reporting.
  Enables system diagnostics (df, ls, ps, journalctl, etc.) from Agent mode.

- Fix orchestrator_engine.run() to use model_name from resolved registry entry
  Previously used settings.orchestrator_model (.env hardcode) regardless of
  what model was assigned to the orchestrator role. Now accepts model_name param
  and falls back to settings value only when registry has no model_name.

- Update ARCH__FUTURE.md: date, running host, local orchestrator status,
  model registry V2 progress, added Cortex Mesh concept (section 9)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 20:29:46 -04:00
Scott Idem
8baab874f1 feat: replace backend/slot toggle with role selector
The backend toggle now cycles through configured roles (chat, coder,
research, distill, etc.) instead of backup model slots within the chat
role. Each role uses its own primary→backup chain from the registry.

- ChatRequest.slot replaced by chat_role (default "chat")
- GET /backend returns available_roles instead of chat_models
- _available_roles_for_toggle() builds list from defined_roles, excluding
  orchestrator (which has its own Agent mode)
- Model label on responses now reflects the actual role's assigned model
- Toggle is inert when only one role is configured (avoids useless cycling)
- Add "Clear browser cache" button to Account Settings (Connected Accounts)
- Add _role_model_label() helper for cleaner response tag labeling

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-28 19:23:18 -04:00
Scott Idem
962d58d2e2 feat: model registry Phase 3 — slot-based backend toggle
Backend toggle now cycles through chat role models by label instead of
cycling service type strings (auto/claude/gemini/local).

- model_registry: get_model_for_slot() — resolves a specific priority
  slot without walking the fallback chain
- llm_client: complete() gains slot param; explicit slot selection
  dispatches directly to that model with no silent fallback
- routers/chat.py: ChatRequest.slot; GET /backend returns chat_models
  [{slot, label, type}] for the UI; _stream_chat uses resolved model
  label for the response tag when a slot is pinned
- app.js: toggle loads chat_models from /backend, cycles by label,
  sends slot in chat payload; legacy model field removed from payload
- app.js: fix Gap B — agent mode placeholder no longer says "Gemini
  tool loop"; now says "orchestrator"
- DESIGN doc: updated to reflect phases 1+2 complete, catalog-as-code
  decision, Gap A/B documented, Phase 3 implementation details

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:43:08 -04:00
Scott Idem
3bc6b45f9f fix: update Anthropic model catalog — add previous versions, fix context sizes
- Add Claude Opus 4.6 and Sonnet 4.5 (previous versions, still available)
- Fix context_k for Opus 4.7 and Sonnet 4.6: both have 1M context (was 200)
- Haiku 4.5 context_k 200 is correct

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:16:22 -04:00
Scott Idem
ef07596955 fix: update Google model catalog to current models (April 2026)
Remove outdated gemini-2.0-flash and gemini-1.5-pro.
Add gemini-2.5-flash-lite (GA) and the three Gemini 3.x preview
models (gemini-3.1-pro-preview, gemini-3-flash-preview,
gemini-3.1-flash-lite-preview). All have 1M token context windows.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:14:23 -04:00
Scott Idem
6e56024815 fix: settings page and help docs updated for model registry V2
settings.html:
- Remove Gemini API Key section (keys now managed in Model Registry)
- Rename "Local Models" → "Model Registry" with updated description
  covering all providers (Anthropic, Google, local hosts)
- Update button text: "Manage local models" → "Manage models"

settings.py: remove dead gemini_key template variable lookups

HELP.md:
- Fix navigation path: ☰ → Account → Model Registry → Manage models
- Restructure Model Registry section as ordered steps (1: providers/hosts,
  2: add models, 3: assign roles) so dependency order is clear
- Add explicit note that accounts/hosts must exist before adding models

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 21:07:05 -04:00
Scott Idem
9f6b162fbd docs: update end-user HELP.md for model registry V2
- Backends section: add local as third backend option, explain model
  tag on responses, clarify auto vs explicit toggle behavior
- Agent Mode: remove hard-coded "Gemini" reference — orchestrator model
  is now configurable via role assignments
- New Model Registry section: step-by-step for adding Google accounts,
  local hosts, cloud/local model entries, and role assignments
- API reference: add local to model field, add /settings/models endpoint
- Remove outdated In Progress section (local backend + multi-user shipped)
- Header controls table: update Backend description

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 20:57:05 -04:00
Scott Idem
f08b033d6c feat: model registry Phase 2 — cloud provider UI (Anthropic + Google)
Adds cloud provider management to /settings/models:
- Google Accounts section: add/remove Gemini API keys with labels
- Add Model form: provider tabs (Local / Google / Anthropic) with
  catalog dropdowns that auto-fill label and context_k
- Provider badges on model rows (Anthropic / Google / Local)
- /settings/local now redirects to /settings/models (canonical URL)
- save_cloud_model() in model_registry for Anthropic/Google entries
- Distill role migration restored in _migrate_from_local_llm
- Test fixes: version assertions updated to V2

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 20:41:06 -04:00
Scott Idem
45c95d20ba feat: model registry V2 — provider-aware schema with multi-account support
Adds a providers section to the per-user model registry for Anthropic and
Google as first-class providers alongside local hosts. Google accounts
(API keys) are now stored as a list so multiple Google accounts can coexist.

Changes:
- model_registry.py: V2 schema, auto migration V1→V2 (pulls gemini_api_key
  from auth.json into providers.google.accounts), _resolve_model() merges
  account API key for gemini_api type models
- routers/orchestrator.py: uses model-resolved api_key when orchestrator
  role resolves to a gemini_api model with account_id
- ANTHROPIC_CATALOG and GOOGLE_CATALOG constants for model picker (Phase 2)
- New functions: get_google_api_key(), save/remove_google_account(), get_catalog()
- Documentation: ARCH__BACKENDS.md updated to V2 schema, DESIGN doc added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 20:21:04 -04:00
Scott Idem
27ca7c7efd fix: apply host_type path correction in OpenAI orchestrator
The AsyncOpenAI client always appends /chat/completions to base_url.
Open WebUI's endpoint is at /api/chat/completions, so for openwebui
host_type the base_url must include the /api prefix — same logic as
_local() in llm_client.py.

Also strip non-standard metadata fields (backend, host, etc.) from
session_messages before passing them to the API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 23:16:33 -04:00
Scott Idem
869a596f4b fix: don't fall back when a model is explicitly configured in registry
Previously any backend error would silently fall back to Claude.
Now if the user has a model configured via the model registry, errors
propagate to the UI so the actual problem is visible rather than hidden
behind a transparent backend switch.

Fallback still applies when using the default/auto backend with no
registry config.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:30:55 -04:00
Scott Idem
3b3456600a feat: store and display backend + host metadata on chat messages
Each assistant message in the session JSON now carries:
  backend, backend_label, host (platform.node())

These fields are shown as model tags in the UI — on live responses and
when loading session history. Session log entries (sessions/YYYY-MM-DD.md)
include the backend label and host in the turn header.

The local (OpenAI-compat) backend strips non-standard fields before
sending messages to the API so extra fields don't leak upstream.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:16:48 -04:00
Scott Idem
6c84d6ae72 docs: update README and CLAUDE.md for install script and backup
- README: add Setup/Install section (install.py), Development Workflow
  (dev-restart.sh), and Backup (restic/backup.sh) sections; fix
  .env.example reference → .env.default
- CLAUDE.md: replace stale Docker start command with install.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 21:36:48 -04:00
Scott Idem
2629983452 fix: create cortex config dir and restic password file during install
Previously the ~/.config/cortex/ directory and restic-password were only
created on the first backup run. Now install.py creates them eagerly so
the user can verify setup and back up the password immediately.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 21:24:08 -04:00
Scott Idem
f668826f79 fix: bootstrap pip via ensurepip if missing from venv
On Arch Linux, venvs can be created without pip seeded in.
Detect the missing module before attempting pip install and
recover with `python -m ensurepip --upgrade`.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 21:17:52 -04:00
Scott Idem
576f22216a feat: restic backup of home/ with systemd daily timer
- backup.sh — backs up home/ (persona data, memory, tasks, crons) to a
  local restic repo; auto-generates password on first run, prunes to
  7d/4w/6m retention; excludes sessions/ and session_data/
- install.py — setup_backup_timer() installs cortex-backup.service +
  cortex-backup.timer (daily 03:00, Persistent=true); skips gracefully
  if restic is not installed

Password lives at ~/.config/cortex/restic-password (chmod 600, not in git).
Repo defaults to ~/backups/cortex-home-restic; override via RESTIC_REPOSITORY.

Per-user encrypted backup is a noted future feature.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:36:22 -04:00
Scott Idem
bf800acca8 chore: remove home/ from git, update gitignore
All user persona data (memory, tasks, crons, identity files, sessions)
was previously committed. This removes it from tracking and adds home/
to .gitignore — files remain on disk and will continue to sync via
Syncthing. Backup should be handled via encrypted per-user means.

Also added: cortex/.env*.bak and cortex/=* (pip artifact) to gitignore.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:19:23 -04:00
Scott Idem
d9a322164a feat: OpenAI-compatible orchestrator + backend auto-routing
- openai_orchestrator.py — new ReAct tool loop engine for any
  OpenAI-compatible endpoint (OpenRouter, Open WebUI, Ollama, LiteLLM);
  model handles both tool loop and final response, no Claude handoff needed
- tools/__init__.py — auto-derive OpenAI JSON Schema from existing Gemini
  FunctionDeclarations so tool definitions have a single source of truth
- routers/orchestrator.py — route to openai_orchestrator when model registry
  "orchestrator" role resolves to a local_openai type host
- routers/chat.py — pass role to _backend_label(); fix fallback_used logic
  (only meaningful for explicit backend overrides, not auto-routing)
- static/app.js — add null/"auto" to backend cycle; fetch local model hint
  without overriding the auto default on page load
- model_registry.py — _normalize() back-fills host_type on old registry files
- requirements.txt — add openai>=1.0.0
- ARCH__BACKENDS.md — document OpenAI-compat backend and routing logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:18:18 -04:00
Scott Idem
8ba5247ef5 tooling: install script, workspace file, and dev-restart helper
- install.py — idempotent setup script (venv, systemd service, linger,
  auth checks); supports --check for read-only status inspection
- .stignore — exclude .venv and runtime dirs from Syncthing so each
  host maintains its own machine-local venv
- Cortex_and_Inara.code-workspace — VS Code workspace (service, personas,
  docs folders; launch config for uvicorn --reload)
- dev-restart.sh — SSH wrapper to restart Cortex on the gaming laptop
  and tail logs; supports restart / logs / status subcommands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 19:11:27 -04:00
Scott Idem
a6e404c143 feat: host_type field for OpenRouter / OpenAI-compatible API support
Adds host_type ("openwebui" | "openai") to the host schema so Cortex can
talk to both Open WebUI/Ollama and OpenRouter/standard-OpenAI endpoints.

Path differences per type:
  openwebui (default): /api/chat/completions, /api/models
  openai:              /chat/completions,     /models

model_registry.py:
  - host_type added to host schema (default "openwebui", backward compat)
  - save_host() accepts host_type parameter
  - _resolve_model() passes host_type through with the merged host fields

llm_client._local():
  - Reads host_type from resolved model_cfg
  - Selects correct chat completions path accordingly

routers/local_llm.py:
  - save_host route accepts host_type form field
  - fetch-models uses /models for openai type, /api/models for openwebui
  - Existing host rows show type selector pre-filled from stored value

local_llm.html:
  - "Add host" form includes type selector

To use OpenRouter:
  - Add host: URL = https://openrouter.ai/api/v1, Type = OpenAI-compatible
  - API key from openrouter.ai (store in .env or model_registry.json only)
  - Fetch models or add manually (e.g. anthropic/claude-sonnet-4-5-20251022)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 21:11:22 -04:00
Scott Idem
2dd94696d5 fix: model-tag color was #334155 (invisible on dark theme) → #475569 2026-04-05 22:25:09 -04:00
Scott Idem
8570e8d852 fix: backend toggle not sent to server; add per-message model tag
Fixes:
  - app.js was tracking primaryBackend locally but never included
    model: primaryBackend in the /chat POST body, so the server always
    used settings.primary_backend regardless of what the user clicked.
    Now model: primaryBackend is sent on every chat request.

  - Responses were only annotated when fallback occurred. Now every
    assistant message shows a small model tag at the bottom right.

chat.py:
  - _backend_label() resolves human-readable name:
      claude → "Claude", gemini → "Gemini",
      local → registry label (e.g. "Gemma 4 E4B") or model_name
  - SSE payload now includes backend_label field

app.js:
  - model: primaryBackend added to /chat fetch body
  - After every response, appends .model-tag div with backend_label
  - Fallback shows " fallback → <label>" in amber; normal is muted
  - Removed separate system message for fallback (tag covers it)

style.css:
  - .model-tag: small muted text, right-aligned, separated by thin line
  - .model-tag.fallback: amber (#f59e0b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 22:10:40 -04:00
Scott Idem
9299ce5ba6 test: model registry unit test suite (45 tests)
Covers model_registry.py without requiring a running service or LLM:

  Empty/fresh state: no files, missing user dir
  Save/load: round-trip, corrupt file fallback
  Migration: v1 hosts/models, v1 no active, v0 flat, v0 empty url,
             distill_backend_mid=local → distill role, saves file after migrate
  Built-in resolution: claude_cli, gemini_api, gemini_cli, unknown → None
  User model resolution: local_openai merges host, missing host → None
  get_model_for_role: registry primary, built-in from registry, skips missing,
                      walks full backup chain, .env fallback, hardcoded fallback,
                      custom roles
  get_best_local_model: prefers role chain, falls back to first local, None if no local
  Host CRUD: create, update, unknown ID creates new, remove + cascades to models
  Model CRUD: create, update, remove + clears role refs
  set_role: assign model, assign built-in, clear with None, invalid slot,
            unknown model ID, creates new role key
  get_defined_roles: returns all settings roles, fills gaps with {}
  Multi-user isolation: registries don't bleed across users

All tests use tmp_path + patch.object(config.settings, ...) — no real files touched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:48:00 -04:00
Scott Idem
608e1de246 feat: model registry UI — hosts, models, role assignments
Replaces the single-host local model settings page with a full model
registry interface at /settings/local.

Hosts section:
- List existing hosts with inline edit + save + remove
- Collapsible "Add host" form
- Per-host "Fetch models" button

Models section:
- List all models with label, model name, host, context_k badge, tags
- Remove button

Add Model section:
- Host dropdown, label, model name, context_k, tags (comma-separated)
- "Fetch models from host" with auto-fill picker

Role Assignments section:
- One row per defined role (chat, orchestrator, distill, coder, research)
- Primary + backup_1 + backup_2 dropdowns per role
- Dropdowns pre-filled from registry on load
- AJAX save on change (POST /api/models/role) with toast confirmation
- Built-in models (claude_cli, gemini_cli, gemini_api) always available in dropdowns

Backend:
- All user_settings references replaced with model_registry
- host/{id}/remove route added
- fetch-models now accepts host_id query param
- POST /api/models/role for AJAX role assignment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:31:32 -04:00
Scott Idem
6a1a1c2686 feat: unified model registry with role-based routing
Introduces model_registry.py as the single source of truth for all LLM
backend configuration. Replaces scattered backend settings across user_settings,
config distill_backend_*, and the UI toggle.

model_registry.py:
- Per-user home/{user}/model_registry.json with version, hosts, models, roles
- Models have: type (local_openai|claude_cli|gemini_cli|gemini_api), label,
  model_name, host_id, context_k (tokens), tags (capability labels)
- Roles map to priority chains: primary, backup_1..backup_4
- Built-in IDs (claude_cli, gemini_cli, gemini_api) always resolvable
- Auto-migrates existing local_llm.json on first access
- CRUD: save_host, remove_host, save_model, remove_model, set_role
- get_model_for_role(): registry → .env default → hardcoded fallback

config.py:
- role_chat/orchestrator/distill/coder/research .env defaults
- defined_roles: comma-separated standard role list (extensible)
- get_defined_roles() and get_role_default() helper methods

llm_client.complete():
- New role= parameter (default "chat") for registry-based routing
- model= still accepted for explicit UI toggle override
- _claude() and _local() accept model_cfg dict instead of raw string
- _local() uses pre-resolved config from registry

memory_distiller.py:
- distill_mid/long now use role="distill" (no more distill_backend_* .env vars needed)

cron_runner.py:
- brief jobs use role="chat"

routers/chat.py + auth.py:
- Use model_registry instead of user_settings for local model info

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 21:25:18 -04:00
Scott Idem
a4daebdc9b feat: local LLM multi-model, session search, cron proactive types, notifications, docs overhaul
Local LLM:
- user_settings.py: per-user hosts/models config (local_llm.json)
- routers/local_llm.py + static/local_llm.html: dedicated settings page
- llm_client.py: local OpenAI-compatible backend via httpx
- config.py: LOCAL_API_URL/KEY/MODEL + per-backend timeouts
- Active model shown near backend toggle (amber hint text)

Memory distillation:
- memory_distiller.py: DISTILL_BACKEND_MID/LONG .env overrides
- scheduler.py + notification.py: notify NC Talk after mid/long distill
- notification.py: outbound channel abstraction (NC Talk, extensible)

Session search:
- routers/files.py: GET /sessions/search?q= with excerpts grouped by date
- static/index.html + app.js: search UI in file sidebar with highlight
- _esc() helper to prevent XSS in search results

Proactive cron:
- cron_runner.py: new job types — message (send directly) and brief (LLM + send)
- Both support optional per-job channel override

Channels:
- routers/nextcloud_talk.py: consolidated using notification._send_nct_message()
- routers/auth.py: local backend status in /auth/status
- routers/chat.py: /backend returns {primary, fallback, local_model} object

UI / UX:
- Copy button for user messages (matching assistant)
- Autocomplete disabled on sensitive form fields
- settings.html: local model section replaced with link to /settings/local

Docs overhaul:
- MASTER.md hub + ARCH__SYSTEM/BACKENDS/PERSONA/CHANNELS/FUTURE.md
- ARCH__Intelligence_Layer.md replaced with redirect table
- CORTEX.md trimmed to vision only; README updated
- OPEN_WEBUI_API.md added to docs/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 20:53:06 -04:00
Scott Idem
bd6532e93a feat: shared nav bar on Help and Settings pages
Replaces the lone "← Back to Cortex" link with a consistent page-nav
on both pages: ← Chat | Help | Settings | Sign out

Active page is highlighted purple; others are muted gray.
Settings page gets a {{ help_href }} template var from settings.py.
Help page builds nav links from the existing cfg JS object.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 22:09:08 -04:00
Scott Idem
a94fdc869d docs: fix Gitea SSH URL to use git.dgrzone.com
cortex subdomain works incidentally but git.dgrzone.com is correct.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:53:54 -04:00
Scott Idem
1fefd42e19 docs: Gitea SSH port 2222 verified working
WAN port forward confirmed end-to-end. Clone URL:
ssh://git@cortex.dgrzone.com:2222/<user>/<repo>.git

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:52:36 -04:00
Scott Idem
0c17b4b1ab docs: overhaul TODO__Agents.md to reflect current state
Moved to completed: token expiry restart, Holly onboarding, per-user
channel config, Google OAuth, per-user Gemini key, session persistence,
persona picker, Lucide icons, favicon, Help shared base, reminders tools,
Brian onboarding.

Updated in-progress: knowledge consolidation tools (ae_journal_* done,
import script still pending). NC Talk and Google Chat notes updated for
per-user routing. Removed stale "default user only" notes.

High priority now: Ollama backend, Gitea SSH verification.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:44:56 -04:00
Scott Idem
cec6d3e23a docs: update README for current state
- .env location: cortex/.env + cortex/.env.example (not project root)
- Webhook endpoints: per-user /webhook/nextcloud/{username} and /channels/google-chat/{username}
- Personas table: added brian/wintermute and scott/developer
- Docs table: added GOOGLE_CHAT_BOT.md, cortex/static/HELP.md
- Channels section: per-user webhook note + links to setup docs
- User management: added google-add command and channels.json note
- Removed stale Inara/Tina-only framing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:42:11 -04:00
Scott Idem
2d3a380d6b docs: add Tools & Modes protocol to all personas and template
Every persona now knows: direct chat has no tools, Agent mode () has
the full tool suite. If asked to write a reminder/task/etc in chat mode,
tell the user to switch modes rather than silently failing.

Updated: inara, tina, donut, wintermute, developer, cleo PROTOCOLS.md
Updated: persona_template.py so all future personas get this by default

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:35:54 -04:00
Scott Idem
662924c6a1 fix: pass user to _run_job so get_user_gemini_key resolves correctly
NameError: name 'user' is not defined in orchestrator._run_job —
user was resolved in the endpoint but not forwarded to the background task.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:21:14 -04:00
Scott Idem
e5b6d58889 feat: reminders_add and reminders_list tools
- New cortex/tools/reminders.py with reminders_add, reminders_list, reminders_clear
- reminders_clear moved here from cron.py (cron still imports from same file)
- __init__.py: wired up new callables and Gemini declarations
- Inara can now add/read reminders in Agent mode via the orchestrator

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 21:14:22 -04:00
Scott Idem
6b725afc3e docs: update NC Talk doc with real container name, commands, and logs section
- Use dgr_zone_nextcloud-app-1 throughout (actual container name)
- talk:bot:uninstall (not remove — wrong command in previous version)
- Added Logs section: occ log:tail + journalctl
- Bruteforce reset command now includes full docker exec form

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 13:47:43 -04:00
Scott Idem
ddf5dd6338 docs: add Google Chat setup guide, update NC Talk for per-user routing
- docs/GOOGLE_CHAT_BOT.md: new step-by-step guide covering channels.json,
  Google Cloud Console config, JWT audience, and troubleshooting
- docs/NEXTCLOUD_TALK_BOT.md: updated for per-user endpoints
  (/webhook/nextcloud/{username}), channels.json config, removed old
  server-level .env references, updated Multi-User note

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 13:24:36 -04:00
Scott Idem
93f7f44e51 feat: per-user channel config for Google Chat and Nextcloud Talk
- New endpoints: POST /channels/google-chat/{username} and /webhook/nextcloud/{username}
- Channel secrets/config live in home/{username}/channels.json (gitignored)
- auth_utils: get_user_channels() helper reads channels.json
- Both routers load persona, audience/secret, backend, timeout per user;
  set_context() wires the correct persona before building the system prompt
- Removed server-level channel settings from config.py and .env —
  no user gets a channel until they create their own channels.json
- .gitignore: home/**/channels.json added

To migrate: update Google Chat Add-on webhook URL to /channels/google-chat/{username}
and re-register NC Talk bot at /webhook/nextcloud/{username}

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 13:02:45 -04:00
Scott Idem
496da58f58 chore: consolidate .env files — one .env in cortex/, one .env.example
- Removed orphaned root .env and .env.default (values already in cortex/.env,
  which is what the systemd service actually loads)
- Replaced outdated cortex/.env.example with the comprehensive .env.default content
- Also tracks: tested/persona/cleo/ (new test persona), Inara memory updates

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 12:22:49 -04:00
Scott Idem
8e20bfbea8 feat: shared Help base, Google OAuth live, new personas, cleanup
- cortex/static/HELP.md: shared Help & Reference base served to all users
- help.html: loads shared base + appends persona-specific HELP.md if present
- inara/HELP.md: cleared (content moved to shared base)
- Google OAuth: registered scott.idem@oneskyit.com; flow now working end-to-end
- .gitignore: exclude home/**/sessions/ (runtime logs)
- New personas tracked: home/holly/persona/donut/, home/scott/persona/developer/
- Removed orphans: holly/, personas/, cortex-holly.service
- CLAUDE.md: updated current state and recently completed list to 2026-03-27

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 22:55:45 -04:00
Scott Idem
3a94df1eaf fix: prevent password managers autofilling Gemini API key field
Change type="password" to type="text" — the main signal password
managers use. Also add autocomplete="off", data-lpignore, data-1p-ignore
for broader coverage across Bitwarden, 1Password, LastPass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:57:14 -04:00
Scott Idem
ce806e52ed fix: google-add now also sets profile.json email
The invite command reads email from profile.json, not auth.json.
google-add was only writing to auth.json so invite had no address
to send to. Now calls set_email() as well.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:32:48 -04:00
Scott Idem
7438031797 feat: connected accounts + Gemini API key in account settings UI
Settings page gains two new sections:
- Connected Accounts: shows linked Google email (read-only)
- Gemini API Key: paste personal key from aistudio.google.com,
  shows masked hint of saved key, remove link to revert to server key

POST /settings/gemini-key saves/clears gemini_api_key in auth.json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:16:37 -04:00
Scott Idem
8aec6aafcc feat: Google OAuth sign-in + per-user Gemini API key
Users with Google accounts can now sign in without a password.

Auth flow:
- GET /auth/google → Google consent page (CSRF state cookie)
- GET /auth/google/callback → exchange code, lookup user, set JWT
- auth.json gains google_sub + google_email fields
- set_password() no longer overwrites unrelated auth.json fields

Admin setup:
  python manage_passwords.py google-add <username> <email>
  # add GOOGLE_CLIENT_ID + GOOGLE_CLIENT_SECRET to .env

Per-user Gemini key:
- get_user_gemini_key() reads gemini_api_key from auth.json
- orchestrator_engine.run() accepts gemini_api_key param
- orchestrator router passes user's key, falls back to server key

login.html: "Sign in with Google" button above the password form.
manage_passwords.py list: now shows auth method columns (pw / google).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:01:52 -04:00
Scott Idem
62fde62653 feat: persona-specific favicon + fix favicon.ico 404
app.js updates the <link rel="icon"> to the active persona's emoji on
load (CORTEX_EMOJI is already injected server-side). /favicon.ico route
added as a fallback for login/settings/help pages that don't have
persona context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:45:36 -04:00
Scott Idem
f8d89bc272 fix: close SSE connection cleanly on page navigation
beforeunload closes the EventSource explicitly so the browser doesn't
log "connection interrupted while page was loading". onerror handler
suppresses auto-reconnect noise if the connection temporarily drops.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:44:12 -04:00
Scott Idem
92350f7a7b feat: persist active session across page navigation with inactivity TTL
Session ID is stored in localStorage keyed to user+persona. On page load
it's silently restored if within 30 min of last activity. Timestamp
updates on every sent message. New session / delete session clears the
stored ID so the TTL logic stays consistent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:38:04 -04:00
Scott Idem
4f09823afe feat: Lucide icons on edit/del/copy and inline edit save/cancel buttons
pencil → edit, trash-2 → del, copy → copy, check → copied feedback,
check → Save, x → Cancel. All small action buttons get inline-flex
alignment for consistent icon+label layout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:32:19 -04:00
Scott Idem
826bd6cfe3 feat: /{username} persona picker landing page
Visiting /scott (or any user root) now shows a clean card page listing
all their personas with emoji + name, each linking to /{user}/{persona}.
Previously the route was unhandled (404 or wildcard match).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:19:04 -04:00
Scott Idem
c3507f8e11 fix: help page back link preserves active persona
Pass ?persona= query param on the help link so the server knows which
persona to return to. Previously always defaulted to personas[0], causing
navigation back to the wrong persona.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:13:52 -04:00
Scott Idem
65548ebf36 feat: Lucide SVG icons throughout main UI
Replace all emoji/unicode icons with Lucide SVG icons:
- Mode select dropdown: message-circle / pencil / lock / bot
- Send button: arrow-up (chat/OTR), pencil (note), zap (agent)
- Stop button: square icon
- Header nav already had Lucide SVGs; render_icons() now called at init

Add icon_html() + render_icons() helpers; update update_mode_ui() and
open_mode_dropdown() to use innerHTML + lucide.createIcons(). CSS: .btn-icon
alignment, inline-flex on .hdr-btn / .hdr-dd-item / #send / #stop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 23:06:01 -04:00
Scott Idem
24c9f52b49 fix: agent mode icon — 🥸 in dropdown, on Run button
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:51:51 -04:00
Scott Idem
3bada5f311 fix: font sizes 21/25/17px, icon labels on send btn, no-wrap fix
- Sync preload script font sizes to match app.js (21/25/17px)
- Send button variants now show icons: ↑ Send, 📝 Note,  Run
- Remove fixed width on send-col; add white-space:nowrap + padding
  so "📝 Note" never wraps regardless of font size

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:43:34 -04:00
Scott Idem
cf277f822e feat: Inter font, weight 450, bumped base sizes across all pages
- Load Inter variable font from Google Fonts on all 5 HTML pages
- font-weight: 450 on body (between regular and medium — fixes thin feel)
- -webkit-font-smoothing: antialiased for cleaner screen rendering
- Base font size: normal 16→17px, large 18→19px, small 14→15px
- Applies consistently to main UI, login, setup, settings, and help pages

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:20:10 -04:00
Scott Idem
6cf10d2755 feat: UI redesign — compact mode select, consolidated header nav
Header:
- Sessions, ⚙ context panel, ≡ settings dropdown (Files, Account,
  Sign Out), and  help — down from 6+ individual buttons
- Responsive: flex-row on desktop, wraps on mobile with labels hidden

Footer (input area):
- 4-way mode select replaces the button row — shows only the active
  mode as [icon] [label] ▲; click opens an upward dropdown
- Options sorted by MRU: most recently used floats to the bottom
  (closest to the trigger button) for quick re-selection
- Current mode marked with ✓
- Note mode shows a small prv/pub sub-toggle below the select button
- Mobile: textarea on top (full width), mode select + send on one row

Mode state consolidated from 3 booleans into a single current_mode
variable with localStorage persistence and MRU tracking.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 22:08:33 -04:00
Scott Idem
fa04b5e6b0 feat: off the record mode (OTR)
Adds a third input mode toggle alongside Note and Agent. When active:
- Textarea gets a subtle purple tint with dashed border
- OTR button highlights purple
- Placeholder reads "Off the record — not logged or distilled…"
- off_record=True is sent to /chat; session_logger is skipped
- In-memory session context is preserved within the session

Switching to Note or Agent mode deactivates OTR, and vice versa.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 21:07:21 -04:00
Scott Idem
b3f40cf437 fix: message input placeholder uses active persona name
Hardcoded 'Inara' replaced with CORTEX_PERSONA in all placeholder
strings (chat mode and agent task mode).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 20:59:37 -04:00
Scott Idem
8487645224 fix: claude auth expiry warning — correct field name and smarter threshold
- Fix 'undefined' in auth banner: read access_token_hours_remaining (not hours_remaining)
- Fix false-positive warning on fresh tokens: when refresh token present, only warn
  within 1 hour of expiry (not 24h) since the CLI should auto-rotate but sometimes misses
- Emit claude_auth_expired SSE event on 401 so UI shows inline red banner immediately
- app.js: handle claude_auth_expired SSE event with persistent top banner + dismiss button

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 23:22:18 -04:00
Scott Idem
0cf0d65e9e feat: session naming, username/persona rename, help page, contrast fixes
- Session name field: PATCH /sessions/{id} endpoint, inline rename button in UI
- Persona rename: inline ✏ toggle form in settings, POST /settings/persona/rename
- Username rename: inline form in settings, POST /settings/username (renames home dir, forces re-login)
- Help page: dedicated /help route replacing modal, collapsible sections
- Per-persona isolation: files.py and session_store.py now scope to correct user/persona
- Contrast/visibility: muted text bumped to slate-400+, session rename btn at 0.4 opacity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:10:12 -04:00
Scott Idem
1b425a539f feat: account settings page + dedicated help page
- Add /settings page with password change form and personas list
- Add /help dedicated page (replaces help modal); renders HELP.md with
  collapsible sections, dark theme, back link to active persona
- Add 👤 account button and convert ? button to link in header
- Remove help modal HTML and ~55 lines of modal JS from main app
- Register settings and help routers in main.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 21:41:18 -04:00
158 changed files with 25269 additions and 4151 deletions

View File

@@ -1,88 +0,0 @@
# Cortex .env reference — copy to .env and fill in values
# DO NOT commit .env — it contains secrets
# ── Agent identity ───────────────────────────────────────────────────────────
# Global display names used in distillation prompts and session logs.
# Individual persona identities live in home/{username}/persona/{name}/IDENTITY.md
AGENT_NAME=Inara
USER_NAME=Scott
# ── Home directory ────────────────────────────────────────────────────────────
# Root for all user/persona data. Layout: home/{username}/persona/{name}/
# Relative paths are resolved from the cortex/ directory.
# Default: ../home (i.e. Cortex_and_Inara_dev/home/)
# HOME_DIR=../home
# ── Session auth ─────────────────────────────────────────────────────────────
# Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
JWT_SECRET=change-me-in-dotenv
JWT_EXPIRE_DAYS=30
# ── SMTP (invite emails + future notifications) ───────────────────────────────
SMTP_SERVER=linode.oneskyit.com
SMTP_PORT=465
SMTP_USERNAME=send_mail
SMTP_PASSWORD=
SMTP_FROM_EMAIL=noreply@oneskyit.com
SMTP_FROM_NAME=Cortex
# Base URL included in invite links
CORTEX_BASE_URL=https://cortex.dgrzone.com
# ── Server ──────────────────────────────────────────────────────────────────
HOST=0.0.0.0
PORT=8000
# ── Google Chat bot ──────────────────────────────────────────────────────────
# JWT audience for verifying inbound Workspace Add-on Chat webhook requests.
# For Workspace Add-on Chat apps, the aud claim = the endpoint URL.
# Leave blank to disable verification (dev/testing only).
GOOGLE_CHAT_AUDIENCE=https://cortex.dgrzone.com/channels/google-chat
# ── Nextcloud Talk bot ───────────────────────────────────────────────────────
NEXTCLOUD_URL=https://cloud.dgrzone.com
NEXTCLOUD_TALK_BOT_SECRET=
# ── LLM backends ────────────────────────────────────────────────────────────
# Primary backend: "claude" or "gemini" (other is always fallback)
PRIMARY_BACKEND=claude
# Timeouts in seconds
TIMEOUT_CLAUDE=60
TIMEOUT_GEMINI=120
# ── Orchestrator (Gemini API — not Gemini CLI) ───────────────────────────────
# Required for /orchestrate endpoint and tool use
# Free tier key: https://aistudio.google.com/apikey
GEMINI_API_KEY=
# Model for the orchestration tool loop (not the user-facing response)
ORCHESTRATOR_MODEL=gemini-2.5-flash
# Safety cap on tool loop iterations
ORCHESTRATOR_MAX_ROUNDS=10
# ── DuckDuckGo search ────────────────────────────────────────────────────────
# Leave blank for free unauthenticated tier
# Set to your API key for higher rate limits (paid DuckDuckGo account)
DDG_API_KEY=
DDG_MAX_RESULTS=5
# ── Aether Platform API ───────────────────────────────────────────────────────
# Used by orchestrator tools: ae_journal_search, ae_journal_entry_create, ae_task_list
# Same values as agents_sync/mcp/.env — copy from there
AE_API_URL=https://dev-api.oneskyit.com
AE_API_KEY=
AE_ACCOUNT_ID=
AE_API_TIMEOUT=15
# ── Distillation schedule ────────────────────────────────────────────────────
SCHEDULER_TIMEZONE=America/New_York
AUTO_DISTILL=true
AUTO_DISTILL_SHORT=true
AUTO_DISTILL_MID=true
AUTO_DISTILL_LONG=false # manual review recommended before enabling
# Memory tier token budgets (soft caps)
MEMORY_BUDGET_SHORT=3000
MEMORY_BUDGET_MID=2000
MEMORY_BUDGET_LONG=2000

28
.gitignore vendored
View File

@@ -3,25 +3,33 @@
__pycache__/
*.pyc
# Secrets — keep .env.example, never commit real .env
# Secrets — keep .env.default, never commit real .env
.env
cortex/.env*.bak
# Session data (runtime state, not source)
# Pip install artifacts
cortex/=*
# Runtime data
cortex/data/
home/**/session_data/
# User credentials and tokens — never commit
home/**/auth.json
home/**/invite.json
home/**/profile.json
# User home directory — all persona data, memory, tasks, and credentials
# are personal and must never be committed. Back up via encrypted means.
home/
# Syncthing Metadata
# Syncthing metadata
.stfolder/
# Temporary Files
# Temporary files
tmp/
*.tmp
*.log
# System Files
# Aider — history files are personal/ephemeral; .aider.conf.yml is project config and IS tracked
.aider.chat.history.md
.aider.input.history
.aider.llm.history
# System files
.DS_Store
.aider*

5
.stignore Normal file
View File

@@ -0,0 +1,5 @@
// Machine-local — never sync across hosts
.venv/
__pycache__/
*.pyc
cortex/data/

128
CLAUDE.md
View File

@@ -22,7 +22,7 @@ Cortex_and_Inara_dev/
main.py ← App entry point, router registration
config.py ← All settings (pydantic-settings, reads .env)
persona.py ← Two-level identity: user + persona, path resolution, ContextVars
llm_client.py ← Claude CLI + Gemini CLI subprocess backends
llm_client.py ← Claude CLI + Gemini CLI subprocess backends + Anthropic SDK direct
orchestrator_engine.py ← Gemini API ReAct tool loop → Claude handoff
context_loader.py ← Builds system prompt from persona files (tier 14)
session_store.py ← In-memory + file session persistence
@@ -45,12 +45,15 @@ Cortex_and_Inara_dev/
google_chat.py ← POST /webhook/google (Google Chat Add-on)
ui.py ← Login/logout, /{user}/{persona} UI route, /api/personas
onboarding.py ← /setup/{token} password step + /setup/persona creation
settings.py ← /settings, /settings/notifications, /settings/integrations (admin)
tools_settings.py ← /settings/tools
crons.py ← /settings/crons — Schedules web UI (list/add/edit/toggle/remove)
tools/
__init__.py ← Tool registry (Gemini FunctionDeclarations + dispatcher)
web.py ← DuckDuckGo web_search tool
scratch.py ← Scratchpad tools (scratch_read/write/append/clear)
tasks.py ← Personal task management (task_create/list/update/complete)
cron.py ← Scheduled job tools (cron_list/add/remove/toggle)
cron.py ← Scheduled job tools (cron_list/add/remove/toggle); 5 types; hourly/daily/weekly/monthly/yearly schedules
system.py ← Local machine tools (claude_allow_dir)
tests/ ← pytest test suite (80 tests)
static/ ← Single-page web UI (index.html, style.css, app.js)
@@ -82,6 +85,7 @@ Cortex_and_Inara_dev/
docs/ ← Integration reference docs
NEXTCLOUD_TALK_BOT.md
OPEN_WEBUI_API.md ← Open WebUI API: tool calling, RAG, model management
documentation/ ← Architecture decisions and agent task list
TODO__Agents.md ← READ THIS FIRST — active task list
@@ -97,8 +101,8 @@ Cortex_and_Inara_dev/
## Run Commands
```bash
# Start (Docker)
docker compose up -d
# First-time setup or update on any machine
python3 install.py
# Restart service (after any Python change)
systemctl --user restart cortex
@@ -135,9 +139,10 @@ http://localhost:8000/docs
- **Orchestrated tasks** go to `POST /orchestrate` — returns a job_id, result is polled
### LLM Backends
- `llm_client.py` manages Claude CLI (`claude --print`) and Gemini CLI (`gemini -p`) subprocesses
- `llm_client.py` manages Claude CLI (`claude --print`), Gemini CLI (`gemini -p`), and Anthropic SDK (`anthropic_api` type) subprocesses/calls
- `orchestrator_engine.py` uses the Gemini **API** (google-genai SDK) — completely separate from the Gemini CLI
- Claude OAuth token is read live from `~/.claude/.credentials.json` (never rely on stale env var)
- `anthropic_api` backend: user-configured API key from `providers.anthropic.credentials` in `model_registry.json` — uses `anthropic.AsyncAnthropic`
### Tool Strategy
- Orchestrator tools live in `cortex/tools/` — separate from the `ae_*` MCP tools
@@ -145,8 +150,8 @@ http://localhost:8000/docs
- Tools are registered in `cortex/tools/__init__.py` as both Gemini FunctionDeclarations and Python callables
### Context / Memory
- `context_loader.py` assembles Inara's system prompt from `inara/` files based on tier (13)
- Tier 1 = minimal (identity only); Tier 2 = standard (+ memory + user profile); Tier 3 = full
- `context_loader.py` assembles Inara's system prompt from `inara/` files based on tier (14)
- Tier 1 = minimal (identity only); Tier 2 = standard (+ memory + user profile); Tier 3 = + last 2 sessions; Tier 4 = + last 7 sessions
- Memory files are written by the distiller or manually — do not delete them
### Security / Safety
@@ -159,6 +164,44 @@ http://localhost:8000/docs
- Passwords are bcrypt-hashed and stored in `home/{username}/auth.json` — never in `.env` or the DB
- Invite tokens are one-time-use, 72-hour expiry, stored in `home/{username}/invite.json`
### Onboarding Flow
New users follow a three-step setup before reaching the chat:
1. `GET /setup/{token}` → password form → `POST /setup/{token}` sets password + session cookie
2. `GET /setup/persona` → persona creation form → `POST /setup/persona` bootstraps persona directory
3. `GET /setup/model` → OpenRouter quick-connect → `POST /setup/model` saves host + model + role assignment
Step 3 is optional (skip link goes straight to `/{user}/{persona}`). `/setup/model` also works
standalone (accessible from Settings) for existing users who haven't configured a model.
All in `cortex/routers/onboarding.py`. Model writes use `model_registry.py`: `save_host()`,
`save_model()`, `set_role(username, "chat", "primary", model_id)`.
### Documentation Philosophy
Cortex is a no-black-box system. Docs must match reality — at all times.
- **Docs first:** When planning significant changes, update `TODO__Agents.md` and the relevant
`ARCH__*.md` to describe the intended design *before* implementing. This creates a spec to
implement against.
- **Verify after:** Once implementation is complete, re-read the pre-written docs and confirm
they match what was actually built. Update anything that drifted.
- **HELP.md is a user contract:** It describes what users can do. Never let it describe
features that don't exist or omit features that do.
- **CLAUDE.md + ARCH__*.md are the developer contract:** Update them as the architecture evolves.
- **Stale docs are bugs.** If you notice drift, fix it before moving on.
### Doc update checklist (run after any significant change)
| Doc | Update when |
|---|---|
| `CLAUDE.md` | New tool, channel, router, major design change, tool count |
| `cortex/static/HELP.md` | Any user-visible feature — tools, settings, UI, API endpoints |
| `documentation/TODO__Agents.md` | Mark completed items; add new planned work |
| `documentation/MASTER.md` | New capability goes live; tool count changes |
| `documentation/ROADMAP.md` | Phase items completed or added |
| `documentation/ARCH__CHANNELS.md` | New channel, notification trigger, or scheduler job |
| `documentation/ARCH__SYSTEM.md` | New module, router, or tools/ file |
| `README.md` | Architecture diagram, channels table, or setup steps change |
---
## Adding a New Tool
@@ -166,7 +209,13 @@ http://localhost:8000/docs
1. Implement the tool function in `cortex/tools/<domain>.py`
- Must be `async def`; use `asyncio.to_thread` for blocking calls
- Return a plain string result
2. Add a `FunctionDeclaration` and register it in `cortex/tools/__init__.py`
2. Add a `FunctionDeclaration` and register it in `cortex/tools/__init__.py`:
- Import the callable
- Add to `TOOL_CATEGORIES` (pick an existing category or create one)
- Add to `_CALLABLES`
- Add a `TOOL_RISK` rating (low/medium/high)
- Add to `TOOL_ROLES` if admin-only; add to `CONFIRM_REQUIRED` if destructive
- Add module to `_ALL_DECLARATIONS`
3. Syntax check: `python3 -m py_compile cortex/tools/<domain>.py`
4. Restart Cortex
@@ -211,37 +260,48 @@ clearly asked for a directory to be unblocked.
---
## Current State (2026-03-20)
## Current State (2026-05-12)
Cortex is running and stable. All three primary channels are live:
Cortex is running and stable. All channels are live:
| Channel | Status | Notes |
|---|---|---|
| Web UI | ✅ Live | `https://cortex.dgrzone.com` (basic auth) |
| Web UI | ✅ Live | `https://cortex.dgrzone.com` — PWA-installable |
| Nextcloud Talk | ✅ Live | HMAC-signed webhook, async reply |
| Google Chat | ✅ Live | Workspace Add-on, `hostAppDataAction` response format |
| Local backend | ✅ Live | Open WebUI/Ollama on scott_gaming, per-user multi-model config |
| Gemini orchestrator | ✅ Live | Gemini API tool loop → Claude response; ⚡ toggle in UI |
| Local orchestrator | ✅ Live | OpenAI-compatible ReAct loop; fires when orchestrator role → local model |
| Tool audit log | ✅ Live | Every tool call logged to `home/{user}/tool_audit/YYYY-MM-DD.jsonl` |
| Token usage tracking | ✅ Live | Per-user `home/{user}/usage.json`; summary in Settings |
| Web push | ✅ Live | VAPID push notifications; `web_push` tool; subscribe via ☰ menu |
| Proactive notifications | ✅ Live | Daily reminder check (09:00); distill/cron completions; `GET /settings/notifications` dedicated page |
| Proactive cron | ✅ Live | 5 job types: `remind`, `note`, `message`, `brief`, `task` (full orchestrator loop); monthly/yearly schedule formats; HA inbound webhook tools toggle |
| Schedules web UI | ✅ Live | `/settings/crons` — list, add, edit, pause/resume, delete scheduled jobs |
### Active Tasks
Active users: scott (inara), holly (tina), brian (wintermute)
See `documentation/TODO__Agents.md` for the full list. Current priorities:
**69 orchestrator tools** across 17 domain modules:
web_search/http_fetch/web_read/http_post,
project_file_read/list + file_stat/grep/diff/syntax_check (project-scoped),
file_read/list/write/session_read/session_search (system-scoped, admin),
git_status/git_log/git_diff (read-only git inspection, project-scoped),
shell_exec/claude_allow_dir,
cortex_restart/logs/status/update,
task_list/create/update/complete, cron_list/add/remove/toggle,
reminders_add/list/remove/clear, scratch_read/write/append/clear,
web_push/email_send/nc_talk_send/nc_talk_history,
ae_journal_list/search/entries_list/entry_read/create/update/disable/append/prepend,
ae_task_list, ae_db_query/describe/show_view (SELECT-only MariaDB access, admin; disable requires confirm),
agent_notes_read/write/append/clear, spawn_agent/aider_run (admin; aider_run requires confirm),
agent_status/agent_list (user-level)/agent_cancel (admin, confirm-required),
ha_get_state/ha_get_states/ha_call_service.
- **[High]** Ollama backend — local LLM via `scott_gaming` over WireGuard
- **[Medium]** NC Talk — complete bot registration docs (`docs/NEXTCLOUD_TALK_BOT.md`)
- **[Medium]** Knowledge consolidation — markdown → AE Journals
Each tool has a `TOOL_RISK` rating (low/medium/high). Configure access at `/settings/tools`
(max_risk threshold + per-tool whitelist/blacklist). Risk policy stored in `home/{user}/tool_policy.json`.
### Recently Completed
- ✅ Session auth — bcrypt passwords, JWT cookies, login/logout, `SessionAuthMiddleware` — 2026-03-20
- ✅ Persona onboarding — invite tokens, self-service password setup, persona creation form — 2026-03-20
- ✅ Multi-persona switcher — dropdown in UI header, `/api/personas` endpoint — 2026-03-20
- ✅ SMTP invite email — `noreply@oneskyit.com`, HTML + plain text, `manage_passwords.py invite` — 2026-03-20
- ✅ CSS routing fix — `/static/*` mount must precede wildcard `/{user}/{persona}` route — 2026-03-20
- ✅ Multi-user/multi-persona support (`home/{username}/persona/{name}/` two-level layout) — 2026-03-20
- ✅ Scratchpad, task management, and cron/scheduled job tools — 2026-03-20
- ✅ Test suite (80 tests) covering API, persona routing, tools, security — 2026-03-20
- ✅ Google Chat bot (Workspace Add-on, JWT auth, `hostAppDataAction` format) — 2026-03-20
- ✅ Orchestrator Agent mode UI + session persistence — 2026-03-18
- ✅ Memory distiller (APScheduler, short/mid/long) — 2026-03
See `documentation/TODO__Agents.md` for the active task list.
See `documentation/ROADMAP.md` for phases and what's next.
---
@@ -249,8 +309,14 @@ See `documentation/TODO__Agents.md` for the full list. Current priorities:
| File | Purpose |
|---|---|
| `documentation/MASTER.md` | **Start here** — index, current state, all doc links |
| `documentation/TODO__Agents.md` | Active task list — read before starting work |
| `documentation/ARCH__Intelligence_Layer.md` | Full architecture design |
| `~/agents_sync/projects/CORTEX.md` | High-level project vision and phases |
| `documentation/ROADMAP.md` | Phases — what's done, what's next |
| `documentation/ARCH__SYSTEM.md` | System architecture and component map |
| `documentation/ARCH__BACKENDS.md` | LLM backends, routing, per-user config |
| `documentation/ARCH__PERSONA.md` | Persona system, context tiers, memory distillation |
| `documentation/ARCH__CHANNELS.md` | Input channels — web, NC Talk, Google Chat, cron |
| `documentation/ARCH__FUTURE.md` | Planned: local orchestrator, dev agents, knowledge layer |
| `~/agents_sync/projects/CORTEX.md` | Project vision and philosophy |
| `~/agents_sync/CLAUDE.md` | Fleet coordination rules |
| `~/CLAUDE.md` | Machine identity (`scott_lpt`) |

View File

@@ -0,0 +1,75 @@
{
"folders": [
{
"name": "cortex (service)",
"path": "cortex"
},
{
"name": "home (personas)",
"path": "home"
},
{
"name": "documentation",
"path": "documentation"
},
{
"name": "docs (integrations)",
"path": "docs"
},
{
"name": "project root",
"path": "."
}
],
"settings": {
"files.exclude": {
"**/__pycache__": true,
"**/*.pyc": true,
"cortex/.venv": true,
"cortex/data": true
},
"search.exclude": {
"**/__pycache__": true,
"cortex/.venv": true,
"cortex/data": true,
"home/**/sessions": true,
"home/**/session_data": true
},
"[python]": {
"editor.formatOnSave": false
},
"editor.rulers": [100],
"files.associations": {
"*.env": "dotenv",
"*.env.default": "dotenv"
}
},
"extensions": {
"recommendations": [
"ms-python.python",
"ms-python.vscode-pylance",
"humao.rest-client",
"tamasfe.even-better-toml"
]
},
"launch": {
"version": "0.2.0",
"configurations": [
{
"name": "Cortex (uvicorn dev)",
"type": "python",
"request": "launch",
"module": "uvicorn",
"args": [
"main:app",
"--host", "0.0.0.0",
"--port", "8000",
"--reload"
],
"cwd": "${workspaceFolder:cortex (service)}",
"envFile": "${workspaceFolder:cortex (service)}/.env",
"justMyCode": false
}
]
}
}

147
README.md
View File

@@ -6,7 +6,44 @@
> *"You can't stop the signal."*
Cortex is a self-hosted multi-agent AI platform. It supports multiple users, each with their own named AI persona. Inara (Scott's persona) and Tina (Holly's persona) are the initial instances.
Cortex is a self-hosted multi-agent AI platform. It supports multiple users, each with their own named AI persona.
---
## Where Cortex Fits
AI tools aren't one-size-fits-all. Cortex exists in a specific niche — it's not trying to be everything.
**Cortex is a self-hosted persona platform.** It gives you a persistent AI companion with its own
identity, memory, and voice — reachable through your chat apps, not just a browser tab. It remembers
who you are across days and weeks. It can proactively message you on a schedule. It runs on your
own hardware, behind your own auth.
### What Cortex is good at
- **Being a consistent AI presence** — same persona, same memory, day after day
- **Multi-channel access** — web, Nextcloud Talk, Google Chat, all routed to the same brain
- **Proactive work** — scheduled messages, reminders, cron jobs that reach out to you
- **Multi-user households** — each person gets their own persona (Scott → Inara, Holly → Tina)
- **Private, offline-capable** — local models via Ollama when you don't want anything leaving the LAN
### What Cortex is not
- **Not a coding assistant.** Cortex lives in chat apps, not in your terminal or IDE.
Use Claude Code, DeepSeek TUI, Gemini CLI, or Copilot for code-level work — they specialize in reading and
editing project files. Cortex can't open a codebase.
- **Not a generic LLM chat UI.** Open WebUI and LibreChat are excellent model-switching frontends.
Cortex isn't a frontend — it's a platform with its own identity system, orchestrator, and memory
pipeline. Two different jobs.
- **Not a SaaS product.** Nobody else hosts your Cortex instance. Nobody else sees your conversations.
The trade-off is you manage the service yourself — `systemctl --user restart cortex`.
- **Not an agent framework.** LangChain, CrewAI, and similar are libraries for building AI pipelines.
Cortex is a running service with concrete personas, not an abstraction layer to build on top of.
### The stack in practice
- Use **Cortex** to talk to Inara — daily assistant, memory keeper, scheduled check-ins
- Use **Claude Code / DeepSeek TUI** to work *on* Cortex — code edits, architecture, debugging
- Use **Open WebUI** when you want to test a new model or run a quick prompt without persona context
Same AI, different interfaces for different jobs.
---
@@ -16,9 +53,7 @@ Cortex is a self-hosted multi-agent AI platform. It supports multiple users, eac
|---|---|
| `cortex/` | FastAPI service — dispatcher, routing, LLM backends, session management |
| `home/` | User and persona data (`home/{username}/persona/{name}/`) |
| `home/scott/persona/inara/` | Inara identity, memory, and context files |
| `home/holly/persona/tina/` | Tina identity, memory, and context files |
| `docs/` | Integration reference docs (NC Talk bot, etc.) |
| `docs/` | Integration reference docs (NC Talk bot, Google Chat bot) |
| `documentation/` | Architecture decisions, project plans, agent task lists |
---
@@ -48,6 +83,20 @@ with a letter or underscore; max 32 characters. Example: `scott`, `holly`, `my_a
---
## Setup / Install
Run `install.py` on any machine to set up or update Cortex. It is idempotent — safe to re-run.
```bash
python3 install.py # install / update everything
python3 install.py --check # status check only, no changes
```
What it does: creates the Python venv, installs dependencies, writes the systemd user service,
enables linger, starts/restarts the service, checks LLM CLI auth, and sets up the daily backup timer.
Config: copy `cortex/.env.default` to `cortex/.env` and fill in secrets before first run.
## Running Cortex
Cortex runs as a **systemd user service** (no sudo required).
@@ -69,49 +118,83 @@ http://localhost:8000 (or cortex.dgrzone.com on WireGuard)
The service starts automatically at boot via `loginctl enable-linger`.
Service file: `~/.config/systemd/user/cortex.service`
Config lives in `cortex/config.py` and a `.env` file at the project root (not tracked — see `.env.default`).
Config lives in `cortex/config.py` and `cortex/.env` (not tracked — see `cortex/.env.default`).
## Development Workflow
The codebase lives in `agents_sync/` and syncs to all fleet machines via Syncthing.
Edit code on any machine; use `dev-restart.sh` to apply changes on the host running the service.
```bash
./dev-restart.sh # restart service, show last 30 log lines
./dev-restart.sh logs # tail live logs (ctrl-c to stop)
./dev-restart.sh status # show service status only
```
## Backup
Persona data (`home/`) is excluded from git and backed up with restic.
`install.py` sets up a systemd timer that runs `backup.sh` daily at 03:00.
```bash
./backup.sh # run a backup manually
# Inspect snapshots (set env vars or export them)
RESTIC_REPOSITORY=~/backups/cortex-home-restic \
RESTIC_PASSWORD_FILE=~/.config/cortex/restic-password \
restic snapshots
```
The restic password is generated at `~/.config/cortex/restic-password` on first install.
Back it up separately — it is required to restore from any snapshot.
---
## Key Documentation
**Start here for a full picture:** [`documentation/MASTER.md`](documentation/MASTER.md)
| File | Purpose |
|---|---|
| `documentation/TODO__Agents.md` | Active task list — read first |
| `documentation/ARCH__Intelligence_Layer.md` | Intelligence layer architecture (orchestrator, dev agents, knowledge) |
| `docs/NEXTCLOUD_TALK_BOT.md` | NC Talk bot setup |
| `home/scott/persona/inara/IDENTITY.md` | Inara persona and identity |
| `home/scott/persona/inara/HELP.md` | In-app help content (rendered in UI) |
| `home/scott/persona/inara/PROTOCOLS.md` | Inara behavioral protocols |
| `~/agents_sync/projects/CORTEX.md` | High-level project vision and phases |
| `documentation/MASTER.md` | Index — current state, all doc links, quick reference |
| `documentation/ROADMAP.md` | Phases — what's done, what's next |
| `documentation/TODO__Agents.md` | Active task list |
| `documentation/ARCH__SYSTEM.md` | System architecture and component map |
| `documentation/ARCH__BACKENDS.md` | LLM backends, routing, fallback |
| `documentation/ARCH__PERSONA.md` | Persona system, context tiers, memory distillation |
| `documentation/ARCH__CHANNELS.md` | Input channels — web, NC Talk, Google Chat, cron |
| `documentation/ARCH__FUTURE.md` | Planned features — local orchestrator, dev agents, knowledge layer |
| `docs/NEXTCLOUD_TALK_BOT.md` | NC Talk bot setup and troubleshooting |
| `docs/GOOGLE_CHAT_BOT.md` | Google Chat Add-on setup |
| `docs/OPEN_WEBUI_API.md` | Open WebUI/Ollama API reference |
---
## Architecture at a Glance
```
[User / Cron / Webhook]
[Web UI / NC Talk / Google Chat / Cron / Webhooks]
Cortex Dispatcher (FastAPI, cortex/)
├─ POST /chat — direct to LLM (streaming SSE)
├─ POST /orchestrate — Gemini tool loop → Claude response
├─ POST /webhook/nextcloud — Nextcloud Talk bot
└─ POST /webhook/google — Google Chat Add-on
├─ POST /webhook/nextcloud/{username} — Nextcloud Talk bot (per-user)
└─ POST /channels/google-chat/{username} — Google Chat Add-on (per-user)
LLM Backend(s)
• Claude CLI — primary reasoning, coding, long-context
• Gemini CLI — secondary / cost routing
• Gemini API — orchestrator tool loop (separate from Gemini CLI)
Ollama — offline/private (scott_gaming, future)
LLM Backends
• Claude CLI — primary, all user-facing responses
• Gemini CLI — fallback
• Gemini API — orchestrator tool loop (two-brain: Gemini plans, Claude responds)
Local OpenAI — Open WebUI/Ollama on scott_gaming; also runs local orchestrator loop
Persona context loaded from home/{user}/persona/{name}/
```
See `documentation/ARCH__Intelligence_Layer.md` for the orchestrator/responder and dev-agent architecture.
See `documentation/ARCH__SYSTEM.md` for the full architecture breakdown.
---
## Inara / Tina
## Personas
Each persona has its own identity, memory, and session history.
They are not tied to a specific LLM model — the name is fixed, the backend varies.
@@ -120,17 +203,24 @@ Context is loaded at request time from `home/{user}/persona/{name}/` via `cortex
| User | Persona | Description |
|---|---|---|
| scott | inara | Scott's primary AI assistant |
| scott | developer | Scott's dev-focused persona |
| holly | tina | Holly's primary AI assistant |
| brian | wintermute | Brian's primary AI assistant |
---
## Channels
| Channel | Status | Notes |
Webhook endpoints are per-user — each user configures their own secrets in `home/{username}/channels.json`.
| Channel | Status | Endpoint / Notes |
|---|---|---|
| Web UI | Live | `https://cortex.dgrzone.com` — session auth (login form + JWT cookie) |
| Nextcloud Talk | Live | HMAC-signed webhook, async reply |
| Google Chat | Live | Workspace Add-on, JWT auth |
| Nextcloud Talk | Live | `POST /webhook/nextcloud/{username}` HMAC-signed, async reply |
| Google Chat | Live | `POST /channels/google-chat/{username}` Workspace Add-on, JWT auth |
| Browser Push | Live | VAPID push notifications — subscribe via ☰ menu; proactive reminders + distill alerts |
See `docs/NEXTCLOUD_TALK_BOT.md` and `docs/GOOGLE_CHAT_BOT.md` for setup instructions.
---
@@ -142,7 +232,10 @@ cd cortex
# Create a user directory and send an invite email
.venv/bin/python manage_passwords.py invite <username> <email>
# List users with password and email status
# Register a Google account for sign-in (run after user completes onboarding)
.venv/bin/python manage_passwords.py google-add <username> <email>
# List users with password, Google, and email status
.venv/bin/python manage_passwords.py list
# Set/check a password directly
@@ -152,6 +245,8 @@ cd cortex
New users receive a link to `/setup/{token}` where they set their own password and create their first persona. Invite tokens expire in 72 hours and are one-time-use.
To enable a channel for a user, create `home/{username}/channels.json` — see the relevant doc in `docs/`.
---
## Testing

70
backup.sh Executable file
View File

@@ -0,0 +1,70 @@
#!/usr/bin/env bash
# backup.sh — restic backup of Cortex persona/home data
#
# Backs up the home/ directory (all user persona files, memory, tasks, crons).
# Code is in git; this covers everything git intentionally excludes.
#
# Config — override via environment or edit here:
REPO_DIR="${RESTIC_REPOSITORY:-$HOME/backups/cortex-home-restic}"
PASSWORD_FILE="${RESTIC_PASSWORD_FILE:-$HOME/.config/cortex/restic-password}"
SOURCE="$(cd "$(dirname "$0")" && pwd)/home"
# Retention policy
KEEP_DAILY=7
KEEP_WEEKLY=4
KEEP_MONTHLY=6
# ── Preflight ─────────────────────────────────────────────────────────────────
set -euo pipefail
if ! command -v restic &>/dev/null; then
echo "ERROR: restic not found. Install with: sudo pacman -S restic" >&2
exit 1
fi
if [[ ! -d "$SOURCE" ]]; then
echo "ERROR: source directory not found: $SOURCE" >&2
exit 1
fi
# ── Password setup ────────────────────────────────────────────────────────────
if [[ ! -f "$PASSWORD_FILE" ]]; then
mkdir -p "$(dirname "$PASSWORD_FILE")"
chmod 700 "$(dirname "$PASSWORD_FILE")"
python3 -c "import secrets; print(secrets.token_urlsafe(32))" > "$PASSWORD_FILE"
chmod 600 "$PASSWORD_FILE"
echo "Generated new restic password: $PASSWORD_FILE"
echo "IMPORTANT: back this file up separately — you need it to restore."
fi
export RESTIC_REPOSITORY="$REPO_DIR"
export RESTIC_PASSWORD_FILE="$PASSWORD_FILE"
# ── Init repo if needed ───────────────────────────────────────────────────────
if [[ ! -d "$REPO_DIR" ]]; then
echo "Initializing restic repository at $REPO_DIR"
restic init
fi
# ── Backup ────────────────────────────────────────────────────────────────────
echo "Backing up $SOURCE$REPO_DIR"
restic backup "$SOURCE" \
--exclude="**/sessions" \
--exclude="**/session_data" \
--tag "cortex-home"
# ── Prune ─────────────────────────────────────────────────────────────────────
restic forget \
--keep-daily "$KEEP_DAILY" \
--keep-weekly "$KEEP_WEEKLY" \
--keep-monthly "$KEEP_MONTHLY" \
--tag "cortex-home" \
--prune
echo "Backup complete."
restic snapshots --tag cortex-home --last 3

View File

@@ -1,15 +0,0 @@
[Unit]
Description=Cortex / Holly LLM Gateway
After=network.target
[Service]
Type=simple
User=scott
WorkingDirectory=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/cortex
EnvironmentFile=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/cortex/.env.holly
ExecStart=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/cortex/.venv/bin/uvicorn main:app --host 0.0.0.0 --port 8001
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target

View File

@@ -1,33 +1,118 @@
# Auth is handled by the claude CLI (claude setup-token) — no API key needed here.
# ANTHROPIC_API_KEY=only_needed_if_switching_to_sdk
# Cortex .env reference — copy to .env and fill in values
# DO NOT commit .env — it contains secrets
# Path to the inara/ identity directory — relative to cortex/ or absolute
INARA_DIR=../inara
# ── Agent identity ───────────────────────────────────────────────────────────
# Global display names used in distillation prompts and session logs.
# Individual persona identities live in home/{username}/persona/{name}/IDENTITY.md
AGENT_NAME=Inara
USER_NAME=Scott
# Path for persistent JSON session files
SESSIONS_DIR=./data/sessions
# ── Home directory ────────────────────────────────────────────────────────────
# Root for all user/persona data. Layout: home/{username}/persona/{name}/
# Relative paths are resolved from the cortex/ directory.
# Default: ../home (i.e. Cortex_and_Inara_dev/home/)
# HOME_DIR=../home
# LLM defaults
DEFAULT_MODEL=claude-sonnet-4-6
DEFAULT_TIER=2
# ── Google OAuth — "Sign in with Google" ────────────────────────────────────
# Create credentials at console.cloud.google.com → APIs & Services → Credentials
# Application type: Web Application
# Authorised redirect URI: https://cortex.dgrzone.com/auth/google/callback
# Pre-register users: cd cortex && .venv/bin/python manage_passwords.py google-add <user> <email>
# Per-user Gemini key: add "gemini_api_key": "AIza..." to home/{username}/auth.json
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
# Session rolling window — number of messages to keep (user + assistant pairs)
# 40 = 20 turns
MAX_HISTORY_MESSAGES=40
# ── Session auth ─────────────────────────────────────────────────────────────
# Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
JWT_SECRET=change-me-in-dotenv
JWT_EXPIRE_DAYS=30
# Per-backend timeouts (seconds)
# Gemini is generous — it frequently takes 30-60s under load
# Local models may need time to load into VRAM before first response
# ── SMTP (invite emails + future notifications) ───────────────────────────────
SMTP_SERVER=linode.oneskyit.com
SMTP_PORT=465
SMTP_USERNAME=send_mail
SMTP_PASSWORD=
SMTP_FROM_EMAIL=noreply@oneskyit.com
SMTP_FROM_NAME=Cortex
# Base URL included in invite links
CORTEX_BASE_URL=https://cortex.dgrzone.com
# ── Server ──────────────────────────────────────────────────────────────────
HOST=0.0.0.0
PORT=8000
# ── Google Chat bot ──────────────────────────────────────────────────────────
# JWT audience for verifying inbound Workspace Add-on Chat webhook requests.
# For Workspace Add-on Chat apps, the aud claim = the endpoint URL.
# Leave blank to disable verification (dev/testing only).
GOOGLE_CHAT_AUDIENCE=https://cortex.dgrzone.com/channels/google-chat
# ── Nextcloud Talk bot ───────────────────────────────────────────────────────
NEXTCLOUD_URL=https://cloud.dgrzone.com
NEXTCLOUD_TALK_BOT_SECRET=
# ── LLM backends ────────────────────────────────────────────────────────────
# Primary backend: "claude", "gemini", or "local" (switchable at runtime via UI)
PRIMARY_BACKEND=claude
# Timeouts in seconds
TIMEOUT_CLAUDE=60
TIMEOUT_GEMINI=120
TIMEOUT_LOCAL=300
TIMEOUT_LOCAL=300 # local models may need time to load
# Google Chat — must respond within 30s or Chat shows an error to the user
GOOGLE_CHAT_TIMEOUT=25
# Backend pinned for Google Chat (claude recommended — more reliable within 25s)
GOOGLE_CHAT_BACKEND=claude
# TODO: add GOOGLE_CHAT_TOKEN for request verification once endpoint is public
# ── Local model (Open WebUI / Ollama — OpenAI-compatible API) ────────────────
# Leave LOCAL_API_URL blank to disable. When set, "local" appears as a backend option.
# API key: Open WebUI → Settings → Account → API Keys
# Model: workspace alias or full Ollama model name
LOCAL_API_URL=http://192.168.32.19:3000
LOCAL_API_KEY=
LOCAL_MODEL=test-agent-simple
# Server
PORT=8000
HOST=0.0.0.0
# ── Orchestrator (Gemini API — not Gemini CLI) ───────────────────────────────
# Required for /orchestrate endpoint and tool use
# Free tier key: https://aistudio.google.com/apikey
GEMINI_API_KEY=
# Model for the orchestration tool loop (not the user-facing response)
ORCHESTRATOR_MODEL=gemini-2.5-flash
# Safety cap on tool loop iterations
ORCHESTRATOR_MAX_ROUNDS=10
# ── DuckDuckGo search ────────────────────────────────────────────────────────
# Leave blank for free unauthenticated tier
# Set to your API key for higher rate limits (paid DuckDuckGo account)
DDG_API_KEY=
DDG_MAX_RESULTS=5
# ── Aether Platform API ───────────────────────────────────────────────────────
# Used by orchestrator tools: ae_journal_search, ae_journal_entry_create, ae_task_list
# Same values as agents_sync/mcp/.env — copy from there
AE_API_URL=https://dev-api.oneskyit.com
AE_API_KEY=
AE_ACCOUNT_ID=
AE_API_TIMEOUT=15
# ── Aether MariaDB (direct — SELECT-only via ae_db_query/describe/show_view tools) ─
# Configured per-user in home/{username}/channels.json — NOT in .env.
# Add this block to the user's channels.json to enable the tools:
#
# "aether_db": {
# "host": "192.168.64.5",
# "port": 3306,
# "name": "aether_dev",
# "user": "aether_dev",
# "password": "..."
# }
# ── Distillation schedule ────────────────────────────────────────────────────
SCHEDULER_TIMEZONE=America/New_York
AUTO_DISTILL=true
AUTO_DISTILL_SHORT=true
AUTO_DISTILL_MID=true
AUTO_DISTILL_LONG=false # manual review recommended before enabling
# Memory tier token budgets (soft caps)
MEMORY_BUDGET_SHORT=3000
MEMORY_BUDGET_MID=2000
MEMORY_BUDGET_LONG=2000

158
cortex/agent_manager.py Normal file
View File

@@ -0,0 +1,158 @@
"""
Agent lifecycle manager — registry for background spawn_agent and aider_run tasks.
Tracks running and recently completed agents in-process. On completion, fires
notification.notify() if notify=True (same channel used by reminders and cron jobs).
Records are kept for 24 hours after completion, then pruned on next registration.
"""
import asyncio
import logging
import uuid
from dataclasses import dataclass, field
from datetime import datetime, timedelta
logger = logging.getLogger(__name__)
_PRUNE_AFTER = timedelta(hours=24)
_RESULT_PREVIEW_CHARS = 500
_TASK_PREVIEW_CHARS = 200
@dataclass
class AgentRecord:
agent_id: str
level: int # 1 = persona, 2 = specialized sub-agent, 3 = support agent
role: str # e.g. "coder", "research", "chat"
task: str # first _TASK_PREVIEW_CHARS of the task
status: str # running / done / failed / cancelled / timeout
started: datetime
user: str
parent_id: str | None = None # agent_id of the spawner (lineage tracking)
finished: datetime | None = None
result: str | None = None # first _RESULT_PREVIEW_CHARS on completion
notify: bool = False # push notification on completion
_task_ref: "asyncio.Task | None" = field(default=None, repr=False)
# Module-level registry — in-process only, not persisted across restarts.
_agents: dict[str, AgentRecord] = {}
_lock = asyncio.Lock()
async def register(
user: str,
role: str,
task: str,
level: int = 2,
parent_id: str | None = None,
notify: bool = False,
) -> AgentRecord:
"""Create and register a new running agent. Returns the record (agent_id is set)."""
agent_id = str(uuid.uuid4())
rec = AgentRecord(
agent_id=agent_id,
level=level,
role=role,
task=task[:_TASK_PREVIEW_CHARS],
status="running",
started=datetime.now(),
user=user,
parent_id=parent_id,
notify=notify,
)
async with _lock:
_prune_locked()
_agents[agent_id] = rec
logger.info(
"agent_manager: registered %s role=%s level=%d user=%s task=%.60s",
agent_id[:8], role, level, user, task,
)
return rec
def set_task_ref(agent_id: str, task_ref: "asyncio.Task") -> None:
"""Store the asyncio.Task reference so it can be cancelled later.
Call immediately after asyncio.create_task() — before the event loop yields.
"""
rec = _agents.get(agent_id)
if rec:
rec._task_ref = task_ref
async def finish(agent_id: str, result: str, status: str = "done") -> None:
"""Mark an agent complete, store the result, and notify the user if requested."""
async with _lock:
rec = _agents.get(agent_id)
if not rec:
return
rec.status = status
rec.finished = datetime.now()
rec.result = (result or "")[:_RESULT_PREVIEW_CHARS]
logger.info("agent_manager: finished %s status=%s", agent_id[:8], status)
if rec.notify and status != "cancelled":
try:
from notification import notify as _notify
elapsed = int((rec.finished - rec.started).total_seconds())
emoji = "" if status == "done" else "⚠️"
preview = (rec.result or "(no output)")[:200]
msg = f"{emoji} Agent done [{rec.role}, {elapsed}s]: {preview}"
await _notify(rec.user, msg)
except Exception as e:
logger.warning("agent_manager: notification failed for %s: %s", agent_id[:8], e)
async def cancel_agent(agent_id: str, user: str) -> str:
"""Cancel a running background agent. Returns a human-readable status message."""
async with _lock:
rec = _agents.get(agent_id)
if not rec:
return f"No agent found: {agent_id}"
if rec.user != user:
return "Access denied."
if rec.status != "running":
return f"Agent {agent_id[:8]}… is already {rec.status}."
task_ref = rec._task_ref
rec.status = "cancelled"
rec.finished = datetime.now()
if task_ref and not task_ref.done():
task_ref.cancel()
logger.info("agent_manager: cancelled %s by user=%s", agent_id[:8], user)
return f"Agent {agent_id[:8]}… cancelled."
def get(agent_id: str) -> AgentRecord | None:
"""Look up an agent record by ID."""
return _agents.get(agent_id)
def list_agents(user: str, status: str | None = None, limit: int = 10) -> list[AgentRecord]:
"""Return recent agents for a user, newest first.
Does not acquire the lock — safe for read-only listing (Python dict iteration is
thread-safe for reads; we don't care about racing with a concurrent registration).
"""
records = [r for r in _agents.values() if r.user == user]
if status:
records = [r for r in records if r.status == status]
records.sort(key=lambda r: r.started, reverse=True)
return records[:limit]
def _prune_locked() -> None:
"""Remove completed agents older than _PRUNE_AFTER. Must be called inside _lock."""
cutoff = datetime.now() - _PRUNE_AFTER
stale = [
aid for aid, r in _agents.items()
if r.status != "running" and r.finished and r.finished < cutoff
]
for aid in stale:
del _agents[aid]
if stale:
logger.debug("agent_manager: pruned %d stale records", len(stale))

View File

@@ -17,10 +17,11 @@ from starlette.responses import RedirectResponse, JSONResponse
from auth_utils import COOKIE_NAME, decode_token
# Paths that don't require a session cookie
_PUBLIC = {"/login", "/logout", "/health"}
_PUBLIC = {"/login", "/logout", "/health", "/manifest.json", "/sw.js", "/favicon.ico",
"/api/push/vapid-key"}
# Path prefixes that are always public (setup flow + webhooks)
_PUBLIC_PREFIXES = ("/setup/", "/channels/", "/webhook/")
# Path prefixes that are always public (setup flow + webhooks + Google OAuth)
_PUBLIC_PREFIXES = ("/setup/", "/channels/", "/webhook/", "/auth/google")
class SessionAuthMiddleware(BaseHTTPMiddleware):

View File

@@ -29,33 +29,102 @@ ALGORITHM = "HS256"
# ---------------------------------------------------------------------------
# Password helpers
# auth.json helpers — read/write without clobbering unrelated fields
# ---------------------------------------------------------------------------
def _auth_path(username: str) -> Path:
return settings.home_root() / username / "auth.json"
def _read_auth(username: str) -> dict:
path = _auth_path(username)
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except Exception:
return {}
def _write_auth(username: str, data: dict) -> None:
path = _auth_path(username)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, indent=2) + "\n")
# ---------------------------------------------------------------------------
# Password helpers
# ---------------------------------------------------------------------------
def set_password(username: str, password: str) -> None:
"""Hash and store a password for a user. Creates auth.json if needed."""
hashed = bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
_auth_path(username).write_text(json.dumps({"password_hash": hashed}) + "\n")
"""Hash and store a password. Preserves any existing fields in auth.json."""
data = _read_auth(username)
data["password_hash"] = bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
_write_auth(username, data)
logger.info("password set for user: %s", username)
def check_credentials(username: str, password: str) -> bool:
"""Return True if username+password are valid, False otherwise."""
path = _auth_path(username)
if not path.exists():
return False
try:
data = json.loads(path.read_text())
stored = data.get("password_hash", "").encode()
stored = _read_auth(username).get("password_hash", "").encode()
if not stored:
return False
return bcrypt.checkpw(password.encode(), stored)
except Exception:
return False
# ---------------------------------------------------------------------------
# Google OAuth helpers
# ---------------------------------------------------------------------------
def find_user_by_google(sub: str, email: str) -> str | None:
"""
Scan all users for one whose auth.json matches the given Google sub or email.
Sub match takes priority (stable); email match is a fallback for first sign-in.
Returns the username, or None if no match.
"""
root = settings.home_root()
if not root.exists():
return None
for user_dir in sorted(root.iterdir()):
if not user_dir.is_dir():
continue
data = _read_auth(user_dir.name)
if not data:
continue
if sub and data.get("google_sub") == sub:
return user_dir.name
if email and data.get("google_email", "").lower() == email.lower():
return user_dir.name
return None
def link_google(username: str, sub: str, email: str) -> None:
"""Store / update Google sub and email in a user's auth.json."""
data = _read_auth(username)
data["google_sub"] = sub
data["google_email"] = email
_write_auth(username, data)
logger.info("Google account linked for user: %s (%s)", username, email)
def get_user_gemini_key(username: str) -> str | None:
"""Return the user's personal Gemini API key, or None to use the server key."""
return _read_auth(username).get("gemini_api_key") or None
def get_user_role(username: str) -> str:
"""Return the user's role: 'admin' or 'user' (default).
Role is stored as auth.json["role"]. Any unrecognised value falls back to 'user'.
Set via: manage_passwords.py role <username> admin|user
"""
role = _read_auth(username).get("role", "user")
return role if role in ("admin", "user") else "user"
# ---------------------------------------------------------------------------
# JWT helpers
# ---------------------------------------------------------------------------
@@ -136,3 +205,56 @@ def consume_invite(username: str) -> None:
path.write_text(json.dumps(data) + "\n")
except Exception:
pass
# ---------------------------------------------------------------------------
# Per-user channel config
# ---------------------------------------------------------------------------
def _channels_path(username: str) -> Path:
return settings.home_root() / username / "channels.json"
def get_user_channels(username: str) -> dict:
"""Return the parsed channels.json for a user, or {} if not found."""
path = _channels_path(username)
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except Exception:
return {}
def get_tool_policy(username: str) -> dict:
"""Return the parsed tool_policy.json for a user.
Confirmation-gate keys (existing):
allow — tools in CONFIRM_REQUIRED that this user has pre-approved (skip gate)
deny — tools always blocked for this user regardless of global CONFIRM_REQUIRED
Risk-policy keys (new):
max_risk — auto-include all tools at/below this level ("low"|"medium"|"high")
whitelist — force-include specific tools above max_risk
blacklist — force-exclude specific tools regardless of max_risk
"""
path = settings.home_root() / username / "tool_policy.json"
try:
return json.loads(path.read_text())
except Exception:
return {}
def get_risk_policy(username: str) -> tuple[str | None, list[str], list[str]]:
"""Return (max_risk, whitelist, blacklist) from the user's tool policy."""
policy = get_tool_policy(username)
return (
policy.get("max_risk") or None,
policy.get("whitelist") or [],
policy.get("blacklist") or [],
)
def save_tool_policy(username: str, data: dict) -> None:
path = settings.home_root() / username / "tool_policy.json"
path.write_text(json.dumps(data, indent=2) + "\n")

View File

@@ -5,6 +5,12 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
anthropic_api_key: str | None = None # not used — claude CLI handles auth
# Google OAuth — "Sign in with Google" for all users
# Create credentials at console.cloud.google.com → APIs & Services → Credentials
# Add https://<your-domain>/auth/google/callback as an authorised redirect URI
google_client_id: str | None = None
google_client_secret: str | None = None
# Orchestrator (Gemini API — separate from Gemini CLI)
# Get a key at: https://aistudio.google.com/apikey (free tier is sufficient)
gemini_api_key: str | None = None
@@ -34,26 +40,17 @@ class Settings(BaseSettings):
max_history_messages: int = 40 # rolling window — 20 turns (user + assistant)
primary_backend: str = "claude" # "claude" or "gemini" — other is always fallback
# Local model backend — OpenAI-compatible API (Open WebUI / Ollama)
# Set LOCAL_API_URL in .env to enable; leave blank to disable
local_api_url: str = "" # e.g. http://192.168.32.19:3000
local_api_key: str = "" # sk-... from Open WebUI → Settings → Account → API Keys
local_model: str = "" # workspace or model name, e.g. test-agent-simple
# Per-backend timeouts in seconds
timeout_claude: int = 60
timeout_gemini: int = 120 # frequently slow under load
timeout_local: int = 300 # local models may need to load first
# Google Chat
# JWT audience (aud) claim to verify on inbound webhook requests.
# Google Chat sets aud = the Google Cloud project number (e.g. "741112865538").
# Set to "" to disable verification (dev/testing only).
google_chat_audience: str = ""
# Google Chat must receive a response within 30s or shows an error to the user
google_chat_timeout: int = 25
# Backend forced for Google Chat — Claude is more reliable within the 25s deadline
google_chat_backend: str = "claude"
# Nextcloud Talk bot
nextcloud_url: str = "https://cloud.dgrzone.com"
nextcloud_talk_bot_secret: str = "" # set in .env
nextcloud_talk_timeout: int = 55
# Auto-distillation schedule — override in .env
# AUTO_DISTILL=false disables entirely
scheduler_timezone: str = "America/New_York" # IANA tz — override in .env if needed
@@ -62,6 +59,26 @@ class Settings(BaseSettings):
auto_distill_mid: bool = True # weekly Sunday at 03:30 — LLM summarizes short → mid
auto_distill_long: bool = False # monthly 1st at 04:00 — off by default (manual review recommended)
# Which backend to use for distillation LLM calls.
# "" = use primary_backend (default); "local" = use local model (saves API credits).
# "long" stays on default (claude/gemini) for best quality.
distill_backend_mid: str = ""
distill_backend_long: str = ""
# Model registry: default backend type per role when user registry has no entry.
# Values: "claude_cli" | "gemini_cli" | "gemini_api" (builtin IDs)
# Override in .env: ROLE_CHAT=claude_cli ROLE_DISTILL=gemini_api etc.
role_chat: str = "claude_cli"
role_orchestrator: str = "gemini_api"
role_distill: str = "claude_cli"
role_coder: str = "claude_cli"
role_research: str = "gemini_api"
# Comma-separated list of standard roles shown in the model settings UI.
# Add custom roles here to extend the UI without code changes.
# Example: DEFINED_ROLES=chat,orchestrator,distill,coder,research,medical
defined_roles: str = "chat,orchestrator,distill,coder,research"
# Memory tier token budgets — soft caps used during distillation
# Override in .env: MEMORY_BUDGET_LONG=4000 etc.
memory_budget_long: int = 2000
@@ -72,6 +89,12 @@ class Settings(BaseSettings):
jwt_secret: str = "change-me-in-dotenv" # override in .env: JWT_SECRET=<random>
jwt_expire_days: int = 30
# Web Push (VAPID) — for browser push notifications
# Generate once with py_vapid; see push_utils.py for key format details
vapid_public_key: str = "" # base64url-encoded uncompressed EC point (for browser)
vapid_private_key_b64: str = "" # base64-encoded PEM private key (single-line .env storage)
vapid_contact: str = "mailto:admin@example.com"
# SMTP — for sending invite emails
smtp_server: str = ""
smtp_port: int = 465
@@ -87,6 +110,14 @@ class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
def get_defined_roles(self) -> list[str]:
"""Return the ordered list of standard roles from the defined_roles setting."""
return [r.strip() for r in self.defined_roles.split(",") if r.strip()]
def get_role_default(self, role: str) -> str:
"""Return the .env default backend type for a role (e.g. 'claude_cli')."""
return getattr(self, f"role_{role.replace('-', '_')}", "claude_cli")
def home_root(self) -> Path:
"""Resolve home_dir relative to this file's location if not absolute."""
if self.home_dir.is_absolute():

View File

@@ -1,4 +1,10 @@
from datetime import datetime
from pathlib import Path
from persona import persona_path
from tools.reminders import load_due_reminders
_STATIC_DIR = Path(__file__).parent / "static"
# Core identity files — always loaded regardless of tier
@@ -13,6 +19,10 @@ def load_context(
include_long: bool = True,
include_mid: bool = True,
include_short: bool = True,
role_append: str = "",
inject_datetime: bool = True,
inject_mode: bool = True,
mode: str = "chat",
) -> str:
"""
Build the system-prompt context block for a given tier and memory toggles.
@@ -24,10 +34,26 @@ def load_context(
Tier 2 — + USER full + PROTOCOLS + memory (~5,000 tokens)
Tier 3 — + last 2 raw session logs (~15,000 tokens)
Tier 4 — + last 7 raw session logs (~50,000 tokens)
role_append — optional text injected last (closest to the turn),
sourced from the active role's system_append config.
"""
inara_dir = persona_path()
parts = []
# ── 0. System block — date/time and session mode (injected first so it's prominent) ──
system_lines = []
if inject_datetime:
now = datetime.now().astimezone()
system_lines.append(f"Current date and time: {now.strftime('%A, %Y-%m-%d at %I:%M %p %Z')}")
if mode == "otr" and inject_mode:
system_lines.append(
"Current mode: Off The Record — "
"this conversation is private and will not be logged or included in memory distillation"
)
if system_lines:
parts.append("--- System ---\n" + "\n".join(system_lines))
# ── 1. Core identity (always) ──────────────────────────────────
for filename in _CORE:
path = inara_dir / filename
@@ -52,15 +78,24 @@ def load_context(
if proto_path.exists():
parts.append(f"--- PROTOCOLS.md ---\n{proto_path.read_text()}")
ops_path = inara_dir / "OPERATIONS.md"
if ops_path.exists():
parts.append(f"--- OPERATIONS.md ---\n{ops_path.read_text()}")
# Global tool reference (same for all personas)
tools_path = _STATIC_DIR / "TOOLS.md"
if tools_path.exists():
parts.append(f"--- TOOLS.md ---\n{tools_path.read_text()}")
# Persona-specific help additions (optional)
help_path = inara_dir / "HELP.md"
if help_path.exists():
if help_path.exists() and help_path.stat().st_size > 10:
parts.append(f"--- HELP.md ---\n{help_path.read_text()}")
# ── 4. Pending reminders (tier 2+) ────────────────────────────
# Written by cron jobs; cleared by Inara after acting on them.
reminders_path = inara_dir / "REMINDERS.md"
if reminders_path.exists() and reminders_path.stat().st_size > 10:
content = reminders_path.read_text().strip()
# Only due and undated reminders are surfaced — future-dated ones
# are stored in REMINDERS.md but suppressed until their date arrives.
content = load_due_reminders()
if content:
parts.append(f"--- REMINDERS.md ---\n{content}")
@@ -97,4 +132,8 @@ def load_context(
for sf in session_files:
parts.append(f"--- Session: {sf.name} ---\n{sf.read_text()}")
# ── 7. Role-specific instructions (always last — closest to the turn) ──
if role_append and role_append.strip():
parts.append(f"--- Role Context ---\n{role_append.strip()}")
return "\n\n".join(parts)

View File

@@ -10,16 +10,25 @@ Job schema:
"id": "c_abc123",
"label": "Human-readable name",
"schedule": "daily:09:00", # see parse_schedule() for all formats
"type": "remind" | "note",
"payload": "Text to write when the job fires",
"type": "remind" | "note" | "message" | "brief" | "task",
"payload": "Text or prompt when the job fires",
"channel": null | "nextcloud" | "google_chat", # for message/brief/task types
"enabled": true,
"created_at": "ISO 8601",
"last_run": null | "ISO 8601"
}
Job types:
remind → appends to inara/REMINDERS.md (auto-loaded into Inara's context)
note → appends to inara/SCRATCH.md (read on demand via scratch_read)
remind → appends to REMINDERS.md (auto-loaded into context at tier 2+)
note → appends to SCRATCH.md (read on demand via scratch_read)
message → sends payload as-is to notification channel
brief → calls LLM (no tools) with payload as prompt, sends response
(good for morning briefings, summaries, proactive check-ins)
task → runs full orchestrator tool loop with payload as the user request,
sends Claude's response to notification channel
(good for agentic scheduled work: research, file updates, checks)
Tools that require confirmation are skipped — pre-approve them
in Settings → Tools to allow them in scheduled tasks.
"""
import logging
@@ -81,6 +90,11 @@ def parse_schedule(schedule: str) -> dict:
"daily:HH:MM" → every day at HH:MM
"weekly:DOW" → every DOW at 09:00
"weekly:DOW:HH:MM" → every DOW at HH:MM
"monthly" → 1st of every month at 09:00
"monthly:DD" → day DD of every month at 09:00
"monthly:DD:HH:MM" → day DD of every month at HH:MM
"yearly:MM:DD" → every year on MM/DD at 09:00 (birthdays, anniversaries)
"yearly:MM:DD:HH:MM" → every year on MM/DD at HH:MM
"""
s = schedule.strip().lower()
@@ -108,9 +122,37 @@ def parse_schedule(schedule: str) -> dict:
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
return {"day_of_week": dow, "hour": h, "minute": m}
if s.startswith("monthly"):
rest = s[7:].lstrip(":")
if not rest:
return {"day": 1, "hour": _DEFAULT_HOUR, "minute": _DEFAULT_MINUTE}
parts = rest.split(":")
day = _parse_day(parts[0], schedule)
if len(parts) == 3:
h, m = _parse_hhmm(f"{parts[1]}:{parts[2]}", schedule)
else:
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
return {"day": day, "hour": h, "minute": m}
if s.startswith("yearly:"):
rest = s[7:].split(":")
if len(rest) < 2:
raise ValueError(
f"yearly requires at least MM:DD in {schedule!r}. "
f"Example: yearly:03:15 or yearly:03:15:09:00"
)
month = _parse_month(rest[0], schedule)
day = _parse_day(rest[1], schedule)
if len(rest) == 4:
h, m = _parse_hhmm(f"{rest[2]}:{rest[3]}", schedule)
else:
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
return {"month": month, "day": day, "hour": h, "minute": m}
raise ValueError(
f"Unrecognised schedule {schedule!r}. "
f"Valid formats: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM"
f"Valid formats: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM | "
f"monthly | monthly:DD | monthly:DD:HH:MM | yearly:MM:DD | yearly:MM:DD:HH:MM"
)
@@ -121,6 +163,26 @@ def _parse_hhmm(s: str, original: str) -> tuple[int, int]:
return int(parts[0]), int(parts[1])
def _parse_day(s: str, original: str) -> int:
try:
d = int(s)
except ValueError:
raise ValueError(f"Expected day number (131) in {original!r}, got {s!r}")
if not 1 <= d <= 31:
raise ValueError(f"Day must be 131 in {original!r}, got {d}")
return d
def _parse_month(s: str, original: str) -> int:
try:
m = int(s)
except ValueError:
raise ValueError(f"Expected month number (112) in {original!r}, got {s!r}")
if not 1 <= m <= 12:
raise ValueError(f"Month must be 112 in {original!r}, got {m}")
return m
# ---------------------------------------------------------------------------
# Execution
# ---------------------------------------------------------------------------
@@ -150,6 +212,89 @@ async def run_job(job: dict) -> None:
p.write_text(existing.rstrip() + "\n" + section)
logger.info("cron [note] fired: %s", label)
elif job_type == "message":
# Send payload text directly to the user's notification channel
from notification import notify
username = job.get("user") or "scott"
channel = job.get("channel") or None
await notify(username, payload, channel=channel)
logger.info("cron [message] sent: %s", label)
elif job_type == "brief":
# Run LLM with payload as the prompt, send response to notification channel.
# Great for morning briefings, reminders, proactive check-ins.
from context_loader import load_context
from llm_client import complete
from notification import notify
from persona import set_context
from config import settings as _s
username = job.get("user") or _s.user_name.lower()
persona_nm = job.get("persona") or _s.agent_name.lower()
channel = job.get("channel") or None
set_context(username, persona_nm)
system_prompt = load_context(2) # tier 2: identity + memory + user profile
try:
response_text, backend = await complete(
system_prompt=system_prompt,
messages=[{"role": "user", "content": payload}],
role="chat",
)
await notify(username, response_text, channel=channel)
logger.info("cron [brief] sent via %s: %s", backend, label)
except Exception as e:
logger.error("cron [brief] LLM error for %s: %s", label, e)
elif job_type == "task":
# Run the full orchestrator tool loop, send Claude's response to the
# notification channel. Tools that require confirmation are skipped in
# cron context — the user is notified to pre-approve them.
from orchestrator_engine import run as _orch_run
from context_loader import load_context
from notification import notify
from persona import set_context
from auth_utils import get_user_gemini_key, get_tool_policy, get_risk_policy
from config import settings as _s
username = job.get("user") or _s.user_name.lower()
persona_nm = job.get("persona") or _s.agent_name.lower()
channel = job.get("channel") or None
set_context(username, persona_nm)
system_prompt = load_context(2)
policy = get_tool_policy(username)
max_risk, whitelist, blacklist = get_risk_policy(username)
gemini_key = get_user_gemini_key(username)
try:
result = await _orch_run(
task=payload,
system_prompt=system_prompt,
gemini_api_key=gemini_key,
respond_with_claude=True,
confirm_allow=set(policy.get("allow") or []),
confirm_deny=set(policy.get("deny") or []),
max_risk=max_risk,
risk_whitelist=whitelist,
risk_blacklist=blacklist,
)
if result.checkpoint:
tool_name = (result.checkpoint.pending_calls[0].name
if result.checkpoint.pending_calls else "unknown tool")
msg = (
f"Scheduled task '{label}' paused — "
f"'{tool_name}' requires confirmation. "
"Pre-approve it in Settings → Tools to allow it in scheduled tasks."
)
await notify(username, msg, channel=channel)
logger.warning("cron [task] %s: confirmation required for %s", label, tool_name)
else:
await notify(username, result.response, channel=channel)
logger.info("cron [task] completed via %s: %s", result.backend, label)
except Exception as e:
logger.error("cron [task] error for %s: %s", label, e)
else:
logger.warning("cron: unknown type %r (job %s)", job_type, job.get("id"))
return

View File

@@ -4,6 +4,7 @@ import os
import signal
import subprocess
from config import settings
import event_bus
logger = logging.getLogger(__name__)
@@ -30,24 +31,83 @@ async def cleanup() -> None:
_active_pgroups.clear()
# Map from registry model type → dispatch function key
_TYPE_TO_BACKEND = {
"claude_cli": "claude",
"gemini_cli": "gemini",
"gemini_api": "gemini", # gemini_api falls back to CLI in this context
"local_openai": "local",
"anthropic_api": "anthropic_api",
}
# Explicit UI toggle values (kept for backward compat)
_EXPLICIT_BACKENDS = ("claude", "gemini", "local")
_FALLBACK = {"claude": "gemini", "gemini": "claude", "local": "claude", "anthropic_api": "claude"}
async def complete(
system_prompt: str,
messages: list[dict],
model: str | None = None,
role: str = "chat",
slot: str | None = None,
max_tokens: int = 2048,
attachment: dict | None = None,
) -> tuple[str, str]:
"""Returns (response_text, actual_backend_used)."""
if model in ("claude", "gemini"):
"""
Returns (response_text, actual_backend_used).
slot: Phase 3 — specific role slot ("primary" | "backup_1" | "backup_2").
Resolves that exact slot, no fallback chain. Takes priority over model.
model: legacy backend override ("claude" | "gemini" | "local") from old toggle.
None = resolve via model registry for the given role.
role: registry role used for slot/auto routing (default: "chat").
"""
import model_registry as _reg
from persona import _user
username = _user.get()
resolved_cfg: dict | None = None
if slot is not None:
# Phase 3: explicit slot selection — no fallback within the role
resolved_cfg = _reg.get_model_for_slot(username, role, slot)
if resolved_cfg:
primary = _TYPE_TO_BACKEND.get(resolved_cfg["type"], "claude")
else:
# Slot not configured — fall through to auto routing
slot = None
if slot is None:
if model in _EXPLICIT_BACKENDS:
# Legacy: explicit backend override from old UI toggle
if model == "local":
resolved_cfg = _reg.get_best_local_model(username, role)
if not resolved_cfg:
raise RuntimeError("No local model configured — add one at /settings/models")
primary = model
else:
# Auto: role-based routing via model registry
resolved = _reg.get_model_for_role(username, role)
if resolved:
resolved_cfg = resolved
primary = _TYPE_TO_BACKEND.get(resolved["type"], "claude")
else:
primary = settings.primary_backend
fallback = "gemini" if primary == "claude" else "claude"
fallback = _FALLBACK.get(primary, "claude")
try:
response = await _dispatch(primary, system_prompt, messages, model)
response = await _dispatch(primary, system_prompt, messages, resolved_cfg, attachment=attachment)
return response, primary
except Exception as e:
err_str = str(e)
if primary == "claude" and any(k in err_str for k in ("401", "authenticate", "expired", "OAuth")):
await event_bus.publish({"type": "claude_auth_expired"})
# Surface errors when a model is explicitly configured or a specific slot was pinned.
if resolved_cfg is not None:
logger.error("%s failed (no fallback — model explicitly configured): %s", primary, e)
raise
logger.warning("%s failed (%s) — falling back to %s", primary, e, fallback)
response = await _dispatch(fallback, system_prompt, messages, None)
return response, fallback
@@ -57,11 +117,16 @@ async def _dispatch(
backend: str,
system_prompt: str,
messages: list[dict],
model: str | None,
model_cfg: dict | None,
attachment: dict | None = None,
) -> str:
if backend == "gemini":
return await _gemini(system_prompt, messages)
return await _claude(system_prompt, messages, model)
if backend == "local":
return await _local(system_prompt, messages, model_cfg, attachment=attachment)
if backend == "anthropic_api":
return await _anthropic_api(system_prompt, messages, model_cfg)
return await _claude(system_prompt, messages, model_cfg)
def _fresh_claude_token() -> str | None:
@@ -81,14 +146,16 @@ def _fresh_claude_token() -> str | None:
return None
async def _claude(system_prompt: str, messages: list[dict], model: str | None) -> str:
async def _claude(system_prompt: str, messages: list[dict], model_cfg: dict | None) -> str:
model_name = (model_cfg or {}).get("model_name") if model_cfg else None
cmd = [
"claude", "--print",
"--no-session-persistence",
"--output-format", "text",
]
if model and model not in ("claude", "gemini"):
cmd.extend(["--model", model])
# Only pass --model if it's a real model name (not a backend type string)
if model_name and model_name not in ("claude", "gemini", "local", ""):
cmd.extend(["--model", model_name])
if system_prompt:
cmd.extend(["--system-prompt", system_prompt])
cmd.append(_build_conversation(messages))
@@ -104,6 +171,137 @@ async def _claude(system_prompt: str, messages: list[dict], model: str | None) -
return await _run(cmd, timeout=settings.timeout_claude, env=env)
async def _local(
system_prompt: str,
messages: list[dict],
model_cfg: dict | None = None,
attachment: dict | None = None,
) -> str:
"""OpenAI-compatible backend — Open WebUI / Ollama.
model_cfg is pre-resolved by complete() via model_registry.
Falls back to registry lookup if not provided.
attachment: optional image dict {filename, mime_type, data} for vision calls.
"""
import httpx
cfg = model_cfg
if not cfg:
# Fallback: resolve directly from registry
import model_registry as _reg
from persona import _user
cfg = _reg.get_best_local_model(_user.get())
if not cfg:
raise RuntimeError("No local model configured — add one at /settings/models")
api_url = cfg["api_url"]
api_key = cfg["api_key"]
model = cfg["model_name"]
if not api_url:
raise RuntimeError("local_api_url not configured — set LOCAL_API_URL in .env or add a host at /settings/models")
if not model:
raise RuntimeError("local_model not configured — add a model at /settings/models")
host_type = cfg.get("host_type", "openwebui")
# "openwebui" uses Open WebUI/Ollama path layout; "openai" uses standard OpenAI layout
chat_path = "/chat/completions" if host_type == "openai" else "/api/chat/completions"
logger.info("local backend (%s): %s @ %s", host_type, model, api_url)
msgs: list[dict] = []
if system_prompt:
msgs.append({"role": "system", "content": system_prompt})
# Build message list; inject image into the last user message when present.
for i, m in enumerate(messages):
is_last = (i == len(messages) - 1)
if is_last and m["role"] == "user" and attachment:
content: list[dict] = [{"type": "text", "text": m["content"]}]
content.append({
"type": "image_url",
"image_url": {"url": attachment["data"]},
})
msgs.append({"role": "user", "content": content})
else:
# Strip non-standard metadata fields before sending to the API
msgs.append({"role": m["role"], "content": m["content"]})
url = api_url.rstrip("/") + chat_path
headers: dict[str, str] = {}
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
payload = {"model": model, "messages": msgs}
async with httpx.AsyncClient(timeout=settings.timeout_local) as client:
resp = await client.post(url, json=payload, headers=headers)
resp.raise_for_status()
data = resp.json()
text = data["choices"][0]["message"]["content"]
if not text or not text.strip():
raise RuntimeError("Local model returned an empty response")
usage = data.get("usage") or {}
if usage.get("prompt_tokens") is not None:
import usage_tracker
from persona import _user
asyncio.create_task(usage_tracker.record(
username=_user.get(),
backend="local",
model_name=model,
prompt_tokens=usage.get("prompt_tokens", 0),
completion_tokens=usage.get("completion_tokens", 0),
))
return text.strip()
async def _anthropic_api(system_prompt: str, messages: list[dict], model_cfg: dict | None) -> str:
"""Direct Anthropic API backend using the anthropic SDK."""
try:
import anthropic
except ImportError:
raise RuntimeError("anthropic SDK not installed — run: pip install 'anthropic>=0.40.0'")
cfg = model_cfg or {}
api_key = cfg.get("api_key", "")
model_name = cfg.get("model_name") or settings.default_model
if not api_key:
raise RuntimeError("No Anthropic API key — add one at /settings/models")
client = anthropic.AsyncAnthropic(api_key=api_key)
msgs = [{"role": m["role"], "content": m["content"]} for m in messages]
kwargs: dict = {
"model": model_name,
"max_tokens": 4096,
"messages": msgs,
}
if system_prompt:
kwargs["system"] = system_prompt
resp = await client.messages.create(**kwargs)
text = resp.content[0].text if resp.content else ""
if not text.strip():
raise RuntimeError("Anthropic API returned an empty response")
if resp.usage:
import usage_tracker
from persona import _user
asyncio.create_task(usage_tracker.record(
username=_user.get(),
backend="anthropic_api",
model_name=model_name,
prompt_tokens=resp.usage.input_tokens,
completion_tokens=resp.usage.output_tokens,
))
return text.strip()
async def _gemini(system_prompt: str, messages: list[dict]) -> str:
# Gemini CLI spawns MCP child processes that keep stdout pipes open after responding.
# start_new_session=True puts the whole tree in its own process group so

View File

@@ -8,8 +8,8 @@ logging.basicConfig(level=logging.INFO, format="%(levelname)s:%(name)s: %(messag
from config import settings
from auth_middleware import SessionAuthMiddleware
from routers import chat, google_chat, nextcloud_talk, files, distill, auth, orchestrator
from routers import ui, onboarding
from routers import chat, google_chat, nextcloud_talk, homeassistant, files, distill, auth, orchestrator
from routers import ui, onboarding, settings, tools_settings, help, auth_google, local_llm, push, audit, usage, crons
@asynccontextmanager
@@ -30,27 +30,44 @@ app.add_middleware(SessionAuthMiddleware)
app.include_router(chat.router)
app.include_router(google_chat.router)
app.include_router(nextcloud_talk.router)
app.include_router(homeassistant.router)
app.include_router(files.router)
app.include_router(distill.router)
app.include_router(auth.router)
app.include_router(orchestrator.router)
app.include_router(push.router)
app.include_router(audit.router)
app.include_router(usage.router)
# Static files — must be mounted BEFORE ui.router so /static/* is matched first.
# ui.router has a wildcard /{username}/{persona} that would otherwise catch /static/style.css etc.
app.mount("/static", StaticFiles(directory="static"), name="static")
# Google OAuth — must be before ui.router (wildcard /{user}/{persona} would swallow it)
app.include_router(auth_google.router)
# Onboarding (invite tokens + persona creation — before ui.router)
app.include_router(onboarding.router)
# UI router (login + /{user}/{persona} — must be last to avoid swallowing API paths)
app.include_router(ui.router)
# Account settings
app.include_router(settings.router)
app.include_router(tools_settings.router)
app.include_router(local_llm.router)
app.include_router(crons.router)
# Help page
app.include_router(help.router)
# Health check — must be before ui.router so /{username} catch-all doesn't swallow it.
@app.get("/health")
async def health() -> dict:
return {"status": "ok"}
# UI router (login + /{user}/{persona} — must be last to avoid swallowing API paths)
app.include_router(ui.router)
if __name__ == "__main__":
uvicorn.run(
"main:app",

View File

@@ -6,9 +6,10 @@ Usage:
python manage_passwords.py set <username> # prompt for password
python manage_passwords.py set <username> <pass> # set directly (avoid in shell history)
python manage_passwords.py check <username> # test a password interactively
python manage_passwords.py list # show users, passwords, and emails
python manage_passwords.py list # show users, auth methods, and emails
python manage_passwords.py invite <username> [email] # generate + optionally email invite link
python manage_passwords.py email <username> <email> # store/update an email address
python manage_passwords.py google-add <username> <email> # register a user for Google sign-in
"""
import json
@@ -18,7 +19,7 @@ import getpass
# Add cortex/ to path so we can import config and auth_utils
sys.path.insert(0, str(__import__('pathlib').Path(__file__).parent))
from auth_utils import set_password, check_credentials, _auth_path, create_invite
from auth_utils import set_password, check_credentials, _auth_path, create_invite, link_google, _read_auth
from persona import list_users
from config import settings
@@ -96,10 +97,14 @@ def cmd_list(_args):
if not users:
print(" No users found in home/")
return
print(f" {'USER':<18} {'PW':<6} {'GOOGLE':<8} {'EMAIL'}")
print(f" {'-'*18} {'-'*6} {'-'*8} {'-'*30}")
for user in users:
has_pw = "✓ pw" if _auth_path(user).exists() else "✗ pw"
auth = _read_auth(user)
has_pw = "" if auth.get("password_hash") else ""
google = auth.get("google_email") or ""
email = get_email(user) or ""
print(f" {user:<20} {has_pw} {email}")
print(f" {user:<18} {has_pw:<6} {google:<36} {email}")
def cmd_email(args):
@@ -149,6 +154,41 @@ def cmd_invite(args):
print("Tip: python manage_passwords.py invite <username> <email> to email it next time.\n")
def cmd_google_add(args):
if len(args) < 2:
print("Usage: manage_passwords.py google-add <username> <google_email>")
sys.exit(1)
username, email = args[0], args[1].lower().strip()
# Ensure the user directory exists
(settings.home_root() / username).mkdir(parents=True, exist_ok=True)
# Store in auth.json (google_sub filled in on first sign-in) + profile.json (for invites)
link_google(username, sub="", email=email)
set_email(username, email)
print(f"Google sign-in registered for {username!r}: {email}")
print(f"They can now sign in at {settings.cortex_base_url}/login using that Google account.")
def cmd_role(args):
if len(args) < 2:
print("Usage: manage_passwords.py role <username> admin|user")
sys.exit(1)
username, role = args[0], args[1].lower().strip()
if role not in ("admin", "user"):
print("Role must be 'admin' or 'user'.")
sys.exit(1)
from auth_utils import _read_auth, _write_auth
data = _read_auth(username)
if not data:
print(f"User '{username}' not found — no auth.json.")
sys.exit(1)
old_role = data.get("role", "user")
data["role"] = role
_write_auth(username, data)
print(f"Role for '{username}': {old_role}{role}")
if __name__ == "__main__":
if len(sys.argv) < 2:
print(__doc__)
@@ -167,6 +207,10 @@ if __name__ == "__main__":
cmd_email(rest)
elif command == "invite":
cmd_invite(rest)
elif command == "google-add":
cmd_google_add(rest)
elif command == "role":
cmd_role(rest)
else:
print(f"Unknown command: {command}")
print(__doc__)

View File

@@ -1,9 +1,17 @@
"""
Inara tiered memory distillation.
Tiered memory distillation.
distill_short() — roll recent session logs → MEMORY_SHORT.md (no LLM)
distill_mid() — summarize MEMORY_SHORT → MEMORY_MID.md (LLM)
distill_long() — integrate MEMORY_MID → MEMORY_LONG.md (LLM)
Before any file is overwritten, two rolling backups are kept:
MEMORY_*.bak1.md — most recent backup (created just before last write)
MEMORY_*.bak2.md — backup before that
LLM responses are sanity-checked before writing. If the response looks like
a refusal, is too short, or is obviously not memory content, the distill is
aborted and the original file is left untouched.
"""
import logging
from datetime import datetime
@@ -16,6 +24,25 @@ logger = logging.getLogger(__name__)
# Rough chars-per-token estimate for budget enforcement
_CHARS_PER_TOKEN = 4
# Phrases that indicate the LLM refused or misunderstood the task
_REFUSAL_PREFIXES = (
"i'm sorry",
"i am sorry",
"i can't",
"i cannot",
"i'm unable",
"i am unable",
"as an ai",
"as a language model",
"i don't have access",
"i do not have access",
"i'm not able",
"i am not able",
)
# Minimum characters for a valid mid/long distill response
_MIN_RESPONSE_CHARS = 80
def _budget_chars(tokens: int) -> int:
return tokens * _CHARS_PER_TOKEN
@@ -25,7 +52,62 @@ def _read(path: Path) -> str:
return path.read_text() if path.exists() else ""
def distill_short(username: str | None = None, persona: str | None = None) -> dict:
def _rotate_backup(path: Path, n: int = 2) -> None:
"""Rotate up to n rolling backups of path before a write.
MEMORY_LONG.md → MEMORY_LONG.bak1.md (most recent), MEMORY_LONG.bak2.md (older)
"""
if not path.exists():
return
# Shift older backups down: bak(n-1) → bak(n), …, bak1 stays as bak1 source
for i in range(n, 1, -1):
older = path.parent / f"{path.stem}.bak{i}.md"
newer = path.parent / f"{path.stem}.bak{i - 1}.md"
if newer.exists():
older.write_text(newer.read_text())
# Current file → bak1
bak1 = path.parent / f"{path.stem}.bak1.md"
bak1.write_text(path.read_text())
def _sanity_check(response_text: str, context: str, existing_content: str = "") -> str | None:
"""Return an error string if the LLM response looks invalid, else None.
Checks:
- Minimum absolute length
- Refusal / AI preamble phrases
- Size shrinkage: new content must be at least 40% of the old (catches truncation)
- Size explosion: new content must not exceed 250% of the old (catches runaway output)
(Both bounds only apply when an existing file is present and reasonably sized.)
"""
stripped = response_text.strip()
if len(stripped) < _MIN_RESPONSE_CHARS:
return f"{context}: response too short ({len(stripped)} chars) — not writing"
first_line = stripped.lower().splitlines()[0]
if any(first_line.startswith(p) for p in _REFUSAL_PREFIXES):
return f"{context}: response looks like a refusal — not writing"
if existing_content:
old_len = len(existing_content.strip())
new_len = len(stripped)
if old_len >= _MIN_RESPONSE_CHARS * 4: # only compare when old file has real content
ratio = new_len / old_len
if ratio < 0.40:
return (
f"{context}: new content is only {ratio:.0%} of the old "
f"({new_len} vs {old_len} chars) — looks truncated, not writing"
)
if ratio > 2.50:
return (
f"{context}: new content is {ratio:.0%} of the old "
f"({new_len} vs {old_len} chars) — looks like runaway output, not writing"
)
return None
def distill_short(username: str, persona: str) -> dict:
"""
Roll the most recent session log files into MEMORY_SHORT.md.
No LLM involved — pure aggregation with budget truncation.
@@ -64,8 +146,9 @@ def distill_short(username: str | None = None, persona: str | None = None) -> di
)
out_path = inara_dir / "MEMORY_SHORT.md"
_rotate_backup(out_path)
out_path.write_text(header + body)
logger.info("distill_short: wrote %d chars from %d files", len(header) + len(body), len(parts))
logger.info("distill_short [%s/%s]: wrote %d chars from %d files", username, persona, len(header) + len(body), len(parts))
return {
"files_included": len(parts),
@@ -74,57 +157,78 @@ def distill_short(username: str | None = None, persona: str | None = None) -> di
}
async def distill_mid(username: str | None = None, persona: str | None = None) -> dict:
async def distill_mid(username: str, persona: str) -> dict:
"""
Ask the LLM to summarize MEMORY_SHORT.md → MEMORY_MID.md.
Backs up the current MEMORY_MID.md before overwriting.
"""
from llm_client import complete
from persona import set_context
inara_dir = _persona_path(username, persona)
u, p = username, persona
set_context(u, p)
inara_dir = _persona_path(u, p)
short_content = _read(inara_dir / "MEMORY_SHORT.md")
existing_mid = _read(inara_dir / "MEMORY_MID.md")
if not short_content.strip() or "Not yet populated" in short_content:
return {"error": "MEMORY_SHORT.md is empty — run distill/short first"}
budget_tokens = settings.memory_budget_mid
persona_name = p.title()
user_name = u.title()
system_prompt = (
f"You are {settings.agent_name}'s memory distillation system. "
f"You are {persona_name}'s memory distillation system. "
"Summarize the following recent session logs into a concise mid-term memory digest. "
f"Target length: under {budget_tokens} tokens. "
"Focus on: recurring themes, important decisions made, ongoing projects, "
f"{settings.user_name}'s current state and priorities, and anything that should persist into future sessions. "
f"Write in first person as {settings.agent_name} (e.g. '{settings.user_name} and I worked on...'). "
f"{user_name}'s current state and priorities, and anything that should persist into future sessions. "
f"Write in first person as {persona_name} (e.g. '{user_name} and I worked on...'). "
"Use markdown headings. Be specific and concrete — no filler."
)
response_text, backend = await complete(
system_prompt=system_prompt,
messages=[{"role": "user", "content": short_content}],
role="distill",
)
err = _sanity_check(response_text, "distill_mid", existing_mid)
if err:
logger.warning(err)
return {"error": err}
now = datetime.now().strftime("%Y-%m-%d %H:%M")
header = (
f"# MEMORY_MID.md — Mid-Term Memory Digest\n\n"
f"*Auto-distilled: {now} via {backend}.*\n\n---\n\n"
)
out_path = inara_dir / "MEMORY_MID.md"
_rotate_backup(out_path)
out_path.write_text(header + response_text)
logger.info("distill_mid: wrote %d chars via %s", len(header) + len(response_text), backend)
logger.info("distill_mid [%s/%s]: wrote %d chars via %s", u, p, len(header) + len(response_text), backend)
return {
"username": u,
"backend": backend,
"chars_written": len(header) + len(response_text),
"budget_tokens": budget_tokens,
}
async def distill_long(username: str | None = None, persona: str | None = None) -> dict:
async def distill_long(username: str, persona: str) -> dict:
"""
Ask the LLM to integrate MEMORY_MID.md into MEMORY_LONG.md.
Backs up the current MEMORY_LONG.md before overwriting.
"""
from llm_client import complete
from persona import set_context
inara_dir = _persona_path(username, persona)
u, p = username, persona
set_context(u, p)
inara_dir = _persona_path(u, p)
long_content = _read(inara_dir / "MEMORY_LONG.md")
mid_content = _read(inara_dir / "MEMORY_MID.md")
@@ -132,8 +236,9 @@ async def distill_long(username: str | None = None, persona: str | None = None)
return {"error": "MEMORY_MID.md is empty — run distill/mid first"}
budget_tokens = settings.memory_budget_long
persona_name = p.title()
system_prompt = (
f"You are {settings.agent_name}'s long-term memory curator. "
f"You are {persona_name}'s long-term memory curator. "
"You will receive the current long-term memory and a recent mid-term digest. "
f"Integrate the new information into the long-term memory. Target: under {budget_tokens} tokens. "
"Rules: preserve important historical facts; update or replace stale information; "
@@ -149,22 +254,30 @@ async def distill_long(username: str | None = None, persona: str | None = None)
response_text, backend = await complete(
system_prompt=system_prompt,
messages=[{"role": "user", "content": user_content}],
role="distill",
)
err = _sanity_check(response_text, "distill_long", long_content)
if err:
logger.warning(err)
return {"error": err}
# Ensure the file has the right header if the LLM dropped it
now = datetime.now().strftime("%Y-%m-%d %H:%M")
if not response_text.lstrip().startswith("# MEMORY_LONG"):
response_text = (
f"# MEMORY_LONG.md — {settings.agent_name} Long-Term Memory\n\n"
f"# MEMORY_LONG.md — {persona_name} Long-Term Memory\n\n"
f"*Last distilled: {now} via {backend}.*\n\n---\n\n"
+ response_text
)
out_path = inara_dir / "MEMORY_LONG.md"
_rotate_backup(out_path)
out_path.write_text(response_text)
logger.info("distill_long: wrote %d chars via %s", len(response_text), backend)
logger.info("distill_long [%s/%s]: wrote %d chars via %s", u, p, len(response_text), backend)
return {
"username": u,
"backend": backend,
"chars_written": len(response_text),
"budget_tokens": budget_tokens,

980
cortex/model_registry.py Normal file
View File

@@ -0,0 +1,980 @@
"""
Per-user unified model registry — V2.
Stored in: home/{user}/model_registry.json
V2 Schema:
{
"version": 2,
# Per-provider accounts / credentials (user-configured)
"providers": {
"anthropic": {
"credentials": [
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
]
},
"google": {
"accounts": [
{"id": "<hex>", "label": "My Google account", "api_key": "AIza..."}
]
}
},
# Local OpenAI-compatible hosts (unchanged from V1)
"hosts": [{"id", "label", "api_url", "api_key", "host_type"}, ...],
# User-registered model entries (all providers)
"models": [
{
"id": str, # unique within this registry
"type": str, # see TYPES below
"label": str, # human-readable
"model_name": str, # identifier sent to the API / CLI
"provider": str | null, # "anthropic" | "google" | "local" | null
"host_id": str | null, # local_openai only — references hosts[].id
"credential_id":str | null, # claude_cli only — references providers.anthropic.credentials
"account_id": str | null, # gemini_api only — references providers.google.accounts
"context_k": int, # context window in k tokens (informational)
"max_rounds": int | null, # per-model tool-loop cap; null = use orchestrator_max_rounds global
"tags": [str], # user-defined capability tags
},
],
# Role assignments — any model (any provider) can fill any role
"roles": {
"<role>": {
"primary": "<model_id>" | null,
"backup_1": "<model_id>" | null,
...
"backup_4": "<model_id>" | null,
},
},
}
Types:
"claude_cli" — Claude CLI subprocess (~/.claude/.credentials.json)
"gemini_cli" — Gemini CLI subprocess
"gemini_api" — Gemini API (google-genai SDK); account_id → api_key from providers.google
"local_openai" — OpenAI-compatible endpoint; host_id → api_url/api_key from hosts[]
"anthropic_api" — Anthropic SDK direct; credential_id → api_key from providers.anthropic.credentials
Built-in model IDs (always resolvable without a registry entry):
"claude_cli" — resolves to the default Claude CLI model
"gemini_cli" — resolves to Gemini CLI
"gemini_api" — resolves to Gemini API using GEMINI_API_KEY from .env
Role resolution for get_model_for_role(username, role):
1. User registry: roles[role].primary → backup_1 → ... → backup_4
2. .env default: ROLE_<ROLE>=<builtin_id>
3. Hardcoded last-resort defaults per role
4. claude_cli (absolute fallback)
"""
import json
import logging
import secrets
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
# ── Role-level tool defaults ───────────────────────────────────────────────────
# Applied when a user hasn't configured a custom tool list for a role.
# None = no restriction (all accessible tools); [] = no tools (pure text processing).
# "chat" is intentionally absent: the /chat endpoint never sends tool schemas anyway,
# and the orchestrator uses chat_role="chat" as its default — restricting it here
# would block all tools from every default orchestration request.
# "orchestrator" is intentionally absent — Phase 2 keyword routing narrows it per message.
ROLE_DEFAULT_TOOLS: dict[str, list[str] | None] = {
"distill": [], # pure text processing — no tools needed
"research": ["web_search", "web_read", "http_fetch"],
"coder": [
"project_file_read", "project_file_list", "file_stat", "file_grep",
"file_diff", "file_syntax_check", "file_read", "file_list", "file_write",
"git_status", "git_log", "git_diff", "shell_exec",
],
}
# ── Provider model catalogs ───────────────────────────────────────────────────
# Server-side defaults. Update here when providers release new models.
# Users can add entries via the settings UI (Phase 2).
ANTHROPIC_CATALOG: list[dict] = [
# Latest
{"id": "claude-opus-4-7", "label": "Claude Opus 4.7", "context_k": 1000},
{"id": "claude-sonnet-4-6", "label": "Claude Sonnet 4.6", "context_k": 1000},
{"id": "claude-haiku-4-5-20251001", "label": "Claude Haiku 4.5", "context_k": 200},
# Previous versions (still available, not deprecated)
{"id": "claude-opus-4-6", "label": "Claude Opus 4.6", "context_k": 1000},
{"id": "claude-sonnet-4-5", "label": "Claude Sonnet 4.5", "context_k": 200},
]
GOOGLE_CATALOG: list[dict] = [
# Stable / generally available
{"id": "gemini-2.5-pro", "label": "Gemini 2.5 Pro", "context_k": 1000},
{"id": "gemini-2.5-flash", "label": "Gemini 2.5 Flash", "context_k": 1000},
{"id": "gemini-2.5-flash-lite", "label": "Gemini 2.5 Flash-Lite", "context_k": 1000},
# Preview
{"id": "gemini-3.1-pro-preview", "label": "Gemini 3.1 Pro (preview)", "context_k": 1000},
{"id": "gemini-3-flash-preview", "label": "Gemini 3 Flash (preview)", "context_k": 1000},
{"id": "gemini-3.1-flash-lite-preview", "label": "Gemini 3.1 Flash-Lite (preview)", "context_k": 1000},
]
# Known OpenAI-compatible cloud inference services.
# All use host_type "openai" (/chat/completions + /models paths).
CLOUD_API_CATALOG: list[dict] = [
{"id": "openrouter", "label": "OpenRouter", "api_url": "https://openrouter.ai/api/v1"},
{"id": "openai", "label": "OpenAI", "api_url": "https://api.openai.com/v1"},
{"id": "groq", "label": "Groq", "api_url": "https://api.groq.com/openai/v1"},
{"id": "xai", "label": "X.ai / Grok", "api_url": "https://api.x.ai/v1"},
{"id": "together", "label": "Together.ai", "api_url": "https://api.together.xyz/v1"},
{"id": "fireworks", "label": "Fireworks.ai", "api_url": "https://api.fireworks.ai/inference/v1"},
{"id": "custom", "label": "Custom", "api_url": ""},
]
# ── Built-in model definitions ────────────────────────────────────────────────
def _builtins() -> dict[str, dict]:
return {
"claude_cli": {
"id": "claude_cli",
"type": "claude_cli",
"label": f"Claude (CLI) — {settings.default_model}",
"model_name": settings.default_model,
"context_k": 200,
"tags": ["chat", "persona", "creative"],
},
"gemini_cli": {
"id": "gemini_cli",
"type": "gemini_cli",
"label": "Gemini (CLI)",
"model_name": "",
"context_k": 1000,
"tags": ["chat", "research", "long_context"],
},
"gemini_api": {
"id": "gemini_api",
"type": "gemini_api",
"label": f"Gemini API — {settings.orchestrator_model}",
"model_name": settings.orchestrator_model,
"context_k": 1000,
"tags": ["orchestrator", "research", "long_context", "tools"],
},
}
_ROLE_LAST_RESORT: dict[str, str] = {
"chat": "claude_cli",
"orchestrator": "gemini_api",
"distill": "claude_cli",
"coder": "claude_cli",
"research": "gemini_api",
}
PRIORITY_KEYS = ["primary", "backup_1", "backup_2", "backup_3", "backup_4"]
REQUIRED_ROLES: list[str] = ["chat", "orchestrator", "distill"]
# ── Storage ───────────────────────────────────────────────────────────────────
def _registry_path(username: str) -> Path:
return settings.home_root() / username / "model_registry.json"
def _local_llm_path(username: str) -> Path:
return settings.home_root() / username / "local_llm.json"
def _auth_path(username: str) -> Path:
return settings.home_root() / username / "auth.json"
def _empty() -> dict:
return {
"version": 2,
"providers": _default_providers(),
"hosts": [],
"models": [],
"roles": {},
}
def _default_providers() -> dict:
return {
"anthropic": {
"credentials": [
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
]
},
"google": {
"accounts": []
},
}
def _normalize(data: dict) -> dict:
"""Back-fill missing fields introduced by schema additions."""
for h in data.get("hosts", []):
h.setdefault("host_type", "openwebui")
h.setdefault("max_concurrent", 3)
data.setdefault("providers", _default_providers())
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
data["providers"].setdefault("google", {"accounts": []})
return data
def _load(username: str) -> dict:
path = _registry_path(username)
if path.exists():
try:
data = json.loads(path.read_text())
if isinstance(data, dict) and "version" in data:
if data["version"] == 1:
data = _migrate_v1_to_v2(username, data)
_save(username, data)
return _normalize(data)
except (json.JSONDecodeError, OSError):
logger.warning("model_registry.json for %s is unreadable — starting fresh", username)
return _empty()
# No registry — try migrating from local_llm.json
legacy = _local_llm_path(username)
if legacy.exists():
data = _migrate_from_local_llm(username, legacy)
_save(username, data)
logger.info("Migrated local_llm.json → model_registry.json for %s", username)
return data
return _empty()
def _save(username: str, data: dict) -> None:
_registry_path(username).write_text(json.dumps(data, indent=2))
# ── Migration ─────────────────────────────────────────────────────────────────
def _migrate_v1_to_v2(username: str, data: dict) -> dict:
"""
Upgrade a V1 registry to V2.
Changes:
- Adds providers section with default structure
- Migrates gemini_api_key from auth.json → first Google account entry
- Sets version to 2
"""
logger.info("Migrating model_registry.json V1 → V2 for %s", username)
data["version"] = 2
if "providers" not in data:
data["providers"] = _default_providers()
else:
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
data["providers"].setdefault("google", {"accounts": []})
# Pull existing Gemini key from auth.json (stored there in V1) → first account entry
accounts = data["providers"]["google"]["accounts"]
if not accounts:
try:
auth = json.loads(_auth_path(username).read_text())
existing_key = auth.get("gemini_api_key")
if existing_key:
accounts.append({
"id": secrets.token_hex(4),
"label": "Gemini API Key",
"api_key": existing_key,
})
logger.info("Migrated gemini_api_key from auth.json → providers.google.accounts for %s", username)
except (OSError, json.JSONDecodeError):
pass
return data
def _migrate_from_local_llm(username: str, path: Path) -> dict:
"""Convert local_llm.json → V2 model_registry format."""
try:
old = json.loads(path.read_text())
except Exception:
return _empty()
data = _empty()
# Handle v0 flat format
if "hosts" not in old:
api_url = old.get("api_url") or settings.local_api_url
api_key = old.get("api_key") or settings.local_api_key
model_name = old.get("model") or settings.local_model
if not api_url:
return data
host_id = secrets.token_hex(4)
old = {
"hosts": [{"id": host_id, "label": "Local Model Server", "api_url": api_url, "api_key": api_key}],
"models": [{"id": secrets.token_hex(4), "host_id": host_id, "label": model_name, "model_name": model_name}] if model_name else [],
"active_model_id": None,
}
if old["models"]:
old["active_model_id"] = old["models"][0]["id"]
data["hosts"] = old.get("hosts", [])
for m in old.get("models", []):
data["models"].append({
"id": m["id"],
"type": "local_openai",
"label": m.get("label") or m.get("model_name", ""),
"model_name": m.get("model_name", ""),
"provider": "local",
"host_id": m.get("host_id"),
"context_k": 0,
"tags": [],
})
active_id = old.get("active_model_id")
if active_id and any(m["id"] == active_id for m in data["models"]):
data["roles"]["chat"] = {"primary": active_id}
if settings.distill_backend_mid == "local":
data["roles"]["distill"] = {"primary": active_id}
# Migrate Gemini key from auth.json
data = _migrate_v1_to_v2(username, {"version": 1, **data})
return data
# ── Model resolution ──────────────────────────────────────────────────────────
def _resolve_model(registry: dict, model_id: str) -> dict | None:
"""Resolve a model_id to its full config dict (credentials merged in), or None."""
builtins = _builtins()
# Built-in IDs take priority over user-defined entries with the same ID
if model_id in builtins:
return dict(builtins[model_id])
model = next((m for m in registry.get("models", []) if m["id"] == model_id), None)
if not model:
return None
model_type = model.get("type")
if model_type == "local_openai":
host_id = model.get("host_id")
host = next((h for h in registry.get("hosts", []) if h["id"] == host_id), None)
if not host:
logger.warning("model %s references missing host_id %s", model_id, host_id)
return None
return {
**model,
"api_url": host.get("api_url", ""),
"api_key": host.get("api_key", ""),
"host_type": host.get("host_type", "openwebui"),
}
if model_type == "gemini_api":
account_id = model.get("account_id")
if account_id:
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
account = next((a for a in accounts if a["id"] == account_id), None)
if account:
return {**model, "api_key": account.get("api_key", "")}
logger.warning("model %s references missing account_id %s", model_id, account_id)
return dict(model)
if model_type == "anthropic_api":
credential_id = model.get("credential_id")
if credential_id:
creds = registry.get("providers", {}).get("anthropic", {}).get("credentials", [])
cred = next((c for c in creds if c["id"] == credential_id), None)
if cred and cred.get("api_key"):
return {**model, "api_key": cred["api_key"]}
logger.warning("model %s references missing/keyless credential_id %s", model_id, credential_id)
return dict(model)
if model_type == "claude_cli":
return dict(model)
return dict(model)
def get_model_for_role(username: str, role: str) -> dict | None:
"""
Return the resolved model config for the given role.
Resolution order:
1. User registry: roles[role].primary → backup_1 → ... → backup_4
2. .env: ROLE_<ROLE> = builtin model ID
3. Hardcoded last-resort default per role
4. claude_cli (absolute fallback)
"""
registry = _load(username)
role_cfg = registry.get("roles", {}).get(role, {})
for key in PRIORITY_KEYS:
model_id = role_cfg.get(key)
if not model_id:
continue
resolved = _resolve_model(registry, model_id)
if resolved:
return resolved
logger.debug("role %s.%s = %s but model not found", role, key, model_id)
# .env default
env_type = settings.get_role_default(role)
builtins = _builtins()
if env_type and env_type in builtins:
return dict(builtins[env_type])
# Hardcoded last resort
fallback_id = _ROLE_LAST_RESORT.get(role, "claude_cli")
return dict(builtins.get(fallback_id, builtins["claude_cli"]))
def get_best_local_model(username: str, role: str = "chat") -> dict | None:
"""
Return the best available local_openai model for the given role.
Used when the user explicitly selects "local" backend in the UI.
"""
registry = _load(username)
role_cfg = registry.get("roles", {}).get(role, {})
for key in PRIORITY_KEYS:
model_id = role_cfg.get(key)
if not model_id:
continue
resolved = _resolve_model(registry, model_id)
if resolved and resolved.get("type") == "local_openai":
return resolved
for model in registry.get("models", []):
if model.get("type") == "local_openai":
resolved = _resolve_model(registry, model["id"])
if resolved:
return resolved
return None
def set_role_config(
username: str,
role: str,
system_append: str,
tools: list[str] | None,
inject_datetime: bool = True,
inject_mode: bool = True,
) -> None:
"""Save system_append, tools allow-list, and per-injection flags for a role.
tools=None clears the allow-list (role uses all accessible tools).
inject_datetime=False suppresses the date/time header for pure processing roles.
inject_mode=False suppresses the session mode (OTR) line for pure processing roles.
"""
data = _load(username)
roles = data.setdefault("roles", {})
if role not in roles:
roles[role] = {}
roles[role]["system_append"] = system_append.strip()
roles[role]["inject_datetime"] = inject_datetime
roles[role]["inject_mode"] = inject_mode
if tools is None:
roles[role].pop("tools", None)
else:
roles[role]["tools"] = [t for t in tools if t]
_save(username, data)
def get_role_config(username: str, role: str) -> dict:
"""
Return supplemental config for a role: system_append, tools, and injection flags.
All keys are optional in the registry — missing means "use defaults":
system_append: str — appended to the system prompt for this role
tools: list[str] | None — explicit tool allow-list (None = no restriction)
inject_datetime: bool — whether to inject current date/time (default True)
inject_mode: bool — whether to inject session mode (OTR) line (default True)
"""
registry = _load(username)
role_cfg = registry.get("roles", {}).get(role, {})
user_tools = role_cfg.get("tools")
if user_tools is None:
# No user-configured list — fall back to system defaults for this role
effective_tools: list[str] | None = ROLE_DEFAULT_TOOLS.get(role)
else:
# User has configured tools; preserve their setting (empty list → no restriction)
effective_tools = user_tools or None
return {
"system_append": role_cfg.get("system_append", ""),
"tools": effective_tools,
"inject_datetime": role_cfg.get("inject_datetime", True),
"inject_mode": role_cfg.get("inject_mode", True),
}
def get_model_for_slot(username: str, role: str, slot: str) -> dict | None:
"""
Resolve a single named priority slot from a role without walking the fallback chain.
Used by Phase 3 explicit slot selection — the user has pinned a specific model;
don't silently redirect to another slot if this one is empty or broken.
Returns None if the slot is unset or the model can't be resolved.
"""
if slot not in PRIORITY_KEYS:
return None
registry = _load(username)
model_id = registry.get("roles", {}).get(role, {}).get(slot)
if not model_id:
return None
return _resolve_model(registry, model_id)
def get_google_api_key(username: str, account_id: str | None = None) -> str | None:
"""
Return the best available Gemini API key for the user.
If account_id is specified, returns that account's key (or None if not found).
Otherwise returns the first configured account key, falling back to the
server-level GEMINI_API_KEY from .env.
"""
registry = _load(username)
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
if account_id:
account = next((a for a in accounts if a["id"] == account_id), None)
return account.get("api_key") if account else None
# First configured account
if accounts:
return accounts[0].get("api_key") or None
# Fall back to .env server key
return settings.gemini_api_key or None
# ── Read API ──────────────────────────────────────────────────────────────────
def get_registry(username: str) -> dict:
"""Return the full registry (providers + hosts + models + roles)."""
return _load(username)
def get_all_models(username: str) -> list[dict]:
"""Return all user-defined models (resolved — credentials/hosts merged in)."""
registry = _load(username)
out = []
for m in registry.get("models", []):
resolved = _resolve_model(registry, m["id"])
if resolved:
out.append(resolved)
return out
def get_defined_roles(username: str) -> dict[str, dict]:
"""Return the roles section, filling gaps with empty dicts."""
registry = _load(username)
roles = registry.get("roles", {})
return {role: roles.get(role, {}) for role in settings.get_defined_roles()}
def get_google_accounts(username: str) -> list[dict]:
"""Return Google account entries (api_key masked for display)."""
registry = _load(username)
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
return [
{
"id": a["id"],
"label": a.get("label", ""),
"hint": (a.get("api_key") or "")[:8] + "" if a.get("api_key") else "",
}
for a in accounts
]
def get_catalog(provider: str, username: str | None = None) -> list[dict]:
"""
Return the model catalog for a provider.
For now returns server defaults. Phase 2 will merge in per-user additions.
"""
if provider == "anthropic":
return list(ANTHROPIC_CATALOG)
if provider == "google":
return list(GOOGLE_CATALOG)
if provider == "cloud":
return list(CLOUD_API_CATALOG)
return []
# ── Write API — Google accounts ───────────────────────────────────────────────
def save_google_account(username: str, account_id: str | None,
label: str, api_key: str) -> str:
"""Create or update a Google account entry. Returns the account ID."""
data = _load(username)
accounts = data["providers"]["google"]["accounts"]
if account_id:
for a in accounts:
if a["id"] == account_id:
a["label"] = label.strip()
if api_key.strip():
a["api_key"] = api_key.strip()
_save(username, data)
return account_id
account_id = secrets.token_hex(4)
accounts.append({
"id": account_id,
"label": label.strip(),
"api_key": api_key.strip(),
})
_save(username, data)
return account_id
def remove_google_account(username: str, account_id: str) -> bool:
"""Remove a Google account. Clears any model entries that reference it."""
data = _load(username)
accounts = data["providers"]["google"]["accounts"]
before = len(accounts)
data["providers"]["google"]["accounts"] = [a for a in accounts if a["id"] != account_id]
# Clear role assignments for models that referenced this account
removed_model_ids = {
m["id"] for m in data.get("models", [])
if m.get("account_id") == account_id
}
data["models"] = [m for m in data.get("models", []) if m["id"] not in removed_model_ids]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["providers"]["google"]["accounts"]) < before
# ── Write API — Anthropic API keys ───────────────────────────────────────────
def get_anthropic_api_keys(username: str) -> list[dict]:
"""Return Anthropic API key credentials (type='api_key') with key masked for display."""
registry = _load(username)
creds = registry.get("providers", {}).get("anthropic", {}).get("credentials", [])
return [
{
"id": c["id"],
"label": c.get("label", ""),
"hint": (c.get("api_key") or "")[:8] + "" if c.get("api_key") else "no key",
}
for c in creds
if c.get("type") == "api_key"
]
def save_anthropic_api_key(username: str, key_id: str | None,
label: str, api_key: str) -> str:
"""Create or update an Anthropic API key credential. Returns the credential ID."""
data = _load(username)
creds = data["providers"]["anthropic"]["credentials"]
if key_id:
for c in creds:
if c["id"] == key_id and c.get("type") == "api_key":
c["label"] = label.strip() or c.get("label", "API Key")
if api_key.strip():
c["api_key"] = api_key.strip()
_save(username, data)
return key_id
key_id = secrets.token_hex(4)
creds.append({
"id": key_id,
"label": label.strip() or "API Key",
"type": "api_key",
"api_key": api_key.strip(),
})
_save(username, data)
return key_id
def remove_anthropic_api_key(username: str, key_id: str) -> bool:
"""Remove an Anthropic API key credential. Clears model entries that reference it."""
data = _load(username)
creds = data["providers"]["anthropic"]["credentials"]
before = len(creds)
data["providers"]["anthropic"]["credentials"] = [
c for c in creds if c["id"] != key_id
]
removed_model_ids = {
m["id"] for m in data.get("models", [])
if m.get("credential_id") == key_id
}
data["models"] = [m for m in data.get("models", []) if m["id"] not in removed_model_ids]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["providers"]["anthropic"]["credentials"]) < before
# ── Write API — Hosts ─────────────────────────────────────────────────────────
def save_host(username: str, host_id: str | None,
label: str, api_url: str, api_key: str,
host_type: str = "openwebui",
max_concurrent: int = 3) -> str:
"""Create or update a host. Returns the host ID."""
data = _load(username)
host_type = host_type if host_type in ("openwebui", "openai") else "openwebui"
max_concurrent = max(1, min(int(max_concurrent), 20))
if host_id:
for h in data["hosts"]:
if h["id"] == host_id:
h["label"] = label.strip()
h["api_url"] = api_url.strip()
h["host_type"] = host_type
h["max_concurrent"] = max_concurrent
if api_key.strip():
h["api_key"] = api_key.strip()
_save(username, data)
return host_id
host_id = None
host_id = secrets.token_hex(4)
data["hosts"].append({
"id": host_id,
"label": label.strip(),
"api_url": api_url.strip(),
"api_key": api_key.strip(),
"host_type": host_type,
"max_concurrent": max_concurrent,
})
_save(username, data)
return host_id
def remove_host(username: str, host_id: str) -> bool:
"""Remove a host and all models that reference it."""
data = _load(username)
before = len(data["hosts"])
removed_model_ids = {m["id"] for m in data["models"] if m.get("host_id") == host_id}
data["hosts"] = [h for h in data["hosts"] if h["id"] != host_id]
data["models"] = [m for m in data["models"] if m.get("host_id") != host_id]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) in removed_model_ids:
role_cfg[key] = None
_save(username, data)
return len(data["hosts"]) < before
# ── Write API — Models ────────────────────────────────────────────────────────
def save_model(username: str, model_id: str | None, host_id: str,
label: str, model_name: str, context_k: int = 0,
tags: list[str] | None = None,
max_rounds: int | None = None,
tools: bool = True,
reasoning_budget_tokens: int | None = None) -> str:
"""Create or update a local_openai model entry. Returns the model ID."""
data = _load(username)
tags = tags or []
if model_id:
for m in data["models"]:
if m["id"] == model_id:
m["host_id"] = host_id
m["label"] = label.strip() or model_name.strip()
m["model_name"] = model_name.strip()
m["context_k"] = context_k
m["max_rounds"] = max_rounds
m["tools"] = tools
m["tags"] = tags
m["reasoning_budget_tokens"] = reasoning_budget_tokens
_save(username, data)
return model_id
model_id = None
model_id = secrets.token_hex(4)
data["models"].append({
"id": model_id,
"type": "local_openai",
"label": label.strip() or model_name.strip(),
"model_name": model_name.strip(),
"provider": "local",
"host_id": host_id,
"context_k": context_k,
"max_rounds": max_rounds,
"tools": tools,
"tags": tags,
"reasoning_budget_tokens": reasoning_budget_tokens,
})
_save(username, data)
return model_id
def save_cloud_model(username: str, model_id: str | None,
provider: str, model_name: str, label: str,
account_id: str | None = None,
credential_id: str | None = None,
context_k: int = 0,
tags: list[str] | None = None,
max_rounds: int | None = None,
tools: bool = True) -> str:
"""
Create or update an Anthropic or Google model entry. Returns the model ID.
provider: "anthropic" | "google"
account_id: Google only — references providers.google.accounts[].id
credential_id: Anthropic only — "cli" for OAuth CLI, or a hex ID for an API key credential
"""
data = _load(username)
# Determine model type from credential (anthropic only)
if provider == "anthropic":
creds = data.get("providers", {}).get("anthropic", {}).get("credentials", [])
cred = next((c for c in creds if c["id"] == credential_id), None) if credential_id else None
entry_type = "anthropic_api" if (cred and cred.get("type") == "api_key") else "claude_cli"
elif provider == "google":
entry_type = "gemini_api"
else:
entry_type = "claude_cli"
tags = tags or []
entry: dict = {
"type": entry_type,
"label": label.strip() or model_name.strip(),
"model_name": model_name.strip(),
"provider": provider,
"context_k": context_k,
"max_rounds": max_rounds,
"tools": tools,
"tags": tags,
}
if account_id:
entry["account_id"] = account_id
if credential_id:
entry["credential_id"] = credential_id
if model_id:
for m in data["models"]:
if m["id"] == model_id:
m.update(entry)
_save(username, data)
return model_id
model_id = None
model_id = secrets.token_hex(4)
entry["id"] = model_id
data["models"].append(entry)
_save(username, data)
return model_id
def remove_model(username: str, model_id: str) -> bool:
"""Remove a model and clear any role assignments pointing to it."""
data = _load(username)
before = len(data["models"])
data["models"] = [m for m in data["models"] if m["id"] != model_id]
for role_cfg in data.get("roles", {}).values():
for key in PRIORITY_KEYS:
if role_cfg.get(key) == model_id:
role_cfg[key] = None
_save(username, data)
return len(data["models"]) < before
def get_custom_roles(username: str) -> list[str]:
"""
Return the user's custom (non-required) roles.
Falls back to config-defined roles minus required ones for migration.
"""
registry = _load(username)
if "custom_roles" in registry:
return [r for r in registry["custom_roles"] if r and r not in REQUIRED_ROLES]
from config import settings as _cfg
return [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
def get_all_roles(username: str) -> list[str]:
"""Return required roles followed by the user's custom roles."""
return list(REQUIRED_ROLES) + get_custom_roles(username)
def add_custom_role(username: str, role_name: str) -> bool:
"""Add a custom role. Returns False if the name is invalid or already a required role."""
role_name = role_name.strip().lower()
if not role_name or role_name in REQUIRED_ROLES:
return False
data = _load(username)
if "custom_roles" not in data:
from config import settings as _cfg
data["custom_roles"] = [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
if role_name not in data["custom_roles"]:
data["custom_roles"].append(role_name)
_save(username, data)
return True
def remove_custom_role(username: str, role_name: str) -> bool:
"""Remove a custom role. Required roles cannot be removed."""
if role_name in REQUIRED_ROLES:
return False
data = _load(username)
if "custom_roles" not in data:
from config import settings as _cfg
data["custom_roles"] = [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
if role_name in data["custom_roles"]:
data["custom_roles"].remove(role_name)
_save(username, data)
return True
def set_role(username: str, role: str, priority: str, model_id: str | None) -> bool:
"""
Assign a model to a role priority slot.
priority must be one of: primary, backup_1, backup_2, backup_3, backup_4
model_id None clears the slot.
Built-in IDs (claude_cli, gemini_cli, gemini_api) are always valid.
"""
if priority not in PRIORITY_KEYS:
return False
data = _load(username)
if model_id and model_id not in _builtins():
if not any(m["id"] == model_id for m in data["models"]):
return False
roles = data.setdefault("roles", {})
if role not in roles:
roles[role] = {}
roles[role][priority] = model_id or None
_save(username, data)
return True
# ── Utility ───────────────────────────────────────────────────────────────────
def fetch_models_from_host(api_url: str, api_key: str,
host_type: str = "openwebui") -> list[str]:
"""Synchronously fetch the model list from an OpenAI-compatible host."""
import httpx
path = "/api/models" if host_type == "openwebui" else "/models"
url = api_url.rstrip("/") + path
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
resp = httpx.get(url, headers=headers, timeout=10)
resp.raise_for_status()
data = resp.json()
models = data.get("data", [])
return sorted(m.get("id", m.get("name", "")) for m in models if m.get("id") or m.get("name"))

176
cortex/notification.py Normal file
View File

@@ -0,0 +1,176 @@
"""
Outbound notification helpers — send messages to user channels proactively.
Channel config lives in home/{user}/channels.json:
{
"notification_channel": "email" | "nextcloud" | "google_chat",
"notification_email": "<override address — defaults to login email>",
"nextcloud": {
"url": "...", "bot_secret": "...", "notification_room": "<token>", ...
},
"google_chat": {
"outbound_webhook": "https://chat.googleapis.com/v1/spaces/...", ...
}
}
If notification_channel is absent, defaults to "nextcloud" if configured.
"""
import asyncio
import hashlib
import hmac
import json
import logging
import secrets
import httpx
logger = logging.getLogger(__name__)
async def _send_nct_message(url: str, secret: str, room: str, message: str) -> None:
"""Post a message to a Nextcloud Talk room as the bot."""
endpoint = f"{url}/ocs/v2.php/apps/spreed/api/v1/bot/{room}/message"
random_str = secrets.token_hex(32)
sig = hmac.new(
secret.encode(),
(random_str + message).encode("utf-8"),
hashlib.sha256,
).hexdigest()
body = json.dumps({"message": message}, ensure_ascii=False).encode("utf-8")
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
endpoint,
content=body,
headers={
"Content-Type": "application/json",
"OCS-APIRequest": "true",
"X-Nextcloud-Talk-Bot-Random": random_str,
"X-Nextcloud-Talk-Bot-Signature": sig,
},
timeout=15,
)
if resp.status_code not in (200, 201):
logger.warning("notify NCT %s → HTTP %d: %s", room, resp.status_code, resp.text[:200])
else:
logger.info("notify NCT → %s (%d chars)", room, len(message))
except Exception as e:
logger.error("notify NCT error: %s", e)
async def _notify_nct(nct: dict, message: str, username: str) -> None:
room = nct.get("notification_room", "").strip()
url = nct.get("url", "").rstrip("/")
secret = nct.get("bot_secret", "")
if not room:
logger.debug("notify: NCT notification_room not set for %s — skipping", username)
return
if not url or not secret:
logger.warning("notify: NCT config incomplete for %s (missing url or secret)", username)
return
await _send_nct_message(url, secret, room, message)
async def _notify_email(username: str, message: str, email_override: str | None = None) -> None:
"""Send notification via email. Address = override → google_email from auth.json."""
from auth_utils import _read_auth
from email_utils import send_email
to_addr = email_override or _read_auth(username).get("google_email", "").strip()
if not to_addr:
logger.warning("notify: no email address for %s — set notification_email in channels.json", username)
return
ok = await asyncio.to_thread(
send_email,
to_email=to_addr,
subject="Cortex",
body_text=message,
body_html=message.replace("\n", "<br>"),
)
if ok:
logger.info("notify email → %s (%d chars)", to_addr, len(message))
else:
logger.warning("notify: email send failed for %s", username)
async def _notify_google_chat(webhook_url: str, message: str, username: str) -> None:
"""POST a message to a Google Chat space via incoming webhook."""
body = json.dumps({"text": message}, ensure_ascii=False).encode("utf-8")
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
webhook_url,
content=body,
headers={"Content-Type": "application/json"},
timeout=15,
)
if resp.status_code not in (200, 201):
logger.warning("notify Google Chat %s → HTTP %d: %s", username, resp.status_code, resp.text[:200])
else:
logger.info("notify Google Chat → %s (%d chars)", username, len(message))
except Exception as e:
logger.error("notify Google Chat error for %s: %s", username, e)
async def _notify_web_push(username: str, message: str) -> None:
"""Send a browser push notification."""
import push_utils
result = await push_utils.send_push(username, "Cortex", message, "")
if "error" in result:
logger.warning("notify web_push error for %s: %s", username, result["error"])
elif result.get("sent", 0) == 0:
logger.debug("notify web_push: no subscriptions for %s", username)
else:
logger.info("notify web_push → %s (%d device(s))", username, result["sent"])
async def notify(username: str, message: str, channel: str | None = None) -> None:
"""Send a notification to the user's preferred outbound channel.
Channel resolution order:
1. `channel` parameter if provided
2. `notification_channel` key in channels.json
3. "nextcloud" if notification_room is configured
4. Silent no-op
Supported channels: "web_push", "email", "nextcloud", "google_chat"
Configure via home/{user}/channels.json — see module docstring.
"""
from auth_utils import get_user_channels
channels = get_user_channels(username)
target = channel or channels.get("notification_channel", "").strip()
if not target:
# Auto-detect: nextcloud if a notification_room is set
nct = channels.get("nextcloud", {})
if nct.get("notification_room", "").strip():
target = "nextcloud"
else:
return
if target == "web_push":
await _notify_web_push(username, message)
elif target == "email":
email_override = channels.get("notification_email", "").strip() or None
await _notify_email(username, message, email_override=email_override)
elif target == "nextcloud":
nct = channels.get("nextcloud")
if not nct:
logger.debug("notify: nextcloud not configured for %s", username)
return
await _notify_nct(nct, message, username)
elif target == "google_chat":
gc = channels.get("google_chat", {})
webhook = gc.get("outbound_webhook", "").strip()
if not webhook:
logger.debug("notify: google_chat outbound_webhook not set for %s", username)
return
await _notify_google_chat(webhook, message, username)
else:
logger.debug("notify: channel %r not supported for outbound (user %s)", target, username)

View File

@@ -0,0 +1,531 @@
"""
OpenAI-compatible orchestrator engine.
Implements the same ReAct tool loop as orchestrator_engine.py but uses the
OpenAI tool calling format, which works with any OpenAI-compatible endpoint:
OpenRouter, LiteLLM, Open WebUI, Ollama (tool-capable models), etc.
The model both runs the tool loop AND writes the final user-facing response —
no separate handoff step needed when a single capable model handles everything.
Flow:
1. POST to {api_url}/chat/completions with tools + user message
2. If finish_reason == "tool_calls": execute tools, feed results back, repeat
3. If finish_reason == "stop": final assistant message is the user-facing response
Used when the "orchestrator" role in the model registry resolves to a local_openai
type model. The Gemini engine (orchestrator_engine.py) is used otherwise.
"""
import asyncio
import json
import logging
from openai import AsyncOpenAI, APIConnectionError, APIStatusError
from config import settings
from orchestrator_engine import OrchestrateCheckpoint, OrchestratorResult
from tools import OPENAI_TOOL_SCHEMAS, call_tool, get_openai_tools_for_role, get_tools_for_role, CONFIRM_REQUIRED, narrow_tools_by_keywords
import tool_audit
logger = logging.getLogger(__name__)
# Appended to the persona system prompt so the model knows it has tools.
# Kept brief — capable models handle tool use without much coaching.
_TOOL_INSTRUCTION = (
"\n\nYou have access to tools. Use them when you need current information, "
"need to read files, or need to take actions on the user's behalf. "
"Respond naturally after gathering what you need."
)
async def run(
task: str,
system_prompt: str = "",
session_messages: list[dict] | None = None,
model_cfg: dict | None = None,
respond_with_final: bool = True,
user_role: str = "user",
tool_list: list[str] | None = None,
confirm_allow: set[str] | None = None,
confirm_deny: set[str] | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> OrchestratorResult:
"""
Run a tool-enabled task using an OpenAI-compatible API.
Args:
task: The user's request (plain text)
system_prompt: Persona system prompt from context_loader (passed through)
session_messages: Recent conversation history for session continuity
model_cfg: Resolved model config from model_registry (local_openai type)
respond_with_final: If False, return just the tool-loop summary without a
full persona-voiced response (faster; for cron/background)
confirm_allow: Tools to bypass the confirmation gate for this user
confirm_deny: Tools to always block for this user
Returns:
OrchestratorResult — if checkpoint is set, the job is awaiting confirmation
"""
if not model_cfg:
raise RuntimeError("model_cfg is required for the OpenAI orchestrator")
_confirm_allow = frozenset(confirm_allow or ())
_confirm_deny = frozenset(confirm_deny or ())
effective_confirm = (CONFIRM_REQUIRED - set(_confirm_allow)) | set(_confirm_deny)
# Keyword routing: narrow schemas to only what this message needs.
# Also scans the last assistant turn so follow-ups like "yes, do that" inherit tool context.
# Returns [] when no keywords match (zero tool overhead — model responds as plain chat).
effective_tool_list = narrow_tools_by_keywords(task, tool_list, context_messages=session_messages)
logger.info(
"Keyword routing: %d tools active (role_tools=%s)",
len(effective_tool_list),
len(tool_list) if tool_list is not None else "all",
)
client, model_name, active_tools = _build_client(
model_cfg, user_role, effective_tool_list,
max_risk=max_risk, risk_whitelist=risk_whitelist, risk_blacklist=risk_blacklist,
)
tool_audit.set_context("openai", model_cfg.get("label") or model_name)
sys_content = (system_prompt or "") + _TOOL_INSTRUCTION
messages: list[dict] = [{"role": "system", "content": sys_content}]
if session_messages:
messages.extend(
{"role": m["role"], "content": m["content"]}
for m in session_messages[-6:]
)
messages.append({"role": "user", "content": task})
tool_call_log: list[dict] = []
final_response, checkpoint = await _run_from_messages(
client=client,
messages=messages,
active_tools=active_tools,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=model_name,
task=task,
model_cfg=model_cfg,
respond_with_final=respond_with_final,
user_role=user_role,
tool_list=effective_tool_list,
confirm_allow=_confirm_allow,
confirm_deny=_confirm_deny,
starting_round=0,
)
if checkpoint:
return OrchestratorResult(
response=final_response,
tool_calls=list(tool_call_log),
backend="local",
gemini_summary=final_response,
checkpoint=checkpoint,
)
model_label = model_cfg.get("label") or model_name
logger.info("OpenAI orchestrator complete — model=%s tools=%d", model_label, len(tool_call_log))
return OrchestratorResult(
response=final_response,
tool_calls=tool_call_log,
backend="local",
backend_label=model_label,
gemini_summary=final_response,
)
async def resume(checkpoint: OrchestrateCheckpoint, confirmed: bool) -> OrchestratorResult:
"""Continue an OpenAI orchestrator job that was paused at a confirmation gate."""
client, model_name, active_tools = _build_client(checkpoint.model_cfg, checkpoint.user_role, checkpoint.tool_list)
effective_confirm = (CONFIRM_REQUIRED - set(checkpoint.confirm_allow)) | set(checkpoint.confirm_deny)
messages = list(checkpoint.pre_fn_state)
tool_call_log = [t for t in checkpoint.tool_call_log if t["result"] != "[awaiting confirmation]"]
# Build tool responses for this round
for er in checkpoint.executed_results:
messages.append({
"role": "tool",
"tool_call_id": er.get("tool_call_id", er["name"]),
"content": er["result"],
})
for pt in checkpoint.pending_tools:
if confirmed:
result_str = await _execute_tool_dict(pt["name"], pt["args"], checkpoint.user_role, checkpoint.tool_list)
logger.info("Confirmed tool %s%d chars", pt["name"], len(result_str))
else:
result_str = "Action denied by user."
logger.info("Tool %s denied by user", pt["name"])
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": result_str})
messages.append({
"role": "tool",
"tool_call_id": pt.get("tool_call_id", pt["name"]),
"content": result_str,
})
final_response, new_checkpoint = await _run_from_messages(
client=client,
messages=messages,
active_tools=active_tools,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=model_name,
task=checkpoint.task,
model_cfg=checkpoint.model_cfg,
respond_with_final=checkpoint.respond_with_final,
user_role=checkpoint.user_role,
tool_list=checkpoint.tool_list,
confirm_allow=checkpoint.confirm_allow,
confirm_deny=checkpoint.confirm_deny,
starting_round=checkpoint.rounds_used,
)
if new_checkpoint:
return OrchestratorResult(
response=final_response,
tool_calls=list(tool_call_log),
backend="local",
gemini_summary=final_response,
checkpoint=new_checkpoint,
)
model_label = (checkpoint.model_cfg or {}).get("label") or model_name
logger.info("OpenAI orchestrator resumed — model=%s tools=%d", model_label, len(tool_call_log))
return OrchestratorResult(
response=final_response,
tool_calls=tool_call_log,
backend="local",
gemini_summary=final_response,
)
_CHARS_PER_TOKEN = 4
# Fixed token overhead budget per call (tool schemas excluded — cached separately)
_TOOL_SCHEMA_OVERHEAD = 500
# Chars to keep per truncated old tool result
_TRUNC_RESULT_CHARS = 400
# Always keep the last N tool-result messages uncompacted
_KEEP_RECENT_TOOL_MSGS = 6 # ~2 rounds of 3 tools each
# Module-level schema cache: key = (user_role, sorted_tools, risk_params)
# Bounded in practice — keyword routing produces at most ~30 distinct tool sets.
_tool_schema_cache: dict[str, list[dict]] = {}
def _get_cached_tools(
user_role: str,
tool_list: list[str] | None,
max_risk: str | None = None,
whitelist: list[str] | None = None,
blacklist: list[str] | None = None,
) -> list[dict]:
key = "|".join([
user_role,
str(sorted(tool_list) if tool_list is not None else "all"),
str(max_risk),
str(sorted(whitelist) if whitelist else ""),
str(sorted(blacklist) if blacklist else ""),
])
if key not in _tool_schema_cache:
_tool_schema_cache[key] = get_openai_tools_for_role(
user_role, tool_list,
max_risk=max_risk, whitelist=whitelist, blacklist=blacklist,
)
return _tool_schema_cache[key]
def _estimate_tokens(messages: list[dict]) -> int:
total = sum(len(json.dumps(m)) for m in messages)
return total // _CHARS_PER_TOKEN + _TOOL_SCHEMA_OVERHEAD
def _compact_messages(messages: list[dict], budget_tokens: int) -> list[dict]:
"""
Truncate old tool result content when approaching the context budget.
Strategy: keep system message, recent assistant/tool rounds, and the
original user task intact. Truncate content of old tool results in the
middle of the conversation — the model only needs recent results to reason.
"""
if _estimate_tokens(messages) <= budget_tokens:
return messages
tool_indices = [i for i, m in enumerate(messages) if m.get("role") == "tool"]
n_to_compact = max(0, len(tool_indices) - _KEEP_RECENT_TOOL_MSGS)
if n_to_compact == 0:
return messages # nothing old enough to trim
compact_set = set(tool_indices[:n_to_compact])
result = []
chars_saved = 0
for i, msg in enumerate(messages):
if i in compact_set:
content = msg.get("content", "")
if len(content) > _TRUNC_RESULT_CHARS:
msg = dict(msg)
saved = len(content) - _TRUNC_RESULT_CHARS
chars_saved += saved
msg["content"] = (
content[:_TRUNC_RESULT_CHARS]
+ f" …[{len(content) - _TRUNC_RESULT_CHARS} chars omitted]"
)
result.append(msg)
new_est = _estimate_tokens(result)
logger.info(
"context compaction: saved ~%d tokens (%d chars), now ~%d / %d tokens",
chars_saved // _CHARS_PER_TOKEN, chars_saved, new_est, budget_tokens,
)
return result
def _context_budget(model_cfg: dict | None) -> int:
"""Return the usable token budget (75% of context window, min 16k, default 32k)."""
context_k = (model_cfg or {}).get("context_k") or 32
return max(16_000, int(context_k * 1000 * 0.75))
async def _run_from_messages(
client,
messages: list[dict],
active_tools: list,
tool_call_log: list[dict],
effective_confirm: set[str],
model_name: str,
task: str,
model_cfg: dict | None,
respond_with_final: bool,
user_role: str,
confirm_allow: frozenset,
confirm_deny: frozenset,
starting_round: int = 0,
tool_list: list[str] | None = None,
) -> tuple[str, OrchestrateCheckpoint | None]:
"""
Run the OpenAI ReAct loop from the current messages state.
Returns (final_response, checkpoint) — checkpoint is set if confirmation is needed.
"""
final_response = ""
budget = _context_budget(model_cfg)
per_model_limit = (model_cfg or {}).get("max_rounds") or settings.orchestrator_max_rounds
effective_limit = min(per_model_limit, settings.orchestrator_max_rounds)
for round_num in range(starting_round, effective_limit):
messages = _compact_messages(messages, budget)
est = _estimate_tokens(messages)
logger.info("OpenAI orchestrator round %d / %d model=%s ~%d tokens",
round_num + 1, effective_limit, model_name, est)
call_kwargs: dict = {"model": model_name, "messages": messages}
if active_tools:
call_kwargs["tools"] = active_tools
call_kwargs["tool_choice"] = "auto"
reasoning_budget = (model_cfg or {}).get("reasoning_budget_tokens")
if reasoning_budget:
call_kwargs["extra_body"] = {"reasoning": {"budget_tokens": reasoning_budget}}
response = await _chat_with_retry(client, **call_kwargs)
choice = response.choices[0]
msg = choice.message
assistant_msg: dict = {"role": "assistant"}
if msg.content:
assistant_msg["content"] = msg.content
if msg.tool_calls:
assistant_msg["tool_calls"] = [
{
"id": tc.id,
"type": "function",
"function": {"name": tc.function.name, "arguments": tc.function.arguments},
}
for tc in msg.tool_calls
]
messages.append(assistant_msg)
# Some models set finish_reason="stop" even when tool_calls are present
if msg.tool_calls and (choice.finish_reason in ("tool_calls", "stop", None)):
# Snapshot state before tool responses for potential checkpoint
pre_fn_state = list(messages)
pending_tools: list[dict] = []
executed_results: list[dict] = []
for tc in msg.tool_calls:
name = tc.function.name
raw_args = tc.function.arguments or "{}"
try:
args_parsed = json.loads(raw_args)
if not isinstance(args_parsed, dict):
raise ValueError("args must be a JSON object")
except (json.JSONDecodeError, ValueError) as e:
logger.warning("Malformed tool args for %s: %s — args: %.200s", name, e, raw_args)
args_parsed = {}
if name in effective_confirm:
pending_tools.append({"name": name, "args": args_parsed, "tool_call_id": tc.id})
logger.info("Tool %s blocked — confirmation required", name)
else:
result_str = await _execute_tool(name, tc.function.arguments, user_role, tool_list)
logger.info("Tool %s%d chars", name, len(result_str))
executed_results.append({"name": name, "args": args_parsed, "result": result_str, "tool_call_id": tc.id})
tool_call_log.append({"tool": name, "args": args_parsed, "result": result_str})
messages.append({"role": "tool", "tool_call_id": tc.id, "content": result_str})
if pending_tools:
# Add placeholder responses
for pt in pending_tools:
placeholder = f"[AWAITING USER CONFIRMATION for {pt['name']}]"
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": "[awaiting confirmation]"})
messages.append({"role": "tool", "tool_call_id": pt["tool_call_id"], "content": placeholder})
messages = _compact_messages(messages, budget)
conf_call: dict = {"model": model_name, "messages": messages, "tool_choice": "none"}
if active_tools:
conf_call["tools"] = active_tools
if reasoning_budget:
conf_call["extra_body"] = {"reasoning": {"budget_tokens": reasoning_budget}}
conf_resp = await _chat_with_retry(client, **conf_call)
final_response = conf_resp.choices[0].message.content or (
"This action requires your explicit confirmation before it can proceed."
)
checkpoint = OrchestrateCheckpoint(
engine="openai",
pre_fn_state=pre_fn_state,
executed_results=executed_results,
pending_tools=pending_tools,
tool_call_log=list(tool_call_log),
task=task,
model_cfg=model_cfg,
respond_with_final=respond_with_final,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
rounds_used=round_num + 2,
)
return final_response, checkpoint
else:
final_response = msg.content or ""
logger.info(
"OpenAI orchestrator done after %d round(s). Tools used: %d",
round_num + 1, len(tool_call_log),
)
break
else:
logger.warning("OpenAI orchestrator hit max rounds (%d)", effective_limit)
final_response = (
f"Reached the tool iteration limit ({effective_limit} rounds). "
"Here is what was gathered:\n\n"
+ "\n\n".join(f"**{t['tool']}**: {t['result'][:500]}" for t in tool_call_log)
)
return final_response, None
_RETRY_STATUSES = {429, 500, 502, 503, 504}
_MAX_API_RETRIES = 3
async def _chat_with_retry(client, **kwargs):
"""Wrap chat.completions.create with exponential backoff on transient errors."""
last_exc: Exception = RuntimeError("No attempts made")
for attempt in range(_MAX_API_RETRIES):
try:
return await client.chat.completions.create(**kwargs)
except APIConnectionError as e:
last_exc = e
logger.warning("OpenAI connection error (attempt %d/%d): %s", attempt + 1, _MAX_API_RETRIES, e)
except APIStatusError as e:
if e.status_code in _RETRY_STATUSES:
last_exc = e
logger.warning("OpenAI status %d (attempt %d/%d): %s", e.status_code, attempt + 1, _MAX_API_RETRIES, e)
else:
raise
if attempt < _MAX_API_RETRIES - 1:
await asyncio.sleep(2 ** attempt) # 1s, 2s
raise last_exc
def _build_client(
model_cfg: dict | None,
user_role: str = "user",
tool_list: list[str] | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> tuple:
"""Build AsyncOpenAI client and return (client, model_name, active_tools)."""
if not model_cfg:
raise RuntimeError("model_cfg is required for the OpenAI orchestrator")
api_url = model_cfg.get("api_url", "")
api_key = model_cfg.get("api_key", "") or "none"
model_name = model_cfg.get("model_name", "")
host_type = model_cfg.get("host_type", "openwebui")
if not api_url or not model_name:
raise RuntimeError(
f"model_cfg missing api_url or model_name: {model_cfg.get('label', model_cfg)}"
)
base_url = api_url.rstrip("/")
if host_type == "openwebui":
base_url = base_url + "/api"
client = AsyncOpenAI(base_url=base_url, api_key=api_key, timeout=settings.timeout_local)
if model_cfg.get("tools") is False:
active_tools = []
else:
active_tools = _get_cached_tools(
user_role, tool_list,
max_risk=max_risk, whitelist=risk_whitelist, blacklist=risk_blacklist,
)
return client, model_name, active_tools
async def _execute_tool(
name: str,
arguments_json: str,
user_role: str = "user",
tool_list: list[str] | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> str:
"""Parse tool arguments and execute with role-filtered callables."""
_, callables = get_tools_for_role(
user_role, tool_list,
max_risk=max_risk, whitelist=risk_whitelist, blacklist=risk_blacklist,
)
try:
args = json.loads(arguments_json)
except json.JSONDecodeError:
args = {}
try:
return await call_tool(name, args, callables)
except Exception as e:
logger.warning("Tool %s failed: %s", name, e)
return f"Tool error: {e}"
async def _execute_tool_dict(
name: str,
args: dict,
user_role: str = "user",
tool_list: list[str] | None = None,
) -> str:
"""Execute a tool from a pre-parsed args dict."""
_, callables = get_tools_for_role(user_role, tool_list)
try:
return await call_tool(name, args, callables)
except Exception as e:
logger.warning("Tool %s failed: %s", name, e)
return f"Tool error: {e}"

View File

@@ -16,6 +16,7 @@ calls llm_client.complete() directly, which is faster and has no orchestration o
"""
import asyncio
import json
import logging
from dataclasses import dataclass, field
@@ -24,7 +25,10 @@ from google.genai import types
from config import settings
from llm_client import complete
from tools import TOOL_DECLARATIONS, call_tool
from tools import TOOL_DECLARATIONS, call_tool, get_tools_for_role, CONFIRM_REQUIRED
import usage_tracker
import tool_audit
from persona import _user
logger = logging.getLogger(__name__)
@@ -43,12 +47,61 @@ Keep your summary factual and complete. Include relevant URLs, data, and specifi
If no tools are needed, return an empty summary."""
def _track_gemini_usage(response, model_name: str | None) -> None:
meta = getattr(response, "usage_metadata", None)
if not meta:
return
prompt_tokens = getattr(meta, "prompt_token_count", 0) or 0
completion_tokens = getattr(meta, "candidates_token_count", 0) or 0
if prompt_tokens or completion_tokens:
try:
asyncio.create_task(usage_tracker.record(
username=_user.get(),
backend="gemini_api",
model_name=model_name or settings.orchestrator_model,
prompt_tokens=prompt_tokens,
completion_tokens=completion_tokens,
))
except Exception:
pass
@dataclass
class OrchestrateCheckpoint:
"""Saved execution state for a job paused at a confirmation gate."""
engine: str # "gemini" | "openai"
pre_fn_state: list # conversation state before function responses
executed_results: list[dict] # tools that already ran this round
pending_tools: list[dict] # [{name, args}] awaiting confirmation
tool_call_log: list[dict] # all tool calls so far
task: str
# Gemini-specific config (unused by openai engine)
system_prompt: str = ""
session_messages: list | None = None
model_name: str | None = None
gemini_api_key: str | None = None
respond_with_claude: bool = True
response_role: str = "chat"
# OpenAI-specific config (unused by gemini engine)
model_cfg: dict | None = None
respond_with_final: bool = True
# Common
user_role: str = "user"
tool_list: list[str] | None = None
confirm_allow: frozenset = field(default_factory=frozenset)
confirm_deny: frozenset = field(default_factory=frozenset)
rounds_used: int = 0
max_rounds: int | None = None
@dataclass
class OrchestratorResult:
response: str # final user-facing response (from Claude)
tool_calls: list[dict] = field(default_factory=list) # [{tool, args, result}]
backend: str = "claude" # model that produced the final response
backend_label: str = "" # human-readable model label for display
gemini_summary: str = "" # what Gemini handed to Claude (debug/display)
checkpoint: OrchestrateCheckpoint | None = None # set when awaiting confirmation
async def run(
@@ -56,6 +109,17 @@ async def run(
system_prompt: str = "",
session_messages: list[dict] | None = None,
respond_with_claude: bool = True,
gemini_api_key: str | None = None,
model_name: str | None = None,
response_role: str = "chat",
user_role: str = "user",
tool_list: list[str] | None = None,
confirm_allow: set[str] | None = None,
confirm_deny: set[str] | None = None,
max_rounds: int | None = None,
max_risk: str | None = None,
risk_whitelist: list[str] | None = None,
risk_blacklist: list[str] | None = None,
) -> OrchestratorResult:
"""
Run the full orchestration loop for a task.
@@ -66,116 +130,331 @@ async def run(
session_messages: Prior conversation history for session continuity
respond_with_claude: If False, return Gemini's summary as the response (useful for
background/cron tasks where a polished reply isn't needed)
gemini_api_key: Per-user Gemini API key (falls back to GEMINI_API_KEY in .env)
tool_list: Optional explicit tool allow-list from role config; intersected
with user_role access-level filter (cannot elevate privileges)
confirm_allow: Tools to bypass the confirmation gate for this user
confirm_deny: Tools to always block for this user
Returns:
OrchestratorResult with response, tool call log, backend used, and Gemini summary
OrchestratorResult — if checkpoint is set, the job is awaiting confirmation
"""
if not settings.gemini_api_key:
api_key = gemini_api_key or settings.gemini_api_key
if not api_key:
raise RuntimeError(
"GEMINI_API_KEY not set — orchestrator requires Gemini API. "
"Get a free key at https://aistudio.google.com/apikey and add it to .env"
"No Gemini API key available — set GEMINI_API_KEY in .env or add a personal key "
"via: manage_passwords.py gemini-key <username> <key>"
)
client = genai.Client(api_key=settings.gemini_api_key)
client = genai.Client(api_key=api_key)
tool_audit.set_context("gemini", model_name or settings.orchestrator_model)
_confirm_allow = frozenset(confirm_allow or ())
_confirm_deny = frozenset(confirm_deny or ())
effective_confirm = (CONFIRM_REQUIRED - set(_confirm_allow)) | set(_confirm_deny)
# Seed Gemini with the task — include recent session context if available
task_with_context = _build_task_prompt(task, session_messages)
contents: list[types.Content] = [
types.Content(role="user", parts=[types.Part(text=task_with_context)])
]
tool_declarations, tool_callables = get_tools_for_role(
user_role, tool_list, max_risk=max_risk,
whitelist=risk_whitelist, blacklist=risk_blacklist,
)
tool_call_log: list[dict] = []
gemini_summary = ""
# --- ReAct tool loop ---
for round_num in range(settings.orchestrator_max_rounds):
gemini_summary, checkpoint = await _run_from_contents(
client=client,
contents=contents,
tool_declarations=tool_declarations,
tool_callables=tool_callables,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=model_name,
task=task,
system_prompt=system_prompt,
session_messages=session_messages,
respond_with_claude=respond_with_claude,
response_role=response_role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=_confirm_allow,
confirm_deny=_confirm_deny,
starting_round=0,
gemini_api_key=api_key,
max_rounds=max_rounds,
)
if checkpoint:
return OrchestratorResult(
response=gemini_summary,
tool_calls=list(tool_call_log),
backend="gemini",
gemini_summary=gemini_summary,
checkpoint=checkpoint,
)
return await _claude_handoff(
task=task,
tool_call_log=tool_call_log,
gemini_summary=gemini_summary,
system_prompt=system_prompt,
session_messages=session_messages,
respond_with_claude=respond_with_claude,
response_role=response_role,
)
async def resume(checkpoint: OrchestrateCheckpoint, confirmed: bool) -> OrchestratorResult:
"""Continue a job that was paused at a confirmation gate."""
api_key = checkpoint.gemini_api_key or settings.gemini_api_key
client = genai.Client(api_key=api_key)
tool_declarations, tool_callables = get_tools_for_role(
checkpoint.user_role, checkpoint.tool_list,
max_risk=getattr(checkpoint, "max_risk", None),
whitelist=getattr(checkpoint, "risk_whitelist", None),
blacklist=getattr(checkpoint, "risk_blacklist", None),
)
effective_confirm = (CONFIRM_REQUIRED - set(checkpoint.confirm_allow)) | set(checkpoint.confirm_deny)
# Rebuild from saved state — strip "[awaiting confirmation]" placeholders
contents = list(checkpoint.pre_fn_state)
tool_call_log = [t for t in checkpoint.tool_call_log if t["result"] != "[awaiting confirmation]"]
# Build function responses for this round
response_parts: list[types.Part] = []
for er in checkpoint.executed_results:
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=er["name"], response={"result": er["result"]}
)))
for pt in checkpoint.pending_tools:
if confirmed:
result_str = await _execute_tool(pt["name"], pt["args"], tool_callables)
logger.info("Confirmed tool %s%d chars", pt["name"], len(result_str))
else:
result_str = "Action denied by user."
logger.info("Tool %s denied by user", pt["name"])
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": result_str})
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=pt["name"], response={"result": result_str}
)))
contents.append(types.Content(role="user", parts=response_parts))
gemini_summary, new_checkpoint = await _run_from_contents(
client=client,
contents=contents,
tool_declarations=tool_declarations,
tool_callables=tool_callables,
tool_call_log=tool_call_log,
effective_confirm=effective_confirm,
model_name=checkpoint.model_name,
task=checkpoint.task,
system_prompt=checkpoint.system_prompt,
session_messages=checkpoint.session_messages,
respond_with_claude=checkpoint.respond_with_claude,
response_role=checkpoint.response_role,
user_role=checkpoint.user_role,
tool_list=checkpoint.tool_list,
confirm_allow=checkpoint.confirm_allow,
confirm_deny=checkpoint.confirm_deny,
starting_round=checkpoint.rounds_used,
gemini_api_key=api_key,
max_rounds=checkpoint.max_rounds,
)
if new_checkpoint:
return OrchestratorResult(
response=gemini_summary,
tool_calls=list(tool_call_log),
backend="gemini",
gemini_summary=gemini_summary,
checkpoint=new_checkpoint,
)
return await _claude_handoff(
task=checkpoint.task,
tool_call_log=tool_call_log,
gemini_summary=gemini_summary,
system_prompt=checkpoint.system_prompt,
session_messages=checkpoint.session_messages,
respond_with_claude=checkpoint.respond_with_claude,
response_role=checkpoint.response_role,
)
async def _run_from_contents(
client,
contents: list,
tool_declarations: list,
tool_callables: dict,
tool_call_log: list[dict],
effective_confirm: set[str],
model_name: str | None,
task: str,
system_prompt: str,
session_messages: list[dict] | None,
respond_with_claude: bool,
response_role: str,
user_role: str,
confirm_allow: frozenset,
confirm_deny: frozenset,
starting_round: int = 0,
gemini_api_key: str | None = None,
tool_list: list[str] | None = None,
max_rounds: int | None = None,
) -> tuple[str, OrchestrateCheckpoint | None]:
"""
Run the ReAct loop from the current contents state.
Returns (gemini_summary, checkpoint) — checkpoint is set if confirmation is needed.
"""
gemini_summary = ""
per_model_limit = max_rounds or settings.orchestrator_max_rounds
effective_limit = min(per_model_limit, settings.orchestrator_max_rounds)
for round_num in range(starting_round, effective_limit):
logger.info("Orchestrator round %d for task: %.80s", round_num + 1, task)
response = await asyncio.to_thread(
client.models.generate_content,
model=settings.orchestrator_model,
model=model_name or settings.orchestrator_model,
contents=contents,
config=types.GenerateContentConfig(
tools=TOOL_DECLARATIONS,
tools=tool_declarations,
system_instruction=_ORCHESTRATOR_SYSTEM,
),
)
_track_gemini_usage(response, model_name)
candidate = response.candidates[0]
parts = candidate.content.parts if candidate.content else []
# Check if Gemini wants to call any tools
tool_call_parts = [
p for p in parts
if hasattr(p, "function_call") and p.function_call and p.function_call.name
]
if not tool_call_parts:
# No more tool calls — extract Gemini's text summary
gemini_summary = "".join(
p.text for p in parts if hasattr(p, "text") and p.text
).strip()
logger.info("Orchestrator done after %d round(s). Tools used: %d",
round_num + 1, len(tool_call_log))
break
return gemini_summary, None
# Add Gemini's response (with function calls) to the conversation
contents.append(candidate.content)
# Execute all tool calls in parallel
tool_tasks = [
_execute_tool(fc.function_call.name, dict(fc.function_call.args))
for fc in tool_call_parts
]
tool_results = await asyncio.gather(*tool_tasks, return_exceptions=True)
# Snapshot state before function responses — used if a checkpoint is needed
pre_fn_state = list(contents)
# Build function response parts and update log
response_parts: list[types.Part] = []
for fc_part, result in zip(tool_call_parts, tool_results):
fc = fc_part.function_call
result_str = str(result) if not isinstance(result, Exception) else f"Error: {result}"
logger.info("Tool %s%d chars", fc.name, len(result_str))
pending_tools: list[dict] = []
executed_results: list[dict] = []
tool_call_log.append({
"tool": fc.name,
"args": dict(fc.args),
"result": result_str,
})
response_parts.append(
types.Part(
function_response=types.FunctionResponse(
name=fc.name,
response={"result": result_str},
for fc_part in tool_call_parts:
fc = fc_part.function_call
name = fc.name
args = dict(fc.args)
if name in effective_confirm:
pending_tools.append({"name": name, "args": args})
logger.info("Tool %s blocked — confirmation required", name)
else:
result_str = await _execute_tool(name, args, tool_callables)
logger.info("Tool %s%d chars", name, len(result_str))
executed_results.append({"name": name, "args": args, "result": result_str})
tool_call_log.append({"tool": name, "args": args, "result": result_str})
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=name, response={"result": result_str}
)))
if pending_tools:
# Add placeholder responses and get Gemini to produce the confirmation message
for pt in pending_tools:
placeholder = f"[AWAITING USER CONFIRMATION for {pt['name']}]"
response_parts.append(types.Part(function_response=types.FunctionResponse(
name=pt["name"], response={"result": placeholder}
)))
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": "[awaiting confirmation]"})
contents.append(types.Content(role="user", parts=response_parts))
conf_response = await asyncio.to_thread(
client.models.generate_content,
model=model_name or settings.orchestrator_model,
contents=contents,
config=types.GenerateContentConfig(
tools=tool_declarations,
system_instruction=_ORCHESTRATOR_SYSTEM,
),
)
_track_gemini_usage(conf_response, model_name)
conf_parts = (
conf_response.candidates[0].content.parts
if conf_response.candidates and conf_response.candidates[0].content
else []
)
gemini_summary = "".join(
p.text for p in conf_parts if hasattr(p, "text") and p.text
).strip() or "This action requires your explicit confirmation before it can proceed."
checkpoint = OrchestrateCheckpoint(
engine="gemini",
pre_fn_state=pre_fn_state,
executed_results=executed_results,
pending_tools=pending_tools,
tool_call_log=list(tool_call_log),
task=task,
system_prompt=system_prompt,
session_messages=session_messages,
model_name=model_name,
gemini_api_key=gemini_api_key,
respond_with_claude=respond_with_claude,
response_role=response_role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
rounds_used=round_num + 2,
max_rounds=max_rounds,
)
return gemini_summary, checkpoint
contents.append(types.Content(role="user", parts=response_parts))
else:
# Hit the round limit — use whatever Gemini produced last
logger.warning("Orchestrator hit max rounds (%d)", settings.orchestrator_max_rounds)
logger.warning("Orchestrator hit max rounds (%d)", effective_limit)
gemini_summary = (
f"Reached the tool iteration limit ({settings.orchestrator_max_rounds} rounds). "
f"Reached the tool iteration limit ({effective_limit} rounds). "
"Here is what was gathered so far:\n\n"
+ "\n\n".join(f"**{t['tool']}**: {t['result'][:500]}" for t in tool_call_log)
)
# --- Claude handoff ---
return gemini_summary, None
async def _claude_handoff(
task: str,
tool_call_log: list[dict],
gemini_summary: str,
system_prompt: str,
session_messages: list[dict] | None,
respond_with_claude: bool,
response_role: str,
) -> OrchestratorResult:
if respond_with_claude:
claude_prompt = _build_claude_prompt(task, tool_call_log, gemini_summary)
# Merge with session history so Claude has conversation context
messages = list(session_messages or [])
messages.append({"role": "user", "content": claude_prompt})
response_text, backend = await complete(
system_prompt=system_prompt,
messages=messages,
model="claude",
role=response_role,
)
else:
# Cron/background tasks: return Gemini's summary directly, no Claude call
response_text = gemini_summary or "No information gathered."
backend = "gemini"
@@ -187,10 +466,10 @@ async def run(
)
async def _execute_tool(name: str, args: dict) -> str:
async def _execute_tool(name: str, args: dict, callables: dict | None = None) -> str:
"""Execute a single tool call, catching all exceptions."""
try:
return await call_tool(name, args)
return await call_tool(name, args, callables)
except Exception as e:
logger.warning("Tool %s failed: %s", name, e)
return f"Tool error: {e}"
@@ -201,12 +480,11 @@ def _build_task_prompt(task: str, session_messages: list[dict] | None) -> str:
if not session_messages:
return task
# Include last few turns for context (don't send the full history to keep tokens low)
recent = session_messages[-6:] # last 3 turns
recent = session_messages[-6:]
history_lines = []
for msg in recent:
label = "User" if msg["role"] == "user" else "Assistant"
history_lines.append(f"{label}: {msg['content'][:300]}") # truncate long messages
history_lines.append(f"{label}: {msg['content'][:300]}")
context = "\n".join(history_lines)
return f"<recent_conversation>\n{context}\n</recent_conversation>\n\nCurrent request: {task}"
@@ -224,7 +502,6 @@ def _build_claude_prompt(
parts.append("## Research gathered\n")
for tc in tool_calls:
parts.append(f"### {tc['tool']}({_format_args(tc['args'])})")
# Truncate very long results — Claude gets the gist
result = tc["result"]
if len(result) > 2000:
result = result[:2000] + "\n… [truncated]"

View File

@@ -135,6 +135,27 @@ def _protocols(display_name: str) -> str:
---
## Tools & Modes
Cortex has two chat modes. Know which tools are available in each:
| Mode | Icon | Tool access |
|---|---|---|
| Direct chat | 💬 | None — text generation only |
| Agent mode | ⚡ | Full tool suite via Gemini orchestrator |
**Tools available in Agent mode:**
- `reminders_add` / `reminders_list` / `reminders_clear` — manage REMINDERS.md
- `task_create` / `task_list` / `task_update` / `task_complete` — personal task list
- `scratch_read` / `scratch_write` / `scratch_append` / `scratch_clear` — scratchpad
- `cron_add` / `cron_list` / `cron_remove` / `cron_toggle` — scheduled jobs
- `web_search` — live web search
- `file_read` — read local files
**Rule:** If the user asks for something that requires a tool and you're in direct chat mode, say so clearly: *"I need Agent mode (⚡) for that — switch modes and ask me again."* Do not attempt workarounds or pretend the action was taken.
---
## Memory
- Long-term memory lives in MEMORY_LONG.md (auto-distilled monthly).

117
cortex/push_utils.py Normal file
View File

@@ -0,0 +1,117 @@
"""
Web Push (VAPID) helpers.
Subscriptions are stored per-user at:
home/{user}/push_subscriptions.json → list of {endpoint, keys:{p256dh, auth}}
send_push(username, title, body, url) iterates all stored subscriptions for that
user and fires a push. Stale endpoints (410 Gone) are pruned automatically.
"""
import asyncio
import base64
import json
import logging
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
def _subs_path(username: str) -> Path:
return settings.home_root() / username / "push_subscriptions.json"
def load_subscriptions(username: str) -> list[dict]:
path = _subs_path(username)
if not path.exists():
return []
try:
return json.loads(path.read_text())
except Exception:
return []
def _save_subscriptions(username: str, subs: list[dict]) -> None:
path = _subs_path(username)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(subs, indent=2))
def add_subscription(username: str, sub: dict) -> None:
"""Upsert a subscription by endpoint URL."""
subs = load_subscriptions(username)
endpoint = sub.get("endpoint", "")
subs = [s for s in subs if s.get("endpoint") != endpoint]
subs.append(sub)
_save_subscriptions(username, subs)
def remove_subscription(username: str, endpoint: str) -> bool:
subs = load_subscriptions(username)
new_subs = [s for s in subs if s.get("endpoint") != endpoint]
if len(new_subs) == len(subs):
return False
_save_subscriptions(username, new_subs)
return True
def _get_private_key_pem() -> str:
"""Decode the base64-encoded PEM private key from settings."""
raw = settings.vapid_private_key_b64.strip()
if not raw:
raise RuntimeError("VAPID_PRIVATE_KEY_B64 is not set in .env")
return base64.b64decode(raw).decode()
def _send_one(sub: dict, payload: dict) -> bool:
"""Send a push to a single subscription. Returns False if the endpoint is stale (410)."""
from pywebpush import webpush, WebPushException
from py_vapid import Vapid
try:
vapid = Vapid.from_pem(_get_private_key_pem().encode())
webpush(
subscription_info=sub,
data=json.dumps(payload),
vapid_private_key=vapid,
vapid_claims={"sub": settings.vapid_contact},
)
return True
except WebPushException as e:
if e.response is not None and e.response.status_code == 410:
logger.info("push endpoint gone (410), pruning: %s", sub.get("endpoint", "")[:60])
return False
logger.warning("push failed: %s", e)
return True # keep the sub; might be transient
async def send_push(username: str, title: str, body: str, url: str = "") -> dict:
"""
Send a push notification to all subscriptions for username.
Returns {"sent": n, "pruned": m}.
"""
if not settings.vapid_public_key or not settings.vapid_private_key_b64:
return {"error": "VAPID keys not configured"}
subs = load_subscriptions(username)
if not subs:
return {"error": f"No push subscriptions for {username}"}
payload = {"title": title, "body": body, "url": url}
keep = []
sent = 0
pruned = 0
for sub in subs:
alive = await asyncio.to_thread(_send_one, sub, payload)
if alive:
keep.append(sub)
sent += 1
else:
pruned += 1
if pruned:
_save_subscriptions(username, keep)
return {"sent": sent, "pruned": pruned}

View File

@@ -16,5 +16,20 @@ bcrypt>=4.0.0
PyJWT>=2.8.0
python-multipart>=0.0.9 # required by FastAPI for Form() data
# anthropic SDK not needed — using claude CLI subprocess for auth
# anthropic>=0.40.0
# Async HTTP client — used for local OpenAI-compatible backend (Open WebUI / Ollama)
httpx>=0.27.0
# Web content extraction — strips ads/nav/boilerplate, returns clean article text
trafilatura>=1.6.0
# OpenAI-compatible client — tool calling for OpenRouter / LiteLLM / any OAI-compat host
openai>=1.0.0
# Web Push / VAPID — browser push notifications
pywebpush>=2.0.0
# MariaDB / MySQL connector — used by ae_db_query orchestrator tool
pymysql>=1.1.0
# Anthropic SDK — direct API key backend (alternative to CLI OAuth)
anthropic>=0.40.0

122
cortex/routers/audit.py Normal file
View File

@@ -0,0 +1,122 @@
"""
Tool audit log endpoints.
Self-service (any authenticated user, own data):
GET /api/audit/files → list of available date strings (newest first)
GET /api/audit/day?date=YYYY-MM-DD → entries for one day
Admin-only (cross-user aggregation):
GET /api/audit/recent?user=scott&days=7&limit=200
GET /api/audit/stats?user=scott&days=7
"""
import jwt
from collections import Counter
from datetime import date, timedelta
from fastapi import APIRouter, HTTPException, Query, Request
from auth_utils import COOKIE_NAME, decode_token, get_user_role
from config import settings
import tool_audit
from persona import list_users
router = APIRouter(prefix="/api/audit")
def _session_user(request: Request) -> str:
"""Return the authenticated username or raise 401."""
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
return decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
def _require_admin(request: Request) -> str:
username = _session_user(request)
if get_user_role(username) != "admin":
raise HTTPException(status_code=403, detail="Admin access required")
return username
@router.get("/files")
async def audit_files(request: Request) -> dict:
"""List available audit log dates for the current user (newest first)."""
username = _session_user(request)
audit_dir = settings.home_root() / username / "tool_audit"
if not audit_dir.exists():
return {"dates": []}
dates = sorted(
[p.stem for p in audit_dir.glob("*.jsonl") if p.stem],
reverse=True,
)
return {"dates": dates}
@router.get("/day")
async def audit_day(
request: Request,
date: str = Query(..., description="YYYY-MM-DD"),
) -> dict:
"""Return all entries for a specific day (current user only)."""
username = _session_user(request)
try:
from datetime import date as _date
d = _date.fromisoformat(date)
except ValueError:
raise HTTPException(status_code=400, detail="date must be YYYY-MM-DD")
entries = tool_audit.read_day(username, date)
return {"date": date, "entries": entries}
@router.get("/recent")
async def audit_recent(
request: Request,
user: str = Query(None, description="Username to filter (omit for all users)"),
days: int = Query(7, ge=1, le=90),
limit: int = Query(200, ge=1, le=1000),
) -> dict:
_require_admin(request)
if user:
if user not in list_users():
raise HTTPException(status_code=404, detail=f"User not found: {user}")
entries = tool_audit.read_recent(user, days=days, limit=limit)
else:
entries = tool_audit.read_recent_all_users(days=days, limit=limit)
return {"entries": entries, "count": len(entries), "days": days}
@router.get("/stats")
async def audit_stats(
request: Request,
user: str = Query(None),
days: int = Query(7, ge=1, le=90),
) -> dict:
_require_admin(request)
if user:
if user not in list_users():
raise HTTPException(status_code=404, detail=f"User not found: {user}")
entries = tool_audit.read_recent(user, days=days, limit=10000)
else:
entries = tool_audit.read_recent_all_users(days=days, limit=10000)
tool_counts: Counter = Counter()
status_counts: Counter = Counter()
user_counts: Counter = Counter()
for e in entries:
tool_counts[e.get("tool", "?")] += 1
status_counts[e.get("status", "?")] += 1
user_counts[e.get("user", "?")] += 1
return {
"total": len(entries),
"days": days,
"by_tool": dict(tool_counts.most_common()),
"by_status": dict(status_counts),
"by_user": dict(user_counts.most_common()),
}

View File

@@ -13,6 +13,7 @@ import logging
from datetime import datetime, timezone
from pathlib import Path
from fastapi import APIRouter
from config import settings
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/auth")
@@ -20,7 +21,8 @@ router = APIRouter(prefix="/auth")
CLAUDE_CREDS = Path.home() / ".claude" / ".credentials.json"
GEMINI_CREDS = Path.home() / ".gemini" / "oauth_creds.json"
GEMINI_ACCTS = Path.home() / ".gemini" / "google_accounts.json"
WARN_HOURS = 24
WARN_HOURS = 24 # no refresh token — warn a day ahead
WARN_HOURS_REFRESH = 1 # refresh token present — only warn if CLI hasn't rotated in time
def _claude_status() -> dict:
@@ -31,11 +33,13 @@ def _claude_status() -> dict:
expires_dt = datetime.fromtimestamp(oauth["expiresAt"] / 1000, tz=timezone.utc)
now = datetime.now(tz=timezone.utc)
hours_remaining = (expires_dt - now).total_seconds() / 3600
# If a refresh token is present the session is long-lived (~1 year).
# expiresAt only reflects the current access token window (~8 h) and
# rotates automatically — do not warn based on it when a refresh token exists.
warning = not has_refresh and hours_remaining < WARN_HOURS
expired = hours_remaining <= 0 and not has_refresh
# When a refresh token is present the CLI *should* auto-rotate the access
# token, but sometimes it doesn't. Use a tight 1-hour window so a fresh
# 8-hour token doesn't immediately trigger a warning, but a stale token
# that the CLI missed will still surface before it expires.
expired = hours_remaining <= 0
threshold = WARN_HOURS_REFRESH if has_refresh else WARN_HOURS
warning = expired or hours_remaining < threshold
return {
"ok": True,
"has_refresh_token": has_refresh,
@@ -68,9 +72,39 @@ def _gemini_status() -> dict:
return {"ok": False, "error": str(e), "warning": True, "authenticated": False}
async def _local_status(username: str = "scott") -> dict:
"""Check reachability of the user's configured local model host."""
import model_registry
cfg = model_registry.get_best_local_model(username)
if not cfg:
return {"configured": False}
api_url = cfg.get("api_url", "")
if not api_url:
return {"configured": False}
try:
import httpx
url = api_url.rstrip("/") + "/api/models"
headers = {}
api_key = cfg.get("api_key", "")
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
async with httpx.AsyncClient(timeout=5) as client:
resp = await client.get(url, headers=headers)
reachable = resp.status_code < 400
return {
"configured": True,
"reachable": reachable,
"model": cfg.get("model_name", ""),
"label": cfg.get("label", ""),
}
except Exception as e:
return {"configured": True, "reachable": False, "error": str(e), "model": cfg.get("model_name", "")}
@router.get("/status")
async def auth_status() -> dict:
return {
"claude": _claude_status(),
"gemini": _gemini_status(),
"local": await _local_status(),
}

View File

@@ -0,0 +1,205 @@
"""
Google OAuth 2.0 sign-in.
Flow:
1. GET /auth/google → redirect to Google's consent page
2. GET /auth/google/callback → exchange code, look up user, set JWT cookie
Users must be pre-registered by Scott before they can sign in:
cd cortex && .venv/bin/python manage_passwords.py google-add <username> <email>
Routes are public (added to _PUBLIC_PREFIXES in auth_middleware.py).
"""
import json
import logging
import secrets
import urllib.parse
import urllib.request
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, RedirectResponse, Response
from auth_utils import COOKIE_NAME, create_token, find_user_by_google, link_google
from config import settings
from persona import list_user_personas
logger = logging.getLogger(__name__)
router = APIRouter()
_GOOGLE_AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth"
_GOOGLE_TOKEN_URL = "https://oauth2.googleapis.com/token"
_GOOGLE_USERINFO = "https://openidconnect.googleapis.com/v1/userinfo"
_STATE_COOKIE = "oauth_state"
_STATE_MAX_AGE = 600 # 10 minutes — plenty of time to complete the flow
@router.get("/auth/google", include_in_schema=False)
async def google_login():
if not settings.google_client_id:
return HTMLResponse("Google sign-in is not configured on this server.", status_code=503)
state = secrets.token_urlsafe(16)
params = urllib.parse.urlencode({
"client_id": settings.google_client_id,
"redirect_uri": f"{settings.cortex_base_url}/auth/google/callback",
"response_type": "code",
"scope": "openid email profile",
"state": state,
"access_type": "online",
"prompt": "select_account",
})
resp = RedirectResponse(f"{_GOOGLE_AUTH_URL}?{params}", status_code=302)
resp.set_cookie(_STATE_COOKIE, state, max_age=_STATE_MAX_AGE, httponly=True, samesite="lax")
return resp
@router.get("/auth/google/callback", include_in_schema=False)
async def google_callback(
request: Request,
code: str = "",
state: str = "",
error: str = "",
):
if error:
return _error_page(f"Google sign-in was cancelled or denied: {error}")
if not code:
return _error_page("No authorisation code returned by Google.")
# CSRF check — state must match what we stored in the cookie
stored_state = request.cookies.get(_STATE_COOKIE)
if not stored_state or stored_state != state:
return _error_page("State mismatch — please try signing in again.")
# Exchange authorisation code for tokens
try:
token_data = _exchange_code(code)
except Exception as e:
logger.error("Google token exchange failed: %s", e)
return _error_page("Could not complete sign-in with Google. Please try again.")
access_token = token_data.get("access_token")
if not access_token:
return _error_page("No access token returned by Google.")
# Fetch the user's profile
try:
userinfo = _get_userinfo(access_token)
except Exception as e:
logger.error("Google userinfo fetch failed: %s", e)
return _error_page("Could not retrieve your Google profile. Please try again.")
google_sub = userinfo.get("sub", "")
google_email = userinfo.get("email", "")
if not google_sub or not google_email:
return _error_page("Your Google account didn't return a usable email address.")
# Match to a Cortex user
username = find_user_by_google(google_sub, google_email)
if not username:
logger.warning("Google sign-in rejected: no account for %s (%s)", google_sub, google_email)
return _error_page(
f"Your Google account (<strong>{google_email}</strong>) isn't registered with Cortex.<br><br>"
"Contact Scott to get access."
)
# Persist the stable sub so future lookups use it (not just email)
link_google(username, google_sub, google_email)
personas = list_user_personas(username)
if not personas:
return _error_page("No personas are configured for your account yet. Contact Scott.")
logger.info("Google sign-in: %s (%s)", username, google_email)
resp = RedirectResponse(f"/{username}/{personas[0]}", status_code=302)
_set_session_cookie(resp, username)
resp.delete_cookie(_STATE_COOKIE)
return resp
# ---------------------------------------------------------------------------
# Private helpers
# ---------------------------------------------------------------------------
def _exchange_code(code: str) -> dict:
body = urllib.parse.urlencode({
"code": code,
"client_id": settings.google_client_id,
"client_secret": settings.google_client_secret,
"redirect_uri": f"{settings.cortex_base_url}/auth/google/callback",
"grant_type": "authorization_code",
}).encode()
req = urllib.request.Request(
_GOOGLE_TOKEN_URL,
data=body,
headers={"Content-Type": "application/x-www-form-urlencoded"},
method="POST",
)
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read())
def _get_userinfo(access_token: str) -> dict:
req = urllib.request.Request(
_GOOGLE_USERINFO,
headers={"Authorization": f"Bearer {access_token}"},
)
with urllib.request.urlopen(req, timeout=10) as resp:
return json.loads(resp.read())
def _set_session_cookie(response: Response, username: str) -> None:
token = create_token(username)
response.set_cookie(
COOKIE_NAME,
token,
max_age=settings.jwt_expire_days * 86400,
httponly=True,
samesite="lax",
secure=False, # set True if terminating TLS at the app layer (not behind a proxy)
)
def _error_page(message: str) -> HTMLResponse:
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Sign In Failed</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after {{ box-sizing: border-box; margin: 0; padding: 0; }}
body {{
min-height: 100vh; display: flex; align-items: center; justify-content: center;
background: #0f1117; font-family: 'Inter', system-ui; font-weight: 450;
-webkit-font-smoothing: antialiased; color: #e2e8f0;
}}
.card {{
background: #1a1d27; border: 1px solid #2d3148; border-radius: 12px;
padding: 2.5rem 2rem; width: 100%; max-width: 420px; text-align: center;
}}
h1 {{ font-size: 1.25rem; font-weight: 700; color: #f87171; margin-bottom: 1rem; }}
p {{ font-size: 0.9rem; color: #94a3b8; margin-bottom: 1.75rem; line-height: 1.65; }}
a {{
display: inline-block; padding: 0.6rem 1.5rem;
background: #7c3aed; border-radius: 6px; color: #fff;
text-decoration: none; font-size: 0.9rem; font-weight: 600;
transition: background 0.15s;
}}
a:hover {{ background: #6d28d9; }}
</style>
</head>
<body>
<div class="card">
<h1>Sign In Failed</h1>
<p>{message}</p>
<a href="/login">← Back to Sign In</a>
</div>
</body>
</html>"""
return HTMLResponse(html, status_code=403)

View File

@@ -1,34 +1,71 @@
import asyncio
import json
from fastapi import APIRouter, HTTPException, Query
import platform
import jwt
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from context_loader import load_context
from llm_client import complete
from session_logger import log_turn
from session_store import load as load_session, save as save_session, list_all, generate_session_id, delete as delete_session
from session_store import load as load_session, save as save_session, list_all, generate_session_id, delete as delete_session, rename as rename_session, get_name as get_session_name
from config import settings
from persona import set_context, validate as validate_persona
from auth_utils import COOKIE_NAME, decode_token
import model_registry
import event_bus
from model_registry import get_role_config
router = APIRouter()
def _backend_label(backend: str, username: str, role: str = "chat") -> str:
"""Human-readable label for the model that handled a request (legacy path)."""
if backend == "claude":
return "Claude"
if backend == "gemini":
return "Gemini"
if backend == "local":
cfg = model_registry.get_best_local_model(username, role)
if cfg:
return cfg.get("label") or cfg.get("model_name") or "Local"
return "Local"
return backend.title()
def _role_model_label(username: str, role: str, actual_backend: str) -> str:
"""Return the model label for a role, falling back to the generic backend label."""
cfg = model_registry.get_model_for_role(username, role)
if cfg:
return cfg.get("label") or cfg.get("model_name") or _backend_label(actual_backend, username, role)
return _backend_label(actual_backend, username, role)
class Attachment(BaseModel):
filename: str
mime_type: str
data: str # base64 data URL for images (e.g. "data:image/png;base64,...")
class ChatRequest(BaseModel):
message: str
session_id: str | None = None
tier: int | None = None
model: str | None = None # "claude" or "gemini" to override; None = use primary_backend
model: str | None = None # legacy backend override ("claude"|"gemini"|"local")
slot: str | None = None # Phase 3: explicit slot ("primary"|"backup_1"|"backup_2")
chat_role: str = "chat" # active role: "chat"|"coder"|"research"|"distill" etc.
include_long: bool = True
include_mid: bool = True
include_short: bool = True
off_record: bool = False # skip session log (in-memory context preserved)
user: str = "scott"
persona: str = "inara"
attachment: Attachment | None = None # image attachment (text files injected client-side)
class BackendRequest(BaseModel):
primary: str # "claude" or "gemini"
primary: str # "claude", "gemini", or "local"
class NoteRequest(BaseModel):
@@ -62,19 +99,39 @@ async def _stream_chat(req: ChatRequest):
session_id = req.session_id or generate_session_id()
tier = req.tier or settings.default_tier
role_cfg = get_role_config(user, req.chat_role)
system_prompt = load_context(
tier,
include_long=req.include_long,
include_mid=req.include_mid,
include_short=req.include_short,
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
mode="otr" if req.off_record else "chat",
)
history = load_session(session_id)
history.append({"role": "user", "content": req.message})
# req.message already contains the full user text:
# - text files: client embedded content as a fenced code block
# - images: client added "📎 filename.png" note; image data is in req.attachment
# History always stores text only — base64 image data is never written to disk.
llm_attachment: dict | None = None
if req.attachment and req.attachment.mime_type.startswith("image/"):
llm_attachment = {
"filename": req.attachment.filename,
"mime_type": req.attachment.mime_type,
"data": req.attachment.data,
}
history.append({"role": "user", "content": req.message, "off_record": req.off_record})
task = asyncio.create_task(complete(
system_prompt=system_prompt,
messages=history,
model=req.model,
role=req.chat_role,
slot=req.slot,
attachment=llm_attachment,
))
try:
@@ -90,17 +147,35 @@ async def _stream_chat(req: ChatRequest):
try:
response_text, actual_backend = task.result()
history.append({"role": "assistant", "content": response_text})
if req.slot:
slot_cfg = model_registry.get_model_for_slot(user, req.chat_role, req.slot)
backend_label = (slot_cfg or {}).get("label") or _role_model_label(user, req.chat_role, actual_backend)
else:
backend_label = _role_model_label(user, req.chat_role, actual_backend)
host = platform.node()
history.append({
"role": "assistant",
"content": response_text,
"backend": actual_backend,
"backend_label": backend_label,
"host": host,
"off_record": req.off_record,
})
save_session(session_id, history)
log_turn(session_id, req.message, response_text)
if not req.off_record:
log_turn(session_id, req.message, response_text, backend_label, host)
requested = req.model or settings.primary_backend
# fallback_used only makes sense for explicit backend selections.
# In auto mode (req.model is None), just report what responded.
fallback_used = bool(req.model and actual_backend != req.model)
payload = {
"type": "response",
"response": response_text,
"session_id": session_id,
"backend": actual_backend,
"fallback_used": actual_backend != requested,
"backend_label": backend_label,
"host": host,
"fallback_used": fallback_used,
}
yield f"data: {json.dumps(payload)}\n\n"
@@ -128,19 +203,111 @@ async def chat(req: ChatRequest) -> StreamingResponse:
)
_BACKEND_CYCLE = ("claude", "gemini", "local")
_BACKEND_FALLBACK = {"claude": "gemini", "gemini": "claude", "local": "claude"}
def _request_user(request: Request) -> str | None:
"""Extract username from JWT cookie, or None."""
try:
token = request.cookies.get(COOKIE_NAME)
return decode_token(token) if token else None
except (jwt.InvalidTokenError, Exception):
return None
def _local_model_info(request: Request) -> dict | None:
"""Return the best local model {label, model_name} for the session user, or None."""
username = _request_user(request)
if not username:
return None
try:
cfg = model_registry.get_best_local_model(username, "chat")
if cfg:
return {"label": cfg.get("label", ""), "model_name": cfg.get("model_name", "")}
except Exception:
pass
return None
def _chat_slot_models(username: str) -> list[dict]:
"""Return [{slot, label, type}] for each configured slot in the chat role, primary first."""
registry = model_registry.get_registry(username)
role_slots = registry.get("roles", {}).get("chat", {})
result = []
for slot_key in model_registry.PRIORITY_KEYS:
model_id = role_slots.get(slot_key)
if not model_id:
continue
resolved = model_registry._resolve_model(registry, model_id)
if resolved:
result.append({
"slot": slot_key,
"label": resolved.get("label") or resolved.get("model_name") or "",
"type": resolved.get("type", ""),
})
return result
def _available_roles_for_toggle(username: str) -> list[dict]:
"""Return roles with a primary model assigned (excluding orchestrator) for the UI toggle.
Returns [{role, label, model_label, type}] ordered by settings.defined_roles.
"""
registry = model_registry.get_registry(username)
roles_cfg = registry.get("roles", {})
result = []
for role_name in settings.get_defined_roles():
if role_name == "orchestrator":
continue
primary_id = roles_cfg.get(role_name, {}).get("primary")
if not primary_id:
continue
resolved = model_registry._resolve_model(registry, primary_id)
if resolved:
result.append({
"role": role_name,
"label": role_name.title(),
"model_label": resolved.get("label") or resolved.get("model_name") or "",
"type": resolved.get("type", ""),
})
return result
@router.get("/backend")
async def get_backend() -> dict:
other = "gemini" if settings.primary_backend == "claude" else "claude"
return {"primary": settings.primary_backend, "fallback": other}
async def get_backend(request: Request) -> dict:
username = _request_user(request)
chat_models = _chat_slot_models(username) if username else []
available_roles = _available_roles_for_toggle(username) if username else []
p = settings.primary_backend
orch_label = None
if username:
orch_cfg = model_registry.get_model_for_role(username, "orchestrator")
if orch_cfg:
orch_label = orch_cfg.get("label") or orch_cfg.get("model_name") or None
return {
"chat_models": chat_models, # Phase 3: [{slot, label, type}] for chat-role slots
"available_roles": available_roles, # kept for banner + backward compat
"orchestrator_model": orch_label,
# Legacy fields kept for backward compat
"primary": p,
"fallback": _BACKEND_FALLBACK.get(p, "claude"),
"local_model": _local_model_info(request),
}
@router.post("/backend")
async def set_backend(req: BackendRequest) -> dict:
if req.primary not in ("claude", "gemini"):
raise HTTPException(status_code=400, detail="primary must be 'claude' or 'gemini'")
async def set_backend(req: BackendRequest, request: Request) -> dict:
if req.primary not in _BACKEND_CYCLE:
raise HTTPException(status_code=400, detail="primary must be 'claude', 'gemini', or 'local'")
settings.primary_backend = req.primary
other = "gemini" if req.primary == "claude" else "claude"
return {"primary": settings.primary_backend, "fallback": other}
return {
"primary": req.primary,
"fallback": _BACKEND_FALLBACK[req.primary],
"local_model": _local_model_info(request),
}
def _set_ctx(user: str, persona: str) -> None:
@@ -159,7 +326,8 @@ async def get_history(
persona: str = Query("inara"),
) -> dict:
_set_ctx(user, persona)
return {"session_id": session_id, "messages": load_session(session_id)}
name = get_session_name(session_id)
return {"session_id": session_id, "name": name, "messages": load_session(session_id)}
@router.get("/sessions")
@@ -171,6 +339,71 @@ async def list_sessions(
return {"sessions": list_all()}
class SessionRename(BaseModel):
name: str
@router.patch("/sessions/{session_id}")
async def rename_session_endpoint(
session_id: str,
req: SessionRename,
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
_set_ctx(user, persona)
found = rename_session(session_id, req.name.strip())
if not found:
raise HTTPException(status_code=404, detail=f"Session {session_id} not found")
return {"ok": True, "session_id": session_id, "name": req.name.strip()}
@router.post("/api/sessions/backfill-names")
async def backfill_session_names(
request: Request,
user: str = Query(""),
persona: str = Query(""),
) -> dict:
"""Name every unnamed session using its first user message (truncated to 60 chars).
Idempotent — only touches sessions that have no name set.
user/persona default to the JWT session user + last-used persona cookie."""
# Resolve user from JWT if not provided
if not user:
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
user = decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
# Resolve persona from cookie if not provided
if not persona:
from persona import list_user_personas
persona_cookie = request.cookies.get("cx_last_persona", "")
available = list_user_personas(user)
persona = persona_cookie if persona_cookie in available else (available[0] if available else "")
if not persona:
raise HTTPException(status_code=400, detail="No persona found for user")
_set_ctx(user, persona)
sessions = list_all()
named = 0
for s in sessions:
if s.get("name"):
continue
messages = load_session(s["session_id"])
first_user = next((m for m in messages if m.get("role") == "user"), None)
if not first_user:
continue
text = (first_user.get("content") or "").strip()
if not text:
continue
auto_name = text[:60].rstrip() + ("" if len(text) > 60 else "")
rename_session(s["session_id"], auto_name)
named += 1
return {"ok": True, "named": named, "total": len(sessions)}
@router.delete("/sessions/{session_id}")
async def delete_session_endpoint(
session_id: str,

479
cortex/routers/crons.py Normal file
View File

@@ -0,0 +1,479 @@
"""
Schedules web UI — GET/POST /settings/crons/*
Lets users view, add, toggle, and remove cron jobs without going through the AI.
Cron data lives in home/{user}/persona/{persona}/CRONS.json.
Scheduler registration mirrors what tools/cron.py does so changes take effect immediately.
"""
import html as _html
import logging
import secrets
from datetime import datetime, timezone
from pathlib import Path
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token
from cron_runner import load_crons, save_crons, parse_schedule
from persona import list_user_personas
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
def _integrations_nav(username: str) -> str:
from auth_utils import _read_auth
role = _read_auth(username).get("role", "user")
if role == "admin":
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
return ""
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _short_id() -> str:
return "c_" + secrets.token_urlsafe(6)
def _scheduler_add(job: dict, sched_kwargs: dict) -> None:
import asyncio
try:
import scheduler as sched_module
from cron_runner import run_job
s = sched_module.get_scheduler()
if s and s.running:
sched_id = f"{job['user']}:{job['persona']}:{job['id']}"
s.add_job(
lambda j=job: asyncio.ensure_future(run_job(j)),
"cron",
id=sched_id,
replace_existing=True,
**sched_kwargs,
)
except Exception as e:
logger.warning("scheduler_add failed: %s", e)
def _scheduler_remove(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.remove_job(job_id)
except Exception:
pass
def _scheduler_pause(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.pause_job(job_id)
except Exception:
pass
def _scheduler_resume(job_id: str) -> None:
try:
import scheduler as sched_module
s = sched_module.get_scheduler()
if s and s.running:
s.resume_job(job_id)
except Exception:
pass
_TYPE_CLASS = {
"remind": "badge-remind", "note": "badge-note", "message": "badge-message",
"brief": "badge-brief", "task": "badge-task",
}
def _render_cron_list(username: str) -> str:
personas = list_user_personas(username)
if not personas:
return '<div class="empty-state">No personas found.</div>'
all_empty = True
html_parts: list[str] = []
for persona in personas:
crons = load_crons(username, persona)
if not crons:
continue
all_empty = False
rows = []
for c in crons:
cid = _html.escape(c["id"])
label = _html.escape(c.get("label", ""))
schedule = _html.escape(c.get("schedule", ""))
job_type = _html.escape(c.get("type", ""))
payload = _html.escape(c.get("payload", ""))
enabled = c.get("enabled", True)
last_run = (c.get("last_run") or "")[:10] or "never"
pers_esc = _html.escape(persona)
type_class = _TYPE_CLASS.get(c.get("type", ""), "badge-note")
status_cls = "badge-enabled" if enabled else "badge-paused"
status_txt = "enabled" if enabled else "paused"
toggle_txt = "Pause" if enabled else "Resume"
rows.append(f"""
<tr>
<td>{label}</td>
<td><code>{schedule}</code></td>
<td><span class="badge {type_class}">{job_type}</span></td>
<td class="payload-cell" title="{payload}">{payload}</td>
<td>{last_run}</td>
<td><span class="badge {status_cls}">{status_txt}</span></td>
<td>
<div class="cron-actions">
<a href="/settings/crons/edit?cron_id={cid}&persona={pers_esc}"
class="btn-cron">Edit</a>
<form method="POST" action="/settings/crons/toggle" style="display:inline">
<input type="hidden" name="cron_id" value="{cid}">
<input type="hidden" name="persona" value="{pers_esc}">
<button type="submit" class="btn-cron">{toggle_txt}</button>
</form>
<form method="POST" action="/settings/crons/remove" style="display:inline"
onsubmit="return confirm('Delete this schedule?')">
<input type="hidden" name="cron_id" value="{cid}">
<input type="hidden" name="persona" value="{pers_esc}">
<button type="submit" class="btn-cron btn-cron-del">Delete</button>
</form>
</div>
</td>
</tr>""")
html_parts.append(f"""
<div class="persona-group">
<p class="persona-group-label">{_html.escape(persona)}</p>
<table class="cron-table">
<thead>
<tr>
<th>Label</th><th>Schedule</th><th>Type</th>
<th>Payload</th><th>Last run</th><th>Status</th><th></th>
</tr>
</thead>
<tbody>{"".join(rows)}
</tbody>
</table>
</div>""")
if all_empty:
return '<div class="empty-state">No schedules yet. Add one below.</div>'
return "\n".join(html_parts)
def _persona_options(username: str, selected: str = "") -> str:
personas = list_user_personas(username)
return "\n".join(
f'<option value="{_html.escape(p)}"{"selected" if p == selected else ""}>{_html.escape(p)}</option>'
for p in personas
)
_TYPE_OPTIONS = ("remind", "note", "message", "brief", "task")
_TYPE_LABELS = {
"remind": "remind — append to REMINDERS.md",
"note": "note — append to SCRATCH.md",
"message": "message — send payload as-is",
"brief": "brief — LLM response, no tools",
"task": "task — full orchestrator tool loop",
}
def _render_edit_form(job: dict, persona: str) -> str:
cid = _html.escape(job["id"])
pers_esc = _html.escape(persona)
label = _html.escape(job.get("label", ""))
schedule = _html.escape(job.get("schedule", ""))
payload = _html.escape(job.get("payload", ""))
cur_type = job.get("type", "remind")
type_opts = "\n".join(
f'<option value="{t}" {"selected" if t == cur_type else ""}>{_html.escape(_TYPE_LABELS.get(t, t))}</option>'
for t in _TYPE_OPTIONS
)
return f"""
<div class="section" style="border: 2px solid var(--pg-accent); border-radius: 8px; padding: 1rem;">
<h2 style="margin-top:0">Edit schedule</h2>
<form method="POST" action="/settings/crons/save">
<input type="hidden" name="cron_id" value="{cid}">
<input type="hidden" name="persona" value="{pers_esc}">
<div class="add-form-grid">
<div class="field">
<label>Persona</label>
<input type="text" value="{pers_esc}" disabled style="opacity:0.5">
</div>
<div class="field">
<label for="edit_job_type">Type</label>
<select id="edit_job_type" name="job_type">{type_opts}</select>
</div>
<div class="field">
<label for="edit_label">Label</label>
<input type="text" id="edit_label" name="label" value="{label}" required autocomplete="off">
</div>
<div class="field">
<label for="edit_schedule">Schedule</label>
<input type="text" id="edit_schedule" name="schedule" value="{schedule}"
required autocomplete="off" spellcheck="false">
<p class="hint">
hourly · daily · daily:HH:MM · weekly:DOW · weekly:DOW:HH:MM ·
monthly · monthly:DD · monthly:DD:HH:MM · yearly:MM:DD · yearly:MM:DD:HH:MM
</p>
</div>
<div class="field field-full">
<label for="edit_payload">Payload / prompt</label>
<textarea id="edit_payload" name="payload" rows="3" required>{payload}</textarea>
</div>
</div>
<div style="display:flex; gap:0.5rem; align-items:center; margin-top:0.5rem">
<button type="submit" class="btn-submit" style="margin-top:0">Save changes</button>
<a href="/settings/crons" style="font-size:0.85rem; color:var(--pg-muted)">Cancel</a>
</div>
</form>
</div>"""
def _render_page(username: str, back_persona: str = "", success: str = "", error: str = "",
edit_html: str = "") -> str:
html = (_STATIC / "crons.html").read_text()
html = html.replace("{{ edit_html }}", edit_html)
html = html.replace("{{ cron_list_html }}", _render_cron_list(username))
html = html.replace("{{ persona_options }}", _persona_options(username, back_persona))
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{_html.escape(success)}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{_html.escape(error)}</p>')
return html
# ---------------------------------------------------------------------------
# Routes
# ---------------------------------------------------------------------------
@router.get("/settings/crons", include_in_schema=False)
async def crons_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_render_page(username, back_persona))
@router.post("/settings/crons/add", include_in_schema=False)
async def cron_add(
request: Request,
persona: str = Form(""),
label: str = Form(""),
schedule: str = Form(""),
job_type: str = Form(""),
payload: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
label = label.strip()
schedule = schedule.strip()
payload = payload.strip()
persona = persona.strip()
_VALID_TYPES = ("remind", "note", "message", "brief", "task")
if job_type not in _VALID_TYPES:
return HTMLResponse(_render_page(username, back_persona, error=f"Invalid type: {job_type}"))
try:
sched_kwargs = parse_schedule(schedule)
except ValueError as e:
return HTMLResponse(_render_page(username, back_persona, error=f"Bad schedule: {e}"))
if not label:
return HTMLResponse(_render_page(username, back_persona, error="Label is required."))
if not payload:
return HTMLResponse(_render_page(username, back_persona, error="Payload is required."))
crons = load_crons(username, persona)
job = {
"id": _short_id(),
"user": username,
"persona": persona,
"label": label,
"schedule": schedule,
"type": job_type,
"payload": payload,
"enabled": True,
"created_at": _now(),
"last_run": None,
}
crons.append(job)
save_crons(crons, username, persona)
_scheduler_add(job, sched_kwargs)
logger.info("cron added via UI: %s %s [%s]", job["id"], schedule, job_type)
return HTMLResponse(_render_page(username, back_persona, success=f"Schedule '{label}' added."))
@router.post("/settings/crons/toggle", include_in_schema=False)
async def cron_toggle(
request: Request,
cron_id: str = Form(""),
persona: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
crons = load_crons(username, persona)
for c in crons:
if c["id"] == cron_id:
c["enabled"] = not c.get("enabled", True)
save_crons(crons, username, persona)
sched_id = f"{username}:{persona}:{cron_id}"
if c["enabled"]:
_scheduler_resume(sched_id)
action = "resumed"
else:
_scheduler_pause(sched_id)
action = "paused"
logger.info("cron %s %s via UI", cron_id, action)
return HTMLResponse(_render_page(username, back_persona, success=f"Schedule {action}."))
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
@router.post("/settings/crons/remove", include_in_schema=False)
async def cron_remove(
request: Request,
cron_id: str = Form(""),
persona: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
crons = load_crons(username, persona)
before = len(crons)
crons = [c for c in crons if c["id"] != cron_id]
if len(crons) == before:
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
save_crons(crons, username, persona)
_scheduler_remove(f"{username}:{persona}:{cron_id}")
logger.info("cron %s removed via UI", cron_id)
return HTMLResponse(_render_page(username, back_persona, success="Schedule deleted."))
@router.get("/settings/crons/edit", include_in_schema=False)
async def cron_edit_page(request: Request, cron_id: str = "", persona: str = ""):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
crons = load_crons(username, persona)
job = next((c for c in crons if c["id"] == cron_id), None)
if not job:
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
edit_html = _render_edit_form(job, persona)
return HTMLResponse(_render_page(username, back_persona, edit_html=edit_html))
@router.post("/settings/crons/save", include_in_schema=False)
async def cron_save(
request: Request,
cron_id: str = Form(""),
persona: str = Form(""),
label: str = Form(""),
schedule: str = Form(""),
job_type: str = Form(""),
payload: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
label = label.strip()
schedule = schedule.strip()
payload = payload.strip()
if job_type not in _TYPE_OPTIONS:
return HTMLResponse(_render_page(username, back_persona, error=f"Invalid type: {job_type}"))
if not label:
return HTMLResponse(_render_page(username, back_persona, error="Label is required."))
if not payload:
return HTMLResponse(_render_page(username, back_persona, error="Payload is required."))
try:
sched_kwargs = parse_schedule(schedule)
except ValueError as e:
# Re-render with the edit form still open so the user can fix the schedule
crons = load_crons(username, persona)
job = next((c for c in crons if c["id"] == cron_id), None)
edit_html = _render_edit_form(job, persona) if job else ""
return HTMLResponse(_render_page(username, back_persona, error=f"Bad schedule: {e}",
edit_html=edit_html))
crons = load_crons(username, persona)
for c in crons:
if c["id"] == cron_id:
c["label"] = label
c["schedule"] = schedule
c["type"] = job_type
c["payload"] = payload
save_crons(crons, username, persona)
# Replace the live scheduler job with the updated schedule
sched_id = f"{username}:{persona}:{cron_id}"
_scheduler_remove(sched_id)
if c.get("enabled", True):
_scheduler_add(c, sched_kwargs)
logger.info("cron %s updated via UI [%s]", cron_id, schedule)
return HTMLResponse(_render_page(username, back_persona,
success=f"Schedule '{label}' updated."))
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))

View File

@@ -5,21 +5,99 @@ Manual memory distillation endpoints.
POST /distill/mid — summarize short → MEMORY_MID.md (LLM)
POST /distill/long — integrate mid → MEMORY_LONG.md (LLM)
POST /distill/all — run all three in sequence
POST /distill/rebuild — wipe mid + long, then run all three from scratch
All endpoints require ?user=<username>&persona=<name> query params.
Concurrency: one distillation at a time per persona. A second request while one
is running returns 409 immediately — no silent queuing.
"""
from fastapi import APIRouter
import asyncio
from datetime import datetime, timedelta
from fastapi import APIRouter, HTTPException, Query
from memory_distiller import distill_short, distill_mid, distill_long
from persona import validate as validate_persona, set_context, persona_path as _persona_path
import scheduler
router = APIRouter(prefix="/distill")
# Per-persona asyncio lock. Key: (user, persona)
_LOCKS: dict[tuple, asyncio.Lock] = {}
_LOCKS_META: dict[tuple, str] = {} # key → which step is currently running
# Minimum time between successive runs of each endpoint, per persona.
# Prevents accidental rapid-fire runs and token waste.
_COOLDOWNS: dict[tuple, timedelta] = {
"short": timedelta(minutes=1),
"mid": timedelta(minutes=30),
"long": timedelta(hours=6),
"all": timedelta(hours=1),
"rebuild": timedelta(hours=6),
}
_LAST_RUN: dict[tuple, datetime] = {} # key: (user, persona, endpoint)
def _get_lock(user: str, persona: str) -> asyncio.Lock:
key = (user, persona)
if key not in _LOCKS:
_LOCKS[key] = asyncio.Lock()
return _LOCKS[key]
def _resolve(user: str, persona: str) -> tuple[str, str]:
try:
u, p = validate_persona(user, persona)
except Exception:
raise HTTPException(status_code=404, detail=f"Persona not found: {user}/{persona}")
set_context(u, p)
return u, p
def _check_lock(user: str, persona: str) -> asyncio.Lock:
"""Return the lock if free, raise 409 if already held."""
lock = _get_lock(user, persona)
if lock.locked():
step = _LOCKS_META.get((user, persona), "distillation")
raise HTTPException(
status_code=409,
detail=f"A {step} is already running for {persona} — please wait for it to finish.",
)
return lock
def _check_cooldown(user: str, persona: str, endpoint: str) -> None:
"""Raise 429 if the endpoint was run too recently for this persona."""
cooldown = _COOLDOWNS.get(endpoint)
if not cooldown:
return
key = (user, persona, endpoint)
last = _LAST_RUN.get(key)
if last:
elapsed = datetime.now() - last
if elapsed < cooldown:
remaining = cooldown - elapsed
mins = int(remaining.total_seconds() // 60)
secs = int(remaining.total_seconds() % 60)
wait = f"{mins}m {secs}s" if mins else f"{secs}s"
raise HTTPException(
status_code=429,
detail=f"{endpoint} was just run — please wait {wait} before running again.",
)
def _record_run(user: str, persona: str, endpoint: str) -> None:
_LAST_RUN[(user, persona, endpoint)] = datetime.now()
@router.get("/status")
async def distill_status() -> dict:
"""Show auto-distillation schedule and next run times."""
from config import settings
# Include which personas are currently distilling
active = [f"{u}/{p}" for (u, p), lock in _LOCKS.items() if lock.locked()]
return {
"enabled": settings.auto_distill,
"jobs": scheduler.status(),
"active": active,
"config": {
"short": settings.auto_distill_short,
"mid": settings.auto_distill_mid,
@@ -29,32 +107,132 @@ async def distill_status() -> dict:
@router.post("/short")
async def do_distill_short() -> dict:
return {"ok": True, **distill_short()}
async def do_distill_short(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "short")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "short distill"
try:
result = distill_short(u, p)
_record_run(u, p, "short")
return {"ok": True, **result}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/mid")
async def do_distill_mid() -> dict:
result = await distill_mid()
async def do_distill_mid(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "mid")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "mid distill"
try:
result = await distill_mid(u, p)
if "error" not in result:
_record_run(u, p, "mid")
return {"ok": "error" not in result, **result}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/long")
async def do_distill_long() -> dict:
result = await distill_long()
async def do_distill_long(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "long")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "long distill"
try:
result = await distill_long(u, p)
if "error" not in result:
_record_run(u, p, "long")
return {"ok": "error" not in result, **result}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/all")
async def do_distill_all() -> dict:
short_result = distill_short()
mid_result = await distill_mid()
async def do_distill_all(
user: str = Query(...),
persona: str = Query(...),
) -> dict:
u, p = _resolve(user, persona)
_check_cooldown(u, p, "all")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "full distill"
try:
short_result = distill_short(u, p)
mid_result = await distill_mid(u, p)
if "error" in mid_result:
return {"ok": False, "short": short_result, "mid": mid_result}
long_result = await distill_long()
long_result = await distill_long(u, p)
ok = "error" not in long_result
if ok:
_record_run(u, p, "all")
return {
"ok": "error" not in long_result,
"ok": ok,
"short": short_result,
"mid": mid_result,
"long": long_result,
}
finally:
_LOCKS_META.pop((u, p), None)
@router.post("/rebuild")
async def do_distill_rebuild(
user: str = Query(...),
persona: str = Query(...),
) -> dict: # noqa: E501
"""Wipe MEMORY_MID and MEMORY_LONG (with backups), then run short → mid → long.
Use when memories have drifted, been corrupted, or you want a clean slate
rebuilt purely from session logs. Hand-edited content will be replaced.
"""
u, p = _resolve(user, persona)
_check_cooldown(u, p, "rebuild")
lock = _check_lock(u, p)
async with lock:
_LOCKS_META[(u, p)] = "memory rebuild"
try:
from memory_distiller import _rotate_backup, _read
inara_dir = _persona_path(u, p)
# Back up then wipe mid and long before rebuilding
for name in ("MEMORY_MID.md", "MEMORY_LONG.md"):
path = inara_dir / name
if path.exists():
_rotate_backup(path)
path.write_text(
f"# {name}\n\n*Cleared for rebuild — {__import__('datetime').datetime.now().strftime('%Y-%m-%d %H:%M')}.*\n"
)
short_result = distill_short(u, p)
mid_result = await distill_mid(u, p)
if "error" in mid_result:
return {"ok": False, "short": short_result, "mid": mid_result, "rebuilt": True}
long_result = await distill_long(u, p)
ok = "error" not in long_result
if ok:
_record_run(u, p, "rebuild")
return {
"ok": ok,
"short": short_result,
"mid": mid_result,
"long": long_result,
"rebuilt": True,
}
finally:
_LOCKS_META.pop((u, p), None)

View File

@@ -1,10 +1,12 @@
"""
Read/write the Inara identity markdown files.
Read/write Inara identity markdown files, and search past session logs.
Only whitelisted filenames are accessible — no path traversal possible.
"""
import re
from fastapi import APIRouter, HTTPException, Query
from pydantic import BaseModel
from persona import persona_path, set_context, validate as validate_persona
from config import settings as _settings
router = APIRouter()
@@ -18,9 +20,29 @@ ALLOWED = {
"MEMORY_LONG.md",
"MEMORY_MID.md",
"MEMORY_SHORT.md",
"MEMORY_LONG.bak1.md",
"MEMORY_LONG.bak2.md",
"MEMORY_MID.bak1.md",
"MEMORY_MID.bak2.md",
"MEMORY_SHORT.bak1.md",
"MEMORY_SHORT.bak2.md",
"HELP.md",
# Agent private notes — backups only; AGENT_NOTES.md itself is agent-only
"AGENT_NOTES.bak1.md",
"AGENT_NOTES.bak2.md",
"AGENT_NOTES.bak3.md",
}
# Files that can be read via the panel but not written by users
READ_ONLY = {
"AGENT_NOTES.bak1.md",
"AGENT_NOTES.bak2.md",
"AGENT_NOTES.bak3.md",
}
# Files served from home/{user}/ instead of persona path
USER_FILES = {"email_allowlist.json", "usage.json"}
def _resolve(user: str, persona: str) -> None:
"""Validate and set context from query params. Raises HTTPException on bad input."""
@@ -31,7 +53,11 @@ def _resolve(user: str, persona: str) -> None:
raise HTTPException(status_code=404, detail=str(e))
def _path(filename: str):
def _path(filename: str, user: str = ""):
if filename in USER_FILES:
if not user:
raise HTTPException(status_code=400, detail="user param required for this file")
return _settings.home_root() / user / filename
if filename not in ALLOWED:
raise HTTPException(status_code=404, detail=f"File not found: {filename}")
return persona_path() / filename
@@ -47,10 +73,22 @@ async def list_files(
files = []
for name in sorted(ALLOWED):
p = persona_dir / name
st = p.stat() if p.exists() else None
files.append({
"name": name,
"exists": p.exists(),
"size": p.stat().st_size if p.exists() else 0,
"size": st.st_size if st else 0,
"modified": st.st_mtime if st else None,
})
for name in sorted(USER_FILES):
p = _settings.home_root() / user / name
st = p.stat() if p.exists() else None
files.append({
"name": name,
"exists": p.exists(),
"size": st.st_size if st else 0,
"modified": st.st_mtime if st else None,
"scope": "user",
})
return {"files": files}
@@ -62,10 +100,14 @@ async def get_file(
persona: str = Query("inara"),
) -> dict:
_resolve(user, persona)
p = _path(filename)
p = _path(filename, user=user)
if not p.exists():
raise HTTPException(status_code=404, detail=f"{filename} does not exist")
return {"name": filename, "content": p.read_text()}
return {
"name": filename,
"content": p.read_text(),
"readonly": filename in READ_ONLY,
}
class FileWrite(BaseModel):
@@ -79,7 +121,65 @@ async def save_file(
user: str = Query("scott"),
persona: str = Query("inara"),
) -> dict:
if filename in READ_ONLY:
raise HTTPException(status_code=403, detail=f"{filename} is read-only.")
_resolve(user, persona)
p = _path(filename)
p = _path(filename, user=user)
p.write_text(req.content)
return {"ok": True, "name": filename, "size": len(req.content)}
# ── Session search ────────────────────────────────────────────────────────────
_CONTEXT_CHARS = 120 # chars of context to include around each match
@router.get("/sessions/search")
async def search_sessions(
q: str = Query(..., min_length=2),
user: str = Query("scott"),
persona: str = Query("inara"),
limit: int = Query(20, ge=1, le=100),
) -> dict:
"""Full-text search across past session logs.
Returns up to `limit` matches, newest sessions first.
Each match includes a short excerpt (120 chars before/after) for context.
"""
_resolve(user, persona)
sessions_dir = persona_path() / "sessions"
if not sessions_dir.exists():
return {"query": q, "matches": [], "total_files_searched": 0}
pattern = re.compile(re.escape(q), re.IGNORECASE)
session_files = sorted(sessions_dir.glob("*.md"), reverse=True) # newest first
matches = []
for sf in session_files:
if len(matches) >= limit:
break
try:
text = sf.read_text()
except OSError:
continue
for m in pattern.finditer(text):
if len(matches) >= limit:
break
start = max(0, m.start() - _CONTEXT_CHARS)
end = min(len(text), m.end() + _CONTEXT_CHARS)
excerpt = text[start:end].strip()
# Prefix with ellipsis if we truncated the left side
if start > 0:
excerpt = "" + excerpt
if end < len(text):
excerpt = excerpt + ""
matches.append({
"date": sf.stem, # YYYY-MM-DD
"excerpt": excerpt,
})
return {
"query": q,
"matches": matches,
"total_files_searched": len(session_files),
}

View File

@@ -3,14 +3,16 @@ import logging
from fastapi import APIRouter, HTTPException, Request, Response
from google.auth.transport import requests as google_requests
from google.oauth2 import id_token
from auth_utils import get_user_channels
from context_loader import load_context
from llm_client import complete
from persona import set_context
from session_logger import log_turn
from session_store import load as load_session, save as save_session
from config import settings
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/channels/google-chat")
router = APIRouter()
# Workspace Add-on Chat apps: JWT is issued by accounts.google.com.
# (Legacy standalone Chat bots used chat@system.gserviceaccount.com — different format.)
@@ -35,7 +37,7 @@ def _msg(text: str) -> dict:
}
def _verify_system_id_token(token: str) -> None:
def _verify_system_id_token(token: str, audience: str) -> None:
"""Verify the systemIdToken from authorizationEventObject.
For Workspace Add-on Chat apps Google sends the token in the request body
@@ -44,13 +46,13 @@ def _verify_system_id_token(token: str) -> None:
Claims verified:
iss = "https://accounts.google.com"
aud = settings.google_chat_audience (the endpoint URL)
aud = the per-user audience from channels.json (the endpoint URL)
"""
try:
claims = id_token.verify_oauth2_token(
token,
google_requests.Request(),
audience=settings.google_chat_audience,
audience=audience,
)
except Exception as exc:
logger.warning("Google Chat JWT verification failed: %s", exc)
@@ -60,17 +62,30 @@ def _verify_system_id_token(token: str) -> None:
raise HTTPException(status_code=401, detail="Wrong issuer")
@router.post("")
async def receive(request: Request):
@router.post("/channels/google-chat/{username}")
async def receive(username: str, request: Request):
channels = get_user_channels(username)
cfg = channels.get("google_chat")
if not cfg:
logger.warning("Google Chat: no channel config for user %r", username)
raise HTTPException(status_code=404, detail="Channel not configured for this user")
persona_name = cfg.get("persona", "inara")
audience = cfg.get("audience", "")
backend = cfg.get("backend", settings.primary_backend)
timeout = cfg.get("timeout", 25)
set_context(username, persona_name)
body = await request.json()
# Verify the systemIdToken embedded in the request body
if settings.google_chat_audience:
if audience:
token = body.get("authorizationEventObject", {}).get("systemIdToken", "")
if not token:
logger.warning("Google Chat: missing systemIdToken")
logger.warning("Google Chat: missing systemIdToken for %s", username)
raise HTTPException(status_code=401, detail="Missing token")
_verify_system_id_token(token)
_verify_system_id_token(token, audience)
chat = body.get("chat", {})
@@ -79,8 +94,8 @@ async def receive(request: Request):
if "addedToSpacePayload" in chat:
space_type = chat["addedToSpacePayload"].get("space", {}).get("type", "")
if space_type == "DM":
return _msg(f"✨ Hello! I'm {settings.agent_name}. What can I help you with?")
return _msg(f"✨ Hello! I'm {settings.agent_name}. Send me a message and I'll do my best to help.")
return _msg(f"✨ Hello! I'm {persona_name.capitalize()}. What can I help you with?")
return _msg(f"✨ Hello! I'm {persona_name.capitalize()}. Send me a message and I'll do my best to help.")
if "removedFromSpacePayload" in chat:
return Response(status_code=200)
@@ -107,7 +122,7 @@ async def receive(request: Request):
logger.warning("Google Chat: empty user_text, ignoring")
return Response(status_code=200)
session_id = "gc_" + space_name.replace("/", "_")
session_id = f"gc_{username}_{space_name.replace('/', '_')}"
system_prompt = load_context(settings.default_tier)
history = load_session(session_id)
history.append({"role": "user", "content": user_text})
@@ -117,9 +132,9 @@ async def receive(request: Request):
complete(
system_prompt=system_prompt,
messages=history,
model=settings.google_chat_backend,
model=backend,
),
timeout=settings.google_chat_timeout,
timeout=timeout,
)
except asyncio.TimeoutError:
logger.warning("Google Chat request timed out for session %s", session_id)

70
cortex/routers/help.py Normal file
View File

@@ -0,0 +1,70 @@
"""
Help page router.
Routes:
GET /help → full-page help viewer (requires auth)
"""
import logging
from pathlib import Path
import jwt
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, _read_auth
from persona import list_user_personas
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
@router.get("/help", include_in_schema=False)
async def help_page(request: Request, persona: str = ""):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
# Use persona from query param if valid, else prefer last-visited from cookie
if persona and persona in personas:
back_persona = persona
else:
back_persona = _preferred_persona(request, username)
back_href = f"/{username}/{back_persona}" if back_persona else "/"
html = (_STATIC / "help.html").read_text()
config_tag = (
f'<script>window.HELP_CONFIG = '
f'{{user: "{username}", persona: "{back_persona}", backHref: "{back_href}"}};</script>'
)
html = html.replace("</head>", f"{config_tag}\n</head>", 1)
nav = '<a href="/settings/integrations" class="nav-link">Integrations</a>' \
if _read_auth(username).get("role", "user") == "admin" else ""
html = html.replace("{{ integrations_nav }}", nav)
return HTMLResponse(html)

View File

@@ -0,0 +1,199 @@
"""
Home Assistant webhook router — POST /webhook/ha/{username}/{webhook_id}
Receives event payloads from HA automations and routes them to Inara.
Auth: the webhook_id in the URL acts as the shared secret (same model HA uses).
Response is delivered async via notify() — NC Talk, web push, etc.
channels.json schema:
"homeassistant": {
"webhook_id": "your-secret-id",
"persona": "inara",
"tier": 2,
"role": "chat",
"tools": false
}
HA automation example (rest_command):
rest_command:
cortex_notify:
url: "https://cortex.dgrzone.com/webhook/ha/scott/your-secret-id"
method: POST
content_type: "application/json"
payload: '{"message": "{{message}}", "entity_id": "{{entity_id}}", "state": "{{state}}"}'
automation:
trigger:
- trigger: state
entity_id: binary_sensor.front_door
to: "on"
action:
- action: rest_command.cortex_notify
data:
message: "Front door opened"
entity_id: "binary_sensor.front_door"
state: "on"
"""
import json
import logging
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response
from auth_utils import get_user_channels, get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
from context_loader import load_context
from llm_client import complete
from notification import notify
from persona import set_context
from session_logger import log_turn
from session_store import load as load_session, save as save_session
from config import settings
import event_bus
import model_registry
import orchestrator_engine
import openai_orchestrator
logger = logging.getLogger(__name__)
router = APIRouter()
def _build_task(body: dict) -> str:
"""Turn an HA event payload into a natural-language prompt for Inara."""
if "message" in body:
msg = str(body["message"])
extras = {k: body[k] for k in ("entity_id", "state", "trigger", "event", "area") if k in body}
if extras:
msg += "\n\nContext: " + json.dumps(extras)
return msg
return "Home Assistant event:\n" + json.dumps(body, indent=2)
async def _process_event(username: str, body: dict, cfg: dict) -> None:
persona_name = cfg.get("persona", "inara")
tier = cfg.get("tier") or settings.default_tier
role = cfg.get("role", "chat")
use_tools = cfg.get("tools", False)
set_context(username, persona_name)
task = _build_task(body)
session_id = f"ha_{username}"
history = load_session(session_id)
session_msgs = list(history)
logger.info("HA event for %s: %r", username, task[:80])
backend = "unknown"
try:
if use_tools:
role_cfg = model_registry.get_role_config(username, role)
system_prompt = load_context(
tier,
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
)
orch_model = model_registry.get_model_for_role(username, "orchestrator")
user_role_val = get_user_role(username)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(username)
c_allow = set(policy.get("allow", []))
c_deny = set(policy.get("deny", []))
max_risk, risk_wl, risk_bl = get_risk_policy(username)
if orch_model and orch_model.get("type") == "local_openai":
result = await openai_orchestrator.run(
task=task,
system_prompt=system_prompt,
session_messages=session_msgs or None,
model_cfg=orch_model,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
else:
gemini_key = (
(orch_model.get("api_key") if orch_model else None)
or get_user_gemini_key(username)
)
result = await orchestrator_engine.run(
task=task,
system_prompt=system_prompt,
session_messages=session_msgs or None,
respond_with_claude=True,
gemini_api_key=gemini_key,
model_name=orch_model.get("model_name") if orch_model else None,
response_role=role,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
response_text = result.response
backend = result.backend
else:
system_prompt = load_context(tier)
msgs = list(session_msgs) + [{"role": "user", "content": task}]
response_text, backend = await complete(system_prompt=system_prompt, messages=msgs)
except Exception as e:
logger.error("HA event error for %s: %s", username, e)
return
logger.info("HA response via %s (%d chars)", backend, len(response_text))
history.append({"role": "user", "content": task})
history.append({"role": "assistant", "content": response_text})
save_session(session_id, history)
log_turn(session_id, task, response_text)
await event_bus.publish({
"type": "ha_event",
"session_id": session_id,
"response": response_text,
"backend": backend,
})
await notify(username, response_text)
@router.post("/webhook/ha/{username}/{webhook_id}")
async def ha_webhook(
username: str,
webhook_id: str,
request: Request,
background_tasks: BackgroundTasks,
) -> Response:
"""Receive an event from a Home Assistant automation and route it to Inara."""
channels = get_user_channels(username)
cfg = channels.get("homeassistant")
if not cfg:
raise HTTPException(status_code=404, detail="Channel not configured")
if webhook_id != cfg.get("webhook_id", ""):
logger.warning("HA webhook: bad webhook_id for user %r", username)
raise HTTPException(status_code=401, detail="Invalid webhook ID")
content_type = request.headers.get("content-type", "")
if "application/json" in content_type:
try:
body = await request.json()
except Exception:
raise HTTPException(status_code=400, detail="Invalid JSON")
else:
form = await request.form()
body = dict(form)
if not body:
return Response(status_code=200)
background_tasks.add_task(_process_event, username, body, cfg)
return Response(status_code=200)

849
cortex/routers/local_llm.py Normal file
View File

@@ -0,0 +1,849 @@
"""
Model Registry settings — providers, hosts, models, and role assignments.
Routes:
GET /settings/models → settings page (canonical)
GET /settings/local → redirect to /settings/models
POST /settings/local/host → save/create a local host
POST /settings/local/host/{id}/remove → remove a host (and its models)
POST /settings/local/google-account → save/create a Google account
POST /settings/local/google-account/{id}/remove → remove a Google account
POST /settings/local/anthropic-key → save/create an Anthropic API key
POST /settings/local/anthropic-key/{id}/remove → remove an Anthropic API key
POST /settings/local/models/add → add a model (any provider)
POST /settings/local/models/{id}/edit → edit an existing model entry
POST /settings/local/models/{id}/remove → remove a model
POST /settings/local/roles/add → add a custom role (redirects to #roles)
POST /settings/local/roles/remove → remove a custom role (redirects to #roles)
POST /api/models/role → AJAX: set a role assignment
GET /api/local-llm/fetch-models → proxy to host /api/models (JSON)
"""
import json as _json
import logging
from pathlib import Path
import httpx
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, _read_auth
from config import settings as app_settings
from persona import list_user_personas
import model_registry as reg
from tools import TOOL_CATEGORIES
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
def _integrations_nav(username: str) -> str:
role = _read_auth(username).get("role", "user")
if role == "admin":
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
return ""
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
def _host_row_html(h: dict) -> str:
"""Return the HTML for one host config row (edit form + remove link)."""
api_key = h.get("api_key", "")
key_hint = f"{api_key[-4:]}" if api_key else "not set"
ht = h.get("host_type", "openwebui")
ow = ' selected' if ht == "openwebui" else ''
ai = ' selected' if ht == "openai" else ''
hid = h["id"]
hlbl = h.get("label", "")
hurl = h.get("api_url", "")
maxc = h.get("max_concurrent", 3)
return f'''
<div class="host-row">
<form method="POST" action="/settings/local/host" class="host-form">
<input type="hidden" name="host_id" value="{hid}">
<div class="field-row">
<div class="field">
<label>Label</label>
<input type="text" name="label" value="{hlbl}"
placeholder="Gaming Laptop" autocomplete="off" data-form-type="other">
</div>
<div class="field" style="flex:2">
<label>API URL</label>
<input type="text" name="api_url" value="{hurl}"
placeholder="http://192.168.x.x:3000"
autocomplete="off" spellcheck="false" data-form-type="other">
</div>
</div>
<div class="field-row">
<div class="field">
<label>API Key</label>
<input type="password" name="api_key" placeholder="Leave blank to keep existing"
autocomplete="new-password" data-1p-ignore data-lpignore="true"
data-form-type="other">
<p class="key-status">Current: {key_hint}</p>
</div>
<div class="field" style="flex:0 0 auto">
<label>Type</label>
<select name="host_type">
<option value="openwebui"{ow}>Open WebUI / Ollama</option>
<option value="openai"{ai}>OpenAI-compatible API</option>
</select>
</div>
<div class="field" style="flex:0 0 auto; width:6rem">
<label>Max parallel</label>
<input type="number" name="max_concurrent" min="1" max="20"
value="{maxc}" style="width:100%">
</div>
</div>
<div class="btn-row">
<button type="submit" class="btn btn-secondary btn-sm">Save</button>
<button type="button" class="btn btn-secondary btn-sm fetch-btn"
data-host-id="{hid}">Fetch models</button>
<span class="fetch-status" id="fetch-{hid}"></span>
</div>
</form>
<form method="POST" action="/settings/local/host/{hid}/remove"
onsubmit="return confirm('Remove host and all its models?')"
style="margin-top:0.5rem">
<button type="submit" class="btn-link danger">Remove host</button>
</form>
</div>'''
# ── Auth helper ───────────────────────────────────────────────────────────────
def _get_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
# ── Page renderer ─────────────────────────────────────────────────────────────
def _render(username: str, request: Request | None = None, success: str = "", error: str = "") -> str:
registry = reg.get_registry(username)
hosts = registry.get("hosts", [])
models = registry.get("models", [])
roles = registry.get("roles", {})
builtins = reg._builtins()
host_by_id = {h["id"]: h for h in hosts}
goog_accts = registry.get("providers", {}).get("google", {}).get("accounts", [])
# ── Google account rows ───────────────────────────────────────────────────
google_account_rows = ""
for a in goog_accts:
hint = (a.get("api_key") or "")[:10] + "" if a.get("api_key") else "no key"
google_account_rows += f'''
<div class="account-row">
<div>
<span class="account-label">{a.get("label") or "Unnamed"}</span>
<span class="account-hint">{hint}</span>
</div>
<form method="POST" action="/settings/local/google-account/{a["id"]}/remove"
onsubmit="return confirm('Remove this Google account?')">
<button type="submit" class="btn-link danger">Remove</button>
</form>
</div>'''
if not google_account_rows:
google_account_rows = '<p class="empty-note">No accounts configured yet.</p>'
# ── Host rows — split cloud (openai) vs local (openwebui) ─────────────────
cloud_hosts = [h for h in hosts if h.get("host_type") == "openai"]
local_hosts = [h for h in hosts if h.get("host_type", "openwebui") != "openai"]
cloud_host_rows = "".join(_host_row_html(h) for h in cloud_hosts)
local_host_rows = "".join(_host_row_html(h) for h in local_hosts)
if not cloud_host_rows:
cloud_host_rows = '<p class="empty-note">No cloud API services configured yet. Add one below.</p>'
if not local_host_rows:
local_host_rows = '<p class="empty-note">No local hosts configured yet. Add one below.</p>'
host_options = "".join(
f'<option value="{h["id"]}">{h.get("label") or h["api_url"]}</option>'
for h in hosts
)
# ── Anthropic API key rows ────────────────────────────────────────────────
anthropic_api_keys = reg.get_anthropic_api_keys(username)
anthropic_keys_js = _json.dumps(anthropic_api_keys)
anthropic_key_rows = ""
for c in anthropic_api_keys:
hint = c.get("hint", "no key")
anthropic_key_rows += f'''
<div class="account-row">
<div>
<span class="account-label">{c.get("label") or "API Key"}</span>
<span class="account-hint">{hint}</span>
</div>
<form method="POST" action="/settings/local/anthropic-key/{c["id"]}/remove"
onsubmit="return confirm('Remove this Anthropic API key?')">
<button type="submit" class="btn-link danger">Remove</button>
</form>
</div>'''
if not anthropic_key_rows:
anthropic_key_rows = '<p class="empty-note">No API keys configured. Add one below or use Claude CLI (OAuth).</p>'
# ── Model rows (all providers) ────────────────────────────────────────────
_PROVIDER_BADGE = {
"claude_cli": ('<span class="pbadge pb-anthropic">Anthropic</span>', "Claude CLI"),
"anthropic_api": ('<span class="pbadge pb-anthropic">Anthropic</span>', "API Key"),
"gemini_api": ('<span class="pbadge pb-google">Google</span>', ""),
"local_openai": ('<span class="pbadge pb-local">Local</span>', ""),
}
model_rows = ""
for m in models:
resolved = reg._resolve_model(registry, m["id"])
if not resolved:
continue
mtype = m.get("type", "local_openai")
badge, default_secondary = _PROVIDER_BADGE.get(mtype, ("", ""))
if mtype == "local_openai":
h = host_by_id.get(m.get("host_id", ""), {})
secondary = h.get("label") or h.get("api_url", "")
elif mtype == "gemini_api":
acct = next((a for a in goog_accts if a["id"] == m.get("account_id")), None)
secondary = acct["label"] if acct else ""
else:
secondary = default_secondary
ctx = f'<span class="ctx-badge">{m.get("context_k",0)}k</span>' if m.get("context_k") else ""
no_tools = '' if m.get("tools", True) else '<span class="pbadge pb-notools">no tools</span>'
tags_html = " ".join(f'<span class="tag">{t}</span>' for t in (m.get("tags") or []))
sec = f'<span class="model-host">{secondary}</span>' if secondary else ""
# ── Inline edit form fields (type-specific) ───────────────────────────
if mtype == "local_openai":
host_opts = "".join(
f'<option value="{h["id"]}"'
f'{" selected" if h["id"] == m.get("host_id") else ""}>'
f'{h.get("label") or h.get("api_url","")}</option>'
for h in hosts
)
mid = m["id"]
extra_fields = (
f'<div class="field"><label>Host</label>'
f'<select name="host_id" id="edit-host-{mid}">{host_opts}</select></div>'
f'<div class="btn-row" style="margin-bottom:0.75rem">'
f'<button type="button" class="btn btn-secondary btn-sm edit-fetch-btn" data-id="{mid}">Fetch models</button>'
f'<span class="fetch-status" id="edit-fetch-status-{mid}"></span>'
f'</div>'
f'<div id="edit-model-select-wrap-{mid}" style="display:none; margin-bottom:0.75rem">'
f'<label>Pick from host</label>'
f'<select id="edit-model-picker-{mid}"><option value="">— fetch first —</option></select>'
f'</div>'
)
elif mtype == "gemini_api":
acct_opts = "".join(
f'<option value="{a["id"]}"'
f'{" selected" if a["id"] == m.get("account_id") else ""}>'
f'{a.get("label","Unnamed")}</option>'
for a in goog_accts
)
extra_fields = (
f'<div class="field"><label>Google Account</label>'
f'<select name="account_id">{acct_opts}</select></div>'
)
elif mtype == "anthropic_api":
key_opts = "".join(
f'<option value="{c["id"]}"'
f'{" selected" if c["id"] == m.get("credential_id") else ""}>'
f'{c.get("label","API Key")} ({c.get("hint","")})</option>'
for c in anthropic_api_keys
)
extra_fields = (
f'<div class="field"><label>API Key</label>'
f'<select name="credential_id">{key_opts or "<option value=\"\">No API keys configured</option>"}</select></div>'
)
else:
extra_fields = '<input type="hidden" name="credential_id" value="cli">'
cur_label = m.get("label", "")
cur_model_name = m.get("model_name", "")
cur_ctx = m.get("context_k", 0) or 0
cur_max_rounds = m.get("max_rounds") or 0
cur_tools = m.get("tools", True)
cur_tags = ", ".join(m.get("tags") or [])
cur_reasoning_budget = m.get("reasoning_budget_tokens") or 0
_rb_levels = [(0, "Off — Non-think"), (1024, "Light"), (4096, "Moderate"), (8192, "High"), (32768, "Max")]
reasoning_opts = "".join(
f'<option value="{v}" {"selected" if cur_reasoning_budget == v else ""}>{lbl}</option>'
for v, lbl in _rb_levels
)
model_rows += f'''
<div class="model-row" id="model-{m["id"]}">
<div class="model-row-header">
<div class="model-info">
<div>{badge}<span class="model-label">{m.get("label") or m.get("model_name","")}</span>{ctx}{no_tools}</div>
<span class="model-name">{m.get("model_name","")}</span>
{sec}
<div class="tag-row">{tags_html}</div>
</div>
<div class="model-btns">
<button type="button" class="row-btn model-edit-btn" data-id="{m["id"]}">Edit</button>
<form method="POST" action="/settings/local/models/{m["id"]}/remove"
onsubmit="return confirm('Remove this model?')" style="margin:0">
<button type="submit" class="row-btn danger">Remove</button>
</form>
</div>
</div>
<form class="model-edit-form" id="edit-form-{m["id"]}" style="display:none"
method="POST" action="/settings/local/models/{m["id"]}/edit">
<input type="hidden" name="mtype" value="{mtype}">
<div class="field-row">
<div class="field">
<label>Display label</label>
<input type="text" name="label" value="{cur_label}"
placeholder="My Model" autocomplete="off" data-form-type="other">
</div>
<div class="field">
<label>Model name / ID</label>
<input type="text" name="model_name" value="{cur_model_name}"
placeholder="provider/model-name" autocomplete="off"
spellcheck="false" data-form-type="other" required>
</div>
</div>
{extra_fields}
<div class="field-row">
<div class="field" style="flex:0 0 auto">
<label title="Context window size in thousands of tokens. 0 = assume 32k.">Context (k)</label>
<input type="number" name="context_k" value="{cur_ctx}" min="0"
title="Context window size in thousands of tokens. 0 = assume 32k (compaction budget ~24k tokens).">
</div>
<div class="field" style="flex:0 0 auto">
<label title="Per-model tool loop cap. 0 = use the global default (orchestrator_max_rounds).">Max rounds</label>
<input type="number" name="max_rounds" value="{cur_max_rounds}" min="0"
title="Per-model tool loop cap. 0 = use the global default (orchestrator_max_rounds).">
</div>
<div class="field" style="flex:0 0 auto">
<label title="Reasoning depth via OpenRouter's reasoning.budget_tokens. Off = Non-think. Light ~1k, Moderate ~4k, High ~8k, Max ~32k tokens.">Reasoning</label>
<select name="reasoning_budget_tokens"
title="Reasoning depth via OpenRouter's reasoning.budget_tokens. Off = Non-think. Light ~1k, Moderate ~4k, High ~8k, Max ~32k tokens.">
{reasoning_opts}
</select>
</div>
<div class="field" style="flex:0 0 auto">
<label title="Whether this model supports tool calling. If not supported, requests skip the tool loop entirely.">Tool calling</label>
<select name="tools"
title="Whether this model supports tool calling. If not supported, requests skip the tool loop entirely.">
<option value="1" {'selected' if cur_tools else ''}>Supported</option>
<option value="0" {'' if cur_tools else 'selected'}>Not supported</option>
</select>
</div>
<div class="field">
<label>Tags</label>
<input type="text" name="tags" value="{cur_tags}"
placeholder="fast, code, vision" autocomplete="off" data-form-type="other">
</div>
</div>
<div class="btn-row" style="margin-top:0.5rem">
<button type="submit" class="btn btn-primary btn-sm">Save</button>
<button type="button" class="model-edit-cancel btn btn-secondary btn-sm"
data-id="{m["id"]}">Cancel</button>
</div>
</form>
</div>'''
if not model_rows:
model_rows = '<p class="empty-note">No models added yet.</p>'
# ── Role assignment rows ──────────────────────────────────────────────────
model_opts = '<option value="">— .env default —</option>\n'
model_opts += '<optgroup label="Built-in">\n'
for bid, bm in builtins.items():
model_opts += f' <option value="{bid}">{bm["label"]}</option>\n'
model_opts += '</optgroup>\n'
if models:
model_opts += '<optgroup label="Configured models">\n'
for m in models:
lbl = m.get("label") or m.get("model_name", m["id"])
model_opts += f' <option value="{m["id"]}">{lbl}</option>\n'
model_opts += '</optgroup>\n'
all_roles = reg.get_all_roles(username)
role_rows = ""
for role in all_roles:
is_required = role in reg.REQUIRED_ROLES
role_cfg = roles.get(role, {})
role_title = role.replace("_", " ").title()
required_badge = (
'<span class="required-badge">required</span>'
if is_required else ''
)
rcp_danger = (
'' if is_required else
f'<div class="rcp-danger">'
f'<form method="POST" action="/settings/local/roles/remove" class="remove-role-form">'
f'<input type="hidden" name="role_name" value="{role}">'
f'<button type="submit" class="btn-link danger" data-role="{role}">Remove this role…</button>'
f'</form>'
f'</div>'
)
role_rows += (
f'<div class="role-row" data-role="{role}">'
f'<div class="role-name-col">'
f'<span class="role-name">{role_title}</span>'
f'{required_badge}'
f'</div>'
f'<div class="role-slots">'
)
for slot in reg.PRIORITY_KEYS[:2]:
slot_label = slot.replace("_", " ").title()
sel = (
f'<select class="role-select" data-role="{role}" '
f'data-slot="{slot}" title="{slot_label}">\n{model_opts}\n</select>'
)
role_rows += f'<div class="role-slot"><span class="slot-label">{slot_label}</span>{sel}</div>'
role_rows += (
f'</div>'
f'<button class="role-cfg-btn" data-role="{role}" title="Configure">⚙</button>'
f'</div>'
f'<div class="role-config-panel" id="rcp-{role}">'
f'<div class="rcp-field">'
f'<label class="rcp-label">System prompt addition</label>'
f'<textarea class="rcp-textarea" data-role="{role}" rows="3" '
f'placeholder="Extra instructions injected into the system prompt when this role is active…"></textarea>'
f'</div>'
f'<div class="rcp-field">'
f'<div style="display:flex;flex-direction:column;gap:0.3rem">'
f'<label class="rcp-check">'
f'<input type="checkbox" class="rcp-datetime-cb" data-role="{role}" checked>'
f'<span>Inject current date &amp; time into system prompt</span>'
f'</label>'
f'<label class="rcp-check">'
f'<input type="checkbox" class="rcp-mode-cb" data-role="{role}" checked>'
f'<span>Inject session mode (Chat / Off The Record) into system prompt</span>'
f'</label>'
f'</div>'
f'<p class="rcp-hint" style="margin-top:0.4rem">Disable both for pure processing roles (summarizer, classifier, translator).</p>'
f'</div>'
f'<div class="rcp-field">'
f'<label class="rcp-label">Tool allow-list '
f'<span class="rcp-hint">— all checked means no restriction (use all accessible tools)</span></label>'
f'<div class="rcp-tools" id="rcp-tools-{role}"></div>'
f'</div>'
f'<div class="rcp-actions">'
f'<button class="btn btn-primary btn-sm rcp-save" data-role="{role}">Save</button>'
f'<button class="btn btn-secondary btn-sm rcp-cancel" data-role="{role}">Cancel</button>'
f'</div>'
f'{rcp_danger}'
f'</div>'
)
role_data_js = _json.dumps({
role: {slot: (roles.get(role, {}).get(slot) or "") for slot in reg.PRIORITY_KEYS[:2]}
for role in all_roles
})
role_config_data_js = _json.dumps({
role: {
"system_append": roles.get(role, {}).get("system_append", ""),
"tools": roles.get(role, {}).get("tools") or None,
"inject_datetime": roles.get(role, {}).get("inject_datetime", True),
"inject_mode": roles.get(role, {}).get("inject_mode", True),
}
for role in all_roles
})
tool_categories_js = _json.dumps(TOOL_CATEGORIES)
# ── Catalog data + Google accounts for JS ─────────────────────────────────
google_accounts_js = _json.dumps(reg.get_google_accounts(username))
google_catalog_js = _json.dumps(reg.get_catalog("google"))
anthropic_catalog_js = _json.dumps(reg.get_catalog("anthropic"))
cloud_catalog_js = _json.dumps(reg.get_catalog("cloud"))
has_hosts = "true" if hosts else "false"
html = (_STATIC / "local_llm.html").read_text()
replacements = {
"{{ username }}": username,
"{{ google_account_rows }}": google_account_rows,
"{{ anthropic_key_rows }}": anthropic_key_rows,
"{{ cloud_host_rows }}": cloud_host_rows,
"{{ local_host_rows }}": local_host_rows,
"{{ model_rows }}": model_rows,
"{{ host_options }}": host_options,
"{{ role_rows }}": role_rows,
"{{ role_data_js }}": role_data_js,
"{{ role_config_data_js }}": role_config_data_js,
"{{ tool_categories_js }}": tool_categories_js,
"{{ google_accounts_js }}": google_accounts_js,
"{{ anthropic_keys_js }}": anthropic_keys_js,
"{{ google_catalog_js }}": google_catalog_js,
"{{ anthropic_catalog_js }}": anthropic_catalog_js,
"{{ cloud_catalog_js }}": cloud_catalog_js,
"{{ has_hosts }}": has_hosts,
}
for key, val in replacements.items():
html = html.replace(key, val)
back_persona = _preferred_persona(request, username) if request else ""
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="msg success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="msg error">{error}</p>')
return html
# ── Routes ────────────────────────────────────────────────────────────────────
@router.get("/settings/models", include_in_schema=False)
async def models_page_canonical(request: Request):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
return HTMLResponse(_render(username, request))
@router.get("/settings/local", include_in_schema=False)
async def models_page_legacy(request: Request):
return RedirectResponse("/settings/models", status_code=301)
@router.post("/settings/local/google-account", include_in_schema=False)
async def save_google_account(
request: Request,
account_id: str = Form(""),
label: str = Form(""),
api_key: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not api_key.strip() and not account_id.strip():
return HTMLResponse(_render(username, request, error="API key is required."))
reg.save_google_account(username, account_id or None, label, api_key)
return HTMLResponse(_render(username, request, success="Google account saved."))
@router.post("/settings/local/google-account/{account_id}/remove", include_in_schema=False)
async def remove_google_account(request: Request, account_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_google_account(username, account_id)
return HTMLResponse(_render(username, request, success="Google account removed."))
@router.post("/settings/local/anthropic-key", include_in_schema=False)
async def save_anthropic_api_key(
request: Request,
key_id: str = Form(""),
label: str = Form(""),
api_key: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not api_key.strip() and not key_id.strip():
return HTMLResponse(_render(username, request, error="API key is required."))
reg.save_anthropic_api_key(username, key_id or None, label, api_key)
return HTMLResponse(_render(username, request, success="Anthropic API key saved."))
@router.post("/settings/local/anthropic-key/{key_id}/remove", include_in_schema=False)
async def remove_anthropic_api_key(request: Request, key_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_anthropic_api_key(username, key_id)
return HTMLResponse(_render(username, request, success="Anthropic API key removed."))
@router.post("/settings/local/host", include_in_schema=False)
async def save_host(
request: Request,
host_id: str = Form(""),
label: str = Form(""),
api_url: str = Form(""),
api_key: str = Form(""),
host_type: str = Form("openwebui"),
max_concurrent: int = Form(3),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not api_url.strip():
return HTMLResponse(_render(username, request, error="API URL is required."))
reg.save_host(username, host_id or None, label, api_url, api_key, host_type, max_concurrent)
return HTMLResponse(_render(username, request, success="Host saved."))
@router.post("/settings/local/host/{host_id}/remove", include_in_schema=False)
async def remove_host(request: Request, host_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_host(username, host_id)
return HTMLResponse(_render(username, request, success="Host removed."))
@router.post("/settings/local/models/add", include_in_schema=False)
async def add_model(
request: Request,
provider: str = Form("local"),
label: str = Form(""),
context_k: int = Form(0),
max_rounds: int = Form(0),
tools: int = Form(1),
tags: str = Form(""),
reasoning_budget_tokens: int = Form(0),
# local-only fields
host_id: str = Form(""),
model_name: str = Form(""),
# cloud-only fields
cloud_model_name: str = Form(""),
account_id: str = Form(""),
credential_id: str = Form("cli"),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
tag_list = [t.strip() for t in tags.split(",") if t.strip()]
max_rounds_ = max_rounds or None
tools_bool = tools != 0
reasoning_budget_ = reasoning_budget_tokens or None
if provider == "local":
if not model_name.strip():
return HTMLResponse(_render(username, request, error="Model name is required."))
if not host_id.strip():
return HTMLResponse(_render(username, request, error="Select a host."))
reg.save_model(username, None, host_id, label, model_name, context_k, tag_list,
max_rounds=max_rounds_, tools=tools_bool,
reasoning_budget_tokens=reasoning_budget_)
display = label or model_name
elif provider in ("google", "anthropic"):
if not cloud_model_name.strip():
return HTMLResponse(_render(username, request, error="Select a model from the catalog."))
if provider == "google" and not account_id.strip():
return HTMLResponse(_render(username, request, error="Select a Google account."))
reg.save_cloud_model(
username, None, provider, cloud_model_name, label,
account_id=account_id or None,
credential_id=credential_id or None,
context_k=context_k, tags=tag_list,
max_rounds=max_rounds_, tools=tools_bool,
)
display = label or cloud_model_name
else:
return HTMLResponse(_render(username, request, error=f"Unknown provider: {provider}"))
logger.info("model added: %s / %s (%s)", username, display, provider)
return HTMLResponse(_render(username, request, success=f'Model "{display}" added.'))
@router.post("/settings/local/models/{model_id}/edit", include_in_schema=False)
async def edit_model(
request: Request,
model_id: str,
mtype: str = Form(""),
label: str = Form(""),
model_name: str = Form(""),
context_k: int = Form(0),
max_rounds: int = Form(0),
tools: int = Form(1),
tags: str = Form(""),
reasoning_budget_tokens: int = Form(0),
host_id: str = Form(""),
account_id: str = Form(""),
credential_id: str = Form("cli"),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not model_name.strip():
return HTMLResponse(_render(username, request, error="Model name is required."))
tag_list = [t.strip() for t in tags.split(",") if t.strip()]
max_rounds_ = max_rounds or None
tools_bool = tools != 0
reasoning_budget_ = reasoning_budget_tokens or None
if mtype == "local_openai":
if not host_id.strip():
return HTMLResponse(_render(username, request, error="Select a host for this model."))
reg.save_model(username, model_id, host_id, label, model_name, context_k, tag_list,
max_rounds=max_rounds_, tools=tools_bool,
reasoning_budget_tokens=reasoning_budget_)
elif mtype == "gemini_api":
reg.save_cloud_model(username, model_id, "google", model_name, label,
account_id=account_id or None, context_k=context_k, tags=tag_list,
max_rounds=max_rounds_, tools=tools_bool)
elif mtype in ("claude_cli", "anthropic_api"):
reg.save_cloud_model(username, model_id, "anthropic", model_name, label,
credential_id=credential_id or "cli", context_k=context_k, tags=tag_list,
max_rounds=max_rounds_, tools=tools_bool)
else:
return HTMLResponse(_render(username, request, error=f"Unknown model type: {mtype}"))
display = label.strip() or model_name.strip()
logger.info("model edited: %s / %s (%s)", username, display, mtype)
return HTMLResponse(_render(username, request, success=f'Model "{display}" updated.'))
@router.post("/settings/local/models/{model_id}/remove", include_in_schema=False)
async def remove_model(request: Request, model_id: str):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
reg.remove_model(username, model_id)
return HTMLResponse(_render(username, request, success="Model removed."))
@router.post("/settings/local/roles/add", include_in_schema=False)
async def add_custom_role_route(
request: Request,
role_name: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
name = role_name.strip().lower()
if not name or not name[0].isalpha():
return HTMLResponse(_render(username, request, error="Role name must start with a letter."))
ok = reg.add_custom_role(username, name)
if not ok:
return HTMLResponse(_render(username, request, error=f'"{name}" is a required role and cannot be re-added.'))
logger.info("custom role added: %s / %s", username, name)
return RedirectResponse("/settings/models#roles", status_code=303)
@router.post("/settings/local/roles/remove", include_in_schema=False)
async def remove_custom_role_route(
request: Request,
role_name: str = Form(""),
):
username = _get_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
name = role_name.strip()
ok = reg.remove_custom_role(username, name)
if not ok:
return HTMLResponse(_render(username, request, error=f'"{name}" is a required role and cannot be removed.'))
logger.info("custom role removed: %s / %s", username, name)
return RedirectResponse("/settings/models#roles", status_code=303)
@router.post("/api/models/role")
async def set_role(request: Request) -> JSONResponse:
"""AJAX: assign a model to a role priority slot.
Body: {"role": "chat", "slot": "primary", "model_id": "abc123" | ""}
"""
username = _get_user(request)
if not username:
return JSONResponse({"error": "Not authenticated"}, status_code=401)
try:
body = await request.json()
except Exception:
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
role = body.get("role", "").strip()
slot = body.get("slot", "").strip()
model_id = body.get("model_id", "").strip() or None
if not role or not slot:
return JSONResponse({"error": "role and slot are required"}, status_code=400)
ok = reg.set_role(username, role, slot, model_id)
if not ok:
return JSONResponse({"error": "Invalid slot or model_id not found"}, status_code=400)
logger.info("role set: %s %s.%s = %s", username, role, slot, model_id)
return JSONResponse({"ok": True})
@router.post("/api/models/role-config")
async def set_role_config(request: Request) -> JSONResponse:
"""AJAX: save system_append, tool allow-list, and inject_datetime flag for a role.
Body: {"role": "coder", "system_append": "...", "tools": [...] | null, "inject_datetime": true}
tools=null clears the allow-list (role uses all accessible tools).
inject_datetime=false suppresses the date/time header for pure processing roles.
"""
username = _get_user(request)
if not username:
return JSONResponse({"error": "Not authenticated"}, status_code=401)
try:
body = await request.json()
except Exception:
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
role = body.get("role", "").strip()
system_append = body.get("system_append", "")
tools = body.get("tools") # list[str] or None
inject_datetime = body.get("inject_datetime", True)
inject_mode = body.get("inject_mode", True)
if not role:
return JSONResponse({"error": "role is required"}, status_code=400)
if tools is not None and not isinstance(tools, list):
return JSONResponse({"error": "tools must be a list or null"}, status_code=400)
reg.set_role_config(username, role, system_append, tools,
inject_datetime=bool(inject_datetime),
inject_mode=bool(inject_mode))
logger.info("role config saved: %s %s (tools=%s inject_datetime=%s inject_mode=%s)",
username, role, len(tools) if tools is not None else "all",
inject_datetime, inject_mode)
return JSONResponse({"ok": True})
@router.get("/api/local-llm/fetch-models")
async def fetch_models(request: Request, host_id: str = "") -> JSONResponse:
"""Proxy to the host's models endpoint. host_id selects which host."""
username = _get_user(request)
if not username:
return JSONResponse({"error": "Not authenticated"}, status_code=401)
registry = reg.get_registry(username)
hosts = registry.get("hosts", [])
host = next((h for h in hosts if h["id"] == host_id), None) if host_id else (hosts[0] if hosts else None)
if host:
api_url, api_key, host_type = host.get("api_url",""), host.get("api_key",""), host.get("host_type","openwebui")
else:
api_url, api_key, host_type = app_settings.local_api_url, app_settings.local_api_key, "openwebui"
if not api_url:
return JSONResponse({"error": "No host configured."}, status_code=400)
models_path = "/models" if host_type == "openai" else "/api/models"
url = api_url.rstrip("/") + models_path
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
try:
async with httpx.AsyncClient(timeout=8) as client:
resp = await client.get(url, headers=headers)
resp.raise_for_status()
data = resp.json()
models = sorted(
[{"id": m["id"], "name": m.get("name") or m["id"]} for m in data.get("data", [])],
key=lambda m: m["name"].lower(),
)
return JSONResponse({"models": models})
except httpx.HTTPStatusError as e:
return JSONResponse({"error": f"Host returned {e.response.status_code}"}, status_code=502)
except Exception as e:
return JSONResponse({"error": str(e)}, status_code=502)

View File

@@ -3,17 +3,21 @@ import hashlib
import hmac
import json
import logging
import secrets
import httpx
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response
from config import settings
from auth_utils import get_user_channels, get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
from context_loader import load_context
from llm_client import complete
from notification import _send_nct_message
from persona import set_context
from session_logger import log_turn
from session_store import load as load_session, save as save_session
from config import settings
import event_bus
import model_registry
import orchestrator_engine
import openai_orchestrator
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
@@ -26,58 +30,44 @@ if not logger.handlers:
router = APIRouter()
def _verify_signature(body: bytes, random_header: str, sig_header: str) -> bool:
def _verify_signature(body: bytes, random_header: str, sig_header: str, secret: str) -> bool:
"""Nextcloud signs requests with HMAC-SHA256(key=secret, msg=random+body)."""
expected = hmac.new(
settings.nextcloud_talk_bot_secret.encode(),
secret.encode(),
(random_header + body.decode("utf-8", errors="replace")).encode(),
hashlib.sha256,
).hexdigest()
return hmac.compare_digest(expected, sig_header.lower())
async def _send_reply(conversation_token: str, message: str) -> None:
async def _send_reply(conversation_token: str, message: str, nextcloud_url: str, secret: str) -> None:
"""Post a message to Nextcloud Talk as the bot."""
url = (
f"{settings.nextcloud_url}/ocs/v2.php/apps/spreed/api/v1"
f"/bot/{conversation_token}/message"
)
# NC Talk verifies HMAC over (random + message_text), NOT the raw body.
# See BotController::getBotFromHeaders → checksumVerificationService::validateRequest($random, $sig, $secret, $message)
body_dict = {"message": message}
body_bytes = json.dumps(body_dict, ensure_ascii=False).encode("utf-8")
random_str = secrets.token_hex(32)
sig = hmac.new(
settings.nextcloud_talk_bot_secret.encode(),
(random_str + message).encode("utf-8"),
hashlib.sha256,
).hexdigest()
logger.info("NCT _send_reply → %s (body: %s)", url, body_bytes.decode())
try:
async with httpx.AsyncClient() as client:
resp = await client.post(
url,
content=body_bytes,
headers={
"Content-Type": "application/json",
"OCS-APIRequest": "true",
"X-Nextcloud-Talk-Bot-Random": random_str,
"X-Nextcloud-Talk-Bot-Signature": sig,
},
timeout=15,
)
logger.info("NCT reply: %s%s", resp.status_code, resp.text[:400])
except Exception as e:
logger.error("NCT reply error: %s", e)
logger.info("NCT _send_reply → room %s (%d chars)", conversation_token, len(message))
await _send_nct_message(nextcloud_url, secret, conversation_token, message)
async def _process_message(conversation_token: str, user_text: str, actor_name: str) -> None:
async def _process_message(
conversation_token: str,
user_text: str,
actor_name: str,
username: str,
persona_name: str,
nextcloud_url: str,
secret: str,
timeout: int,
cfg: dict,
) -> None:
logger.info("NCT process: token=%s user=%s text=%r", conversation_token, actor_name, user_text)
session_id = f"nct_{conversation_token}"
system_prompt = load_context(settings.default_tier)
set_context(username, persona_name)
tier = cfg.get("tier") or settings.default_tier
role = cfg.get("role", "chat")
use_tools = cfg.get("tools", False)
session_id = f"nct_{username}_{conversation_token}"
history = load_session(session_id)
history.append({"role": "user", "content": user_text})
session_msgs = list(history) # snapshot before we append
await event_bus.publish({
"type": "nct_message",
@@ -87,21 +77,88 @@ async def _process_message(conversation_token: str, user_text: str, actor_name:
"actor": actor_name,
})
backend = "unknown"
try:
response_text, backend = await asyncio.wait_for(
complete(system_prompt=system_prompt, messages=history),
timeout=settings.nextcloud_talk_timeout,
if use_tools:
await _send_reply(conversation_token, "⏳ Working on it…", nextcloud_url, secret)
role_cfg = model_registry.get_role_config(username, role)
system_prompt = load_context(
tier,
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
)
orch_model = model_registry.get_model_for_role(username, "orchestrator")
user_role_val = get_user_role(username)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(username)
c_allow = set(policy.get("allow", []))
c_deny = set(policy.get("deny", []))
max_risk, risk_wl, risk_bl = get_risk_policy(username)
if orch_model and orch_model.get("type") == "local_openai":
result = await openai_orchestrator.run(
task=user_text,
system_prompt=system_prompt,
session_messages=session_msgs or None,
model_cfg=orch_model,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
else:
gemini_key = (
(orch_model.get("api_key") if orch_model else None)
or get_user_gemini_key(username)
)
result = await orchestrator_engine.run(
task=user_text,
system_prompt=system_prompt,
session_messages=session_msgs or None,
respond_with_claude=True,
gemini_api_key=gemini_key,
model_name=orch_model.get("model_name") if orch_model else None,
response_role=role,
user_role=user_role_val,
tool_list=tool_list,
confirm_allow=c_allow,
confirm_deny=c_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
response_text = result.response
backend = result.backend
if result.checkpoint:
response_text += "\n\n_(This action requires confirmation — use the web UI to approve or deny.)_"
else:
system_prompt = load_context(tier)
history_for_llm = list(session_msgs) + [{"role": "user", "content": user_text}]
response_text, backend = await asyncio.wait_for(
complete(system_prompt=system_prompt, messages=history_for_llm),
timeout=timeout,
)
except asyncio.TimeoutError:
logger.warning("NCT timeout for %s", conversation_token)
await _send_reply(conversation_token, "⏳ Still thinking — this is taking longer than usual.")
await _send_reply(conversation_token, "⏳ Still thinking — this is taking longer than usual.", nextcloud_url, secret)
return
except Exception as e:
logger.error("NCT LLM error for %s: %s", conversation_token, e)
await _send_reply(conversation_token, "⚠️ Something went wrong on my end.")
await _send_reply(conversation_token, "⚠️ Something went wrong on my end.", nextcloud_url, secret)
return
logger.info("NCT LLM responded via %s (%d chars)", backend, len(response_text))
history.append({"role": "user", "content": user_text})
history.append({"role": "assistant", "content": response_text})
save_session(session_id, history)
log_turn(session_id, user_text, response_text)
@@ -114,22 +171,33 @@ async def _process_message(conversation_token: str, user_text: str, actor_name:
"backend": backend,
})
await _send_reply(conversation_token, response_text)
await _send_reply(conversation_token, response_text, nextcloud_url, secret)
@router.post("/inara-nextcloud-talk-webhook")
async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundTasks):
body = await request.body()
@router.post("/webhook/nextcloud/{username}")
async def nextcloud_talk_webhook(username: str, request: Request, background_tasks: BackgroundTasks):
channels = get_user_channels(username)
cfg = channels.get("nextcloud")
if not cfg:
logger.warning("NCT webhook: no channel config for user %r", username)
raise HTTPException(status_code=404, detail="Channel not configured for this user")
if not settings.nextcloud_talk_bot_secret:
logger.error("nextcloud_talk_bot_secret not configured")
persona_name = cfg.get("persona", "inara")
nextcloud_url = cfg.get("url", "")
secret = cfg.get("bot_secret", "")
timeout = cfg.get("timeout", 55)
if not secret:
logger.error("NCT webhook: bot_secret missing for user %r", username)
return Response(status_code=500)
body = await request.body()
random_header = request.headers.get("X-Nextcloud-Talk-Random", "")
sig_header = request.headers.get("X-Nextcloud-Talk-Signature", "")
if not _verify_signature(body, random_header, sig_header):
logger.warning("NCT webhook: signature mismatch")
if not _verify_signature(body, random_header, sig_header, secret):
logger.warning("NCT webhook: signature mismatch for %s", username)
raise HTTPException(status_code=401, detail="Invalid signature")
try:
@@ -158,7 +226,7 @@ async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundT
except (json.JSONDecodeError, AttributeError):
user_text = (obj.get("name") or obj.get("content", "")).strip()
mention_prefix = f"@{settings.agent_name.lower()}"
mention_prefix = f"@{persona_name.lower()}"
if user_text.lower().startswith(mention_prefix):
user_text = user_text[len(mention_prefix):].strip()
@@ -168,5 +236,9 @@ async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundT
actor_name = actor.get("name", "User")
logger.info("NCT message from %s in %s: %r", actor_name, conversation_token, user_text[:60])
background_tasks.add_task(_process_message, conversation_token, user_text, actor_name)
background_tasks.add_task(
_process_message,
conversation_token, user_text, actor_name,
username, persona_name, nextcloud_url, secret, timeout, cfg,
)
return Response(status_code=200)

View File

@@ -1,11 +1,13 @@
"""
Onboarding router — invite-based setup + persona creation.
Onboarding router — invite-based setup + persona creation + model connect.
Routes:
GET /setup/{token} → show password setup form (step 1)
POST /setup/{token} → set password, redirect to persona step
GET /setup/persona → show persona creation form (step 2, requires auth)
POST /setup/persona → create persona, redirect to /{user}/{persona}
POST /setup/persona → create persona, redirect to /setup/model
GET /setup/model → OpenRouter quick-connect (step 3, also standalone)
POST /setup/model → save host + model + assign to chat role, redirect to chat
"""
import logging
@@ -21,6 +23,7 @@ from auth_utils import (
)
from persona_template import create_persona
from persona import list_user_personas, validate as validate_persona
import model_registry
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/setup")
@@ -114,7 +117,11 @@ async def persona_submit(
description=description.strip(),
)
logger.info("persona created: %s/%s", username, persona_name)
return RedirectResponse(f"/{username}/{persona_name}", status_code=302)
# Step 3: guided model setup before entering the chat
resp = RedirectResponse("/setup/model", status_code=302)
# Remember which persona to land on after model setup
resp.set_cookie("cx_setup_persona", f"{username}/{persona_name}", max_age=3600, httponly=True, samesite="lax")
return resp
# ---------------------------------------------------------------------------
@@ -178,3 +185,126 @@ async def setup_submit(
return resp
return HTMLResponse(_setup_page("Unknown step."), status_code=400)
# ---------------------------------------------------------------------------
# Step 3 — model connect (OpenRouter quick-connect, also standalone)
# ---------------------------------------------------------------------------
# Curated model list shown in the Step 3 dropdown.
_OPENROUTER_MODELS = [
("anthropic/claude-3-5-haiku-20241022", "Claude 3.5 Haiku — Fast & affordable"),
("anthropic/claude-3-7-sonnet-20250219", "Claude 3.7 Sonnet — Smarter Claude"),
("google/gemini-2.0-flash-001", "Gemini 2.0 Flash — Fast Google model"),
("meta-llama/llama-3.3-70b-instruct", "Llama 3.3 70B — Open source"),
]
def _model_page(error: str = "", from_setup: bool = False) -> str:
html = (_STATIC / "setup.html").read_text()
# Hide steps 1 and 2 inline; show step 3
html = html.replace('<div id="step-password">', '<div id="step-password" style="display:none">')
html = html.replace('<div id="step-persona" style="display:none">', '<div id="step-persona" style="display:none">')
html = html.replace('<div id="step-model" style="display:none">', '<div id="step-model">')
if from_setup:
html = html.replace("<!-- SETUP_STEP3_LABEL -->", "Step 3 of 3")
if error:
html = html.replace("<!-- ERROR_MODEL -->", f'<p class="error">{error}</p>')
return html
@router.post("/model/skip", include_in_schema=False)
async def model_skip(request: Request):
"""Skip model setup — redirect to the remembered persona or user root."""
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
username = None
if token:
try:
username = decode_token(token)
except jwt.InvalidTokenError:
pass
dest_cookie = request.cookies.get("cx_setup_persona", "")
dest = f"/{dest_cookie}" if dest_cookie else (f"/{username}" if username else "/")
resp = RedirectResponse(dest, status_code=302)
resp.delete_cookie("cx_setup_persona")
return resp
@router.get("/model", include_in_schema=False)
async def model_page(request: Request):
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
if not token:
return RedirectResponse("/login", status_code=302)
try:
decode_token(token)
except jwt.InvalidTokenError:
return RedirectResponse("/login", status_code=302)
from_setup = bool(request.cookies.get("cx_setup_persona"))
return HTMLResponse(_model_page(from_setup=from_setup))
@router.post("/model", include_in_schema=False)
async def model_submit(
request: Request,
api_key: str = Form(...),
model_name: str = Form(...),
):
from auth_utils import decode_token
import jwt
token = request.cookies.get(COOKIE_NAME)
if not token:
return RedirectResponse("/login", status_code=302)
try:
username = decode_token(token)
except jwt.InvalidTokenError:
return RedirectResponse("/login", status_code=302)
api_key = api_key.strip()
model_name = model_name.strip()
if not api_key:
from_setup = bool(request.cookies.get("cx_setup_persona"))
return HTMLResponse(_model_page("API key is required.", from_setup=from_setup), status_code=422)
# Save OpenRouter as a host
host_id = model_registry.save_host(
username=username,
host_id=None,
label="OpenRouter",
api_url="https://openrouter.ai/api/v1",
api_key=api_key,
host_type="openai",
)
# Find label for selected model
label = next((lbl for mn, lbl in _OPENROUTER_MODELS if mn == model_name), model_name)
label = label.split("")[0] # keep just the model name part
# Save model entry
mid = model_registry.save_model(
username=username,
model_id=None,
host_id=host_id,
label=label,
model_name=model_name,
context_k=128,
tools=True,
)
# Assign as chat role primary
model_registry.set_role(username, "chat", "primary", mid)
logger.info("openrouter setup complete: %s%s", username, model_name)
# Redirect to chat (use remembered persona, or user root)
dest_cookie = request.cookies.get("cx_setup_persona", "")
dest = f"/{dest_cookie}" if dest_cookie else f"/{username}"
resp = RedirectResponse(dest, status_code=302)
resp.delete_cookie("cx_setup_persona")
return resp

View File

@@ -12,28 +12,36 @@ Designed to be triggered from:
import asyncio
import logging
import platform
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from auth_utils import get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
from config import settings
from context_loader import load_context
from persona import set_context, validate as validate_persona
import model_registry
import orchestrator_engine
import openai_orchestrator
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/orchestrate", tags=["orchestrator"])
# ---------------------------------------------------------------------------
# In-memory job store
# Jobs are keyed by UUID. For this phase, memory is fine — jobs are short-lived.
# ---------------------------------------------------------------------------
_jobs: dict[str, dict] = {}
_jobs_lock = asyncio.Lock()
# Checkpoints are stored separately — they hold Python objects (types.Content, etc.)
# that can't be included in the JSON-serializable job dict.
_checkpoints: dict[str, orchestrator_engine.OrchestrateCheckpoint] = {}
_checkpoints_lock = asyncio.Lock()
# ---------------------------------------------------------------------------
# Request / response models
@@ -49,11 +57,13 @@ class OrchestrateRequest(BaseModel):
include_short: bool = True
user: str = "scott"
persona: str = "inara"
chat_role: str = "chat" # role used for the final response (decoupled from tool-loop model)
off_record: bool = False # skip session log; inject OTR mode line into system prompt
class OrchestrateResponse(BaseModel):
job_id: str
status: str # "queued" | "running" | "complete" | "error"
status: str # "queued" | "running" | "complete" | "error" | "awaiting_confirmation"
class JobStatusResponse(BaseModel):
@@ -66,8 +76,11 @@ class JobStatusResponse(BaseModel):
response: str | None = None
tool_calls: list[dict] | None = None
backend: str | None = None
backend_label: str | None = None
host: str | None = None
gemini_summary: str | None = None
error: str | None = None
pending_confirmation: dict | None = None # {tools: [{name, args}], message: str}
# ---------------------------------------------------------------------------
@@ -81,7 +94,6 @@ async def orchestrate(req: OrchestrateRequest) -> OrchestrateResponse:
user, persona = validate_persona(req.user, req.persona)
set_context(user, persona)
except ValueError as e:
from fastapi import HTTPException
raise HTTPException(status_code=400, detail=str(e))
job_id = str(uuid.uuid4())
@@ -93,18 +105,21 @@ async def orchestrate(req: OrchestrateRequest) -> OrchestrateResponse:
"task": req.task,
"created_at": now,
"completed_at": None,
"session_id": None,
"response": None,
"tool_calls": None,
"backend": None,
"gemini_summary": None,
"error": None,
"pending_confirmation": None,
"_user": user,
"_off_record": req.off_record,
}
async with _jobs_lock:
_jobs[job_id] = job
# Run in background — caller polls GET /orchestrate/{job_id}
asyncio.create_task(_run_job(job_id, req))
asyncio.create_task(_run_job(job_id, req, user))
logger.info("Orchestrator job queued: %s%.80s", job_id, req.task)
return OrchestrateResponse(job_id=job_id, status="queued")
@@ -116,10 +131,9 @@ async def job_status(job_id: str) -> JobStatusResponse:
job = _jobs.get(job_id)
if job is None:
from fastapi import HTTPException
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
return JobStatusResponse(**job)
return JobStatusResponse(**{k: v for k, v in job.items() if not k.startswith("_")})
@router.get("", response_model=list[JobStatusResponse])
@@ -127,14 +141,58 @@ async def list_jobs() -> list[JobStatusResponse]:
"""List all jobs (most recent first). Useful for debugging."""
async with _jobs_lock:
jobs = sorted(_jobs.values(), key=lambda j: j["created_at"], reverse=True)
return [JobStatusResponse(**j) for j in jobs]
return [JobStatusResponse(**{k: v for k, v in j.items() if not k.startswith("_")}) for j in jobs]
@router.post("/{job_id}/confirm", response_model=OrchestrateResponse)
async def confirm_job(job_id: str) -> OrchestrateResponse:
"""Confirm a pending tool call — the blocked tool will execute and the job continues."""
async with _checkpoints_lock:
checkpoint = _checkpoints.pop(job_id, None)
if checkpoint is None:
raise HTTPException(status_code=404, detail="No pending confirmation for this job")
async with _jobs_lock:
job = _jobs.get(job_id)
if not job or job["status"] != "awaiting_confirmation":
raise HTTPException(status_code=409, detail="Job is not awaiting confirmation")
_jobs[job_id]["status"] = "running"
_jobs[job_id]["pending_confirmation"] = None
user = job.get("_user", "scott")
asyncio.create_task(_resume_job(job_id, checkpoint, confirmed=True, user=user))
logger.info("Orchestrator job %s confirmed — resuming", job_id)
return OrchestrateResponse(job_id=job_id, status="running")
@router.post("/{job_id}/deny", response_model=OrchestrateResponse)
async def deny_job(job_id: str) -> OrchestrateResponse:
"""Deny a pending tool call — the tool is skipped and the job produces a final response."""
async with _checkpoints_lock:
checkpoint = _checkpoints.pop(job_id, None)
if checkpoint is None:
raise HTTPException(status_code=404, detail="No pending confirmation for this job")
async with _jobs_lock:
job = _jobs.get(job_id)
if not job or job["status"] != "awaiting_confirmation":
raise HTTPException(status_code=409, detail="Job is not awaiting confirmation")
_jobs[job_id]["status"] = "running"
_jobs[job_id]["pending_confirmation"] = None
user = job.get("_user", "scott")
asyncio.create_task(_resume_job(job_id, checkpoint, confirmed=False, user=user))
logger.info("Orchestrator job %s denied — resuming with skip", job_id)
return OrchestrateResponse(job_id=job_id, status="running")
# ---------------------------------------------------------------------------
# Background runner
# Background runners
# ---------------------------------------------------------------------------
async def _run_job(job_id: str, req: OrchestrateRequest) -> None:
async def _run_job(job_id: str, req: OrchestrateRequest, user: str) -> None:
"""Execute the orchestration job and update the job store."""
async with _jobs_lock:
_jobs[job_id]["status"] = "running"
@@ -142,47 +200,91 @@ async def _run_job(job_id: str, req: OrchestrateRequest) -> None:
try:
from session_store import load as load_session, save as save_session, generate_session_id
# Load Inara's system prompt (same as the chat router does)
tier = req.tier or settings.default_tier
role_cfg = model_registry.get_role_config(user, req.chat_role)
system_prompt = load_context(
tier,
include_long=req.include_long,
include_mid=req.include_mid,
include_short=req.include_short,
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
inject_mode=role_cfg.get("inject_mode", True),
mode="otr" if req.off_record else "chat",
)
# Load session history if a session_id was provided
session_id = req.session_id or generate_session_id()
history = load_session(session_id)
session_messages = history or None
orch_model = model_registry.get_model_for_role(user, "orchestrator")
user_role = get_user_role(user)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(user)
confirm_allow = set(policy.get("allow", []))
confirm_deny = set(policy.get("deny", []))
max_risk, risk_wl, risk_bl = get_risk_policy(user)
if orch_model and orch_model.get("type") == "local_openai":
result = await openai_orchestrator.run(
task=req.task,
system_prompt=system_prompt,
session_messages=session_messages,
model_cfg=orch_model,
respond_with_final=req.respond_with_claude,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
else:
gemini_key = (
(orch_model.get("api_key") if orch_model else None)
or get_user_gemini_key(user)
)
result = await orchestrator_engine.run(
task=req.task,
system_prompt=system_prompt,
session_messages=session_messages,
respond_with_claude=req.respond_with_claude,
gemini_api_key=gemini_key,
model_name=orch_model.get("model_name") if orch_model else None,
response_role=req.chat_role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
max_rounds=orch_model.get("max_rounds") if orch_model else None,
max_risk=max_risk,
risk_whitelist=risk_wl,
risk_blacklist=risk_bl,
)
# Save the turn to the session store so it survives a page refresh
history.append({"role": "user", "content": req.task})
history.append({"role": "assistant", "content": result.response})
save_session(session_id, history)
from session_logger import log_turn
log_turn(session_id, req.task, result.response)
now = datetime.now(timezone.utc).isoformat()
if result.checkpoint:
async with _checkpoints_lock:
_checkpoints[job_id] = result.checkpoint
async with _jobs_lock:
_jobs[job_id].update({
"status": "complete",
"completed_at": now,
"session_id": session_id,
"status": "awaiting_confirmation",
"response": result.response,
"tool_calls": result.tool_calls,
"backend": result.backend,
"gemini_summary": result.gemini_summary,
"session_id": session_id,
"pending_confirmation": {
"tools": result.checkpoint.pending_tools,
"message": result.response,
},
})
logger.info("Orchestrator job complete: %s (%d tool calls)", job_id, len(result.tool_calls))
logger.info("Orchestrator job %s awaiting confirmation — %d tool(s) blocked",
job_id, len(result.checkpoint.pending_tools))
return
await _finalize_job(job_id, result, session_id, req.task, history, off_record=req.off_record)
except Exception as e:
logger.exception("Orchestrator job failed: %s", job_id)
@@ -193,3 +295,100 @@ async def _run_job(job_id: str, req: OrchestrateRequest) -> None:
"completed_at": now,
"error": str(e),
})
async def _resume_job(
job_id: str,
checkpoint: orchestrator_engine.OrchestrateCheckpoint,
confirmed: bool,
user: str,
) -> None:
"""Resume a job after the user confirms or denies a pending tool call."""
try:
if checkpoint.engine == "gemini":
result = await orchestrator_engine.resume(checkpoint, confirmed)
else:
result = await openai_orchestrator.resume(checkpoint, confirmed)
if result.checkpoint:
# Another confirmation needed (chained gates)
async with _checkpoints_lock:
_checkpoints[job_id] = result.checkpoint
async with _jobs_lock:
_jobs[job_id].update({
"status": "awaiting_confirmation",
"response": result.response,
"tool_calls": result.tool_calls,
"backend": result.backend,
"gemini_summary": result.gemini_summary,
"pending_confirmation": {
"tools": result.checkpoint.pending_tools,
"message": result.response,
},
})
logger.info("Orchestrator job %s awaiting another confirmation", job_id)
return
async with _jobs_lock:
session_id = _jobs[job_id].get("session_id") or ""
task = _jobs[job_id].get("task", "")
off_record = _jobs[job_id].get("_off_record", False)
from session_store import load as load_session
history = load_session(session_id) if session_id else []
await _finalize_job(job_id, result, session_id, task, history, off_record=off_record)
except Exception as e:
logger.exception("Orchestrator resume failed: %s", job_id)
now = datetime.now(timezone.utc).isoformat()
async with _jobs_lock:
_jobs[job_id].update({
"status": "error",
"completed_at": now,
"error": str(e),
})
async def _finalize_job(
job_id: str,
result: orchestrator_engine.OrchestratorResult,
session_id: str,
task: str,
history: list,
off_record: bool = False,
) -> None:
"""Save session, log the turn, and mark the job complete."""
from session_store import save as save_session, generate_session_id
from session_logger import log_turn
if not session_id:
session_id = generate_session_id()
host = platform.node()
history.append({"role": "user", "content": task, "off_record": off_record})
history.append({
"role": "assistant",
"content": result.response,
"backend": result.backend,
"backend_label": result.backend_label,
"host": host,
"off_record": off_record,
})
save_session(session_id, history)
if not off_record:
log_turn(session_id, task, result.response)
now = datetime.now(timezone.utc).isoformat()
async with _jobs_lock:
_jobs[job_id].update({
"status": "complete",
"completed_at": now,
"session_id": session_id,
"response": result.response,
"tool_calls": result.tool_calls,
"backend": result.backend,
"backend_label": result.backend_label,
"host": host,
"gemini_summary": result.gemini_summary,
})
logger.info("Orchestrator job complete: %s (%d tool calls)", job_id, len(result.tool_calls))

120
cortex/routers/push.py Normal file
View File

@@ -0,0 +1,120 @@
"""
Web Push endpoints.
GET /api/push/vapid-key → public VAPID key for browser PushManager.subscribe()
POST /api/push/subscribe → save a push subscription for the logged-in user
DELETE /api/push/subscribe → remove a subscription by endpoint
"""
import jwt
from fastapi import APIRouter, HTTPException, Request
from pydantic import BaseModel
from auth_utils import COOKIE_NAME, decode_token
from config import settings
import push_utils
router = APIRouter(prefix="/api/push")
def _require_user(request: Request) -> str:
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
return decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
@router.get("/vapid-key")
async def get_vapid_key() -> dict:
"""Return the VAPID public key. Public endpoint — needed before login to subscribe."""
key = settings.vapid_public_key
if not key:
raise HTTPException(status_code=503, detail="Push notifications not configured")
return {"public_key": key}
class SubscribeRequest(BaseModel):
subscription: dict # full PushSubscription JSON from browser
class UnsubscribeRequest(BaseModel):
endpoint: str
@router.post("/subscribe")
async def subscribe(req: SubscribeRequest, request: Request) -> dict:
username = _require_user(request)
sub = req.subscription
if not sub.get("endpoint"):
raise HTTPException(status_code=400, detail="subscription.endpoint is required")
push_utils.add_subscription(username, sub)
return {"ok": True}
@router.delete("/subscribe")
async def unsubscribe(req: UnsubscribeRequest, request: Request) -> dict:
username = _require_user(request)
found = push_utils.remove_subscription(username, req.endpoint)
return {"ok": True, "found": found}
@router.post("/test")
async def notify_test(request: Request) -> dict:
"""Send a test notification via the user's configured notification channel.
Useful for verifying channel setup (web push, NCT, email, etc.) without
waiting for a cron job or reminder to fire naturally.
"""
username = _require_user(request)
from notification import notify
await notify(username, "Test notification from Cortex — your notification channel is working.")
return {"ok": True, "user": username}
@router.post("/reminders/check")
async def reminder_check_now(request: Request) -> dict:
"""Run the reminder check for the current user immediately.
Same logic as the daily 09:00 scheduler job, but scoped to one user
and fired on demand. Returns how many reminders were found and whether
a notification was sent.
"""
import re
username = _require_user(request)
from persona import list_user_personas, set_context
from notification import notify
total_sent = 0
for persona_name in list_user_personas(username):
set_context(username, persona_name)
from tools.reminders import load_due_reminders
content = load_due_reminders()
if not content:
continue
entries = []
for line in content.splitlines():
m = re.match(r"^\d+\.\s+(.+)", line.strip())
if m:
text = re.sub(r"\[(OVERDUE|due TODAY|due: \S+)\]", "", m.group(1)).strip()
if text:
entries.append(text)
if not entries:
continue
count = len(entries)
if count == 1:
msg = f"Reminder: {entries[0]}"
else:
bullet_list = "\n".join(f"{e}" for e in entries[:3])
tail = f"\n…and {count - 3} more" if count > 3 else ""
msg = f"{count} reminders due:\n{bullet_list}{tail}"
await notify(username, msg)
total_sent += count
return {"ok": True, "user": username, "reminders_found": total_sent}

499
cortex/routers/settings.py Normal file
View File

@@ -0,0 +1,499 @@
"""
Account settings router.
Routes:
GET /settings → show account settings page (requires auth)
POST /settings/password → change password
POST /settings/username → rename the user account (forces re-login)
POST /settings/persona/rename → rename a persona directory
"""
import html as _html
import json
import logging
import re
from pathlib import Path
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, check_credentials, set_password, _read_auth, _write_auth, get_user_channels
from persona import list_user_personas
from config import settings as app_settings
_SLUG_RE = re.compile(r"^[a-z_][a-z0-9_-]{0,31}$")
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
def _integrations_nav(username: str) -> str:
"""Return the Integrations nav link for admin users, empty string otherwise."""
role = _read_auth(username).get("role", "user")
if role == "admin":
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
return ""
def _notifications_page(username: str, back_persona: str = "", success: str = "", error: str = "") -> str:
html = (_STATIC / "notifications.html").read_text()
channels = get_user_channels(username)
nct = channels.get("nextcloud") or {}
notify_ch = _html.escape(channels.get("notification_channel", "") or "")
notify_email = _html.escape(channels.get("notification_email", "") or "")
nc_url = _html.escape(nct.get("url", "") or "")
nc_bot_secret = _html.escape(nct.get("bot_secret", "") or "")
nc_room = _html.escape(nct.get("notification_room", "") or "")
nc_username = _html.escape(nct.get("nc_username", "") or "")
nc_app_password = _html.escape(nct.get("nc_app_password", "") or "")
gc_webhook = _html.escape((channels.get("google_chat") or {}).get("outbound_webhook", "") or "")
ha = channels.get("homeassistant") or {}
ha_url = _html.escape(ha.get("url", "") or "")
ha_webhook_id = _html.escape(ha.get("webhook_id", "") or "")
ha_tools_checked = "checked" if ha.get("tools", False) else ""
html = html.replace("{{ notify_channel }}", notify_ch)
html = html.replace("{{ notify_email_override }}", notify_email)
html = html.replace("{{ nc_url }}", nc_url)
html = html.replace("{{ nc_bot_secret }}", nc_bot_secret)
html = html.replace("{{ nc_notify_room }}", nc_room)
html = html.replace("{{ nc_username }}", nc_username)
html = html.replace("{{ nc_app_password }}", nc_app_password)
html = html.replace("{{ gc_webhook }}", gc_webhook)
html = html.replace("{{ ha_url }}", ha_url)
html = html.replace("{{ ha_webhook_id }}", ha_webhook_id)
html = html.replace("{{ ha_tools_checked }}", ha_tools_checked)
html = html.replace("{{ ha_username }}", username)
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
def _settings_page(username: str, personas: list[str], back_persona: str = "", success: str = "", error: str = "") -> str:
html = (_STATIC / "settings.html").read_text()
html = html.replace("{{ username }}", username)
# Connected Google account (OAuth sign-in)
auth_data = _read_auth(username)
google_email = auth_data.get("google_email") or ""
html = html.replace("{{ google_email }}", google_email)
role = auth_data.get("role", "user")
html = html.replace("{{ user_role }}", role)
al_path = app_settings.home_root() / username / "email_allowlist.json"
try:
patterns = json.loads(al_path.read_text())
allowlist_text = _html.escape("\n".join(str(p) for p in patterns if str(p).strip()))
except Exception:
allowlist_text = ""
html = html.replace("{{ email_allowlist }}", allowlist_text)
http_al_path = app_settings.home_root() / username / "http_allowlist.json"
try:
http_prefixes = json.loads(http_al_path.read_text())
http_allowlist_text = _html.escape("\n".join(str(p) for p in http_prefixes if str(p).strip()))
except Exception:
http_allowlist_text = ""
html = html.replace("{{ http_allowlist }}", http_allowlist_text)
persona_items = "\n".join(
f'''<li>
<a href="/{username}/{p}" class="persona-link">{p}</a>
<button class="persona-rename-toggle" data-persona="{p}" title="Rename">✏</button>
<form class="persona-rename-form" data-persona="{p}"
method="POST" action="/settings/persona/rename" style="display:none">
<input type="hidden" name="old_name" value="{p}">
<input type="text" name="new_name" value="{p}"
pattern="[a-z_][a-z0-9_\\-]{{0,31}}" required>
<button type="submit" class="btn-save">Save</button>
<button type="button" class="btn-cancel persona-rename-cancel">Cancel</button>
</form>
</li>''' for p in personas
)
html = html.replace("{{ persona_items }}", persona_items or "<li><em>No personas yet.</em></li>")
if not back_persona:
back_persona = personas[0] if personas else ""
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
def _integrations_page(username: str, back_persona: str = "", success: str = "", error: str = "") -> str:
html = (_STATIC / "integrations.html").read_text()
channels = get_user_channels(username)
ae_db = channels.get("aether_db") or {}
html = html.replace("{{ ae_db_host }}", _html.escape(ae_db.get("host", "") or ""))
html = html.replace("{{ ae_db_port }}", _html.escape(str(ae_db.get("port", 3306))))
html = html.replace("{{ ae_db_name }}", _html.escape(ae_db.get("name", "") or ""))
html = html.replace("{{ ae_db_user }}", _html.escape(ae_db.get("user", "") or ""))
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
@router.get("/settings", include_in_schema=False)
async def settings_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_settings_page(username, personas, back_persona=back_persona))
@router.post("/settings/password", include_in_schema=False)
async def change_password(
request: Request,
current_password: str = Form(...),
new_password: str = Form(...),
confirm_password: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
if not check_credentials(username, current_password):
return HTMLResponse(_settings_page(username, personas, back_persona, error="Current password is incorrect."))
if len(new_password) < 8:
return HTMLResponse(_settings_page(username, personas, back_persona, error="New password must be at least 8 characters."))
if new_password != confirm_password:
return HTMLResponse(_settings_page(username, personas, back_persona, error="New passwords do not match."))
set_password(username, new_password)
logger.info("password changed: %s", username)
return HTMLResponse(_settings_page(username, personas, back_persona, success="Password updated successfully."))
@router.post("/settings/username", include_in_schema=False)
async def rename_username(
request: Request,
new_username: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
new_username = new_username.strip().lower()
if not _SLUG_RE.match(new_username):
return HTMLResponse(_settings_page(
username, personas, back_persona,
error="Invalid username. Use lowercase letters, digits, _ or - only."))
if new_username == username:
return RedirectResponse("/settings", status_code=302)
home_root = app_settings.home_root()
old_dir = home_root / username
new_dir = home_root / new_username
if new_dir.exists():
return HTMLResponse(_settings_page(
username, personas, back_persona,
error=f"Username '{new_username}' is already taken."))
old_dir.rename(new_dir)
logger.info("user renamed: %s%s", username, new_username)
# Clear the auth cookie — old JWT now refers to a non-existent user
resp = RedirectResponse("/login?msg=username_changed", status_code=302)
resp.delete_cookie(COOKIE_NAME)
return resp
@router.post("/settings/gemini-key", include_in_schema=False)
async def save_gemini_key(
request: Request,
gemini_api_key: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
gemini_api_key = gemini_api_key.strip()
data = _read_auth(username)
if gemini_api_key:
data["gemini_api_key"] = gemini_api_key
msg = "Gemini API key saved."
else:
data.pop("gemini_api_key", None)
msg = "Gemini API key removed — using server key."
_write_auth(username, data)
logger.info("gemini key updated: %s", username)
return HTMLResponse(_settings_page(username, personas, back_persona, success=msg))
@router.post("/settings/persona/rename", include_in_schema=False)
async def rename_persona(
request: Request,
old_name: str = Form(...),
new_name: str = Form(...),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
new_name = new_name.strip().lower()
if not _SLUG_RE.match(new_name):
return HTMLResponse(_settings_page(
username, personas, back_persona,
error="Invalid name. Use lowercase letters, digits, _ or - only."))
if new_name == old_name:
return RedirectResponse("/settings", status_code=302)
persona_root = app_settings.home_root() / username / "persona"
old_dir = persona_root / old_name
new_dir = persona_root / new_name
if not old_dir.exists():
return HTMLResponse(_settings_page(username, personas, back_persona, error=f"Persona '{old_name}' not found."))
if new_dir.exists():
return HTMLResponse(_settings_page(
username, personas, back_persona,
error=f"A persona named '{new_name}' already exists."))
old_dir.rename(new_dir)
logger.info("persona renamed: %s/%s%s", username, old_name, new_name)
return RedirectResponse("/settings", status_code=302)
@router.get("/settings/notifications", include_in_schema=False)
async def notifications_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_notifications_page(username, back_persona))
@router.post("/settings/notifications", include_in_schema=False)
async def save_notifications(
request: Request,
notification_channel: str = Form(""),
notification_email: str = Form(""),
nc_url: str = Form(""),
nc_bot_secret: str = Form(""),
nc_notification_room: str = Form(""),
nc_username: str = Form(""),
nc_app_password: str = Form(""),
gc_outbound_webhook: str = Form(""),
ha_url: str = Form(""),
ha_token: str = Form(""),
ha_webhook_id: str = Form(""),
ha_tools: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
channels_path = app_settings.home_root() / username / "channels.json"
try:
channels = json.loads(channels_path.read_text())
except Exception:
channels = {}
# Top-level notification preference
notification_channel = notification_channel.strip()
if notification_channel in ("web_push", "email", "nextcloud", "google_chat"):
channels["notification_channel"] = notification_channel
else:
channels.pop("notification_channel", None)
# Optional email address override (blank = use login email)
notification_email = notification_email.strip()
if notification_email:
channels["notification_email"] = notification_email
else:
channels.pop("notification_email", None)
# Nextcloud Talk — full config nested under "nextcloud"
if "nextcloud" not in channels:
channels["nextcloud"] = {}
nct = channels["nextcloud"]
if nc_url.strip():
nct["url"] = nc_url.strip().rstrip("/")
# Only overwrite secrets if a new value was provided (blank = keep existing)
if nc_bot_secret.strip():
nct["bot_secret"] = nc_bot_secret.strip()
nct["notification_room"] = nc_notification_room.strip()
if nc_username.strip():
nct["nc_username"] = nc_username.strip()
if nc_app_password.strip():
nct["nc_app_password"] = nc_app_password.strip()
# Google Chat outbound webhook — nested under "google_chat"
if "google_chat" not in channels:
channels["google_chat"] = {}
channels["google_chat"]["outbound_webhook"] = gc_outbound_webhook.strip()
# Home Assistant — nested under "homeassistant"
if "homeassistant" not in channels:
channels["homeassistant"] = {}
ha = channels["homeassistant"]
if ha_url.strip():
ha["url"] = ha_url.strip().rstrip("/")
if ha_token.strip():
ha["token"] = ha_token.strip()
if ha_webhook_id.strip():
ha["webhook_id"] = ha_webhook_id.strip()
ha["tools"] = ha_tools == "1"
channels_path.write_text(json.dumps(channels, indent=2) + "\n")
logger.info("notifications updated for %s (channel=%s)", username, notification_channel or "none")
return HTMLResponse(_notifications_page(username, back_persona, success="Notification settings saved."))
@router.post("/settings/email-allowlist", include_in_schema=False)
async def save_email_allowlist(
request: Request,
patterns: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
lines = [ln.strip() for ln in patterns.splitlines() if ln.strip()]
path = app_settings.home_root() / username / "email_allowlist.json"
path.write_text(json.dumps(lines, indent=2))
logger.info("email allowlist updated for %s (%d patterns)", username, len(lines))
return HTMLResponse(_settings_page(username, personas, back_persona, success=f"Email allowlist saved ({len(lines)} pattern{'s' if len(lines) != 1 else ''})."))
@router.post("/settings/http-allowlist", include_in_schema=False)
async def save_http_allowlist(
request: Request,
prefixes: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
personas = list_user_personas(username)
back_persona = _preferred_persona(request, username)
lines = [ln.strip() for ln in prefixes.splitlines() if ln.strip()]
path = app_settings.home_root() / username / "http_allowlist.json"
path.write_text(json.dumps(lines, indent=2))
logger.info("http allowlist updated for %s (%d prefixes)", username, len(lines))
return HTMLResponse(_settings_page(username, personas, back_persona, success=f"HTTP allowlist saved ({len(lines)} prefix{'es' if len(lines) != 1 else ''})."))
def _require_admin(username: str) -> bool:
return _read_auth(username).get("role", "user") == "admin"
@router.get("/settings/integrations", include_in_schema=False)
async def integrations_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not _require_admin(username):
return RedirectResponse("/settings", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_integrations_page(username, back_persona))
@router.post("/settings/integrations", include_in_schema=False)
async def save_integrations(
request: Request,
ae_db_host: str = Form(""),
ae_db_port: str = Form("3306"),
ae_db_name: str = Form(""),
ae_db_user: str = Form(""),
ae_db_password: str = Form(""),
):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
if not _require_admin(username):
return RedirectResponse("/settings", status_code=302)
back_persona = _preferred_persona(request, username)
channels_path = app_settings.home_root() / username / "channels.json"
try:
channels = json.loads(channels_path.read_text())
except Exception:
channels = {}
if "aether_db" not in channels:
channels["aether_db"] = {}
db = channels["aether_db"]
if ae_db_host.strip():
db["host"] = ae_db_host.strip()
try:
db["port"] = int(ae_db_port.strip()) if ae_db_port.strip() else 3306
except ValueError:
db["port"] = 3306
if ae_db_name.strip():
db["name"] = ae_db_name.strip()
if ae_db_user.strip():
db["user"] = ae_db_user.strip()
if ae_db_password.strip():
db["password"] = ae_db_password.strip()
channels_path.write_text(json.dumps(channels, indent=2) + "\n")
logger.info("integrations updated for %s", username)
return HTMLResponse(_integrations_page(username, back_persona, success="Integration settings saved."))

View File

@@ -0,0 +1,193 @@
"""
Tool settings router.
Routes:
GET /settings/tools → tool risk policy page
POST /settings/tools → save max_risk + per-tool overrides
"""
import html as _html
import json
import logging
from pathlib import Path
import jwt
from fastapi import APIRouter, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from auth_utils import COOKIE_NAME, decode_token, get_tool_policy, save_tool_policy, _read_auth
from persona import list_user_personas
from tools import TOOL_CATEGORIES, TOOL_RISK, CONFIRM_REQUIRED
logger = logging.getLogger(__name__)
router = APIRouter()
_STATIC = Path(__file__).parent.parent / "static"
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _get_session_user(request: Request) -> str | None:
token = request.cookies.get(COOKIE_NAME)
if not token:
return None
try:
return decode_token(token)
except jwt.InvalidTokenError:
return None
def _preferred_persona(request: Request, username: str) -> str:
names = list_user_personas(username)
if not names:
return ""
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
return cookie_val if cookie_val in names else (names[0] if names else "")
def _build_tool_table(policy: dict) -> str:
"""Generate the per-tool override table rows grouped by category."""
whitelist = set(policy.get("whitelist") or [])
blacklist = set(policy.get("blacklist") or [])
rows: list[str] = []
for category, tools in TOOL_CATEGORIES.items():
# Category header spanning all columns
escaped_cat = _html.escape(category)
rows.append(f'<tr class="tool-cat-row"><td colspan="4">{escaped_cat}</td></tr>')
for tool in tools:
risk = TOOL_RISK.get(tool, "medium")
risk_cls = f"risk-{risk}"
risk_html = f'<span class="risk {risk_cls}">{_html.escape(risk)}</span>'
# Override select value
if tool in whitelist:
override_val = "whitelist"
elif tool in blacklist:
override_val = "blacklist"
else:
override_val = "default"
def _opt(val: str, label: str) -> str:
sel = 'selected' if override_val == val else ''
return f'<option value="{val}" {sel}>{label}</option>'
override_sel = (
f'<select name="override_{_html.escape(tool)}" '
f'class="override-sel" data-tool="{_html.escape(tool)}">'
+ _opt("default", "Default (auto)")
+ _opt("whitelist", "Force include")
+ _opt("blacklist", "Force exclude")
+ '</select>'
)
rows.append(
f'<tr data-tool-risk="{_html.escape(risk)}">'
f'<td class="tool-name">{_html.escape(tool)}</td>'
f'<td>{risk_html}</td>'
f'<td><span class="auto-pill"></span></td>'
f'<td>{override_sel}</td>'
f'</tr>'
)
table_body = "\n".join(rows)
return (
'<table class="tool-table">'
'<thead><tr>'
'<th>Tool</th><th>Risk</th><th>Auto status</th><th>Override</th>'
'</tr></thead>'
f'<tbody>{table_body}</tbody>'
'</table>'
)
def _tools_page(
username: str,
back_persona: str = "",
success: str = "",
error: str = "",
) -> str:
html = (_STATIC / "tools_settings.html").read_text()
policy = get_tool_policy(username)
max_risk = policy.get("max_risk") or ""
# Max risk select options
html = html.replace("{{ sel_none }}", "selected" if max_risk == "" else "")
html = html.replace("{{ sel_low }}", "selected" if max_risk == "low" else "")
html = html.replace("{{ sel_medium }}", "selected" if max_risk == "medium" else "")
html = html.replace("{{ sel_high }}", "selected" if max_risk == "high" else "")
html = html.replace("{{ tool_table_html }}", _build_tool_table(policy))
html = html.replace("{{ tool_risk_json }}", json.dumps(TOOL_RISK))
html = html.replace("{{ confirm_required_tools }}", _html.escape(", ".join(sorted(CONFIRM_REQUIRED))))
html = html.replace("{{ tool_allow }}", _html.escape("\n".join(policy.get("allow") or [])))
html = html.replace("{{ tool_deny }}", _html.escape("\n".join(policy.get("deny") or [])))
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
nav = '<a href="/settings/integrations" class="nav-link">Integrations</a>' \
if _read_auth(username).get("role", "user") == "admin" else ""
html = html.replace("{{ integrations_nav }}", nav)
if success:
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
if error:
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
return html
@router.get("/settings/tools", include_in_schema=False)
async def tools_page(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
return HTMLResponse(_tools_page(username, back_persona))
@router.post("/settings/tools", include_in_schema=False)
async def save_tools(request: Request):
username = _get_session_user(request)
if not username:
return RedirectResponse("/login", status_code=302)
back_persona = _preferred_persona(request, username)
form = await request.form()
max_risk = (form.get("max_risk") or "").strip()
if max_risk not in ("", "low", "medium", "high"):
max_risk = ""
whitelist: list[str] = []
blacklist: list[str] = []
all_tools = [t for tools in TOOL_CATEGORIES.values() for t in tools]
for tool in all_tools:
val = (form.get(f"override_{tool}") or "").strip()
if val == "whitelist":
whitelist.append(tool)
elif val == "blacklist":
blacklist.append(tool)
allow_tools = [ln.strip() for ln in (form.get("allow_list") or "").splitlines() if ln.strip()]
deny_tools = [ln.strip() for ln in (form.get("deny_list") or "").splitlines() if ln.strip()]
policy = get_tool_policy(username)
if max_risk:
policy["max_risk"] = max_risk
else:
policy.pop("max_risk", None)
policy["whitelist"] = whitelist
policy["blacklist"] = blacklist
policy["allow"] = allow_tools
policy["deny"] = deny_tools
save_tool_policy(username, policy)
logger.info(
"tool policy saved for %s: max_risk=%s whitelist=%d blacklist=%d allow=%d deny=%d",
username, max_risk or "none", len(whitelist), len(blacklist), len(allow_tools), len(deny_tools),
)
return HTMLResponse(_tools_page(
username, back_persona,
success=f"Tool policy saved — max risk: {max_risk or 'none'}, "
f"{len(whitelist)} whitelisted, {len(blacklist)} blacklisted.",
))

View File

@@ -56,12 +56,52 @@ def _set_cookie(response: Response, username: str) -> None:
)
_LAST_PERSONA_COOKIE = "cx_last_persona"
def _first_persona(username: str) -> str | None:
"""Return the first available persona for a user, or None."""
names = list_user_personas(username)
return names[0] if names else None
def _preferred_persona(request: Request, username: str) -> str | None:
"""Return the last-visited persona from cookie if valid, else the first available."""
names = list_user_personas(username)
if not names:
return None
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
if cookie_val in names:
return cookie_val
return names[0]
# ---------------------------------------------------------------------------
# Favicon — default sparkle; persona pages override via JS
# ---------------------------------------------------------------------------
_FAVICON_SVG = (
"<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'>"
"<text y='.9em' font-size='90'>✨</text></svg>"
)
@router.get("/favicon.ico", include_in_schema=False)
async def favicon():
return Response(content=_FAVICON_SVG, media_type="image/svg+xml")
@router.get("/sw.js", include_in_schema=False)
async def service_worker():
from fastapi.responses import FileResponse
return FileResponse(str(_STATIC / "sw.js"), media_type="application/javascript")
@router.get("/manifest.json", include_in_schema=False)
async def web_manifest():
from fastapi.responses import FileResponse
return FileResponse(str(_STATIC / "manifest.json"), media_type="application/manifest+json")
# ---------------------------------------------------------------------------
# Root redirect
# ---------------------------------------------------------------------------
@@ -71,7 +111,7 @@ async def root(request: Request):
user = _get_session_user(request)
if not user:
return RedirectResponse("/login", status_code=302)
persona = _first_persona(user)
persona = _preferred_persona(request, user)
if not persona:
return HTMLResponse("<h1>No personas configured for your account.</h1>", status_code=500)
return RedirectResponse(f"/{user}/{persona}", status_code=302)
@@ -86,7 +126,7 @@ async def login_page(request: Request):
user = _get_session_user(request)
if user:
# Already logged in — redirect home
persona = _first_persona(user)
persona = _preferred_persona(request, user)
if persona:
return RedirectResponse(f"/{user}/{persona}", status_code=302)
return HTMLResponse((_STATIC / "login.html").read_text())
@@ -123,6 +163,112 @@ async def logout():
return resp
# ---------------------------------------------------------------------------
# User landing — /{username} → persona picker
# ---------------------------------------------------------------------------
@router.get("/{username}", include_in_schema=False)
async def user_landing(username: str, request: Request):
session_user = _get_session_user(request)
if not session_user:
return RedirectResponse("/login", status_code=302)
if session_user != username:
return RedirectResponse(f"/{session_user}", status_code=302)
personas = list_user_personas(username)
if not personas:
return HTMLResponse("<h1>No personas configured.</h1>", status_code=404)
cards_html = ""
for p in personas:
emoji = ""
identity_path = persona_path(username, p) / "IDENTITY.md"
if identity_path.exists():
m = re.search(r"\|\s*Emoji\s*\|\s*(.+?)\s*\|", identity_path.read_text())
if m:
emoji = m.group(1).strip()
cards_html += (
f'<a href="/{username}/{p}" class="persona-card">'
f'<span class="p-emoji">{emoji}</span>'
f'<span class="p-name">{p.capitalize()}</span>'
f'</a>\n'
)
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — {username}</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after {{ box-sizing: border-box; margin: 0; padding: 0; }}
body {{
min-height: 100vh;
display: flex;
align-items: center;
justify-content: center;
background: #1a1228;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
color: #e8e0f0;
padding: 2rem 1.5rem;
}}
.card {{
background: #221840;
border: 1px solid #3a2852;
border-radius: 14px;
padding: 2.5rem 2rem;
width: 100%;
max-width: 400px;
text-align: center;
}}
h1 {{ font-size: 1.3rem; font-weight: 700; color: #c4935a; margin-bottom: 0.4rem; }}
.sub {{ font-size: 0.82rem; color: #b0a2c8; margin-bottom: 2rem; }}
.personas {{ display: flex; flex-direction: column; gap: 0.75rem; }}
.persona-card {{
display: flex;
align-items: center;
gap: 1rem;
padding: 0.85rem 1.2rem;
background: #1a1228;
border: 1px solid #3a2852;
border-radius: 10px;
color: #e8e0f0;
text-decoration: none;
font-size: 1rem;
font-weight: 500;
transition: border-color 0.15s, background 0.15s;
}}
.persona-card:hover {{ border-color: #c4935a; background: #261d42; }}
.p-emoji {{ font-size: 1.6rem; line-height: 1; }}
.p-name {{ color: #c4935a; font-weight: 600; }}
.settings-link {{
display: inline-block;
margin-top: 1.5rem;
font-size: 0.78rem;
color: #b0a2c8;
text-decoration: none;
}}
.settings-link:hover {{ color: #e8e0f0; }}
</style>
</head>
<body>
<div class="card">
<h1>Cortex</h1>
<p class="sub">Signed in as <strong>{username}</strong> — choose a persona</p>
<div class="personas">
{cards_html} </div>
<a href="/settings" class="settings-link">Account settings</a>
</div>
</body>
</html>"""
return HTMLResponse(html)
# ---------------------------------------------------------------------------
# Main UI — /{username}/{persona}
# ---------------------------------------------------------------------------
@@ -134,7 +280,16 @@ async def api_personas(request: Request) -> dict:
if not user:
from fastapi import HTTPException
raise HTTPException(status_code=401, detail="Not authenticated")
return {"user": user, "personas": list_user_personas(user)}
personas_with_emoji = []
for p in list_user_personas(user):
emoji = ""
identity_path = persona_path(user, p) / "IDENTITY.md"
if identity_path.exists():
m = re.search(r"\|\s*Emoji\s*\|\s*(.+?)\s*\|", identity_path.read_text())
if m:
emoji = m.group(1).strip()
personas_with_emoji.append({"name": p, "emoji": emoji})
return {"user": user, "personas": personas_with_emoji}
@router.get("/{username}/{persona}", include_in_schema=False)
@@ -168,4 +323,6 @@ async def serve_ui(username: str, persona: str, request: Request):
f'{{user: "{username}", persona: "{persona}", emoji: "{emoji}"}};</script>'
)
html = html.replace("</head>", f"{config_tag}\n</head>", 1)
return HTMLResponse(html)
resp = HTMLResponse(html)
resp.set_cookie(_LAST_PERSONA_COOKIE, persona, max_age=365 * 86400, httponly=False, samesite="lax")
return resp

104
cortex/routers/usage.py Normal file
View File

@@ -0,0 +1,104 @@
"""
Usage / token-tracking endpoints.
Self-service (any authenticated user, own data):
GET /api/usage → full usage dict {date: {model_key: {calls, prompt_tokens, completion_tokens}}}
GET /api/usage/summary → aggregate totals per model key, with friendly labels resolved from registry
Admin-only (cross-user aggregation):
GET /api/usage/all → summary for every user {username: summary_dict}
"""
import jwt
from fastapi import APIRouter, HTTPException, Request
from auth_utils import COOKIE_NAME, decode_token, get_user_role
from persona import list_users
import model_registry
import usage_tracker
router = APIRouter(prefix="/api/usage")
def _session_user(request: Request) -> str:
token = request.cookies.get(COOKIE_NAME)
if not token:
raise HTTPException(status_code=401, detail="Not authenticated")
try:
return decode_token(token)
except jwt.InvalidTokenError:
raise HTTPException(status_code=401, detail="Invalid session")
def _build_label_map(username: str) -> dict[str, str]:
"""Build a map from usage key (backend/model_name) → registered label."""
label_map: dict[str, str] = {}
try:
for m in model_registry.get_all_models(username):
model_name = m.get("model_name", "")
label = m.get("label", "")
host_type = m.get("host_type", "")
if not model_name or not label:
continue
# local models: key is "local/{model_name}"
if host_type in ("openwebui", "ollama", "openai_compatible"):
label_map[f"local/{model_name}"] = label
# cloud Gemini: key is "gemini_api/{model_name}"
elif host_type == "google":
label_map[f"gemini_api/{model_name}"] = label
except Exception:
pass
return label_map
def _summarize(data: dict, label_map: dict[str, str] | None = None) -> list[dict]:
"""Collapse date-keyed usage dict into per-model totals, sorted by total tokens desc."""
totals: dict[str, dict] = {}
for _date, models in data.items():
for key, counts in models.items():
t = totals.setdefault(key, {"calls": 0, "prompt_tokens": 0, "completion_tokens": 0})
t["calls"] += counts.get("calls", 0)
t["prompt_tokens"] += counts.get("prompt_tokens", 0)
t["completion_tokens"] += counts.get("completion_tokens", 0)
result = []
for key, counts in totals.items():
entry = {
"key": key,
"label": (label_map or {}).get(key) or key,
"calls": counts["calls"],
"prompt_tokens": counts["prompt_tokens"],
"completion_tokens": counts["completion_tokens"],
"total_tokens": counts["prompt_tokens"] + counts["completion_tokens"],
}
result.append(entry)
result.sort(key=lambda x: x["total_tokens"], reverse=True)
return result
@router.get("")
async def get_usage(request: Request) -> dict:
"""Return the raw daily usage log for the authenticated user."""
username = _session_user(request)
return usage_tracker.read_usage(username)
@router.get("/summary")
async def get_usage_summary(request: Request) -> list:
"""Return per-model totals (all time) for the authenticated user, with friendly labels."""
username = _session_user(request)
label_map = _build_label_map(username)
return _summarize(usage_tracker.read_usage(username), label_map)
@router.get("/all")
async def get_all_usage(request: Request) -> dict:
"""Admin: return per-model summary for every user."""
username = _session_user(request)
if get_user_role(username) != "admin":
raise HTTPException(status_code=403, detail="Admin access required")
result = {}
for user in list_users():
label_map = _build_label_map(user)
result[user] = _summarize(usage_tracker.read_usage(user), label_map)
return result

View File

@@ -19,37 +19,95 @@ logger = logging.getLogger(__name__)
_scheduler: AsyncIOScheduler | None = None
def _all_personas() -> list[tuple[str, str]]:
"""Return [(username, persona_name)] for every persona on this instance."""
from persona import list_users, list_user_personas
pairs = []
for u in list_users():
for p in list_user_personas(u):
pairs.append((u, p))
return pairs
async def _run_short() -> None:
from memory_distiller import distill_short
for u, p in _all_personas():
try:
result = distill_short()
logger.info("auto distill short: %d files, %d chars", result["files_included"], result["chars_written"])
result = distill_short(u, p)
logger.info("auto distill short [%s/%s]: %d files, %d chars", u, p, result["files_included"], result["chars_written"])
except Exception as e:
logger.error("auto distill short failed: %s", e)
logger.error("auto distill short [%s/%s] failed: %s", u, p, e)
async def _run_mid() -> None:
from memory_distiller import distill_mid
from notification import notify
for u, p in _all_personas():
try:
result = await distill_mid()
result = await distill_mid(u, p)
if "error" in result:
logger.warning("auto distill mid skipped: %s", result["error"])
logger.warning("auto distill mid [%s/%s] skipped: %s", u, p, result["error"])
else:
logger.info("auto distill mid: %d chars via %s", result["chars_written"], result["backend"])
logger.info("auto distill mid [%s/%s]: %d chars via %s", u, p, result["chars_written"], result["backend"])
await notify(u, f"📝 Weekly memory digest complete ({result['chars_written']} chars via {result['backend']}).")
except Exception as e:
logger.error("auto distill mid failed: %s", e)
logger.error("auto distill mid [%s/%s] failed: %s", u, p, e)
async def _run_long() -> None:
from memory_distiller import distill_long
from notification import notify
for u, p in _all_personas():
try:
result = await distill_long()
result = await distill_long(u, p)
if "error" in result:
logger.warning("auto distill long skipped: %s", result["error"])
logger.warning("auto distill long [%s/%s] skipped: %s", u, p, result["error"])
else:
logger.info("auto distill long: %d chars via %s", result["chars_written"], result["backend"])
logger.info("auto distill long [%s/%s]: %d chars via %s", u, p, result["chars_written"], result["backend"])
await notify(u, f"🧠 Monthly long-term memory integration complete ({result['chars_written']} chars via {result['backend']}). Worth a quick review.")
except Exception as e:
logger.error("auto distill long failed: %s", e)
logger.error("auto distill long [%s/%s] failed: %s", u, p, e)
async def _run_reminder_check() -> None:
"""Notify users of any due or overdue reminders (fires once daily at 09:00)."""
import re
from notification import notify
from persona import set_context
for u, p in _all_personas():
try:
set_context(u, p)
from tools.reminders import load_due_reminders
content = load_due_reminders()
if not content:
continue
# Extract numbered entries (lines like "1. [label] text" or "1. text")
entries = []
for line in content.splitlines():
m = re.match(r"^\d+\.\s+(.+)", line.strip())
if m:
# Strip status tags ([OVERDUE], [due TODAY], etc.) for display
text = re.sub(r"\[(OVERDUE|due TODAY|due: \S+)\]", "", m.group(1)).strip()
if text:
entries.append(text)
if not entries:
continue
count = len(entries)
if count == 1:
msg = f"Reminder: {entries[0]}"
else:
bullet_list = "\n".join(f"{e}" for e in entries[:3])
tail = f"\n…and {count - 3} more" if count > 3 else ""
msg = f"{count} reminders due:\n{bullet_list}{tail}"
await notify(u, msg)
logger.info("reminder check [%s/%s]: notified %d reminder(s)", u, p, count)
except Exception as e:
logger.error("reminder check [%s/%s] failed: %s", u, p, e)
def get_scheduler() -> AsyncIOScheduler | None:
@@ -76,6 +134,10 @@ def start() -> None:
_scheduler.add_job(_run_long, "cron", day=1, hour=4, minute=0, id="distill_long")
logger.info("scheduled: distill_long monthly on 1st at 04:00")
# Daily reminder notification check — 09:00
_scheduler.add_job(_run_reminder_check, "cron", hour=9, minute=0, id="reminder_check")
logger.info("scheduled: reminder_check daily at 09:00")
# Load user-defined cron jobs from CRONS.json
_load_user_crons()

View File

@@ -1,9 +1,14 @@
from datetime import datetime
from config import settings
from persona import persona_path
from persona import persona_path, get_user, get_persona
def log_turn(session_id: str, user_msg: str, assistant_msg: str) -> None:
def log_turn(
session_id: str,
user_msg: str,
assistant_msg: str,
backend_label: str = "",
host: str = "",
) -> None:
today = datetime.now().strftime("%Y-%m-%d")
sessions_dir = persona_path() / "sessions"
sessions_dir.mkdir(exist_ok=True)
@@ -12,11 +17,18 @@ def log_turn(session_id: str, user_msg: str, assistant_msg: str) -> None:
timestamp = datetime.now().strftime("%H:%M")
is_new = not log_file.exists()
meta_parts = [p for p in [backend_label, host] if p]
meta = f" · {' / '.join(meta_parts)}" if meta_parts else ""
# Use the actual user/persona names from the current request context
user_label = get_user().title()
persona_label = get_persona().title()
with open(log_file, "a") as f:
if is_new:
f.write(f"# Session Log — {today}\n")
f.write(
f"\n### [{timestamp}] `{session_id}`\n"
f"**{settings.user_name}:** {user_msg}\n\n"
f"**{settings.agent_name}:** {assistant_msg}\n"
f"\n### [{timestamp}] `{session_id}`{meta}\n"
f"**{user_label}:** {user_msg}\n\n"
f"**{persona_label}:** {assistant_msg}\n"
)

View File

@@ -62,12 +62,40 @@ def save(session_id: str, messages: list[dict]) -> None:
# Enforce rolling window
windowed = messages[-settings.max_history_messages:]
path.write_text(json.dumps({
data = {
"session_id": session_id,
"created": existing.get("created", datetime.now().isoformat()),
"updated": datetime.now().isoformat(),
"messages": windowed,
}, indent=2))
}
if "name" in existing:
data["name"] = existing["name"]
path.write_text(json.dumps(data, indent=2))
def get_name(session_id: str) -> str:
"""Return the friendly name for a session, or '' if none set."""
path = _path(session_id)
if not path.exists():
return ""
try:
return json.loads(path.read_text()).get("name", "")
except Exception:
return ""
def rename(session_id: str, name: str) -> bool:
"""Set (or clear) the friendly name on a session. Returns False if not found."""
path = _path(session_id)
if not path.exists():
return False
data = json.loads(path.read_text())
if name:
data["name"] = name
else:
data.pop("name", None)
path.write_text(json.dumps(data, indent=2))
return True
def delete(session_id: str) -> bool:
@@ -84,14 +112,17 @@ def list_all() -> list[dict]:
if not d.exists():
return []
results = []
for f in sorted(d.glob("*.json"), key=lambda p: p.stat().st_mtime, reverse=True):
for f in d.glob("*.json"):
try:
data = json.loads(f.read_text())
results.append({
"session_id": data["session_id"],
"name": data.get("name", ""),
"updated": data.get("updated"),
"message_count": len(data.get("messages", [])),
"_sort_key": data.get("updated") or f.stat().st_mtime,
})
except Exception:
pass
results.sort(key=lambda s: s.pop("_sort_key"), reverse=True)
return results

518
cortex/static/HELP.md Normal file
View File

@@ -0,0 +1,518 @@
# Cortex UI — Help & Reference
<!-- SHARED BASE: cortex/static/HELP.md
This file is served to all users regardless of persona.
Persona-specific additions live in home/{username}/persona/{name}/HELP.md
and are appended automatically by help.html when present.
-->
*Last updated: 2026-05-13*
---
## Getting Started
If this is your first time using Cortex, you need one thing before the chat will work: an AI model connected to your account.
**Fastest path — OpenRouter:**
OpenRouter gives you access to Claude, Gemini, and dozens of other models with a single API key.
1. Get a free API key at [openrouter.ai/keys](https://openrouter.ai/keys)
2. Go to **☰ → Account → [Set up OpenRouter →]** (shown automatically if no model is configured)
3. Paste your key, pick a starting model, click **Connect**
That's it — you're ready to chat.
**Already past setup but seeing errors?** Go to **☰ → Account → Model Registry → Manage models** and confirm a model is assigned to the **Chat** role (Primary slot). If all slots are empty, add a model first.
---
## Header Controls
| Button | What it does |
|---|---|
| **Sessions** | Open the sessions panel — list, resume, or start sessions |
| **N** (sliders icon) | Open the Context & Memory panel (N = current context tier) |
| **☰** | Settings menu — Files, push notification toggle, Account, Sign Out |
| **?** | Open this help panel |
The **Context & Memory** panel (sliders icon with tier number) contains all configuration options:
| Section | Controls |
|---|---|
| **Context Tier** | T1 T4 context depth |
| **Memory Layers** | Toggle Long / Mid / Short memory on/off |
| **Distill Memory** | Manually trigger Short / Mid / Long / All distillation |
| **Model** | Active chat model — click to cycle through your configured slot models (Primary → Backup 1 → …) |
| **Display** | **Aa** cycles font size · **☾** toggles theme · **S/M/L** cycles input area height · **⌃↵** toggles send shortcut |
All settings persist in `localStorage` across page refreshes.
---
## Chat
- **Send:** `Ctrl+Enter` by default. Click `⌃↵` in the input controls to toggle to plain `Enter` mode.
- **Stop:** Click **Stop** to cancel an in-progress response at any time.
- **Edit a message:** Hover over any message → click **edit**. `Ctrl+Enter` saves, `Esc` cancels.
- **Delete a message:** Hover over any message → click **del**, then **confirm delete**.
- **Copy:** Hover over any message → click **copy**.
- **New line while typing:** `Shift+Enter` (in `Ctrl+Enter` mode) or `Shift+Enter` / Enter (in Enter mode).
Each assistant response shows a small **model tag** below the message identifying which model and host responded.
---
## Tools (⚡)
Click the **⚡** button in the input row to enable the Tools toggle. When lit (amber), **Send** changes to **Run** and messages are routed through the **orchestrator** instead of directly to the chat model.
The orchestrator runs a multi-step tool loop:
1. The **orchestrator model** reasons about the request and calls tools as needed
2. Tool results are fed back into the conversation; the loop continues until the model has what it needs
3. The model produces the final user-facing reply — when the orchestrator role uses Gemini, Claude writes the final response; when it uses a local model, that same model writes it
4. Expandable tool-call cards appear above the response — click any card to see the arguments sent and the result returned
The ⚡ toggle is **independent of the Role selector** — you can use any role (chat, coder, research, etc.) with or without tools. The orchestrator model is configured in **Account → Model Registry → Role Assignments → Orchestrator**.
Tools mode is best for tasks requiring research, multi-step reasoning, or side effects (e.g. "search for X", "add a task", "what's on my list?", "append this to my journal"). Regular chat is faster for conversational turns.
Orchestrated sessions persist to history exactly like regular chat.
### Available Tools
69 tools across 17 categories. Tool schemas are narrowed per-message using keyword routing — only categories relevant to your request are sent, keeping token overhead low. Per-role tool sets provide additional filtering.
| Category | Tools |
|---|---|
| **Web** | `web_search`, `http_fetch`, `web_read`, `http_post` |
| **Project Files** | `project_file_read`, `project_file_list`, `file_stat`, `file_grep`, `file_diff`, `file_syntax_check` |
| **Files** (admin) | `file_read`, `file_list`, `file_write`, `session_read`, `session_search` |
| **Git** | `git_status`, `git_log`, `git_diff` |
| **Shell** | `shell_exec`, `claude_allow_dir` |
| **System** | `cortex_restart`, `cortex_logs`, `cortex_status`, `cortex_update` |
| **Tasks** | `task_list`, `task_create`, `task_update`, `task_complete` |
| **Cron** | `cron_list`, `cron_add`, `cron_remove`, `cron_toggle` |
| **Reminders** | `reminders_add`, `reminders_list`, `reminders_remove`, `reminders_clear` |
| **Scratchpad** | `scratch_read`, `scratch_write`, `scratch_append`, `scratch_clear` |
| **Notifications** | `web_push`, `email_send`, `nc_talk_send`, `nc_talk_history` |
| **Aether Journals** | `ae_journal_list/search`, `ae_journal_entries_list`, `ae_journal_entry_read/create/update/disable/append/prepend` |
| **Aether Tasks** | `ae_task_list` |
| **Aether Database** (admin) | `ae_db_query`, `ae_db_describe`, `ae_db_show_view` |
| **Agent Notes** | `agent_notes_read`, `agent_notes_write`, `agent_notes_append`, `agent_notes_clear` |
| **Agents** | `spawn_agent`, `aider_run` |
| **Home Assistant** | `ha_get_state`, `ha_get_states`, `ha_call_service` |
Files, Shell, System, Aether Database, Agents, and some Notification/Web tools are **admin-only** and not visible to regular users.
`http_post` requires a URL prefix allowlist in `home/{user}/http_allowlist.json`.
`nc_talk_history` requires `nc_username` and `nc_app_password` in `channels.json` under `nextcloud`.
`ae_db_*` tools require Aether DB credentials configured in **Integrations** settings. All queries are SELECT-only — no writes possible.
`aider_run` requires Aider installed (`pip install aider-chat`) and a model configured via `AIDER_MODEL` env var or the project's `.aider.conf.yml`. Supports any OpenAI-compatible backend — DeepSeek, OpenRouter, Ollama, etc.
### Per-Role Tool Sets
Each role can be configured with a specific subset of tool categories. When a role has a tool subset configured, only those tools are sent to the orchestrator — the rest are invisible to the model for that session.
**Example:** a Coder role might only need Web, Files, Shell, and Agent Notes. A Research role might only need Web. Configuring this avoids sending schemas for 30+ irrelevant tools on every call.
Configure per-role tool sets in **Account → Model Registry → Role Assignments** — expand a role card to see the category checkboxes. The default (no checkboxes selected) sends all tools the user has access to.
---
## Sessions
Sessions are named conversation threads that persist across page refreshes.
- Click **Sessions****+ New** to start a fresh session.
- Click any listed session to resume it — full history loads instantly.
- Sessions from Nextcloud Talk appear as `nct_*` prefixed IDs.
- A blue **●** badge appears on the Sessions button when Talk activity arrives in a session you're not currently viewing.
---
## Notes
Notes are injected into a session without triggering an LLM response.
- Click **Note** to toggle note mode. The input border changes colour.
- **Private note** (amber border) — visible only in the UI, never sent to the LLM.
- **Context note** (teal border) — persisted to session history so the LLM sees it on the next turn. Useful for nudging context without a full message.
- Click the `private / public` label to switch between note types.
---
## Install as App (PWA)
Cortex supports installation as a Progressive Web App — it runs in its own window with no browser chrome.
- **Chrome / Edge (desktop):** Look for the install icon in the address bar, or open the browser menu → **Install Cortex…**
- **Android (Chrome):** Tap ⋮ → **Add to Home Screen**
- **iOS (Safari):** Tap the Share button → **Add to Home Screen**
Once installed, opening Cortex from the home screen or app launcher skips the browser UI entirely.
---
## Switching Models
The **Model** button in the Context & Memory panel cycles through the slot models configured for your active role (Primary → Backup 1). Click it to switch between models mid-session.
- The button label shows the active model (e.g. "GPT-4o", "Gemini 2.5 Flash")
- The selected slot is sent with each chat request so the correct model is used
- If only one model is configured, the toggle does nothing
- A system message appears in the chat when you switch models
If the active model fails, the next configured backup slot is tried automatically.
Each response shows a **model tag** (bottom-right of message) with the model label and host, so you always know what responded.
---
## Account Settings
**Navigate to:** ☰ (top-right menu) → **Account**
| Section | What you can do |
|---|---|
| **Account** | View your username, role badge (Admin / User), rename your username |
| **Connected Accounts** | See which Google account is linked for OAuth sign-in |
| **Email Allowlist** | Regex patterns controlling which addresses the `email_send` tool can reach |
| **Notifications** | Dedicated page — set channel (Browser Push, NC Talk, Google Chat, email) for proactive messages; configure Home Assistant inbound webhook; test buttons for instant verification |
| **Schedules** | View, add, edit, pause, and delete scheduled jobs directly — without going through the AI |
| **Tool Permissions** | Allow or block specific orchestrator tools for your account |
| **Usage** | Token consumption by model — see below |
| **Browser Cache** | Clear UI preferences stored locally (theme, font size, session ID, etc.) |
| **Model Registry** | Configure AI providers, local hosts, and role assignments |
| **Change Password** | Update your login password |
| **Personas** | List and rename your personas |
---
## Usage
Token consumption is tracked automatically for API-backed models. **Navigate to:** ☰ → **Account****Usage** section.
The table shows all-time totals per model key, with columns for:
| Column | Meaning |
|---|---|
| **Model** | `backend/model-name` key (e.g. `gemini_api/gemini-2.5-flash`, `local/deepseek-v4`) |
| **Calls** | Number of API calls made |
| **Prompt** | Input tokens sent |
| **Output** | Completion tokens received |
| **Total** | Prompt + Output |
Values ≥ 1,000 are displayed as `k` (e.g. `24.3k`).
**What is and isn't tracked:**
- ✅ Gemini API calls (orchestrator, distillation)
- ✅ Local OpenAI-compatible calls (Open WebUI, Ollama, OpenRouter)
- ✗ Claude CLI — no structured token data is returned by the subprocess
- ✗ Gemini CLI — same reason
The raw data lives in `home/{username}/usage.json` and is also accessible via the Files panel or the API.
---
## Model Registry
Configure which AI models are available and which handles each task type.
**New user quick path:** ☰ → **Account****Set up OpenRouter →** (the guided wizard adds a host, model, and role assignment in one step).
**Full manual path:** ☰ → **Account** → scroll to **Model Registry****Manage models →**
---
### Step 1 — Set up providers and hosts
Do this before adding models — models need a provider account or local host to attach to.
**Anthropic (Claude):** Two options:
- **CLI (OAuth):** Nothing to configure — uses your existing `claude auth login` session. If Claude isn't working, run `claude auth login` in a terminal.
- **Direct API key:** Scroll to **Cloud Providers → Anthropic** → click **+ Add API key**. Enter a label and your `sk-ant-…` key from [console.anthropic.com/keys](https://console.anthropic.com/keys). When you add a model using an API key credential, it routes through the Anthropic SDK instead of the CLI.
**Google (Gemini):** Add one entry per API key you want to use:
1. Scroll to **Cloud Providers → Google** → click **+ Add Google account**
2. Enter a label (e.g. "Work", "Personal") and your API key
3. Get a free key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey)
**OpenRouter** (recommended for new users — one key for many models):
1. Get a key at [openrouter.ai/keys](https://openrouter.ai/keys)
2. Scroll to **Local Hosts****+ Add host**
3. Label: "OpenRouter", URL: `https://openrouter.ai/api/v1`, paste your key, Type: OpenAI-compatible
4. Click **Fetch models** to verify, then add models from the fetched list
**Other local hosts** (Open WebUI, Ollama, LM Studio, etc.):
1. Scroll to **Local Hosts** → click **+ Add host** to expand the form
2. Enter a label, the API URL (e.g. `http://192.168.1.100:3000`), and optional API key
3. Set **Type**: Open WebUI / Ollama, or OpenAI-compatible
4. Click **Fetch models** on the saved host card to verify connectivity
---
### Step 2 — Add models
Scroll to **Add Model**. Select the provider tab, fill in the details, click **Add Model**:
| Tab | What you need |
|---|---|
| **Local** | Select a host (from Step 1) → enter model name, or use **Fetch from host** to pick from a live list |
| **Google** | Select a Gemini model from the catalog → select a Google account (from Step 1) |
| **Anthropic** | Select a credential (CLI OAuth or an API key added in Step 1) → select a Claude model from the catalog |
The label and context window size auto-fill from the catalog — edit them if you want. Tags are optional.
---
### Step 3 — Assign models to roles
Scroll to **Role Assignments** at the bottom of the page. Each role has **Primary** and **Backup 1** slots — Primary is tried first, then Backup 1. Changes save automatically.
**Required roles** (always present, cannot be removed):
| Role | Used for |
|---|---|
| **Chat** | Regular conversation |
| **Orchestrator** | Agent mode tool loop |
| **Distill** | Memory distillation (short / mid / long) |
**Custom roles** — Click **+ Add custom role** to create your own. Each custom role gets its own model selection, tool set, and system prompt addition. Good examples:
| Example | Purpose |
|---|---|
| **Coder** | Code-focused tasks — larger context window, code-aware model |
| **Research** | Long-context research — high-token model, web tools prioritized |
Switch roles via the **Role** selector in the Context & Memory panel (⚙). Leave all slots empty to use the server default.
**Per-role tool sets:** Expand any role card to configure which tool categories the orchestrator can use when that role is active. Unchecked categories are hidden from the model entirely — reducing token overhead on every orchestrated call. Leaving all categories unchecked means all tools the user has access to are available (the default).
**Inject timestamp:** Each role card has an "Inject current date & time into system prompt" checkbox (default on). Disable it for pure processing roles (summarizer, classifier, translator) that don't need clock awareness.
---
## Nextcloud Talk Bot
The Cortex bot is registered in Nextcloud Talk.
- Messages sent in enabled Talk conversations are received by Cortex, processed, and replied to.
- The webhook returns `200 OK` immediately; the reply happens asynchronously.
- Real-time updates stream to the web UI via SSE — you see Talk messages and responses appear live.
- To enable the bot in a conversation: open Talk conversation settings → Bots → enable the bot.
---
## Google Chat Bot
The Cortex bot is available in Google Chat (One Sky IT Workspace).
- Send the bot a direct message in Google Chat to start a conversation.
- Each DM thread is its own session (`gc_spaces/*` prefix) — history persists across messages.
- Responses are synchronous — Google Chat displays the reply directly in the thread.
- To add the bot to a space: open the space, click **Add people & apps**, and search for the Cortex bot.
- Sessions from Google Chat appear as `gc_*` prefixed IDs in the Sessions panel.
---
## Files (Identity Editor)
The **Files** button opens an editor for your persona's identity and memory files:
| File | Purpose |
|---|---|
| `SOUL.md` | Core personality, values, and voice |
| `IDENTITY.md` | Role, capabilities, and context |
| `USER.md` | Your profile, preferences, and history |
| `PROTOCOLS.md` | Behavioural rules and communication protocols |
| `CONTEXT_TIERS.md` | Defines what gets loaded at each context tier |
| `MEMORY_LONG.md` | Permanent curated long-term memory |
| `MEMORY_MID.md` | Rolling mid-term digest (LLM-distilled) |
| `MEMORY_SHORT.md` | Recent session rollup (auto-aggregated) |
| `HELP.md` | This file — persona-specific additions appended below |
| `email_allowlist.json` | Regex patterns for permitted `email_send` recipients (one per line) |
Toggle **preview** / **edit** to switch between rendered markdown and raw text. **Ctrl+S** saves, **Esc** closes.
The **Audit Log** group at the bottom of the sidebar (collapsed by default) lists tool call logs by date (`YYYY-MM-DD.jsonl`). Click any date to view a read-only table of every orchestrator tool call: time, tool name, status, model, args, and result snippet. Status is colour-coded: green = ok, red = error, amber = denied.
---
## Push Notifications
Cortex can send browser push notifications — even when the tab is closed.
- Open **☰ → Enable notifications** and accept the browser permission prompt.
- Once enabled, the button shows **Notifications on** (in accent colour).
- Click again to disable. Subscriptions are stored per-device.
- The orchestrator's `web_push` tool lets your persona send you a push proactively (e.g. when a long task completes).
**Notification channel settings:** ☰ → **Account****Notification settings →** — choose Browser Push, Email, Nextcloud Talk, or Google Chat as the channel your persona uses for scheduled reminders, cron job completions, and memory digests. Use the **Send Test Notification** button to verify your setup, or **Check Reminders Now** to trigger the reminder check immediately.
---
## Context & Memory ( ⚙ panel )
### Context Tiers
Controls how much context is prepended to each LLM call:
| Tier | Loads | ~Tokens |
|---|---|---|
| **Min** | SOUL + IDENTITY + USER summary | ~1,500 |
| **Std** | + USER full + PROTOCOLS + HELP + memory layers | ~5,000 |
| **Ext** | + last 2 raw session logs | ~15,000 |
| **Full** | + last 7 raw session logs | ~50,000 |
Default is **Std**. Use **Min** for small/local models. Use **Ext** or **Full** for complex multi-session tasks.
### Memory Layers
Three independently toggleable memory files, loaded **Long → Mid → Short**:
| Layer | File | Contents |
|---|---|---|
| **Long** | `MEMORY_LONG.md` | Permanent facts — origin, key decisions, profile highlights |
| **Mid** | `MEMORY_MID.md` | Rolling digest of recent weeks — LLM-distilled from Short |
| **Short** | `MEMORY_SHORT.md` | Recent session rollup — auto-aggregated from session logs |
Toggle any layer off to save tokens for a focused conversation.
### Memory Distillation
Distillation builds up the memory layers from raw session logs. Runs automatically on a schedule; trigger manually via the ⚙ panel:
| Button | What it does |
|---|---|
| **short** | Rolls recent session log files → `MEMORY_SHORT.md` (fast, no LLM) |
| **mid** | LLM summarizes `MEMORY_SHORT.md``MEMORY_MID.md` |
| **long** | LLM integrates `MEMORY_MID.md``MEMORY_LONG.md` |
| **all** | Runs short → mid → long in sequence |
**Recommended workflow:** run **short** after any productive session; **mid** weekly; **long** monthly.
---
## Scheduled Jobs
Cortex can run recurring jobs on a schedule — reminders, daily briefings, automated research, and more. Manage them by asking your persona to set them up, or go directly to **☰ → Account → Schedules**.
### Job Types
| Type | What it does |
|---|---|
| `remind` | Appends to `REMINDERS.md` — automatically surfaced in chat context |
| `note` | Appends to `SCRATCH.md` — read on demand via the scratchpad |
| `message` | Sends the payload text directly to your notification channel |
| `brief` | Calls the AI with your payload as the prompt, sends the response to your notification channel. Good for morning briefings, check-ins. |
| `task` | Runs the full orchestrator tool loop with your payload as the request, sends Claude's response to your notification channel. Use this for agentic scheduled work: research, file updates, summaries that need tool access. |
For `task` jobs: tools that require confirmation are skipped in scheduled context. Pre-approve them in **Settings → Tools** to allow them in scheduled tasks.
### Schedule Formats
| Format | When it runs |
|---|---|
| `hourly` | Every hour at :00 |
| `daily` | Every day at 09:00 |
| `daily:HH:MM` | Every day at the specified time |
| `weekly:DOW` | Every specified day at 09:00 (e.g. `weekly:mon`) |
| `weekly:DOW:HH:MM` | Every specified day at the specified time (e.g. `weekly:fri:17:00`) |
| `monthly` | 1st of every month at 09:00 |
| `monthly:DD` | Specific day of month at 09:00 (e.g. `monthly:15`) |
| `monthly:DD:HH:MM` | Specific day of month at the specified time |
| `yearly:MM:DD` | Every year on that date at 09:00 — for birthdays, anniversaries (e.g. `yearly:03:15`) |
| `yearly:MM:DD:HH:MM` | Every year on that date at the specified time |
DOW values: `mon tue wed thu fri sat sun`. All times are server-local.
Schedules take effect immediately when added or edited — no restart needed. Paused jobs stay in the list and can be resumed at any time.
### Home Assistant Integration
HA automations can trigger your persona via webhook. Configure in **Notifications → Home Assistant → Inbound webhook**:
- Set a **Webhook ID** (long random string — this is your secret URL component)
- Your endpoint: `https://cortex.dgrzone.com/webhook/ha/{username}/{webhook_id}`
- **Enable orchestrator tools** — when checked, HA events trigger the full tool loop; when unchecked, events get a direct LLM response (faster, no tools)
HA payload fields recognized: `message`, `entity_id`, `state`, `trigger`, `event`, `area`.
---
## Keyboard Shortcuts
| Keys | Action |
|---|---|
| `Ctrl+Enter` | Send message (default mode) |
| `Enter` | Send (when in Enter mode) |
| `Shift+Enter` | New line in message input |
| `Ctrl+Enter` | Save inline message edit |
| `Esc` | Cancel inline edit / close any open modal |
| `Ctrl+S` | Save file (Files modal) |
---
## API Reference
For direct access or scripting:
| Method | Endpoint | Description |
|---|---|---|
| `POST` | `/chat` | Send a message — returns SSE stream |
| `GET` | `/backend` | Get current primary/fallback backends |
| `POST` | `/backend` | Set primary backend (`{"primary": "claude"}`) |
| `GET` | `/sessions` | List all sessions |
| `GET` | `/history/{id}` | Get session message history |
| `PUT` | `/history/{id}` | Replace full session history |
| `GET` | `/events` | SSE stream for real-time Talk activity |
| `POST` | `/note` | Inject a context note into a session |
| `GET` | `/files` | List identity files |
| `GET` | `/files/{name}` | Read a file |
| `PUT` | `/files/{name}` | Write a file |
| `POST` | `/distill/short` | Aggregate session logs → MEMORY_SHORT |
| `POST` | `/distill/mid` | Summarize short → MEMORY_MID (LLM) |
| `POST` | `/distill/long` | Integrate mid → MEMORY_LONG (LLM) |
| `POST` | `/distill/all` | Run all three distillation steps |
| `GET` | `/distill/status` | Scheduler status and next run times |
| `POST` | `/orchestrate` | Submit an agent task — returns `{"job_id": "..."}` |
| `GET` | `/orchestrate/{job_id}` | Poll job status and result |
| `GET` | `/settings/models` | Model registry UI |
| `POST` | `/api/models/role` | Set a role assignment (JSON body) |
| `POST` | `/api/models/role-config` | Set per-role tool list and system prompt append |
| `GET` | `/api/push/vapid-key` | VAPID public key (for push subscription) |
| `POST` | `/api/push/subscribe` | Register a push subscription |
| `DELETE` | `/api/push/subscribe` | Remove a push subscription |
| `POST` | `/api/push/test` | Send a test notification via configured channel |
| `POST` | `/api/push/reminders/check` | Run reminder check immediately; returns `{"reminders_found": n}` |
| `GET` | `/api/audit/files` | List available audit log dates (own data) |
| `GET` | `/api/audit/day?date=` | Tool call entries for a specific date (own data) |
| `GET` | `/api/audit/recent` | Recent tool calls across days (admin) |
| `GET` | `/api/audit/stats` | Tool call counts by tool/status/user (admin) |
| `GET` | `/api/usage` | Full daily token usage log (own data) |
| `GET` | `/api/usage/summary` | Per-model token totals, all time (own data) |
| `GET` | `/api/usage/all` | Per-model totals for all users (admin) |
| `GET` | `/setup/model` | Guided OpenRouter setup form (Step 3 / standalone) |
| `POST` | `/setup/model` | Save OpenRouter host + model + assign to chat role |
| `GET` | `/health` | Health check — returns `{"status": "ok"}` |
Chat request body (`POST /chat`):
```json
{
"message": "string",
"session_id": "string | null",
"tier": 2,
"chat_role": "chat",
"slot": "primary | backup_1 | backup_2 | null",
"include_long": true,
"include_mid": true,
"include_short": true,
"off_record": false
}
```
---
*Cortex is a self-hosted personal AI platform. Named after the 'verse-wide communications network in Firefly.*

123
cortex/static/TOOLS.md Normal file
View File

@@ -0,0 +1,123 @@
# Tool Reference
> This reference covers all 45 orchestrator tools available when the ⚡ toggle is on.
> Tools are invoked automatically by the orchestrator — you don't call them directly.
¹ **Admin only** — requires the `admin` role. Invisible to regular users.
² **Confirmation required** — the orchestrator pauses and shows **Confirm / Deny** buttons before executing.
---
## Web
| Tool | What it does |
|---|---|
| `web_search` | DuckDuckGo search — returns titles, URLs, and snippets for the top results |
| `http_fetch` | Fetch a specific URL and return the response body (8 192 char cap) |
## Files ¹
| Tool | What it does |
|---|---|
| `file_read` ¹ | Read any file under the persona home directory |
| `file_list` ¹ | List files and directories with sizes (200 entry cap) |
| `file_write` ¹ ² | Write or append to a file under the persona home directory |
## Shell ¹
| Tool | What it does |
|---|---|
| `shell_exec` ¹ ² | Run any shell command on the Cortex host; timeout 1120 s |
| `claude_allow_dir` ¹ | Add a directory to Claude Code's auto-allowed paths |
## System ¹
| Tool | What it does |
|---|---|
| `cortex_restart` ¹ ² | Restart the Cortex service (5 s delay); connection drops — refresh the page |
| `cortex_logs` ¹ | Recent lines from the systemd journal (default 50, max 200) |
| `cortex_status` ¹ | Current git branch, commit, ahead/behind remote, and service state |
| `cortex_update` ¹ ² | `git pull` + syntax check all `.py` files; reports what changed. Does **not** restart automatically — call `cortex_restart` after reviewing |
## Tasks
| Tool | What it does |
|---|---|
| `task_list` | List personal tasks; pass `include_done=true` to include completed |
| `task_create` | Create a task with title, optional notes and due date |
| `task_update` | Update any fields on an existing task |
| `task_complete` | Mark a task as complete |
## Cron
| Tool | What it does |
|---|---|
| `cron_list` | List all scheduled jobs for this persona |
| `cron_add` | Add a scheduled job — accepts cron syntax or plain-English interval |
| `cron_remove` ² | Remove a scheduled job by ID |
| `cron_toggle` | Enable or disable a job without removing it |
## Reminders
| Tool | What it does |
|---|---|
| `reminders_add` | Add a reminder with optional label; surfaced in context at Tier 2+ |
| `reminders_list` | List all pending reminders, numbered for easy removal |
| `reminders_remove` | Remove a single reminder by number (call `reminders_list` first) |
| `reminders_clear` ² | Clear all reminders at once |
## Scratchpad
| Tool | What it does |
|---|---|
| `scratch_read` | Read the current scratchpad |
| `scratch_write` | Overwrite the scratchpad with new content |
| `scratch_append` | Append a timestamped section to the scratchpad |
| `scratch_clear` | Erase the scratchpad |
## Notifications ¹
| Tool | What it does |
|---|---|
| `web_push` | Send a browser push notification to the active user's registered devices |
| `email_send` ¹ | Send an email via SMTP; recipient must match your `email_allowlist.json` |
| `nc_talk_send` ¹ | Send a message to a Nextcloud Talk conversation |
## Aether Journals
| Tool | What it does |
|---|---|
| `ae_journal_list` | List all journals for the configured AE account (returns names + IDs) |
| `ae_journal_search` | Search entries by keyword, tag, date range, type, status, or priority |
| `ae_journal_entries_list` | Browse all entries in a specific journal, newest first; paginated |
| `ae_journal_entry_read` | Read the full content of a single entry by ID |
| `ae_journal_entry_create` | Create a new entry with title, content, tags, and summary |
| `ae_journal_entry_update` | Patch any fields on an existing entry (title, content, tags, summary, enable) |
| `ae_journal_entry_disable` | Soft-delete an entry (`enable=false`) without permanently removing it |
| `ae_journal_entry_append` | Append a timestamped section to the bottom of an entry's content |
| `ae_journal_entry_prepend` | Prepend a timestamped section to the top of an entry's content |
## Aether Tasks ¹
| Tool | What it does |
|---|---|
| `ae_task_list` ¹ | List tasks from the agents_sync Kanban board |
## Agent Notes
Private, durable notes visible only to the orchestrator — not surfaced to users. Persist across sessions. Only available in orchestrated (tool-enabled) sessions.
| Tool | What it does |
|---|---|
| `agent_notes_read` | Read the current private notes file |
| `agent_notes_write` | Overwrite the notes file completely |
| `agent_notes_append` | Append a timestamped entry (keeps last 3 backups automatically) |
| `agent_notes_clear` | Erase all notes (backs up first) |
## Agents ¹
Spawn sub-agents that run their own tool loop using a specific role's model and tools.
| Tool | What it does |
|---|---|
| `spawn_agent` ¹ | Spawn a sub-agent synchronously — blocks until the task completes or times out. Params: `task`, `role` (default `chat`), `tier` (14, default 1), `timeout` seconds, `max_rounds` override. Only works with `local_openai` and `gemini_api` models. |

File diff suppressed because it is too large Load Diff

172
cortex/static/crons.html Normal file
View File

@@ -0,0 +1,172 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Schedules</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Server-generated table + badges ── */
.cron-table {
width: 100%; border-collapse: collapse; font-size: 0.82rem;
margin-bottom: 1.5rem;
}
.cron-table th {
text-align: left; padding: 0.4rem 0.6rem;
border-bottom: 2px solid var(--pg-border);
color: var(--pg-muted); font-weight: 600; font-size: 0.75rem;
text-transform: uppercase; letter-spacing: 0.04em;
}
.cron-table td {
padding: 0.5rem 0.6rem; border-bottom: 1px solid var(--pg-border);
vertical-align: middle;
}
.cron-table tr:last-child td { border-bottom: none; }
.cron-table tr:hover td { background: var(--pg-hover); }
.badge {
display: inline-block; padding: 0.15rem 0.45rem;
border-radius: 4px; font-size: 0.72rem; font-weight: 600;
text-transform: uppercase; letter-spacing: 0.03em;
}
.badge-enabled { background: color-mix(in srgb, var(--pg-accent) 18%, transparent); color: var(--pg-accent); }
.badge-paused { background: color-mix(in srgb, var(--pg-muted) 18%, transparent); color: var(--pg-muted); }
.badge-remind { background: color-mix(in srgb, #a78bfa 15%, transparent); color: #a78bfa; }
.badge-note { background: color-mix(in srgb, #60a5fa 15%, transparent); color: #60a5fa; }
.badge-message { background: color-mix(in srgb, #34d399 15%, transparent); color: #34d399; }
.badge-brief { background: color-mix(in srgb, #fb923c 15%, transparent); color: #fb923c; }
.badge-task { background: color-mix(in srgb, #f472b6 15%, transparent); color: #f472b6; }
.cron-actions { display: flex; gap: 0.35rem; }
.btn-cron {
padding: 0.2rem 0.55rem; border-radius: 4px; border: 1px solid var(--pg-border);
background: transparent; color: var(--pg-muted); font-size: 0.75rem; cursor: pointer;
font-family: inherit;
}
.btn-cron:hover { border-color: var(--pg-accent); color: var(--pg-accent); }
.btn-cron-del { color: var(--pg-dimmer); }
.btn-cron-del:hover { border-color: #ef4444; color: #ef4444; }
.payload-cell {
max-width: 240px; overflow: hidden; text-overflow: ellipsis;
white-space: nowrap; color: var(--pg-dimmer);
}
.persona-group-label {
font-size: 0.72rem; font-weight: 700; text-transform: uppercase;
letter-spacing: 0.06em; color: var(--pg-dimmer); margin: 1.25rem 0 0.5rem;
}
.empty-state {
text-align: center; padding: 2rem 1rem;
color: var(--pg-dimmer); font-size: 0.85rem;
border: 1px dashed var(--pg-border); border-radius: 8px;
margin-bottom: 1.5rem;
}
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link active">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Schedules</h1>
<p class="page-subtitle">Recurring jobs — reminders, notes, briefings, and agentic tasks.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<!-- Edit form (shown only when editing) -->
{{ edit_html }}
<!-- Cron list -->
{{ cron_list_html }}
<!-- Add new schedule -->
<div class="section">
<h2>Add schedule</h2>
<form method="POST" action="/settings/crons/add">
<div class="grid grid-cols-2 gap-x-3">
<div class="field">
<label for="add_persona">Persona</label>
<select id="add_persona" name="persona">
{{ persona_options }}
</select>
</div>
<div class="field">
<label for="add_job_type">Type</label>
<select id="add_job_type" name="job_type">
<option value="remind">remind — append to REMINDERS.md</option>
<option value="note">note — append to SCRATCH.md</option>
<option value="message">message — send payload as-is</option>
<option value="brief">brief — LLM response, no tools</option>
<option value="task">task — full orchestrator tool loop</option>
</select>
</div>
<div class="field">
<label for="add_label">Label</label>
<input type="text" id="add_label" name="label"
placeholder="Monday morning summary"
required autocomplete="off">
</div>
<div class="field">
<label for="add_schedule">Schedule</label>
<input type="text" id="add_schedule" name="schedule"
placeholder="weekly:mon:08:00"
required autocomplete="off" spellcheck="false">
<p class="hint">
hourly · daily · daily:HH:MM · weekly:DOW · weekly:DOW:HH:MM ·
monthly · monthly:DD · monthly:DD:HH:MM · yearly:MM:DD · yearly:MM:DD:HH:MM
</p>
</div>
<div class="field col-span-2">
<label for="add_payload">Payload / prompt</label>
<textarea id="add_payload" name="payload" rows="3"
placeholder="Check my open tasks and send a summary." required></textarea>
</div>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Add schedule</button>
</form>
</div>
</div>
</body>
</html>

239
cortex/static/help.html Normal file
View File

@@ -0,0 +1,239 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Help &amp; Reference</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Tab panels (JS-toggled display) ── */
.tab-panel { display: none; }
.tab-panel.active { display: block; }
/* ── Dynamically-rendered markdown content ── */
.help-body { line-height: 1.7; }
details {
margin-bottom: 0.75rem;
background: var(--pg-surface);
border: 1px solid var(--pg-border);
border-radius: 8px; overflow: hidden;
}
summary {
padding: 0.85rem 1rem; font-weight: 600; font-size: 0.95rem;
color: var(--pg-bright); cursor: pointer; list-style: none;
display: flex; align-items: center; gap: 0.5rem;
}
summary::before {
content: '▶'; font-size: 0.65rem; color: var(--pg-muted);
transition: transform 0.15s;
}
details[open] summary::before { transform: rotate(90deg); }
summary::-webkit-details-marker { display: none; }
details > *:not(summary) { padding: 0 1rem 1rem; }
.help-body p { margin: 0.5rem 0; font-size: 0.9rem; color: var(--pg-bright); }
.help-body ul { margin: 0.5rem 0 0.5rem 1.25rem; }
.help-body li { font-size: 0.9rem; color: var(--pg-bright); margin-bottom: 0.25rem; }
.help-body strong { color: var(--pg-text); }
.help-body code {
background: var(--pg-bg); border: 1px solid var(--pg-border);
border-radius: 4px; padding: 0.1em 0.4em;
font-size: 0.85em; color: var(--pg-accent);
}
.help-body a { color: var(--pg-accent); }
.help-body h1 { font-size: 1.1rem; font-weight: 700; color: var(--pg-text); margin: 0.75rem 0 0.5rem; }
.help-body h3 {
font-size: 0.8rem; font-weight: 600; color: var(--pg-muted);
text-transform: uppercase; letter-spacing: 0.05em; margin: 0.75rem 0 0.25rem;
}
.help-body table { width: 100%; border-collapse: collapse; font-size: 0.88rem; margin: 0.5rem 0 0.75rem; }
.help-body th, .help-body td { padding: 0.45rem 0.7rem; text-align: left; border-bottom: 1px solid var(--pg-border); }
.help-body th { color: var(--pg-muted); font-weight: 600; font-size: 0.8rem; text-transform: uppercase; letter-spacing: 0.04em; }
.help-body td { color: var(--pg-bright); }
.help-body pre { background: var(--pg-bg); border: 1px solid var(--pg-border); border-radius: 6px; padding: 0.75rem 1rem; overflow-x: auto; margin: 0.5rem 0; }
.help-body pre code { background: none; border: none; padding: 0; font-size: 0.85em; color: var(--pg-muted); }
.help-body hr { border: none; border-top: 1px solid var(--pg-border); margin: 0.5rem 0; }
</style>
</head>
<body>
<nav class="page-nav" id="page-nav">
<a id="nav-chat" href="/" class="nav-link">← Chat</a>
<a href="/help" class="nav-link active">Help</a>
<a href="/settings" class="nav-link" id="nav-settings">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="max-w-3xl mx-auto px-6 py-8 pb-16">
<div class="mb-6 pb-4 border-b border-pg-border">
<h1 class="text-xl font-bold text-pg-accent">Help &amp; Reference</h1>
<p id="persona-label" class="text-xs text-pg-muted mt-1"></p>
</div>
<!-- Tab bar -->
<div class="tab-bar">
<button class="tab-btn active" data-tab="ui">UI Guide</button>
<button class="tab-btn" data-tab="tools">Tools</button>
<button class="tab-btn" data-tab="persona" id="tab-btn-persona">Persona</button>
</div>
<div id="tab-ui" class="tab-panel active"><div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
<div id="tab-tools" class="tab-panel"> <div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
<div id="tab-persona" class="tab-panel"> <div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
</div>
<style>
.tab-bar {
display: flex; gap: 0.25rem; margin-bottom: 1.5rem;
border-bottom: 1px solid var(--pg-border);
}
.tab-btn {
padding: 0.45rem 1rem; font-size: 0.85rem; font-weight: 500;
color: var(--pg-dim); background: none; border: none;
border-bottom: 2px solid transparent; margin-bottom: -1px;
cursor: pointer; transition: color 0.15s, border-color 0.15s;
font-family: inherit;
}
.tab-btn:hover { color: var(--pg-bright); }
.tab-btn.active { color: var(--pg-accent); border-bottom-color: var(--pg-accent); }
</style>
<script>
const cfg = window.HELP_CONFIG || {};
const user = cfg.user || 'scott';
const persona = cfg.persona || 'inara';
const params = `user=${encodeURIComponent(user)}&persona=${encodeURIComponent(persona)}`;
document.getElementById('nav-chat').href = cfg.backHref || '/';
if (persona) {
document.getElementById('persona-label').textContent =
`${persona.charAt(0).toUpperCase() + persona.slice(1)} · ${user}`;
}
// Rename Persona tab to the actual persona name
const personaTabBtn = document.getElementById('tab-btn-persona');
personaTabBtn.textContent = persona.charAt(0).toUpperCase() + persona.slice(1);
// ── Tab switching ────────────────────────────────────────────────
const tabBtns = document.querySelectorAll('.tab-btn');
const tabPanels = document.querySelectorAll('.tab-panel');
const TAB_KEY = `cx_help_tab_${user}_${persona}`;
function activateTab(name) {
tabBtns.forEach(b => b.classList.toggle('active', b.dataset.tab === name));
tabPanels.forEach(p => p.classList.toggle('active', p.id === `tab-${name}`));
try { localStorage.setItem(TAB_KEY, name); } catch (_) {}
}
tabBtns.forEach(btn => btn.addEventListener('click', () => activateTab(btn.dataset.tab)));
// Restore last active tab
try {
const saved = localStorage.getItem(TAB_KEY);
if (saved) activateTab(saved);
} catch (_) {}
// ── Collapsible h2 sections ──────────────────────────────────────
function makeCollapsible(container, openAll = false, openSet = null) {
container.querySelectorAll('h2').forEach(h2 => {
const title = h2.textContent.trim();
const details = document.createElement('details');
if (openAll || (openSet && openSet.has(title))) details.open = true;
const summary = document.createElement('summary');
summary.textContent = title;
details.appendChild(summary);
const siblings = [];
let node = h2.nextSibling;
while (node && node.nodeName !== 'H2') { siblings.push(node); node = node.nextSibling; }
siblings.forEach(s => details.appendChild(s));
h2.parentNode.replaceChild(details, h2);
});
}
// ── Render markdown into a panel ────────────────────────────────
function render(panelId, markdown, openAll, openSet) {
const panel = document.querySelector(`#${panelId} .help-body`);
panel.innerHTML = marked.parse(markdown);
panel.querySelectorAll('a').forEach(a => { a.target = '_blank'; a.rel = 'noopener noreferrer'; });
makeCollapsible(panel, openAll, openSet);
}
// ── Load all three tabs in parallel ─────────────────────────────
const UI_OPEN = new Set(['Getting Started', 'Chat', 'Sessions', 'Model Registry']);
async function loadAll() {
// UI Guide
fetch('/static/HELP.md')
.then(r => r.ok ? r.text() : Promise.reject(r.status))
.then(md => render('tab-ui', md, false, UI_OPEN))
.catch(e => { document.querySelector('#tab-ui .help-body').innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">Failed to load: ${e}</p>`; });
// Tools
fetch('/static/TOOLS.md')
.then(r => r.ok ? r.text() : Promise.reject(r.status))
.then(md => render('tab-tools', md, true, null))
.catch(e => { document.querySelector('#tab-tools .help-body').innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">Failed to load: ${e}</p>`; });
// Persona-specific HELP.md
const personaPanel = document.querySelector('#tab-persona .help-body');
try {
const res = await fetch(`/files/HELP.md?${params}`);
if (res.ok) {
const data = await res.json();
const content = (data.content || '').trim();
if (content) {
render('tab-persona', content, true, null);
} else {
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet. Edit <code>HELP.md</code> in the Files panel to add them.</p>`;
}
} else {
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet.</p>`;
}
} catch (_) {
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet.</p>`;
}
}
loadAll();
</script>
</body>
</html>

BIN
cortex/static/icon-192.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.2 KiB

BIN
cortex/static/icon-512.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.7 KiB

4
cortex/static/icon.svg Normal file
View File

@@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
<rect width="512" height="512" rx="96" fill="#1a1228"/>
<text x="256" y="390" font-size="340" text-anchor="middle" font-family="system-ui, -apple-system, sans-serif"></text>
</svg>

After

Width:  |  Height:  |  Size: 251 B

View File

@@ -5,19 +5,35 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Inara</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>✨</text></svg>">
<link rel="manifest" href="/manifest.json">
<meta name="theme-color" content="#1a1228" id="meta-theme-color">
<meta name="mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<meta name="apple-mobile-web-app-title" content="Cortex">
<link rel="apple-touch-icon" href="/static/icon-192.png">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<!-- Apply saved theme + font size before first paint to avoid flash -->
<script>
(function(){
var t = localStorage.getItem('theme');
if (!t) t = window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
document.documentElement.setAttribute('data-theme', t);
var sizes = { normal: '16px', large: '18px', small: '14px' };
var sizes = { normal: '21px', large: '25px', small: '17px' };
var fs = localStorage.getItem('font-size') || 'normal';
document.documentElement.style.fontSize = sizes[fs] || '16px';
document.documentElement.style.fontSize = sizes[fs] || '21px';
})();
</script>
<link rel="stylesheet" href="/static/style.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/codemirror@5.65.17/lib/codemirror.min.css">
<script src="https://cdn.jsdelivr.net/npm/codemirror@5.65.17/lib/codemirror.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/codemirror@5.65.17/mode/markdown/markdown.min.js"></script>
<script src="/static/marked.min.js"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/styles/atom-one-dark.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/highlight.min.js"></script>
<script src="https://unpkg.com/lucide@latest/dist/umd/lucide.min.js"></script>
</head>
<body>
<header>
@@ -25,15 +41,45 @@
<div class="persona-switcher" id="persona-switcher">
<div class="name" id="persona-name">Inara</div>
<div class="subtitle">Cortex · Local</div>
<div id="session-id"></div>
<div class="persona-dropdown" id="persona-dropdown"></div>
</div>
<button id="sessions-btn" class="hdr-btn">Sessions</button>
<button id="files-btn" class="hdr-btn">Files</button>
<button id="ctx-open-btn" class="hdr-btn" title="Settings"><span class="tier-badge">2</span></button>
<button id="help-btn" class="hdr-btn" title="Help &amp; reference">?</button>
<nav id="hdr-nav">
<button id="sessions-btn" class="hdr-btn" title="Sessions">
<svg data-lucide="history" class="btn-icon"></svg>
<span class="btn-label">Sessions</span>
</button>
<button id="ctx-open-btn" class="hdr-btn" title="Context &amp; memory">
<svg data-lucide="sliders-horizontal" class="btn-icon"></svg><span class="tier-badge">2</span>
</button>
<div class="hdr-dropdown-wrap" id="settings-wrap">
<button class="hdr-btn" id="settings-btn" title="Settings">
<svg data-lucide="menu" class="btn-icon"></svg>
</button>
<div class="hdr-dropdown" id="settings-dropdown">
<button id="files-btn" class="hdr-dd-item">
<svg data-lucide="folder-open" class="btn-icon"></svg> Files
</button>
<a href="/settings" class="hdr-dd-item">
<svg data-lucide="user" class="btn-icon"></svg> Account
</a>
<button id="push-btn" class="hdr-dd-item" style="display:none">
<svg data-lucide="bell" class="btn-icon"></svg>
<span id="push-btn-label">Enable notifications</span>
</button>
<div class="hdr-dd-divider"></div>
<form method="POST" action="/logout" style="margin:0">
<button type="submit" class="hdr-btn" title="Sign out" id="logout-btn"></button>
<button type="submit" class="hdr-dd-item">
<svg data-lucide="log-out" class="btn-icon"></svg> Sign Out
</button>
</form>
</div>
</div>
<a id="help-link" href="/help" class="hdr-btn" title="Help &amp; reference" style="text-decoration:none">
<svg data-lucide="circle-help" class="btn-icon"></svg>
</a>
</nav>
<div id="sessions-panel"></div>
@@ -42,10 +88,10 @@
<div class="ctx-section">
<div class="ctx-section-title">Context Tier</div>
<div class="ctx-row">
<button class="ctx-btn" data-tier="1" id="tier-1" title="Minimal (~1.5k tokens)">T1</button>
<button class="ctx-btn active" data-tier="2" id="tier-2" title="Standard (~5k tokens)">T2</button>
<button class="ctx-btn" data-tier="3" id="tier-3" title="Extended (~15k tokens)">T3</button>
<button class="ctx-btn" data-tier="4" id="tier-4" title="Full (~50k tokens)">T4</button>
<button class="ctx-btn" data-tier="1" id="tier-1" title="Minimal — identity only (~1.5k tokens)">Min</button>
<button class="ctx-btn active" data-tier="2" id="tier-2" title="Standard — memory + user profile (~5k tokens)">Std</button>
<button class="ctx-btn" data-tier="3" id="tier-3" title="Extended — + last 2 sessions (~15k tokens)">Ext</button>
<button class="ctx-btn" data-tier="4" id="tier-4" title="Full — + last 7 sessions (~50k tokens)">Full</button>
</div>
</div>
<div class="ctx-section">
@@ -59,25 +105,29 @@
<div class="ctx-section">
<div class="ctx-section-title">Distill Memory</div>
<div class="ctx-row">
<button class="ctx-btn" id="distill-short-btn" title="Roll session logs → MEMORY_SHORT (no LLM)">short</button>
<button class="ctx-btn" id="distill-mid-btn" title="Summarize short → MEMORY_MID (LLM)">mid</button>
<button class="ctx-btn" id="distill-long-btn" title="Integrate mid → MEMORY_LONG (LLM)">long</button>
<button class="ctx-btn" id="distill-all-btn" title="Run all three steps in sequence">all</button>
<button class="ctx-btn" id="distill-short-btn" title="Roll today's sessions → MEMORY_SHORT.md (fast, no LLM)">Short</button>
<button class="ctx-btn" id="distill-mid-btn" title="Summarize SHORT → MID memory (uses LLM)">Mid</button>
<button class="ctx-btn" id="distill-long-btn" title="Integrate MID → LONG memory (uses LLM)">Long</button>
<button class="ctx-btn" id="distill-all-btn" title="Run Short → Mid → Long in sequence">All</button>
<button class="ctx-btn ctx-btn-danger" id="distill-rebuild-btn" title="⚠ Wipe Mid + Long memories and rebuild from session logs. Hand-edited content will be replaced.">Rebuild</button>
</div>
<div id="ctx-distill-status"></div>
<div id="ctx-schedule"></div>
</div>
<div class="ctx-section">
<div class="ctx-section-title">Backend</div>
<div class="ctx-section-title">Role</div>
<div class="ctx-row">
<button id="backend-toggle" class="ctx-btn" title="Click to switch primary backend">claude</button>
<button id="backend-toggle" class="ctx-btn" title="Active role — click to cycle">chat</button>
</div>
<div id="backend-model-hint"></div>
</div>
<div class="ctx-section">
<div class="ctx-section-title">Display</div>
<div class="ctx-row">
<button id="font-size-btn" class="ctx-btn" title="Cycle font size: normal → large → small">Aa</button>
<button id="theme-btn" class="ctx-btn" title="Toggle light/dark mode"></button>
<button id="font-size-btn" class="ctx-btn" title="Cycle font size: Normal → Large → Small">Aa</button>
<button id="theme-btn" class="ctx-btn" title="Toggle light / dark theme"></button>
<button id="height-cycle-btn" class="ctx-btn" title="Input size: Compact — click to cycle">S</button>
<button id="enter-toggle" class="ctx-btn" title="Toggle send shortcut: Ctrl+Enter ↔ Enter">⌃↵</button>
</div>
</div>
</div>
@@ -88,65 +138,71 @@
<div id="file-modal-inner">
<div id="file-modal-header">
<span id="file-modal-title">Context Files</span>
<select id="file-select"></select>
<span class="fm-spacer"></span>
<button class="fm-btn" id="file-raw-btn">edit</button>
<button class="fm-btn active" id="file-preview-btn">preview</button>
<button class="fm-btn save" id="file-save-btn">Save</button>
<span id="file-saved-msg">saved ✓</span>
<button class="fm-btn" id="file-close-btn"></button>
</div>
<div id="file-modal-content">
<div id="file-sidebar-wrap">
<div id="file-sidebar"></div>
<div id="session-search-wrap">
<div id="session-search-label">Session Search</div>
<div id="session-search-row">
<input id="session-search-input" type="search" placeholder="Search sessions…" autocomplete="off">
<button id="session-search-btn">Go</button>
</div>
</div>
</div>
<div id="file-modal-body">
<textarea id="file-editor" spellcheck="false"></textarea>
<div id="file-editor-wrap"></div>
<div id="file-preview"></div>
<div id="session-search-results" style="display:none"></div>
</div>
</div>
</div>
<!-- Help modal -->
<div id="help-modal">
<div id="help-modal-inner">
<div id="help-modal-header">
<h2>Cortex — Help &amp; Reference</h2>
<button class="fm-btn" id="help-close-btn"></button>
</div>
<div id="help-modal-body"></div>
</div>
</div>
<!-- Auth warning banner — shown when Claude CLI token is near expiry -->
<div id="auth-banner">
<div id="auth-banner-text">
<span id="auth-banner-msg"></span>
<span id="auth-banner-hint"></span>
</div>
<button id="auth-banner-close" title="Dismiss"></button>
</div>
<div id="messages"></div>
<div id="session-id"></div>
<div id="input-area">
<textarea id="input" rows="1" placeholder="Message Inara… (Ctrl+Enter to send)" autofocus></textarea>
<div id="right-col">
<!-- Semi-hidden: appear when content > ~3 lines -->
<div id="height-row">
<span></span>
<select id="height-sel">
<option value="120">5 lines</option>
<option value="240">10 lines</option>
<option value="480">20 lines</option>
</select>
<!-- Mode select — compact dropdown, opens upward, MRU sorted -->
<div id="mode-select">
<button id="mode-select-btn" title="Input mode">
<span id="mode-icon">💬</span>
<span id="mode-label">Chat</span>
<span class="mode-arrow"></span>
</button>
<!-- Populated dynamically in MRU order -->
<div id="mode-dropdown"></div>
<!-- Note visibility sub-toggle — only shown when note mode is active -->
<button id="note-vis-btn" title="Toggle note visibility (private / public)">prv</button>
<!-- Tools toggle — routes through the orchestrator tool loop when active -->
<button id="tools-toggle" title="Tools disabled — click to enable"></button>
<!-- Attach file — images (vision) or text/code files -->
<button id="attach-btn" title="Attach image or text file">📎</button>
<input type="file" id="file-input" style="display:none"
accept="image/png,image/jpeg,image/webp,image/gif,text/plain,text/markdown,.md,.txt,.py,.js,.ts,.jsx,.tsx,.json,.yaml,.yml,.toml,.html,.css,.sh,.csv,.xml,.rs,.go,.java,.c,.cpp,.h,.rb,.php,.swift,.kt,.sql">
</div>
<button id="enter-toggle" title="Toggle send shortcut">⌃↵</button>
<!-- Note mode controls -->
<button id="note-type-btn">private</button>
<button id="note-btn">Note</button>
<button id="agent-mode-btn" title="Agent mode — Gemini tool loop + Claude response">Agent</button>
<!-- Attachment preview — shown when a file is pending -->
<div id="attachment-row" style="display:none">
<div id="attachment-preview">
<img id="attachment-thumb" alt="" style="display:none">
<span id="attachment-icon">📎</span>
<span id="attachment-name"></span>
<button id="attachment-clear" title="Remove attachment"></button>
</div>
</div>
<textarea id="input" rows="1" placeholder="Message…" autofocus></textarea>
<div id="send-col">
<button id="send">Send</button>
<button id="stop">Stop</button>
<button id="stop"><svg data-lucide="square" width="14" height="14" class="btn-icon"></svg> Stop</button>
</div>
</div>
<div id="sessions-backdrop"></div>
<div id="toast-container"></div>
<script src="/static/app.js"></script>
</body>
</html>

View File

@@ -0,0 +1,134 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Integrations</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
details.channel-block summary::-webkit-details-marker { display: none; }
details.channel-block summary::before {
content: '▶'; font-size: 0.65rem; color: var(--pg-dimmer);
transition: transform 0.15s; flex-shrink: 0;
}
details.channel-block[open] summary::before { transform: rotate(90deg); }
details.channel-block[open] summary { border-bottom: 1px solid var(--pg-border); }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
<a href="/settings/integrations" class="nav-link active">Integrations</a>
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Integrations</h1>
<p class="page-subtitle">External service connections — admin only.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<form method="POST" action="/settings/integrations">
<div class="section">
<h2>Aether Platform Database</h2>
<p class="section-note">
Gives the orchestrator direct read-only access to the Aether MariaDB via the
<code>ae_db_query</code>, <code>ae_db_describe</code>, and <code>ae_db_show_view</code> tools.
Only SELECT, SHOW, DESCRIBE, and EXPLAIN are permitted — no writes possible.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ ae_db_host and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Connection
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer mb-4 -mt-1 leading-relaxed">
Use the same credentials as
<code class="font-mono text-pg-accent bg-pg-bg border border-pg-border rounded px-1 text-xs">agents_sync/mcp/scripts/sql_inspector.py</code>.
Leave the password blank to keep the stored value.
</p>
<div class="grid grid-cols-[1fr_7rem] gap-3 items-start">
<div class="field">
<label for="ae_db_host">Host</label>
<input type="text" id="ae_db_host" name="ae_db_host"
value="{{ ae_db_host }}"
placeholder="192.168.64.5"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ae_db_port">Port</label>
<input type="number" id="ae_db_port" name="ae_db_port"
value="{{ ae_db_port }}"
placeholder="3306" min="1" max="65535"
autocomplete="off">
</div>
</div>
<div class="field">
<label for="ae_db_name">Database name</label>
<input type="text" id="ae_db_name" name="ae_db_name"
value="{{ ae_db_name }}"
placeholder="aether_dev"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ae_db_user">Username</label>
<input type="text" id="ae_db_user" name="ae_db_user"
value="{{ ae_db_user }}"
placeholder="aether_dev"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ae_db_password">Password</label>
<input type="password" id="ae_db_password" name="ae_db_password"
value=""
placeholder="Leave blank to keep existing value"
autocomplete="new-password" spellcheck="false">
</div>
</div>
</details>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save integrations</button>
</form>
</div>
</body>
</html>

1055
cortex/static/local_llm.html Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -4,6 +4,9 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Sign In</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
@@ -13,7 +16,10 @@
align-items: center;
justify-content: center;
background: #0f1117;
font-family: system-ui, -apple-system, sans-serif;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
color: #e2e8f0;
}
@@ -40,7 +46,7 @@
.logo p {
font-size: 0.8rem;
color: #64748b;
color: #94a3b8;
margin-top: 0.25rem;
}
@@ -84,6 +90,40 @@
button[type="submit"]:hover { background: #6d28d9; }
.divider {
display: flex;
align-items: center;
gap: 0.75rem;
margin: 1.25rem 0;
color: #475569;
font-size: 0.78rem;
}
.divider::before, .divider::after {
content: '';
flex: 1;
border-top: 1px solid #2d3148;
}
.google-btn {
display: flex;
align-items: center;
justify-content: center;
gap: 0.6rem;
width: 100%;
padding: 0.65rem;
background: #fff;
border: 1px solid #dadce0;
border-radius: 6px;
color: #3c4043;
font-size: 0.95rem;
font-weight: 500;
font-family: inherit;
cursor: pointer;
text-decoration: none;
transition: background 0.15s, box-shadow 0.15s;
}
.google-btn:hover { background: #f8f9fa; box-shadow: 0 1px 4px rgba(0,0,0,0.2); }
.error {
color: #f87171;
font-size: 0.85rem;
@@ -101,6 +141,18 @@
<!-- ERROR -->
<a href="/auth/google" class="google-btn">
<svg width="18" height="18" viewBox="0 0 18 18" xmlns="http://www.w3.org/2000/svg">
<path d="M17.64 9.2c0-.637-.057-1.251-.164-1.84H9v3.481h4.844c-.209 1.125-.843 2.078-1.796 2.717v2.258h2.908c1.702-1.567 2.684-3.875 2.684-6.615z" fill="#4285F4"/>
<path d="M9 18c2.43 0 4.467-.806 5.956-2.18l-2.908-2.259c-.806.54-1.837.86-3.048.86-2.344 0-4.328-1.584-5.036-3.711H.957v2.332A8.997 8.997 0 0 0 9 18z" fill="#34A853"/>
<path d="M3.964 10.71A5.41 5.41 0 0 1 3.682 9c0-.593.102-1.17.282-1.71V4.958H.957A8.996 8.996 0 0 0 0 9c0 1.452.348 2.827.957 4.042l3.007-2.332z" fill="#FBBC05"/>
<path d="M9 3.58c1.321 0 2.508.454 3.44 1.345l2.582-2.58C13.463.891 11.426 0 9 0A8.997 8.997 0 0 0 .957 4.958L3.964 7.29C4.672 5.163 6.656 3.58 9 3.58z" fill="#EA4335"/>
</svg>
Sign in with Google
</a>
<div class="divider">or</div>
<form method="POST" action="/login">
<div class="field">
<label for="username">Username</label>

View File

@@ -0,0 +1,30 @@
{
"name": "Cortex · Inara",
"short_name": "Cortex",
"description": "Personal AI assistant",
"start_url": "/",
"scope": "/",
"display": "standalone",
"background_color": "#1a1228",
"theme_color": "#1a1228",
"icons": [
{
"src": "/static/icon-192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "any"
},
{
"src": "/static/icon-512.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "any maskable"
},
{
"src": "/static/icon.svg",
"sizes": "any",
"type": "image/svg+xml",
"purpose": "any"
}
]
}

View File

@@ -0,0 +1,348 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Notifications</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Channel collapsible arrow ── */
details.channel-block summary::-webkit-details-marker { display: none; }
details.channel-block summary::before {
content: '▶'; font-size: 0.65rem; color: var(--pg-dimmer);
transition: transform 0.15s; flex-shrink: 0;
}
details.channel-block[open] summary::before { transform: rotate(90deg); }
details.channel-block[open] summary { border-bottom: 1px solid var(--pg-border); }
/* ── Test result feedback (JS-toggled display) ── */
#test-result { display: none; }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link active">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Notifications</h1>
<p class="page-subtitle">How your persona reaches out proactively — reminders, cron jobs, and memory digests.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<form method="POST" action="/settings/notifications">
<!-- Channel selector -->
<div class="section">
<h2>Channel</h2>
<div class="field">
<label for="notification_channel">Default outbound channel</label>
<select id="notification_channel" name="notification_channel"
data-value="{{ notify_channel }}">
<option value="">None (disabled)</option>
<option value="web_push">Browser Push Notification</option>
<option value="email">Email</option>
<option value="nextcloud">Nextcloud Talk</option>
<option value="google_chat">Google Chat</option>
</select>
<p class="hint">Used for reminder alerts, distillation summaries, and cron job notifications.</p>
</div>
<div class="field">
<label for="notification_email">
Email address override
<span class="font-normal text-pg-dim">(optional)</span>
</label>
<input type="email" id="notification_email" name="notification_email"
value="{{ notify_email_override }}"
placeholder="Leave blank to use your login email"
autocomplete="off">
</div>
</div>
<!-- Nextcloud Talk -->
<div class="section">
<h2>Nextcloud Talk</h2>
<p class="section-note">
Configure to send and receive messages via your Nextcloud Talk bot.
<strong>Sending</strong> requires the bot URL, secret, and notification room.
<strong>Reading history</strong> (<code>nc_talk_history</code> tool) additionally
requires a Nextcloud username and app password.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ nc_url and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Bot credentials (sending)
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
Set these up in your Nextcloud Talk room → Bot settings.
See the <a href="/help" class="text-pg-accent">setup guide</a> for step-by-step instructions.
</p>
<div class="field">
<label for="nc_url">Nextcloud URL</label>
<input type="url" id="nc_url" name="nc_url"
value="{{ nc_url }}"
placeholder="https://cloud.example.com"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="nc_bot_secret">Bot secret</label>
<input type="password" id="nc_bot_secret" name="nc_bot_secret"
value="{{ nc_bot_secret }}"
placeholder="Leave blank to keep existing value"
autocomplete="new-password" spellcheck="false">
<p class="hint">Generated when you registered the bot in Nextcloud Talk.</p>
</div>
<div class="field">
<label for="nc_notification_room">Notification room token</label>
<input type="text" id="nc_notification_room" name="nc_notification_room"
value="{{ nc_notify_room }}"
placeholder="Token from the Talk room URL"
autocomplete="off" spellcheck="false">
<p class="hint">The token at the end of the Talk room URL — e.g. <code>abc123def</code>.</p>
</div>
</div>
</details>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ nc_username and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
API credentials (reading history)
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
Required for the <code>nc_talk_history</code> orchestrator tool.
Generate an app password in Nextcloud → Settings → Security → App passwords.
</p>
<div class="field">
<label for="nc_username">Nextcloud username</label>
<input type="text" id="nc_username" name="nc_username"
value="{{ nc_username }}"
placeholder="Your Nextcloud login username"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="nc_app_password">App password</label>
<input type="password" id="nc_app_password" name="nc_app_password"
value="{{ nc_app_password }}"
placeholder="Leave blank to keep existing value"
autocomplete="new-password" spellcheck="false">
</div>
</div>
</details>
</div>
<!-- Home Assistant -->
<div class="section">
<h2>Home Assistant</h2>
<p class="section-note">
Receive events from HA automations and let your persona call the HA REST API
(read states, control devices). Webhook ID is the shared secret used in your
HA <code>rest_command</code> URL.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ ha_url and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Connection
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
HA URL and a Long-Lived Access Token (Profile → scroll to bottom →
Long-Lived Access Tokens → Create Token).
</p>
<div class="field">
<label for="ha_url">Home Assistant URL</label>
<input type="url" id="ha_url" name="ha_url"
value="{{ ha_url }}"
placeholder="https://ha.yourdomain.com"
autocomplete="off" spellcheck="false">
</div>
<div class="field">
<label for="ha_token">Long-Lived Access Token</label>
<input type="password" id="ha_token" name="ha_token"
value=""
placeholder="Leave blank to keep existing token"
autocomplete="new-password" spellcheck="false">
</div>
</div>
</details>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ ha_webhook_id and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Inbound webhook (HA → Cortex)
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
The webhook ID is the shared secret in your HA <code>rest_command</code> URL.
Your endpoint: <code>https://cortex.dgrzone.com/webhook/ha/{{ ha_username }}/&lt;webhook_id&gt;</code>
</p>
<div class="field">
<label for="ha_webhook_id">Webhook ID</label>
<input type="text" id="ha_webhook_id" name="ha_webhook_id"
value="{{ ha_webhook_id }}"
placeholder="Paste or generate a random secret"
autocomplete="off" spellcheck="false">
<p class="hint">Treat this like a password — use a long, random string.</p>
</div>
<div class="field">
<label class="checkbox-label">
<input type="checkbox" name="ha_tools" value="1" {{ ha_tools_checked }}>
Enable orchestrator tools
</label>
<p class="hint">When checked, HA events trigger the full tool loop (research, home control, tasks). When unchecked, events get a direct LLM response — faster but no tools.</p>
</div>
</div>
</details>
</div>
<!-- Google Chat -->
<div class="section">
<h2>Google Chat</h2>
<p class="section-note">
Outbound webhook for proactive messages to a Google Chat space.
Incoming messages are handled separately via the Google Chat Add-on.
</p>
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
{{ gc_webhook and 'open' or '' }}>
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
Outbound webhook
</summary>
<div class="px-4 pt-4 pb-2">
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
Create a webhook in your Google Chat space → Manage webhooks. Paste the full URL here.
</p>
<div class="field">
<label for="gc_outbound_webhook">Webhook URL</label>
<input type="url" id="gc_outbound_webhook" name="gc_outbound_webhook"
value="{{ gc_webhook }}"
placeholder="https://chat.googleapis.com/v1/spaces/…"
autocomplete="off" spellcheck="false">
</div>
</div>
</details>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save notification settings</button>
</form>
<!-- Test -->
<div class="section" style="margin-top:2rem;">
<h2>Test</h2>
<p class="section-note">
Fire a notification via your configured channel or run the reminder check
immediately — no need to wait for the daily 09:00 scheduler job.
</p>
<div class="flex gap-3 mt-2">
<button class="flex-1 px-3 py-2.5 text-sm font-medium border border-pg-border rounded-md bg-pg-bg text-pg-text hover:border-pg-action hover:text-pg-accent transition-colors cursor-pointer disabled:opacity-50"
id="btn-test-notify">Send Test Notification</button>
<button class="flex-1 px-3 py-2.5 text-sm font-medium border border-pg-border rounded-md bg-pg-bg text-pg-text hover:border-pg-action hover:text-pg-accent transition-colors cursor-pointer disabled:opacity-50"
id="btn-check-reminders">Check Reminders Now</button>
</div>
<div id="test-result"
class="mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed"></div>
</div>
</div>
<script>
// Set channel select to saved value
const sel = document.getElementById('notification_channel');
if (sel) {
const saved = sel.dataset.value;
if (saved) {
for (const opt of sel.options) {
if (opt.value === saved) { opt.selected = true; break; }
}
}
}
// Test buttons
const resultEl = document.getElementById('test-result');
function showResult(ok, msg) {
resultEl.textContent = msg;
resultEl.className = ok
? 'mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed bg-green-950 text-green-400 border border-green-800'
: 'mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed bg-red-950 text-red-400 border border-red-800';
resultEl.style.display = 'block';
}
async function apiPost(url, btnEl, label) {
btnEl.disabled = true;
btnEl.textContent = label + '…';
resultEl.style.display = 'none';
try {
const r = await fetch(url, { method: 'POST' });
const data = await r.json();
if (r.ok && data.ok) {
if (url.includes('reminders')) {
const n = data.reminders_found ?? 0;
showResult(true, n > 0
? `Found ${n} due reminder${n !== 1 ? 's' : ''} — notification sent.`
: 'No due reminders found — nothing sent.');
} else {
showResult(true, 'Notification sent. Check your configured channel.');
}
} else {
showResult(false, data.detail || 'Request failed.');
}
} catch (e) {
showResult(false, 'Network error: ' + e.message);
} finally {
btnEl.disabled = false;
btnEl.textContent = label;
}
}
document.getElementById('btn-test-notify').addEventListener('click', function() {
apiPost('/api/push/test', this, 'Send Test Notification');
});
document.getElementById('btn-check-reminders').addEventListener('click', function() {
apiPost('/api/push/reminders/check', this, 'Check Reminders Now');
});
</script>
</body>
</html>

189
cortex/static/pg.css Normal file
View File

@@ -0,0 +1,189 @@
/* ─── Cortex settings pages — shared stylesheet ───────────────────────────── */
/* ── Variables ── */
:root {
--pg-bg: #0f1117; --pg-surface: #1a1d27; --pg-border: #2d3148;
--pg-text: #e2e8f0; --pg-muted: #94a3b8;
--pg-dim: #64748b; --pg-dimmer: #475569;
--pg-bright: #cbd5e1; --pg-nav-hover: rgba(255,255,255,0.05);
--pg-accent: #a78bfa; /* heading/highlight purple */
--pg-action: #7c3aed; /* button/focus purple */
}
[data-theme="light"] {
--pg-bg: #f4f2fa; --pg-surface: #ffffff; --pg-border: #d0c8e8;
--pg-text: #1a1228; --pg-muted: #5a5478;
--pg-dim: #7a7290; --pg-dimmer: #9e98b0;
--pg-bright: #1a1228; --pg-nav-hover: rgba(0,0,0,0.05);
--pg-accent: #7c3aed;
--pg-action: #6d28d9;
}
/* ── Reset ── */
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
/* ── Base ── */
body {
min-height: 100vh;
background: var(--pg-bg);
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
color: var(--pg-text);
}
/* ── Top nav ── */
.page-nav {
display: flex; align-items: center; gap: 0.25rem;
padding: 0.5rem 1rem; background: var(--pg-surface);
border-bottom: 1px solid var(--pg-border); flex-wrap: wrap;
}
.nav-link {
display: inline-flex; align-items: center;
padding: 0.3rem 0.6rem; border-radius: 6px;
font-size: 0.8rem; font-weight: 500; color: var(--pg-dim);
text-decoration: none; transition: color 0.15s, background 0.15s;
white-space: nowrap;
}
.nav-link:hover { color: var(--pg-bright); background: var(--pg-nav-hover); }
.nav-link.active { color: var(--pg-accent); }
.nav-spacer { flex: 1; min-width: 0.5rem; }
.nav-link.nav-logout { color: var(--pg-dimmer); }
.nav-link.nav-logout:hover { color: var(--pg-muted); background: none; }
/* ── Page container ── */
.page-wrap {
max-width: 860px; margin: 0 auto;
padding: 2rem 1.5rem 4rem; width: 100%;
}
/* ── Page heading ── */
.page-title {
font-size: 1.4rem; font-weight: 700; color: var(--pg-accent);
}
.page-subtitle {
font-size: 0.8rem; color: var(--pg-muted);
margin-top: 0.2rem; margin-bottom: 1.75rem; line-height: 1.5;
}
/* ── Sections (settings-style, bottom-bordered h2) ── */
.section { margin-bottom: 2rem; }
.section > h2 {
font-size: 0.9rem; font-weight: 600; color: var(--pg-muted);
margin-bottom: 1rem; padding-bottom: 0.4rem;
border-bottom: 1px solid var(--pg-border);
}
/* ── Form elements ── */
.field { margin-bottom: 1rem; }
label {
display: block; font-size: 0.8rem; font-weight: 500;
color: var(--pg-muted); margin-bottom: 0.4rem;
}
input, select, textarea {
width: 100%; padding: 0.65rem 0.85rem;
background: var(--pg-bg); border: 1px solid var(--pg-border);
border-radius: 6px; color: var(--pg-text); font-size: 0.95rem;
font-family: inherit; outline: none; transition: border-color 0.15s;
}
input:focus, select:focus, textarea:focus { border-color: var(--pg-action); }
input[readonly] { color: var(--pg-muted); cursor: default; }
input[type="password"] { font-family: monospace; letter-spacing: 0.05em; }
input[type="checkbox"], input[type="radio"] { width: auto; padding: 0; }
textarea {
font-family: 'SF Mono', 'Fira Mono', 'Menlo', monospace;
font-size: 0.88rem; line-height: 1.55; resize: vertical;
}
/* ── Buttons ── */
/* Primary form submit */
.btn-submit {
padding: 0.6rem 1.5rem; margin-top: 0.25rem;
background: var(--pg-action); border: none; border-radius: 6px;
color: #fff; font-size: 0.9rem; font-weight: 600;
cursor: pointer; transition: opacity 0.15s;
}
.btn-submit:hover { opacity: 0.88; }
/* Compact inline primary (e.g. rename save, inline forms) */
.btn-save {
padding: 0.4rem 0.9rem; background: var(--pg-action); border: none;
border-radius: 6px; color: #fff; font-size: 0.9rem;
font-weight: 600; cursor: pointer; transition: opacity 0.15s;
}
.btn-save:hover { opacity: 0.88; }
/* Outline secondary (e.g. clear cache, cancel, test actions) */
.btn-secondary {
padding: 0.5rem 1rem; background: none;
border: 1px solid var(--pg-border); border-radius: 6px;
color: var(--pg-muted); font-size: 0.88rem; font-weight: 500;
cursor: pointer; transition: border-color 0.15s, color 0.15s;
}
.btn-secondary:hover { border-color: var(--pg-muted); color: var(--pg-text); }
.btn-secondary:disabled { opacity: 0.5; cursor: default; }
/* Inline cancel */
.btn-cancel {
padding: 0.4rem 0.75rem; background: none;
border: 1px solid var(--pg-border); border-radius: 6px;
color: var(--pg-muted); font-size: 0.9rem;
cursor: pointer; transition: border-color 0.15s, color 0.15s;
}
.btn-cancel:hover { border-color: var(--pg-muted); color: var(--pg-text); }
/* Button-styled link (purple, used for "Settings →" style CTAs) */
.action-link {
display: inline-block; padding: 0.5rem 1rem;
background: var(--pg-action); border-radius: 6px;
color: #fff; font-size: 0.88rem; font-weight: 600;
text-decoration: none; transition: opacity 0.15s;
}
.action-link:hover { opacity: 0.88; }
/* Inline button row */
.btn-row { display: flex; gap: 0.5rem; margin-top: 0.5rem; }
/* ── Text utilities ── */
/* Small muted helper text below inputs */
.hint { font-size: 0.78rem; color: var(--pg-dim); margin-top: 0.35rem; line-height: 1.5; }
/* Section-level description paragraph */
.section-note { font-size: 0.8rem; color: var(--pg-muted); margin-bottom: 0.85rem; line-height: 1.55; }
/* Inline code */
code {
font-size: 0.82rem; font-family: 'SF Mono', 'Fira Mono', 'Menlo', monospace;
background: var(--pg-bg); border: 1px solid var(--pg-border);
padding: 0.1rem 0.35rem; border-radius: 4px; color: var(--pg-accent);
}
/* ── Feedback messages ── */
.success { color: #4ade80; font-size: 0.85rem; text-align: center; margin-bottom: 1rem; }
.error { color: #f87171; font-size: 0.85rem; text-align: center; margin-bottom: 1rem; }
/* ── Usage table (JS-rendered in settings) ── */
.usage-table { border-collapse: collapse; width: 100%; min-width: 360px; }
.usage-table th {
padding: 0.35rem 0.5rem; font-size: 0.75rem; color: var(--pg-muted);
font-weight: 600; text-align: right; border-bottom: 1px solid var(--pg-border);
}
.usage-table th:first-child { padding-left: 0; text-align: left; }
.usage-table td {
padding: 0.4rem 0.5rem; font-size: 0.82rem; color: var(--pg-muted); text-align: right;
}
.usage-table td:first-child { padding-left: 0; color: var(--pg-text); text-align: left; white-space: nowrap; }
.usage-table td:last-child { color: var(--pg-text); font-weight: 600; }
/* ── Tool category header row (tools_settings.py generated) ── */
.tool-cat-row td {
padding: 0.75rem 0.9rem 0.3rem;
font-size: 0.72rem; font-weight: 700; letter-spacing: 0.07em;
text-transform: uppercase; color: var(--pg-dimmer);
border-bottom: 1px solid var(--pg-border);
}

397
cortex/static/settings.html Normal file
View File

@@ -0,0 +1,397 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Account Settings</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Server-generated persona list ── */
.persona-list {
list-style: none; display: flex; flex-direction: column;
gap: 0.5rem; margin-top: 0.5rem;
}
.persona-list li { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
.persona-link {
display: inline-block; padding: 0.3rem 0.75rem;
background: var(--pg-bg); border: 1px solid var(--pg-border);
border-radius: 20px; color: var(--pg-accent); font-size: 0.85rem;
text-decoration: none; transition: border-color 0.15s;
}
.persona-link:hover { border-color: var(--pg-action); }
.persona-list li em { color: var(--pg-muted); font-size: 0.85rem; }
.persona-rename-toggle {
background: none; border: 1px solid var(--pg-border);
border-radius: 6px; color: var(--pg-muted); font-size: 0.8rem;
padding: 0.3rem 0.6rem; margin-top: 0.25rem;
cursor: pointer; opacity: 0.7; transition: opacity 0.15s, color 0.15s;
}
.persona-rename-toggle:hover { opacity: 1; color: var(--pg-accent); }
.persona-rename-form { display: flex; align-items: center; gap: 0.4rem; }
.persona-rename-form input[type="text"] {
width: 12rem; padding: 0.3rem 0.6rem;
border-color: var(--pg-action); font-size: 0.9rem;
}
.persona-rename-form .btn-save { padding: 0.3rem 0.75rem; font-size: 0.85rem; }
/* ── Server-generated role badge ── */
.role-badge {
display: inline-block; padding: 0.25rem 0.75rem;
border-radius: 20px; font-size: 0.78rem; font-weight: 600;
text-transform: uppercase; letter-spacing: 0.06em;
}
.role-badge.role-admin {
background: rgba(124,58,237,0.15); color: var(--pg-accent);
border: 1px solid rgba(124,58,237,0.4);
}
.role-badge.role-user {
background: rgba(100,116,139,0.12); color: var(--pg-muted);
border: 1px solid var(--pg-border);
}
/* ── JS-toggled states ── */
#clear-ls-ok { display: none; margin-left: 0.75rem; font-size: 0.8rem; color: #4ade80; }
.usage-wrap { overflow-x: auto; }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link active">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Account Settings</h1>
<p class="page-subtitle">Manage your account and personas.</p>
<!-- SUCCESS -->
<!-- ERROR -->
<!-- OpenRouter quickstart (shown by JS when no model is configured) -->
<div id="openrouter-quickstart"
class="hidden rounded-xl border border-amber-800 bg-amber-950 p-4 mb-5">
<p class="text-xs font-semibold text-amber-400 mb-1">⚡ You're on the server default model</p>
<p class="text-xs text-amber-600 mb-3 leading-relaxed">
You can chat now, but adding your own model gives you more choices, lets you pick
role-specific models, and tracks your usage separately.
OpenRouter is the easiest way to get started — one key, many models.
</p>
<a href="/setup/model"
class="inline-block px-3 py-2 rounded-md bg-amber-900 text-amber-100 text-sm font-medium hover:bg-amber-800 transition-colors">
Set up OpenRouter →
</a>
</div>
<!-- Account info -->
<div class="section">
<h2>Account</h2>
<div class="field">
<label>Username</label>
<input type="text" value="{{ username }}" readonly>
</div>
<div class="field">
<label>Role</label>
<span class="role-badge role-{{ user_role }}">{{ user_role }}</span>
</div>
<button type="button" id="show-rename-user" class="persona-rename-toggle">
✏ Change username
</button>
<form id="rename-user-form" method="POST" action="/settings/username" style="display:none; margin-top:0.75rem;">
<div class="field">
<label for="new_username">New username</label>
<input type="text" id="new_username" name="new_username"
value="{{ username }}"
pattern="[a-z_][a-z0-9_\-]{0,31}" required autofocus
autocomplete="off" data-form-type="other">
<p class="hint">Lowercase letters, digits, _ or - only. You will be logged out after renaming.</p>
</div>
<div class="btn-row">
<button type="submit" class="btn-save">Save</button>
<button type="button" id="cancel-rename-user" class="btn-cancel">Cancel</button>
</div>
</form>
</div>
<!-- Connected accounts -->
<div class="section">
<h2>Connected Accounts</h2>
<div class="field">
<label>Google Account</label>
<input type="text" value="{{ google_email }}" readonly
placeholder="No Google account linked"
style="{{ google_email == '' and 'color:var(--pg-dimmer)' or '' }}">
</div>
<p class="hint" style="margin-top:-0.5rem;">To link or change your Google account, contact Scott.</p>
</div>
<!-- Email Allowlist -->
<div class="section">
<h2>Email Allowlist</h2>
<p class="section-note">
One regex pattern per line. The <code>email_send</code> tool will only send to addresses
that match at least one pattern. Leave blank to block all outbound email.
</p>
<form method="POST" action="/settings/email-allowlist">
<div class="field">
<label for="email_allowlist_ta">Allowed patterns</label>
<textarea id="email_allowlist_ta" name="patterns" rows="6"
placeholder=".*@example\.com&#10;alice@example\.com"
spellcheck="false">{{ email_allowlist }}</textarea>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save allowlist</button>
</form>
</div>
<!-- HTTP POST Allowlist -->
<div class="section">
<h2>HTTP POST Allowlist</h2>
<p class="section-note">
One URL prefix per line. The <code>http_post</code> tool will only POST to URLs that
start with a listed prefix. Leave blank to block all outbound POST requests.
</p>
<form method="POST" action="/settings/http-allowlist">
<div class="field">
<label for="http_allowlist_ta">Allowed URL prefixes</label>
<textarea id="http_allowlist_ta" name="prefixes" rows="5"
placeholder="https://ha.dgrzone.com/api/webhook/&#10;https://n8n.dgrzone.com/webhook/"
spellcheck="false">{{ http_allowlist }}</textarea>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Save allowlist</button>
</form>
</div>
<!-- Usage summary -->
<div class="section" id="usage-section">
<h2>Usage</h2>
<p class="section-note">
Token consumption tracked for API-backed models (Gemini API, local OpenAI-compatible).
Claude CLI calls are not metered.
</p>
<div id="usage-table-wrap" class="usage-wrap">
<p class="section-note">Loading…</p>
</div>
</div>
<!-- Browser Cache -->
<div class="section">
<h2>Browser Cache</h2>
<p class="section-note">
Clears UI preferences stored in this browser: active mode, session ID, memory toggles,
theme, font size, and context tier. Does not sign you out.
</p>
<button type="button" id="clear-ls-btn" class="btn-secondary">Clear browser cache</button>
<span id="clear-ls-ok">Cleared.</span>
</div>
<!-- Change Password -->
<div class="section">
<h2>Change Password</h2>
<form method="POST" action="/settings/password" id="password-form">
<div class="field">
<label for="current_password">Current password</label>
<input type="password" id="current_password" name="current_password"
autocomplete="current-password" required>
</div>
<div class="field">
<label for="new_password">New password</label>
<input type="password" id="new_password" name="new_password"
autocomplete="new-password" required minlength="8">
</div>
<div class="field">
<label for="confirm_password">Confirm new password</label>
<input type="password" id="confirm_password" name="confirm_password"
autocomplete="new-password" required>
</div>
<button type="submit" class="btn-submit w-full md:w-96">Update password</button>
</form>
</div>
<!-- Sessions -->
<div class="section">
<h2>Sessions</h2>
<p class="section-note">
Auto-name any sessions that still show a random ID, using their first message as the name.
Only unnamed sessions are affected — existing names are left alone.
</p>
<button type="button" id="backfill-names-btn" class="btn-secondary">Auto-name old sessions</button>
<span id="backfill-names-ok"
class="ml-3 text-xs hidden"
style="color:#4ade80"></span>
</div>
<!-- Personas -->
<div class="section">
<h2>Personas</h2>
<ul class="persona-list">
{{ persona_items }}
</ul>
<a href="/setup/persona"
class="inline-block mt-3 text-xs text-pg-muted hover:text-pg-accent transition-colors">
+ Add new persona
</a>
</div>
</div>
<script>
// Password confirmation check
document.getElementById('password-form').addEventListener('submit', e => {
const np = document.getElementById('new_password').value;
const cfm = document.getElementById('confirm_password').value;
if (np !== cfm) {
e.preventDefault();
alert('New passwords do not match.');
}
});
// Username rename toggle
document.getElementById('show-rename-user').addEventListener('click', () => {
document.getElementById('show-rename-user').style.display = 'none';
document.getElementById('rename-user-form').style.display = 'block';
document.getElementById('new_username').focus();
});
document.getElementById('cancel-rename-user').addEventListener('click', () => {
document.getElementById('rename-user-form').style.display = 'none';
document.getElementById('show-rename-user').style.display = '';
});
// Clear localStorage (keeps JWT cookie — no sign-out)
document.getElementById('clear-ls-btn').addEventListener('click', () => {
localStorage.clear();
document.getElementById('clear-ls-ok').style.display = 'inline';
});
// Show OpenRouter quick-start card if no model is configured
(async () => {
try {
const d = await fetch('/backend').then(r => r.json());
if ((d.available_roles || []).length === 0) {
const el = document.getElementById('openrouter-quickstart');
el.classList.remove('hidden');
el.style.display = 'block';
}
} catch (_) {}
})();
// Usage summary table
(async () => {
const wrap = document.getElementById('usage-table-wrap');
try {
const resp = await fetch('/api/usage/summary');
if (!resp.ok) throw new Error(resp.statusText);
const rows_data = await resp.json();
if (!rows_data.length) {
wrap.innerHTML = '<p class="section-note">No usage recorded yet.</p>';
return;
}
const fmt = n => n >= 1000 ? (n / 1000).toFixed(1) + 'k' : String(n);
const rows = rows_data.map(d => {
const labelCell = d.label !== d.key
? `<span title="${d.key}">${d.label}</span>`
: `<span>${d.key}</span>`;
return `<tr>
<td>${labelCell}</td>
<td>${d.calls}</td>
<td>${fmt(d.prompt_tokens)}</td>
<td>${fmt(d.completion_tokens)}</td>
<td>${fmt(d.total_tokens)}</td>
</tr>`;
}).join('');
wrap.innerHTML = `<table class="usage-table">
<thead><tr>
<th style="text-align:left">Model</th>
<th>Calls</th><th>Prompt</th><th>Output</th><th>Total</th>
</tr></thead>
<tbody>${rows}</tbody>
</table>`;
} catch (e) {
wrap.innerHTML = '<p class="section-note">Could not load usage data.</p>';
}
})();
// Auto-name old sessions backfill
document.getElementById('backfill-names-btn').addEventListener('click', async () => {
const btn = document.getElementById('backfill-names-btn');
const ok = document.getElementById('backfill-names-ok');
btn.disabled = true;
btn.textContent = 'Working…';
try {
const params = new URLSearchParams(window.location.search);
const user = params.get('user') || document.querySelector('input[value]')?.value || '';
const persona = params.get('persona') || '';
const qs = user ? `?user=${encodeURIComponent(user)}&persona=${encodeURIComponent(persona)}` : '';
const res = await fetch(`/api/sessions/backfill-names${qs}`, { method: 'POST' });
const data = await res.json();
if (!res.ok) throw new Error(data.detail || res.statusText);
const n = data.named ?? 0;
ok.textContent = `Named ${n} session${n !== 1 ? 's' : ''}.`;
ok.style.display = 'inline';
ok.classList.remove('hidden');
} catch (e) {
ok.textContent = 'Error — check console.';
ok.style.color = '#f87171';
ok.style.display = 'inline';
ok.classList.remove('hidden');
}
btn.textContent = 'Auto-name old sessions';
btn.disabled = false;
});
// Persona rename toggle
document.querySelectorAll('.persona-rename-toggle').forEach(btn => {
btn.addEventListener('click', () => {
const p = btn.dataset.persona;
const form = document.querySelector(`.persona-rename-form[data-persona="${p}"]`);
btn.style.display = 'none';
form.style.display = 'flex';
form.querySelector('input[type="text"]').focus();
});
});
document.querySelectorAll('.persona-rename-cancel').forEach(btn => {
btn.addEventListener('click', () => {
const form = btn.closest('.persona-rename-form');
const p = form.dataset.persona;
const toggle = document.querySelector(`.persona-rename-toggle[data-persona="${p}"]`);
form.style.display = 'none';
toggle.style.display = '';
});
});
</script>
</body>
</html>

View File

@@ -4,6 +4,9 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cortex — Setup</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<style>
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
@@ -13,7 +16,10 @@
align-items: center;
justify-content: center;
background: #0f1117;
font-family: system-ui, -apple-system, sans-serif;
font-family: 'Inter', system-ui, -apple-system, sans-serif;
font-weight: 450;
-webkit-font-smoothing: antialiased;
-moz-osx-font-smoothing: grayscale;
color: #e2e8f0;
padding: 1.5rem;
}
@@ -33,7 +39,7 @@
}
.logo h1 { font-size: 1.6rem; font-weight: 700; letter-spacing: 0.05em; color: #a78bfa; }
.logo p { font-size: 0.8rem; color: #64748b; margin-top: 0.25rem; }
.logo p { font-size: 0.8rem; color: #94a3b8; margin-top: 0.25rem; }
h2 {
font-size: 1rem;
@@ -52,7 +58,7 @@
margin-bottom: 0.4rem;
}
label small { font-weight: 400; color: #475569; }
label small { font-weight: 400; color: #94a3b8; }
input, select {
width: 100%;
@@ -71,7 +77,7 @@
.field { margin-bottom: 1rem; }
.hint { font-size: 0.75rem; color: #475569; margin-top: 0.3rem; }
.hint { font-size: 0.75rem; color: #94a3b8; margin-top: 0.3rem; }
button[type="submit"] {
width: 100%;
@@ -98,7 +104,7 @@
.step-label {
font-size: 0.7rem;
color: #475569;
color: #94a3b8;
text-align: right;
margin-bottom: 1rem;
}
@@ -121,6 +127,36 @@
.emoji-opt.selected { border-color: #7c3aed; background: #2d1f52; }
#emoji-hidden { display: none; }
.provider-badge {
display: inline-flex;
align-items: center;
gap: 0.4rem;
background: #2d1f52;
border: 1px solid #7c3aed;
border-radius: 6px;
padding: 0.3rem 0.6rem;
font-size: 0.78rem;
color: #a78bfa;
margin-bottom: 1rem;
}
.skip-link {
display: block;
text-align: center;
margin-top: 1rem;
font-size: 0.8rem;
color: #64748b;
text-decoration: none;
}
.skip-link:hover { color: #94a3b8; }
.model-hint {
font-size: 0.72rem;
color: #64748b;
margin-top: 0.75rem;
text-align: center;
}
</style>
</head>
<body>
@@ -131,10 +167,11 @@
</div>
<!-- ERROR -->
<!-- ERROR_MODEL -->
<!-- ── Step 1: password ───────────────────────────────────────── -->
<div id="step-password">
<div class="step-label">Step 1 of 2</div>
<div class="step-label">Step 1 of 3</div>
<h2>Set your password</h2>
<form method="POST" action="" id="password-form">
<input type="hidden" name="step" value="password">
@@ -155,7 +192,7 @@
<!-- ── Step 2: persona ────────────────────────────────────────── -->
<div id="step-persona" style="display:none">
<div class="step-label">Step 2 of 2</div>
<div class="step-label">Step 2 of 3</div>
<h2>Create your persona</h2>
<form method="POST" action="" id="persona-form">
<input type="hidden" name="step" value="persona">
@@ -197,6 +234,39 @@
<button type="submit">Create my persona →</button>
</form>
</div>
<!-- ── Step 3: model connect ─────────────────────────────────── -->
<div id="step-model" style="display:none">
<div class="step-label"><!-- SETUP_STEP3_LABEL --></div>
<h2>Connect an AI model</h2>
<div class="provider-badge">⚡ Recommended: OpenRouter</div>
<p style="font-size:0.82rem;color:#94a3b8;margin-bottom:1rem;">
One API key gives you access to Claude, Gemini, Llama, and dozens of other models.
Get a free key at <a href="https://openrouter.ai/keys" target="_blank" style="color:#a78bfa;">openrouter.ai/keys</a>.
</p>
<form method="POST" action="/setup/model" id="model-form">
<div class="field">
<label for="api_key">OpenRouter API key</label>
<input type="password" id="api_key" name="api_key"
autocomplete="off" placeholder="sk-or-v1-..." required>
</div>
<div class="field">
<label for="model_name">Starting model</label>
<select id="model_name" name="model_name">
<option value="anthropic/claude-3-5-haiku-20241022">Claude 3.5 Haiku — Fast &amp; affordable</option>
<option value="anthropic/claude-3-7-sonnet-20250219">Claude 3.7 Sonnet — Smarter Claude</option>
<option value="google/gemini-2.0-flash-001">Gemini 2.0 Flash — Fast Google model</option>
<option value="meta-llama/llama-3.3-70b-instruct">Llama 3.3 70B — Open source</option>
</select>
<p class="hint">You can add more models or switch anytime in Account → Model Registry.</p>
</div>
<button type="submit">Connect &amp; start chatting →</button>
</form>
<p class="model-hint">
Using Ollama, a local model, or something else?
<a href="#" id="skip-model-link" style="color:#64748b;">Skip this step →</a>
</p>
</div>
</div>
<script>
@@ -226,6 +296,11 @@
document.getElementById('step-password').style.display = 'none';
document.getElementById('step-persona').style.display = 'block';
}
if (params.get('step') === '3') {
document.getElementById('step-password').style.display = 'none';
document.getElementById('step-persona').style.display = 'none';
document.getElementById('step-model').style.display = 'block';
}
// ── Client-side confirm password check ───────────────────────────
document.getElementById('password-form').addEventListener('submit', e => {
@@ -237,6 +312,15 @@
}
});
// ── Skip model setup — navigate to user home ─────────────────────
document.getElementById('skip-model-link')?.addEventListener('click', e => {
e.preventDefault();
// Ask server for skip target (the cx_setup_persona cookie has the path)
fetch('/setup/model/skip', { method: 'POST', credentials: 'same-origin' })
.then(r => { if (r.redirected) location.href = r.url; else location.href = '/'; })
.catch(() => { location.href = '/'; });
});
// ── Auto-generate persona slug from display name ─────────────────
document.getElementById('display_name').addEventListener('input', function() {
const slugField = document.getElementById('persona_name');

File diff suppressed because it is too large Load Diff

106
cortex/static/sw.js Normal file
View File

@@ -0,0 +1,106 @@
const CACHE = 'cortex-v2';
const PRECACHE = [
'/static/style.css',
'/static/app.js',
'/static/marked.min.js',
'/static/icon-192.png',
'/static/icon-512.png',
'/static/icon.svg',
'/static/manifest.json',
];
self.addEventListener('install', evt => {
evt.waitUntil(
caches.open(CACHE)
.then(c => c.addAll(PRECACHE))
.then(() => self.skipWaiting())
);
});
self.addEventListener('activate', evt => {
evt.waitUntil(
caches.keys()
.then(keys => Promise.all(
keys.filter(k => k !== CACHE).map(k => caches.delete(k))
))
.then(() => self.clients.claim())
);
});
self.addEventListener('push', evt => {
let data = { title: 'Cortex', body: '', url: '/' };
if (evt.data) {
try { data = { ...data, ...evt.data.json() }; } catch (_) {}
}
evt.waitUntil(
self.registration.showNotification(data.title, {
body: data.body,
icon: '/static/icon-192.png',
badge: '/static/icon-192.png',
data: { url: data.url },
})
);
});
self.addEventListener('notificationclick', evt => {
evt.notification.close();
const url = evt.notification.data?.url || '/';
evt.waitUntil(
clients.matchAll({ type: 'window', includeUncontrolled: true }).then(list => {
for (const c of list) {
if (c.url.includes(self.location.origin) && 'focus' in c) {
c.navigate(url);
return c.focus();
}
}
if (clients.openWindow) return clients.openWindow(url);
})
);
});
self.addEventListener('fetch', evt => {
const url = new URL(evt.request.url);
// Only handle same-origin GETs
if (evt.request.method !== 'GET' || url.origin !== self.location.origin) return;
// Never intercept streaming or API calls
if (
url.pathname.startsWith('/chat') ||
url.pathname.startsWith('/orchestrate') ||
url.pathname.startsWith('/api/') ||
url.pathname.startsWith('/distill') ||
url.pathname.startsWith('/webhook') ||
url.pathname.startsWith('/auth/')
) return;
// Static assets — cache first, refresh in background (stale-while-revalidate)
if (url.pathname.startsWith('/static/')) {
evt.respondWith(
caches.open(CACHE).then(cache =>
cache.match(evt.request).then(cached => {
const network = fetch(evt.request).then(resp => {
if (resp.ok) cache.put(evt.request, resp.clone());
return resp;
});
return cached || network;
})
)
);
return;
}
// HTML pages — network first, cached shell fallback
evt.respondWith(
fetch(evt.request)
.then(resp => {
if (resp.ok) {
const clone = resp.clone();
caches.open(CACHE).then(c => c.put(evt.request, clone));
}
return resp;
})
.catch(() => caches.match(evt.request))
);
});

View File

@@ -0,0 +1,213 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Tool Settings — Cortex</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
<script src="https://cdn.tailwindcss.com"></script>
<script>
tailwind.config = {
corePlugins: { preflight: false },
darkMode: ['selector', '[data-theme="dark"]'],
theme: {
extend: {
colors: {
pg: {
bg: 'var(--pg-bg)',
surface: 'var(--pg-surface)',
border: 'var(--pg-border)',
text: 'var(--pg-text)',
muted: 'var(--pg-muted)',
dim: 'var(--pg-dim)',
dimmer: 'var(--pg-dimmer)',
bright: 'var(--pg-bright)',
accent: 'var(--pg-accent)',
action: 'var(--pg-action)',
}
},
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
}
}
}
</script>
<link rel="stylesheet" href="/static/pg.css">
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
<style>
/* ── Server-generated tool table ── */
.table-section-label {
font-size: 0.7rem; font-weight: 700; letter-spacing: 0.08em;
text-transform: uppercase; color: var(--pg-dimmer);
margin: 1.75rem 0 0.6rem;
}
.tool-table {
width: 100%; border-collapse: collapse;
background: var(--pg-surface); border: 1px solid var(--pg-border);
border-radius: 0.75rem; overflow: hidden; margin-bottom: 0.5rem;
font-size: 0.85rem;
}
.tool-table th {
text-align: left; padding: 0.55rem 0.9rem;
border-bottom: 1px solid var(--pg-border);
color: var(--pg-muted); font-weight: 600; font-size: 0.78rem;
text-transform: uppercase; letter-spacing: 0.04em;
}
.tool-table td { padding: 0.5rem 0.9rem; border-bottom: 1px solid var(--pg-border); vertical-align: middle; }
.tool-table tr:last-child td { border-bottom: none; }
.tool-table tr:hover td { background: rgba(124,58,237,0.04); }
.tool-name { font-family: monospace; font-size: 0.82rem; }
/* Risk badges (server-generated) */
.risk { display: inline-block; font-size: 0.7rem; font-weight: 700;
padding: 0.15rem 0.45rem; border-radius: 9999px; letter-spacing: 0.04em; }
.risk-low { background: rgba(34,197,94,0.12); color: #4ade80; }
.risk-medium { background: rgba(234,179,8,0.12); color: #fbbf24; }
.risk-high { background: rgba(239,68,68,0.12); color: #f87171; }
[data-theme="light"] .risk-low { background: rgba(34,197,94,0.15); color: #16a34a; }
[data-theme="light"] .risk-medium { background: rgba(234,179,8,0.15); color: #ca8a04; }
[data-theme="light"] .risk-high { background: rgba(239,68,68,0.15); color: #dc2626; }
/* Auto-status pill (server-generated, updated by JS) */
.auto-pill {
display: inline-block; font-size: 0.68rem; font-weight: 600;
padding: 0.12rem 0.4rem; border-radius: 9999px;
}
.auto-on { background: rgba(124,58,237,0.12); color: #a78bfa; }
.auto-off { background: rgba(148,163,184,0.12); color: var(--pg-dimmer); }
[data-theme="light"] .auto-on { color: #7c3aed; }
/* Override select (server-generated) */
.override-sel {
font-size: 0.78rem; padding: 0.25rem 0.5rem;
border-radius: 0.3rem; min-width: 7rem; width: auto;
}
.override-sel.forced-on { border-color: #7c3aed; color: #7c3aed; }
.override-sel.forced-off { border-color: #dc2626; color: #dc2626; }
</style>
</head>
<body>
<nav class="page-nav">
<a href="{{ back_href }}" class="nav-link">← Chat</a>
<a href="{{ help_href }}" class="nav-link">Help</a>
<a href="/settings" class="nav-link">Settings</a>
<a href="/settings/models" class="nav-link">Models</a>
<a href="/settings/notifications" class="nav-link">Notifications</a>
<a href="/settings/tools" class="nav-link active">Tools</a>
<a href="/settings/crons" class="nav-link">Schedules</a>
{{ integrations_nav }}
<span class="nav-spacer"></span>
<a href="/logout" class="nav-link nav-logout">Sign out</a>
</nav>
<div class="page-wrap">
<h1 class="page-title">Tool Settings</h1>
<p class="page-subtitle">
Control which orchestrator tools are available. The risk level sets an automatic threshold;
whitelist and blacklist let you fine-tune individual tools beyond that.
</p>
<!-- SUCCESS -->
<!-- ERROR -->
<form method="POST" action="/settings/tools" id="tools-form">
<!-- Risk policy card -->
<div class="rounded-xl border border-pg-border bg-pg-surface p-5 mb-5">
<h2 class="text-sm font-semibold text-pg-bright mb-4">Risk Policy</h2>
<div class="flex items-center gap-4 flex-wrap mb-3">
<span class="text-sm font-medium text-pg-text min-w-[6rem]">Max risk level</span>
<select name="max_risk" id="max-risk-sel" class="w-auto">
<option value="" {{ sel_none }}>No filter — use all role-permitted tools</option>
<option value="low" {{ sel_low }}>Low — read-only and sandboxed tools only</option>
<option value="medium" {{ sel_medium }}>Medium — low + medium risk (recommended)</option>
<option value="high" {{ sel_high }}>High — all tools including destructive ones</option>
</select>
</div>
<p class="text-xs text-pg-muted leading-relaxed mb-2">
<strong class="text-pg-text">Low</strong> tools are read-only and sandboxed (web search, project file reads, HA status checks).<br>
<strong class="text-pg-text">Medium</strong> tools write to local data or send notifications to you (cron jobs, scratch, task management).<br>
<strong class="text-pg-text">High</strong> tools affect external systems or the host (shell exec, email, device control, service restart).
</p>
<p class="text-xs text-pg-muted leading-relaxed">
The <em>Auto</em> column below shows each tool's status at your current max risk level.
Use the override column to force-include or force-exclude individual tools.
</p>
</div>
<!-- Legend -->
<div class="flex gap-5 flex-wrap mb-4 text-xs text-pg-muted">
<span><span class="inline-block w-2 h-2 rounded-full bg-[#a78bfa] mr-1.5"></span>Auto-included by risk level</span>
<span><span class="inline-block w-2 h-2 rounded-full bg-pg-dimmer mr-1.5"></span>Auto-excluded by risk level</span>
</div>
<!-- Tool table (server-generated) -->
{{ tool_table_html }}
<!-- Confirmation gate card -->
<div class="rounded-xl border border-pg-border bg-pg-surface p-5 mt-5 mb-5">
<h2 class="text-sm font-semibold text-pg-bright mb-2">Confirmation Gate</h2>
<p class="text-xs text-pg-muted leading-relaxed mb-4">
Some tools require explicit confirmation before executing. Override the defaults here.<br>
Tools requiring confirmation by default: <code class="font-mono text-pg-accent bg-pg-bg border border-pg-border rounded px-1">{{ confirm_required_tools }}</code>
</p>
<div class="flex gap-6 flex-wrap items-start">
<div class="flex-1 min-w-[200px]">
<label class="block text-xs font-semibold text-pg-muted mb-1">Allow list — bypass confirmation</label>
<textarea name="allow_list" rows="4"
placeholder="reminders_clear&#10;cron_remove"
autocomplete="off" spellcheck="false">{{ tool_allow }}</textarea>
<p class="hint">One tool name per line. These tools skip the confirmation prompt.</p>
</div>
<div class="flex-1 min-w-[200px]">
<label class="block text-xs font-semibold text-pg-muted mb-1">Deny list — always block</label>
<textarea name="deny_list" rows="4"
placeholder="shell_exec&#10;file_write"
autocomplete="off" spellcheck="false">{{ tool_deny }}</textarea>
<p class="hint">These tools are always blocked regardless of risk policy.</p>
</div>
</div>
</div>
<div class="mt-4">
<button type="submit" class="btn-submit w-full md:w-96">Save tool settings</button>
</div>
</form>
</div>
<script>
const riskRank = { "": 99, "low": 0, "medium": 1, "high": 2 };
const toolRisk = {{ tool_risk_json }};
const sel = document.getElementById('max-risk-sel');
function updateAutoPills() {
const maxRank = riskRank[sel.value] ?? 99;
document.querySelectorAll('[data-tool-risk]').forEach(row => {
const risk = row.dataset.toolRisk;
const pill = row.querySelector('.auto-pill');
const isAuto = riskRank[risk] <= maxRank;
pill.textContent = isAuto ? 'auto ✓' : 'excluded';
pill.className = 'auto-pill ' + (isAuto ? 'auto-on' : 'auto-off');
});
}
sel.addEventListener('change', updateAutoPills);
updateAutoPills();
// Color the override selects
document.querySelectorAll('.override-sel').forEach(s => {
function refresh() {
s.className = 'override-sel';
if (s.value === 'whitelist') s.classList.add('forced-on');
if (s.value === 'blacklist') s.classList.add('forced-off');
}
s.addEventListener('change', refresh);
refresh();
});
</script>
</body>
</html>

View File

@@ -0,0 +1,876 @@
"""
Tests for agent_manager.py and the spawn_agent / aider_run background paths.
Run with:
cd cortex && .venv/bin/python -m pytest tests/test_agent_manager.py -v
No browser, no LLM calls, no Cortex service needed. All LLM interactions are mocked.
The agent_manager tests need no mocks at all — the module is pure asyncio.
"""
import asyncio
import pytest
import pytest_asyncio
from datetime import datetime, timedelta
from unittest.mock import AsyncMock, MagicMock, patch
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_mock_result(response: str = "Agent done."):
"""Build a mock OrchestratorResult returned by openai_orchestrator.run."""
r = MagicMock()
r.checkpoint = None
r.response = response
return r
def _mock_spawn_deps(
model_type: str = "local_openai",
user_role: str = "admin",
tool_policy: dict | None = None,
role_tools: list | None = None,
):
"""Return a context-manager stack that patches all spawn_agent external deps."""
if tool_policy is None:
tool_policy = {"allow": [], "deny": []}
model_cfg = {
"type": model_type,
"api_url": "http://localhost:3000",
"model_name": "test-model",
"api_key": "x",
}
role_cfg = {
"tools": role_tools,
"system_append": "",
"inject_datetime": True,
"inject_mode": True,
}
class _Stack:
def __enter__(self_):
self_._patches = [
patch("model_registry.get_role_config", return_value=role_cfg),
patch("model_registry.get_model_for_role", return_value=model_cfg),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("context_loader.load_context", return_value="Test system prompt"),
patch("auth_utils.get_user_role", return_value=user_role),
patch("auth_utils.get_tool_policy", return_value=tool_policy),
patch("persona.get_user", return_value="scott"),
]
for p in self_._patches:
p.start()
return self_
def __exit__(self_, *args):
for p in self_._patches:
p.stop()
return _Stack()
# ---------------------------------------------------------------------------
# Fixture — reset agent_manager state between tests
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def clear_agent_registry():
"""Wipe the in-process agent registry before each test."""
import agent_manager
agent_manager._agents.clear()
yield
agent_manager._agents.clear()
# ---------------------------------------------------------------------------
# agent_manager — core CRUD
# ---------------------------------------------------------------------------
class TestAgentManagerCore:
@pytest.mark.asyncio
async def test_register_creates_record(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="research", task="Investigate topic X", level=2
)
assert rec.agent_id in agent_manager._agents
assert rec.status == "running"
assert rec.level == 2
assert rec.role == "research"
assert rec.task == "Investigate topic X"
assert rec.user == "scott"
assert rec.finished is None
@pytest.mark.asyncio
async def test_register_truncates_long_task(self):
import agent_manager
long_task = "x" * 500
rec = await agent_manager.register(user="scott", role="chat", task=long_task, level=2)
assert len(rec.task) == 200
@pytest.mark.asyncio
async def test_finish_updates_record(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "All done!", "done")
updated = agent_manager.get(rec.agent_id)
assert updated.status == "done"
assert updated.result == "All done!"
assert updated.finished is not None
@pytest.mark.asyncio
async def test_finish_truncates_result(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "y" * 2000)
updated = agent_manager.get(rec.agent_id)
assert len(updated.result) <= agent_manager._RESULT_PREVIEW_CHARS
@pytest.mark.asyncio
async def test_finish_failed_status(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "Boom", "failed")
assert agent_manager.get(rec.agent_id).status == "failed"
@pytest.mark.asyncio
async def test_cancel_own_agent(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
msg = await agent_manager.cancel_agent(rec.agent_id, "scott")
assert "cancelled" in msg
assert agent_manager.get(rec.agent_id).status == "cancelled"
@pytest.mark.asyncio
async def test_cancel_wrong_user_denied(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
msg = await agent_manager.cancel_agent(rec.agent_id, "holly")
assert "denied" in msg.lower()
assert agent_manager.get(rec.agent_id).status == "running"
@pytest.mark.asyncio
async def test_cancel_nonexistent_agent(self):
import agent_manager
msg = await agent_manager.cancel_agent("does-not-exist", "scott")
assert "No agent found" in msg
@pytest.mark.asyncio
async def test_cancel_already_done(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "done", "done")
msg = await agent_manager.cancel_agent(rec.agent_id, "scott")
assert "already" in msg or "done" in msg
@pytest.mark.asyncio
async def test_cancel_kills_real_task(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
sleep_task = asyncio.create_task(asyncio.sleep(60))
agent_manager.set_task_ref(rec.agent_id, sleep_task)
await agent_manager.cancel_agent(rec.agent_id, "scott")
await asyncio.sleep(0) # let the event loop process the cancellation
assert sleep_task.cancelled() or sleep_task.done()
def test_list_agents_returns_users_agents(self):
import agent_manager
# Manually populate the registry
agent_manager._agents["a1"] = _make_record("a1", "scott", "running")
agent_manager._agents["a2"] = _make_record("a2", "scott", "done")
agent_manager._agents["a3"] = _make_record("a3", "holly", "running")
records = agent_manager.list_agents("scott")
ids = {r.agent_id for r in records}
assert "a1" in ids
assert "a2" in ids
assert "a3" not in ids
def test_list_agents_filters_by_status(self):
import agent_manager
agent_manager._agents["a1"] = _make_record("a1", "scott", "running")
agent_manager._agents["a2"] = _make_record("a2", "scott", "done")
running = agent_manager.list_agents("scott", status="running")
assert len(running) == 1
assert running[0].agent_id == "a1"
def test_list_agents_respects_limit(self):
import agent_manager
for i in range(20):
agent_manager._agents[f"a{i}"] = _make_record(f"a{i}", "scott", "done")
records = agent_manager.list_agents("scott", limit=5)
assert len(records) == 5
@pytest.mark.asyncio
async def test_prune_removes_old_completed(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "done")
# Manually backdate the finished time past the prune threshold
agent_manager._agents[rec.agent_id].finished = (
datetime.now() - agent_manager._PRUNE_AFTER - timedelta(seconds=1)
)
# Trigger pruning via a new registration
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
assert agent_manager.get(rec.agent_id) is None
@pytest.mark.asyncio
async def test_prune_keeps_running_agents(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
# Running agent — finished is None so it should never be pruned
assert rec.agent_id in agent_manager._agents
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
assert agent_manager.get(rec.agent_id) is not None
@pytest.mark.asyncio
async def test_finish_unknown_agent_is_noop(self):
import agent_manager
# Should not raise
await agent_manager.finish("ghost-id", "result", "done")
# ---------------------------------------------------------------------------
# agent_manager — notification hook
# ---------------------------------------------------------------------------
class TestAgentManagerNotify:
@pytest.mark.asyncio
async def test_notify_called_on_done(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=True
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "All good", "done")
mock_notify.assert_called_once()
call_args = mock_notify.call_args
assert call_args[0][0] == "scott" # user
assert "" in call_args[0][1] # success emoji
@pytest.mark.asyncio
async def test_notify_called_on_failed(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=True
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "Oops", "failed")
mock_notify.assert_called_once()
assert "⚠️" in mock_notify.call_args[0][1]
@pytest.mark.asyncio
async def test_no_notify_when_cancelled(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=True
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
mock_notify.assert_not_called()
@pytest.mark.asyncio
async def test_no_notify_when_flag_false(self):
import agent_manager
rec = await agent_manager.register(
user="scott", role="chat", task="t", level=2, notify=False
)
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
await agent_manager.finish(rec.agent_id, "Done", "done")
mock_notify.assert_not_called()
# ---------------------------------------------------------------------------
# spawn_agent — background mode
# ---------------------------------------------------------------------------
class TestSpawnAgentBackground:
@pytest.mark.asyncio
async def test_background_returns_agent_id_immediately(self):
import agent_manager
from tools.agents import spawn_agent
mock_result = _make_mock_result("Research complete.")
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
result = await spawn_agent(
task="Test background research",
role="research",
background=True,
)
assert "Agent started in background" in result
assert "ID:" in result
@pytest.mark.asyncio
async def test_background_registers_agent(self):
import agent_manager
from tools.agents import spawn_agent
mock_result = _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
await spawn_agent(task="Background task", background=True)
agents = agent_manager.list_agents("scott")
assert len(agents) >= 1
@pytest.mark.asyncio
async def test_background_agent_eventually_completes(self):
import agent_manager
from tools.agents import spawn_agent
mock_result = _make_mock_result("Task done!")
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
result = await spawn_agent(task="Quick task", background=True)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
# Poll while patches are still active
for _ in range(40):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
rec = agent_manager.get(agent_id)
assert rec is not None
assert rec.status == "done"
assert "Task done!" in (rec.result or "")
@pytest.mark.asyncio
async def test_background_sync_path_unchanged(self):
"""Verify that background=False still blocks and returns the result string."""
from tools.agents import spawn_agent
mock_result = _make_mock_result("Sync result here.")
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
result = await spawn_agent(task="Sync task", background=False)
assert result == "Sync result here."
@pytest.mark.asyncio
async def test_background_agent_timeout(self):
import agent_manager
from tools.agents import spawn_agent
async def _slow(*args, **kwargs):
await asyncio.sleep(60)
return _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", side_effect=_slow):
result = await spawn_agent(task="Slow task", background=True, timeout=1)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
# Poll while patches are still active (timeout=1s so this completes quickly)
for _ in range(60):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
rec = agent_manager.get(agent_id)
assert rec.status == "timeout"
@pytest.mark.asyncio
async def test_background_agent_failure(self):
import agent_manager
from tools.agents import spawn_agent
with _mock_spawn_deps():
with patch("openai_orchestrator.run", new_callable=AsyncMock, side_effect=RuntimeError("Boom")):
result = await spawn_agent(task="Failing task", background=True)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
for _ in range(20):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
assert agent_manager.get(agent_id).status == "failed"
# ---------------------------------------------------------------------------
# spawn_agent — level enforcement
# ---------------------------------------------------------------------------
class TestLevelEnforcement:
@pytest.mark.asyncio
async def test_l2_parent_denies_spawn_in_l3_child(self):
"""Level 2 agent spawning a child: spawn_agent and aider_run must be denied."""
from tools.agents import spawn_agent
captured_kwargs = {}
async def _capture_run(**kwargs):
captured_kwargs.update(kwargs)
return _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", side_effect=_capture_run):
await spawn_agent(
task="Test L3 enforcement",
background=False,
_agent_level=2, # this agent is Level 2; its child would be Level 3
)
# The orchestrator should have received spawn_agent and aider_run in confirm_deny
confirm_deny = captured_kwargs.get("confirm_deny", set())
assert "spawn_agent" in confirm_deny, "spawn_agent must be blocked for L3 children"
assert "aider_run" in confirm_deny, "aider_run must be blocked for L3 children"
@pytest.mark.asyncio
async def test_l1_parent_does_not_deny_spawn(self):
"""Level 1 agent (persona) spawning a Level 2 child: no extra denies."""
from tools.agents import spawn_agent
captured_kwargs = {}
async def _capture_run(**kwargs):
captured_kwargs.update(kwargs)
return _make_mock_result()
with _mock_spawn_deps():
with patch("openai_orchestrator.run", side_effect=_capture_run):
await spawn_agent(
task="Test L2 spawn",
background=False,
_agent_level=1, # persona is Level 1; child would be Level 2
)
confirm_deny = captured_kwargs.get("confirm_deny", set())
assert "spawn_agent" not in confirm_deny, "L2 agents must be allowed to spawn"
@pytest.mark.asyncio
async def test_l2_deny_intersected_with_tool_list(self):
"""When the role has an explicit tool_list, L3 deny removes from list directly."""
from tools.agents import spawn_agent
captured_kwargs = {}
async def _capture_run(**kwargs):
captured_kwargs.update(kwargs)
return _make_mock_result()
# Role has an explicit tool_list that includes spawn_agent
with _mock_spawn_deps(role_tools=["web_search", "spawn_agent", "aider_run"]):
with patch("openai_orchestrator.run", side_effect=_capture_run):
await spawn_agent(
task="Test",
background=False,
_agent_level=2,
)
# spawn_agent and aider_run must be absent from the tool_list passed to orchestrator
tool_list = captured_kwargs.get("tool_list", [])
assert "spawn_agent" not in tool_list
assert "aider_run" not in tool_list
assert "web_search" in tool_list # unrelated tools must survive
# ---------------------------------------------------------------------------
# Agent lifecycle tools — output formatting
# ---------------------------------------------------------------------------
class TestAgentLifecycleTools:
@pytest.mark.asyncio
async def test_agent_status_running(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="research", task="Do research", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_status
output = await agent_status(rec.agent_id)
assert "running" in output
assert "research" in output
assert rec.agent_id[:8] in output
@pytest.mark.asyncio
async def test_agent_status_done(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="Task", level=2)
await agent_manager.finish(rec.agent_id, "The result text", "done")
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_status
output = await agent_status(rec.agent_id)
assert "done" in output
assert "The result text" in output
@pytest.mark.asyncio
async def test_agent_status_wrong_user(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
with patch("persona.get_user", return_value="holly"):
from tools.agents import agent_status
output = await agent_status(rec.agent_id)
assert "denied" in output.lower()
@pytest.mark.asyncio
async def test_agent_status_not_found(self):
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_status
output = await agent_status("nonexistent-id")
assert "No agent found" in output
@pytest.mark.asyncio
async def test_agent_list_shows_running(self):
import agent_manager
await agent_manager.register(user="scott", role="research", task="Research X", level=2)
await agent_manager.register(user="scott", role="coder", task="Fix bug", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_list
output = await agent_list()
assert "2 agent(s)" in output
assert "research" in output
assert "coder" in output
@pytest.mark.asyncio
async def test_agent_list_status_filter(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
await agent_manager.finish(rec.agent_id, "done", "done")
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_list
output = await agent_list(status="running")
assert "1 agent(s)" in output
@pytest.mark.asyncio
async def test_agent_list_empty(self):
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_list
output = await agent_list()
assert "No agents found" in output
@pytest.mark.asyncio
async def test_agent_cancel_tool(self):
import agent_manager
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
with patch("persona.get_user", return_value="scott"):
from tools.agents import agent_cancel
output = await agent_cancel(rec.agent_id)
assert "cancelled" in output
assert agent_manager.get(rec.agent_id).status == "cancelled"
# ---------------------------------------------------------------------------
# aider_run — background mode
# ---------------------------------------------------------------------------
class TestAiderRunBackground:
@pytest.mark.asyncio
async def test_background_returns_agent_id(self):
import agent_manager
async def _fake_proc(*args, **kwargs):
mock_proc = MagicMock()
mock_proc.communicate = AsyncMock(return_value=(b"All changes applied.", b""))
mock_proc.returncode = 0
return mock_proc
with (
patch("persona.get_user", return_value="scott"),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
):
from tools.aider import aider_run
result = await aider_run(
project=str(_CORTEX_DIR.parent), # use actual project root (exists)
task="Test background task",
background=True,
)
assert "Aider task started in background" in result
assert "ID:" in result
@pytest.mark.asyncio
async def test_background_agent_completes(self):
import agent_manager
async def _fake_proc(*args, **kwargs):
mock_proc = MagicMock()
mock_proc.communicate = AsyncMock(return_value=(b"Edits applied.", b""))
mock_proc.returncode = 0
return mock_proc
from tools.aider import aider_run
with (
patch("persona.get_user", return_value="scott"),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
):
result = await aider_run(
project=str(_CORTEX_DIR.parent),
task="Test",
background=True,
)
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
# Poll while patches are still active
for _ in range(40):
rec = agent_manager.get(agent_id)
if rec and rec.status != "running":
break
await asyncio.sleep(0.05)
rec = agent_manager.get(agent_id)
assert rec.status == "done"
assert "Edits applied" in (rec.result or "")
@pytest.mark.asyncio
async def test_invalid_project_directory(self):
from tools.aider import aider_run
result = await aider_run(project="/this/does/not/exist", task="Test")
assert "does not exist" in result
@pytest.mark.asyncio
async def test_sync_path_still_works(self):
async def _fake_proc(*args, **kwargs):
mock_proc = MagicMock()
mock_proc.communicate = AsyncMock(return_value=(b"Done.", b""))
mock_proc.returncode = 0
return mock_proc
with (
patch("persona.get_user", return_value="scott"),
patch("model_registry.get_registry", return_value={"hosts": []}),
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
):
from tools.aider import aider_run
result = await aider_run(
project=str(_CORTEX_DIR.parent),
task="Sync test",
background=False,
)
assert "Done." in result
# ---------------------------------------------------------------------------
# aider_run — credential resolver (_resolve_credentials)
# ---------------------------------------------------------------------------
class TestAiderCredentialResolver:
"""Pure unit tests for _resolve_credentials — no subprocess, no registry I/O."""
def _registry(self, hosts=None, anthropic_key=None):
reg = {"hosts": hosts or [], "providers": {}}
if anthropic_key:
reg["providers"]["anthropic"] = {
"credentials": [{"api_key": anthropic_key}]
}
return reg
def _host(self, label, api_url, api_key="sk-test", host_type="openai"):
return {"id": "x", "label": label, "api_url": api_url,
"api_key": api_key, "host_type": host_type}
# --- Provider detection ---
def test_openrouter_host_gets_api_key_flag(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, model = _resolve_credentials(reg, None, None)
assert "--api-key" in flags
assert "openrouter=or-key" in flags
def test_anthropic_model_hint_uses_provider_key(self):
from tools.aider import _resolve_credentials
reg = self._registry(
hosts=[self._host("OpenRouter", "https://openrouter.ai/api/v1")],
anthropic_key="ant-key",
)
flags, model = _resolve_credentials(reg, "claude-3-5-sonnet-20241022", None)
assert "anthropic=ant-key" in flags
assert model == "claude-3-5-sonnet-20241022"
def test_anthropic_slash_prefix_hint(self):
from tools.aider import _resolve_credentials
reg = self._registry(anthropic_key="ant-key")
flags, _ = _resolve_credentials(reg, "anthropic/claude-opus-4", None)
assert "anthropic=ant-key" in flags
def test_local_openwebui_host_gets_base_url(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://192.168.32.19:3000", "localkey", host_type="openwebui"),
])
flags, model = _resolve_credentials(reg, None, None)
assert "--openai-api-base" in flags
base = flags[flags.index("--openai-api-base") + 1]
assert base == "http://192.168.32.19:3000/api"
assert "--openai-api-key" in flags
def test_local_host_appends_api_suffix_for_openwebui(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenWebUI", "http://localhost:3000", host_type="openwebui"),
])
flags, _ = _resolve_credentials(reg, None, None)
base = flags[flags.index("--openai-api-base") + 1]
assert base.endswith("/api")
def test_generic_openai_host_no_api_suffix(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Custom", "http://localhost:8080/v1", host_type="openai"),
])
flags, _ = _resolve_credentials(reg, None, None)
base = flags[flags.index("--openai-api-base") + 1]
assert not base.endswith("/api")
assert base == "http://localhost:8080/v1"
# --- Model name adjustment ---
def test_local_host_prefixes_model_without_slash(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
])
_, model = _resolve_credentials(reg, "gemma-4-27b-it", None)
assert model == "openai/gemma-4-27b-it"
def test_local_host_leaves_model_with_slash(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
])
_, model = _resolve_credentials(reg, "ollama/gemma4", None)
assert model == "ollama/gemma4" # already prefixed, don't touch
def test_cloud_provider_does_not_prefix_model(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1"),
])
_, model = _resolve_credentials(reg, "google/gemma-3-27b-it", None)
assert model == "google/gemma-3-27b-it"
# --- Host label override ---
def test_host_label_selects_local_over_openrouter(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
self._host("Local RTX", "http://192.168.32.19:3000", "local-key", host_type="openwebui"),
])
flags, _ = _resolve_credentials(reg, None, "Local")
assert "--openai-api-base" in flags
assert "--api-key" not in flags
def test_host_label_case_insensitive(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, _ = _resolve_credentials(reg, None, "openrouter")
assert "openrouter=or-key" in flags
# --- Model prefix routing ---
def test_model_openrouter_prefix_routes_to_openrouter(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, model = _resolve_credentials(reg, "openrouter/google/gemma-3-27b-it", None)
assert "openrouter=or-key" in flags
assert model == "openrouter/google/gemma-3-27b-it"
def test_model_groq_prefix_routes_to_groq_host(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Groq", "https://api.groq.com/openai/v1", "groq-key"),
])
flags, _ = _resolve_credentials(reg, "groq/llama-3.3-70b", None)
assert "groq=groq-key" in flags
# --- Default fallback priority ---
def test_prefers_openrouter_over_local_when_no_hint(self):
from tools.aider import _resolve_credentials
reg = self._registry(hosts=[
self._host("Local", "http://localhost:3000", host_type="openwebui"),
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
])
flags, _ = _resolve_credentials(reg, None, None)
assert "openrouter=or-key" in flags
def test_prefers_anthropic_over_local_when_no_openrouter(self):
from tools.aider import _resolve_credentials
reg = self._registry(
hosts=[self._host("Local", "http://localhost:3000", host_type="openwebui")],
anthropic_key="ant-key",
)
flags, _ = _resolve_credentials(reg, None, None)
assert "anthropic=ant-key" in flags
def test_empty_registry_returns_no_flags(self):
from tools.aider import _resolve_credentials
flags, model = _resolve_credentials({}, "gemma-4", None)
assert flags == []
assert model == "gemma-4"
# ---------------------------------------------------------------------------
# Helpers for manual test record creation (used in list tests above)
# ---------------------------------------------------------------------------
import agent_manager as _am
_CORTEX_DIR = _am.__file__ and _am and __import__("pathlib").Path(_am.__file__).parent
def _make_record(agent_id: str, user: str, status: str) -> "_am.AgentRecord":
from datetime import datetime
import agent_manager
rec = agent_manager.AgentRecord(
agent_id=agent_id,
level=2,
role="chat",
task="test task",
status=status,
started=datetime.now(),
user=user,
finished=datetime.now() if status != "running" else None,
)
return rec

View File

@@ -25,7 +25,10 @@ async def test_files_get_allowed(client):
@pytest.mark.anyio
async def test_files_get_not_in_allowed(client):
"""Files outside the ALLOWED set should return 404, not the file content."""
for name in ("TASKS.json", "CRONS.json", "SCRATCH.md", "../config.py", ".env"):
# Note: paths with '..' are normalized at the ASGI layer (e.g. /files/../config.py
# becomes /config.py which hits the /{username} UI catch-all, not the files router).
# Only test paths that stay within the files router's scope.
for name in ("TASKS.json", "CRONS.json", "SCRATCH.md", ".env"):
r = await client.get(f"/files/{name}")
assert r.status_code == 404, f"Expected 404 for {name}, got {r.status_code}"

View File

@@ -30,5 +30,7 @@ async def test_distill_status(client):
@pytest.mark.anyio
async def test_unknown_route_404(client):
r = await client.get("/does-not-exist")
# Single-segment paths hit the /{username} persona-picker catch-all (302 redirect).
# Three-segment paths don't match any route pattern → genuine 404.
r = await client.get("/totally/unknown/deep-path")
assert r.status_code == 404

View File

@@ -0,0 +1,805 @@
"""
Unit tests for model_registry.py — no HTTP, no LLM calls, no running service.
All file I/O is redirected to tmp_path via patch.object(config.settings, "home_dir", ...).
Coverage:
- Empty registry (no files)
- Save/load round-trip
- Migration from local_llm.json (v0 flat and v1 hosts/models)
- Host CRUD
- Model CRUD (including role reference cleanup on remove)
- Role assignment (set_role, validation)
- Model resolution (_resolve_model: built-ins, local_openai, missing host/model)
- get_model_for_role: registry chain → .env fallback → hardcoded fallback
- get_best_local_model: role chain, first-local fallback, no-local case
- Backup chain: skips missing models, returns next valid
"""
import json
import pytest
from pathlib import Path
from unittest.mock import patch
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _home(tmp_path: Path) -> Path:
"""Create a minimal home directory and return the root."""
root = tmp_path / "home"
root.mkdir()
return root
def _user_dir(home: Path, username: str = "scott") -> Path:
d = home / username
d.mkdir(exist_ok=True)
return d
def _write_registry(home: Path, data: dict, username: str = "scott") -> Path:
_user_dir(home, username)
path = home / username / "model_registry.json"
path.write_text(json.dumps(data))
return path
def _write_local_llm(home: Path, data: dict, username: str = "scott") -> Path:
_user_dir(home, username)
path = home / username / "local_llm.json"
path.write_text(json.dumps(data))
return path
def _read_registry(home: Path, username: str = "scott") -> dict:
path = home / username / "model_registry.json"
return json.loads(path.read_text())
# ---------------------------------------------------------------------------
# Empty / fresh state
# ---------------------------------------------------------------------------
def test_empty_registry_no_files(tmp_path):
"""With no files, _load returns an empty structure."""
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert data["version"] == 2
assert data["hosts"] == []
assert data["models"] == []
assert data["roles"] == {}
def test_empty_registry_missing_user_dir(tmp_path):
"""Even with no user dir, _load returns an empty structure gracefully."""
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("nobody")
assert data["hosts"] == []
# ---------------------------------------------------------------------------
# Save / load round-trip
# ---------------------------------------------------------------------------
def test_save_and_load(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
registry = {
"version": 1,
"hosts": [{"id": "h1", "label": "ML Box", "api_url": "http://10.0.0.1:3000", "api_key": "sk-test"}],
"models": [{"id": "m1", "type": "local_openai", "label": "Gemma Small",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": ["fast"]}],
"roles": {"chat": {"primary": "m1"}},
}
with patch.object(config.settings, "home_dir", home):
reg._save("scott", registry)
loaded = reg._load("scott")
assert loaded["hosts"][0]["label"] == "ML Box"
assert loaded["models"][0]["model_name"] == "gemma4:e4b"
assert loaded["roles"]["chat"]["primary"] == "m1"
def test_corrupt_registry_falls_back_to_empty(tmp_path):
home = _home(tmp_path)
path = _user_dir(home) / "model_registry.json"
path.write_text("{bad json{{")
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert data["hosts"] == []
# ---------------------------------------------------------------------------
# Migration from local_llm.json
# ---------------------------------------------------------------------------
def test_migrate_v1_hosts_models(tmp_path):
"""v1 local_llm.json (hosts/models/active_model_id) migrates correctly."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Home", "api_url": "http://10.0.0.1:3000", "api_key": "sk-1"}],
"models": [
{"id": "m1", "host_id": "h1", "label": "Gemma Small", "model_name": "gemma4:e4b"},
{"id": "m2", "host_id": "h1", "label": "Gemma Med", "model_name": "gemma4:26b"},
],
"active_model_id": "m1",
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["api_url"] == "http://10.0.0.1:3000"
assert len(data["models"]) == 2
assert all(m["type"] == "local_openai" for m in data["models"])
# active_model_id → roles.chat.primary
assert data["roles"].get("chat", {}).get("primary") == "m1"
def test_migrate_v1_no_active_model(tmp_path):
"""Migration with active_model_id=null: chat role stays unset."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "host_id": "h1", "label": "Model", "model_name": "llama3"}],
"active_model_id": None,
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert "chat" not in data["roles"] or data["roles"]["chat"].get("primary") is None
def test_migrate_v0_flat_format(tmp_path):
"""v0 flat local_llm.json is wrapped into hosts/models structure."""
home = _home(tmp_path)
_write_local_llm(home, {
"api_url": "http://10.0.0.2:3000",
"api_key": "sk-flat",
"model": "qwen3:8b",
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["api_url"] == "http://10.0.0.2:3000"
assert len(data["models"]) == 1
assert data["models"][0]["model_name"] == "qwen3:8b"
def test_migrate_v0_empty_url_returns_empty(tmp_path):
"""v0 with no api_url and no .env fallback → nothing to migrate, empty registry."""
home = _home(tmp_path)
_write_local_llm(home, {"api_url": "", "api_key": "", "model": ""})
import config
import model_registry as reg
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "local_api_url", ""), # ensure no .env fallback
patch.object(config.settings, "local_model", ""),
):
data = reg._load("scott")
assert data["hosts"] == []
assert data["models"] == []
def test_migrate_v1_distill_local_sets_role(tmp_path):
"""When DISTILL_BACKEND_MID=local and active model exists, distill role is set."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "host_id": "h1", "label": "G", "model_name": "gemma4:e4b"}],
"active_model_id": "m1",
})
import config
import model_registry as reg
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "distill_backend_mid", "local"),
):
data = reg._load("scott")
assert data["roles"].get("distill", {}).get("primary") == "m1"
def test_migration_saves_registry_file(tmp_path):
"""After migration, model_registry.json is written so next load skips migration."""
home = _home(tmp_path)
_write_local_llm(home, {
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [],
"active_model_id": None,
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg._load("scott") # triggers migration + save
# Second load should read model_registry.json, not re-run migration
data2 = reg._load("scott")
assert (home / "scott" / "model_registry.json").exists()
assert data2["version"] == 2
# ---------------------------------------------------------------------------
# Built-in model resolution
# ---------------------------------------------------------------------------
def test_builtin_claude_cli(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "claude_cli")
assert result is not None
assert result["type"] == "claude_cli"
assert result["id"] == "claude_cli"
def test_builtin_gemini_api(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "gemini_api")
assert result["type"] == "gemini_api"
def test_builtin_gemini_cli(tmp_path):
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "gemini_cli")
assert result["type"] == "gemini_cli"
def test_builtin_unknown_returns_none(tmp_path):
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "does_not_exist")
assert result is None
# ---------------------------------------------------------------------------
# User model resolution
# ---------------------------------------------------------------------------
def test_resolve_local_openai_merges_host(tmp_path):
"""local_openai model gets api_url and api_key merged from its host."""
home = _home(tmp_path)
registry = {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": "sk-test"}],
"models": [{"id": "m1", "type": "local_openai", "label": "G", "model_name": "gemma4:e4b",
"host_id": "h1", "context_k": 72, "tags": []}],
"roles": {},
}
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(registry, "m1")
assert result["api_url"] == "http://10.0.0.1:3000"
assert result["api_key"] == "sk-test"
assert result["model_name"] == "gemma4:e4b"
def test_resolve_local_openai_missing_host_returns_none(tmp_path):
"""A model pointing to a non-existent host_id returns None."""
home = _home(tmp_path)
registry = {
"version": 1, "hosts": [], "roles": {},
"models": [{"id": "m1", "type": "local_openai", "host_id": "missing",
"label": "X", "model_name": "x", "context_k": 0, "tags": []}],
}
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(registry, "m1")
assert result is None
def test_resolve_unknown_model_id_returns_none(tmp_path):
home = _home(tmp_path)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg._resolve_model(reg._empty(), "no_such_model")
assert result is None
# ---------------------------------------------------------------------------
# get_model_for_role
# ---------------------------------------------------------------------------
def test_get_model_for_role_uses_registry(tmp_path):
"""Registry primary assignment is returned first."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
"roles": {"chat": {"primary": "m1"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["model_name"] == "gemma4:e4b"
assert result["api_url"] == "http://10.0.0.1:3000"
def test_get_model_for_role_uses_builtin_from_registry(tmp_path):
"""Registry can assign built-in IDs (claude_cli, gemini_api, etc.)."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["type"] == "claude_cli"
def test_get_model_for_role_skips_missing_primary(tmp_path):
"""If primary model_id is not found, falls through to backup_1."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m2", "type": "local_openai", "label": "Backup",
"model_name": "llama3:8b", "host_id": "h1", "context_k": 8, "tags": []}],
"roles": {"chat": {"primary": "gone", "backup_1": "m2"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["model_name"] == "llama3:8b"
def test_get_model_for_role_env_fallback(tmp_path):
"""No registry entry for role → falls back to .env setting."""
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "role_chat", "gemini_cli"),
):
result = reg.get_model_for_role("scott", "chat")
assert result["type"] == "gemini_cli"
def test_get_model_for_role_hardcoded_fallback(tmp_path):
"""No registry + no .env for role → hardcoded last resort."""
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
# Clear the .env default for 'chat' to simulate unset
with (
patch.object(config.settings, "home_dir", home),
patch.object(config.settings, "role_chat", ""),
):
result = reg.get_model_for_role("scott", "chat")
# claude_cli is the hardcoded last resort for 'chat'
assert result["type"] == "claude_cli"
def test_get_model_for_role_custom_role(tmp_path):
"""Custom roles not in DEFINED_ROLES can still be assigned and resolved."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"therapy": {"primary": "gemini_api"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "therapy")
assert result["type"] == "gemini_api"
def test_get_model_for_role_full_backup_chain(tmp_path):
"""Walks the entire priority chain before falling back."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m4", "type": "local_openai", "label": "Last",
"model_name": "tiny:1b", "host_id": "h1", "context_k": 4, "tags": []}],
"roles": {"chat": {
"primary": "gone1",
"backup_1": "gone2",
"backup_2": "gone3",
"backup_3": "gone4",
"backup_4": "m4",
}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_model_for_role("scott", "chat")
assert result["model_name"] == "tiny:1b"
# ---------------------------------------------------------------------------
# get_best_local_model
# ---------------------------------------------------------------------------
def test_get_best_local_prefers_role_chain(tmp_path):
"""Returns the first local_openai model in the chat role chain."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [
{"id": "m1", "type": "local_openai", "label": "Preferred",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []},
],
"roles": {"chat": {"primary": "claude_cli", "backup_1": "m1"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
# primary is claude_cli (not local), backup_1 is m1 (local)
result = reg.get_best_local_model("scott", "chat")
assert result["model_name"] == "gemma4:e4b"
def test_get_best_local_falls_back_to_first_model(tmp_path):
"""No local model in role chain → returns first configured local model."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [
{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []},
],
"roles": {}, # no chat role assigned
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_best_local_model("scott", "chat")
assert result["model_name"] == "gemma4:e4b"
def test_get_best_local_returns_none_when_no_local_models(tmp_path):
"""No local_openai models configured → returns None."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
result = reg.get_best_local_model("scott", "chat")
assert result is None
# ---------------------------------------------------------------------------
# Host CRUD
# ---------------------------------------------------------------------------
def test_save_host_creates_new(tmp_path):
home = _home(tmp_path)
_user_dir(home)
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
host_id = reg.save_host("scott", None, "ML Box", "http://10.0.0.1:3000", "sk-abc")
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["id"] == host_id
assert data["hosts"][0]["label"] == "ML Box"
assert data["hosts"][0]["api_key"] == "sk-abc"
def test_save_host_updates_existing(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Old Label", "api_url": "http://10.0.0.1:3000", "api_key": "sk-old"}],
"models": [], "roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_host("scott", "h1", "New Label", "http://10.0.0.2:3000", "")
data = reg._load("scott")
assert len(data["hosts"]) == 1
assert data["hosts"][0]["label"] == "New Label"
assert data["hosts"][0]["api_url"] == "http://10.0.0.2:3000"
# Empty api_key → existing key preserved
assert data["hosts"][0]["api_key"] == "sk-old"
def test_save_host_unknown_id_creates_new(tmp_path):
"""Passing a host_id that doesn't exist creates a new host instead of crashing."""
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_host("scott", "ghost-id", "New", "http://10.0.0.3:3000", "")
data = reg._load("scott")
assert len(data["hosts"]) == 1
def test_remove_host_also_removes_models(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "host_id": "h1",
"label": "G", "model_name": "gemma4:e4b", "context_k": 72, "tags": []}],
"roles": {"chat": {"primary": "m1"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_host("scott", "h1")
data = reg._load("scott")
assert found is True
assert data["hosts"] == []
assert data["models"] == []
def test_remove_host_not_found_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_host("scott", "nope")
assert found is False
# ---------------------------------------------------------------------------
# Model CRUD
# ---------------------------------------------------------------------------
def test_save_model_creates(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [], "roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
model_id = reg.save_model("scott", None, "h1", "Gemma Small", "gemma4:e4b", 72, ["fast", "distill"])
data = reg._load("scott")
assert len(data["models"]) == 1
assert data["models"][0]["id"] == model_id
assert data["models"][0]["context_k"] == 72
assert data["models"][0]["tags"] == ["fast", "distill"]
assert data["models"][0]["type"] == "local_openai"
def test_save_model_updates_existing(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "Old",
"model_name": "llama3", "host_id": "h1", "context_k": 8, "tags": []}],
"roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_model("scott", "m1", "h1", "New Label", "llama3:latest", 128, ["updated"])
data = reg._load("scott")
assert len(data["models"]) == 1
assert data["models"][0]["label"] == "New Label"
assert data["models"][0]["context_k"] == 128
def test_remove_model_clears_role_refs(tmp_path):
"""Removing a model clears it from any role assignments."""
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
"roles": {
"chat": {"primary": "m1", "backup_1": "m1"},
"distill": {"primary": "claude_cli", "backup_1": "m1"},
},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_model("scott", "m1")
data = reg._load("scott")
assert found is True
assert data["models"] == []
assert data["roles"]["chat"].get("primary") is None
assert data["roles"]["chat"].get("backup_1") is None
assert data["roles"]["distill"].get("backup_1") is None
# claude_cli assignment preserved
assert data["roles"]["distill"]["primary"] == "claude_cli"
def test_remove_model_not_found_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
found = reg.remove_model("scott", "ghost")
assert found is False
# ---------------------------------------------------------------------------
# set_role
# ---------------------------------------------------------------------------
def test_set_role_assigns_model(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1,
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
"models": [{"id": "m1", "type": "local_openai", "label": "G",
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
"roles": {},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "primary", "m1")
data = reg._load("scott")
assert ok is True
assert data["roles"]["chat"]["primary"] == "m1"
def test_set_role_assigns_builtin(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "orchestrator", "primary", "gemini_api")
data = reg._load("scott")
assert ok is True
assert data["roles"]["orchestrator"]["primary"] == "gemini_api"
def test_set_role_clears_with_none(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "primary", None)
data = reg._load("scott")
assert ok is True
assert data["roles"]["chat"]["primary"] is None
def test_set_role_invalid_slot_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "backup_99", "claude_cli")
assert ok is False
def test_set_role_unknown_model_id_returns_false(tmp_path):
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
ok = reg.set_role("scott", "chat", "primary", "nonexistent_model")
assert ok is False
def test_set_role_creates_role_key_if_missing(tmp_path):
"""set_role on a role that isn't in roles{} yet creates it."""
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.set_role("scott", "medical", "primary", "claude_cli")
data = reg._load("scott")
assert data["roles"]["medical"]["primary"] == "claude_cli"
# ---------------------------------------------------------------------------
# get_defined_roles
# ---------------------------------------------------------------------------
def test_get_defined_roles_returns_registry_roles(tmp_path):
home = _home(tmp_path)
_write_registry(home, {
"version": 1, "hosts": [], "models": [],
"roles": {"chat": {"primary": "claude_cli"}, "distill": {}},
})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
roles = reg.get_defined_roles("scott")
# Should include all settings.defined_roles, filling gaps with {}
for role in config.settings.get_defined_roles():
assert role in roles
def test_get_defined_roles_fills_gaps(tmp_path):
"""Roles in settings.defined_roles that aren't in registry get empty dicts."""
home = _home(tmp_path)
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
roles = reg.get_defined_roles("scott")
assert "chat" in roles
assert roles["chat"] == {}
# ---------------------------------------------------------------------------
# Multi-user isolation
# ---------------------------------------------------------------------------
def test_registries_are_isolated_per_user(tmp_path):
"""Each user has their own registry file — changes don't bleed across users."""
home = _home(tmp_path)
(home / "scott").mkdir()
(home / "holly").mkdir()
import config
import model_registry as reg
with patch.object(config.settings, "home_dir", home):
reg.save_host("scott", None, "Scott Host", "http://10.0.0.1:3000", "")
scott_data = reg._load("scott")
holly_data = reg._load("holly")
assert len(scott_data["hosts"]) == 1
assert holly_data["hosts"] == []

View File

@@ -69,10 +69,11 @@ async def test_nct_replayed_request_rejected(client):
payload = json.dumps({"type": "Create", "actor": {}, "object": {}, "target": {}}).encode()
# Use wrong secret to generate sig
wrong_sig = hmac_lib.new(b"wrong-secret", b"abc123" + payload, hashlib.sha256).hexdigest()
_channels = {"nextcloud": {"bot_secret": "correct-secret", "url": "https://nc.example.com"}}
from unittest.mock import patch
with patch("config.settings.nextcloud_talk_bot_secret", "correct-secret"):
with patch("routers.nextcloud_talk.get_user_channels", return_value=_channels):
r = await client.post(
"/inara-nextcloud-talk-webhook",
"/webhook/nextcloud/scott",
content=payload,
headers={
"Content-Type": "application/json",
@@ -118,9 +119,11 @@ async def test_known_gap__gchat_no_audience_bypass(client, mock_llm):
LLM responses without a valid token.
Fix: make audience required; fail loudly if not set.
"""
# Channel config with no audience — JWT check is skipped (the known gap).
_channels = {"google_chat": {"persona": "inara"}}
from unittest.mock import patch
with patch("config.settings.google_chat_audience", ""):
r = await client.post("/channels/google-chat", json={
with patch("routers.google_chat.get_user_channels", return_value=_channels):
r = await client.post("/channels/google-chat/scott", json={
"chat": {
"messagePayload": {
"message": {"text": "Exploit"},

View File

@@ -101,19 +101,19 @@ class TestTasks:
def test_list_empty(self):
from tools.tasks import _task_list
assert "No tasks" in _task_list(status=None)
assert "No tasks" in _task_list(status=None, priority=None)
def test_create_and_list(self):
from tools.tasks import _task_list
self._mk("Buy coffee", description="Dark roast", priority="high")
result = _task_list(status=None)
result = _task_list(status=None, priority=None)
assert "Buy coffee" in result
assert "[high]" in result
def test_create_bad_priority_defaults_to_normal(self):
from tools.tasks import _task_list
self._mk("Test task", priority="urgent") # invalid — becomes "normal"
result = _task_list(status=None)
result = _task_list(status=None, priority=None)
assert "Test task" in result
assert "[normal]" not in result # normal priority not shown in brackets
@@ -121,20 +121,20 @@ class TestTasks:
from tools.tasks import _task_update, _task_list
tid = self._id(self._mk("Work item"))
_task_update(tid, status="in_progress", title=None, description=None, priority=None)
assert "Work item" in _task_list(status="in_progress")
assert "Work item" in _task_list(status="in_progress", priority=None)
def test_complete(self):
from tools.tasks import _task_complete, _task_list
tid = self._id(self._mk("Finish this"))
_task_complete(tid)
assert "Finish this" in _task_list(status="done")
assert "Finish this" not in _task_list(status="todo")
assert "Finish this" in _task_list(status="done", priority=None)
assert "Finish this" not in _task_list(status="todo", priority=None)
def test_filter_by_status(self):
from tools.tasks import _task_list
self._mk("A task")
assert "A task" in _task_list(status="todo")
assert "A task" not in _task_list(status="done")
assert "A task" in _task_list(status="todo", priority=None)
assert "A task" not in _task_list(status="done", priority=None)
def test_update_unknown_id(self):
from tools.tasks import _task_update
@@ -231,7 +231,8 @@ class TestCronTools:
def _extract_id(self, result: str) -> str:
import re
m = re.search(r'c_\w+', result)
# token_urlsafe can include '-'; use [\w-]+ to capture the full ID
m = re.search(r'c_[\w-]+', result)
assert m, f"No cron ID in: {result}"
return m.group()

View File

@@ -2,6 +2,10 @@
Webhook auth tests — NC Talk HMAC, Google Chat JWT.
These tests verify that auth is enforced, not that full LLM responses work.
Architecture note: channel config (secrets, audience) lives in per-user channels.json,
not in settings. Tests mock get_user_channels() rather than patching settings fields.
Endpoints are per-user: /webhook/nextcloud/{username} and /channels/google-chat/{username}.
"""
import hashlib
import hmac
@@ -26,6 +30,14 @@ _VALID_NC_PAYLOAD = {
"target": {"id": "abc123token"},
}
_NCT_CHANNELS = {
"nextcloud": {
"bot_secret": _NC_SECRET,
"notification_room": "abc123token",
"url": "https://nc.example.com",
}
}
def _nc_headers(body: bytes, secret: str) -> dict:
random_str = "abc123"
@@ -43,11 +55,11 @@ def _nc_headers(body: bytes, secret: str) -> dict:
@pytest.mark.anyio
async def test_nct_valid_signature(client, mock_llm):
body = json.dumps(_VALID_NC_PAYLOAD).encode()
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
with patch("routers.nextcloud_talk._send_reply", new_callable=AsyncMock):
headers = _nc_headers(body, _NC_SECRET)
r = await client.post(
"/inara-nextcloud-talk-webhook",
"/webhook/nextcloud/scott",
content=body,
headers={**headers, "Content-Type": "application/json"},
)
@@ -57,9 +69,9 @@ async def test_nct_valid_signature(client, mock_llm):
@pytest.mark.anyio
async def test_nct_wrong_signature(client):
body = json.dumps(_VALID_NC_PAYLOAD).encode()
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
r = await client.post(
"/inara-nextcloud-talk-webhook",
"/webhook/nextcloud/scott",
content=body,
headers={
"Content-Type": "application/json",
@@ -73,9 +85,9 @@ async def test_nct_wrong_signature(client):
@pytest.mark.anyio
async def test_nct_missing_signature(client):
body = json.dumps(_VALID_NC_PAYLOAD).encode()
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
r = await client.post(
"/inara-nextcloud-talk-webhook",
"/webhook/nextcloud/scott",
content=body,
headers={"Content-Type": "application/json"},
)
@@ -84,11 +96,13 @@ async def test_nct_missing_signature(client):
@pytest.mark.anyio
async def test_nct_no_secret_configured(client):
"""Service should return 500 if secret is not set, not process the message."""
"""Service should return 500 if bot_secret is missing, not process the message."""
body = json.dumps(_VALID_NC_PAYLOAD).encode()
with patch("config.settings.nextcloud_talk_bot_secret", ""):
# cfg must be non-empty (truthy) to get past the 404 guard; missing bot_secret → 500
empty_cfg = {"nextcloud": {"url": "https://nc.example.com"}}
with patch("routers.nextcloud_talk.get_user_channels", return_value=empty_cfg):
r = await client.post(
"/inara-nextcloud-talk-webhook",
"/webhook/nextcloud/scott",
content=body,
headers={"Content-Type": "application/json"},
)
@@ -100,10 +114,10 @@ async def test_nct_bot_message_ignored(client):
"""Messages from other bots should be silently ignored (not processed)."""
payload = {**_VALID_NC_PAYLOAD, "actor": {"type": "bots", "id": "otherbot", "name": "Bot"}}
body = json.dumps(payload).encode()
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
headers = _nc_headers(body, _NC_SECRET)
r = await client.post(
"/inara-nextcloud-talk-webhook",
"/webhook/nextcloud/scott",
content=body,
headers={**headers, "Content-Type": "application/json"},
)
@@ -124,21 +138,29 @@ _GCHAT_PAYLOAD = {
}
}
_GCHAT_CHANNELS_NO_AUDIENCE = {
# cfg must be non-empty (truthy) to pass the 404 guard; no audience → JWT skipped
"google_chat": {"persona": "inara"}
}
_GCHAT_CHANNELS_WITH_AUDIENCE = {
"google_chat": {"audience": "123456789"}
}
@pytest.mark.anyio
async def test_gchat_no_audience_configured(client, mock_llm):
"""When audience is not set, JWT check is skipped (current behaviour — documented bypass)."""
with patch("config.settings.google_chat_audience", ""):
r = await client.post("/channels/google-chat", json=_GCHAT_PAYLOAD)
# Should process the message (no auth enforcement when audience is empty)
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_NO_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=_GCHAT_PAYLOAD)
assert r.status_code == 200
@pytest.mark.anyio
async def test_gchat_missing_token_with_audience(client):
"""When audience IS configured, requests without a token must be rejected."""
with patch("config.settings.google_chat_audience", "123456789"):
r = await client.post("/channels/google-chat", json=_GCHAT_PAYLOAD)
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_WITH_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=_GCHAT_PAYLOAD)
assert r.status_code == 401
@@ -149,8 +171,8 @@ async def test_gchat_invalid_token_with_audience(client):
**_GCHAT_PAYLOAD,
"authorizationEventObject": {"systemIdToken": "not.a.valid.jwt"},
}
with patch("config.settings.google_chat_audience", "123456789"):
r = await client.post("/channels/google-chat", json=payload_with_token)
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_WITH_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=payload_with_token)
assert r.status_code == 401
@@ -158,7 +180,7 @@ async def test_gchat_invalid_token_with_audience(client):
async def test_gchat_added_to_space(client, mock_llm):
"""Bot added to a space — should return a greeting, no auth when audience empty."""
payload = {"chat": {"addedToSpacePayload": {"space": {"type": "ROOM"}}}}
with patch("config.settings.google_chat_audience", ""):
r = await client.post("/channels/google-chat", json=payload)
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_NO_AUDIENCE):
r = await client.post("/channels/google-chat/scott", json=payload)
assert r.status_code == 200
assert "hostAppDataAction" in r.json()

156
cortex/tool_audit.py Normal file
View File

@@ -0,0 +1,156 @@
"""
Tool call audit log.
One JSONL file per user per day:
home/{user}/tool_audit/YYYY-MM-DD.jsonl
Each line is a JSON object:
ts ISO timestamp (seconds)
user username
tool tool name
args call arguments (string values truncated at ARG_MAX chars)
status "ok" | "error" | "denied"
result_chars length of full result string
result_snippet first SNIPPET_MAX chars of result
"""
import asyncio
import json
import logging
from contextvars import ContextVar
from datetime import datetime, date
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
_ARG_MAX = 500 # truncate individual arg string values longer than this
_SNIPPET_MAX = 300 # chars of result to keep as snippet
# Per-file write locks — prevents interleaved lines under concurrent tool calls
_locks: dict[str, asyncio.Lock] = {}
# ContextVars set by orchestrators before their tool loop runs
_audit_engine: ContextVar[str] = ContextVar("_audit_engine", default="")
_audit_model: ContextVar[str] = ContextVar("_audit_model", default="")
def set_context(engine: str, model: str) -> None:
"""Call at the start of each orchestrator run to tag subsequent tool calls."""
_audit_engine.set(engine)
_audit_model.set(model)
def _truncate_args(args: dict) -> dict:
out = {}
for k, v in args.items():
if isinstance(v, str) and len(v) > _ARG_MAX:
out[k] = v[:_ARG_MAX] + f" …[{len(v)} chars total]"
else:
out[k] = v
return out
def _audit_path(user: str, day: date | None = None) -> Path:
d = day or date.today()
audit_dir = settings.home_root() / user / "tool_audit"
audit_dir.mkdir(parents=True, exist_ok=True)
return audit_dir / f"{d.isoformat()}.jsonl"
async def record(
user: str,
tool: str,
args: dict,
status: str, # "ok" | "error" | "denied"
result: str = "",
) -> None:
"""Append one audit entry. Fire with asyncio.create_task — never awaited directly."""
path = _audit_path(user)
key = str(path)
if key not in _locks:
_locks[key] = asyncio.Lock()
entry = {
"ts": datetime.now().isoformat(timespec="seconds"),
"user": user,
"engine": _audit_engine.get(),
"model": _audit_model.get(),
"tool": tool,
"args": _truncate_args(args),
"status": status,
"result_chars": len(result),
"result_snippet": result[:_SNIPPET_MAX],
}
async with _locks[key]:
try:
with path.open("a", encoding="utf-8") as f:
f.write(json.dumps(entry) + "\n")
except Exception as e:
logger.warning("audit log write failed for %s: %s", user, e)
def read_recent(user: str, days: int = 7, limit: int = 200) -> list[dict]:
"""Read the most recent `limit` entries across the last `days` days.
Returns entries sorted newest-first (by ts field, file order within a day).
"""
from datetime import timedelta
today = date.today()
entries: list[dict] = []
for offset in range(days):
day = today - timedelta(days=offset)
path = settings.home_root() / user / "tool_audit" / f"{day.isoformat()}.jsonl"
if not path.exists():
continue
try:
lines = path.read_text(encoding="utf-8").splitlines()
except Exception:
continue
day_entries = []
for line in lines:
line = line.strip()
if not line:
continue
try:
day_entries.append(json.loads(line))
except json.JSONDecodeError:
pass
# Newest within the day first
entries.extend(reversed(day_entries))
if len(entries) >= limit:
break
return entries[:limit]
def read_day(user: str, day_str: str) -> list[dict]:
"""Read all entries for a specific date string (YYYY-MM-DD), chronological order."""
path = settings.home_root() / user / "tool_audit" / f"{day_str}.jsonl"
if not path.exists():
return []
entries = []
try:
for line in path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line:
continue
try:
entries.append(json.loads(line))
except json.JSONDecodeError:
pass
except Exception:
pass
return entries
def read_recent_all_users(days: int = 7, limit: int = 500) -> list[dict]:
"""Read recent entries across all users, sorted newest-first."""
from persona import list_users
all_entries: list[dict] = []
for user in list_users():
all_entries.extend(read_recent(user, days=days, limit=limit))
all_entries.sort(key=lambda e: e.get("ts", ""), reverse=True)
return all_entries[:limit]

File diff suppressed because it is too large Load Diff

253
cortex/tools/ae_database.py Normal file
View File

@@ -0,0 +1,253 @@
"""
Aether MariaDB tools — SELECT-only access to the Aether Platform database.
Credentials are read from the current user's channels.json:
"aether_db": {
"host": "192.168.64.5",
"port": 3306,
"name": "aether_dev",
"user": "aether_dev",
"password": "..."
}
Configure per-user in Settings → Notifications (or edit channels.json directly).
Only SELECT, SHOW, DESCRIBE, and EXPLAIN statements are permitted — no writes possible.
"""
import asyncio
import logging
import re
from google.genai import types
from auth_utils import get_user_channels
from persona import get_user
logger = logging.getLogger(__name__)
_MAX_ROWS = 200
_MAX_CELL = 120
_ALLOWED = {"select", "show", "describe", "desc", "explain"}
_SAFE_ID = re.compile(r'^[a-zA-Z0-9_]+$')
def _get_db_cfg() -> tuple[dict, str | None]:
"""Return (cfg_dict, error_string). cfg is empty dict on error."""
channels = get_user_channels(get_user())
cfg = channels.get("aether_db") or {}
if not cfg.get("host") or not cfg.get("user"):
return {}, (
"Aether DB not configured for this user. "
"Add an 'aether_db' block to channels.json: "
'{"host": "...", "port": 3306, "name": "aether_dev", "user": "...", "password": "..."}'
)
return cfg, None
def _is_read_only(sql: str) -> bool:
stripped = sql.strip()
if not stripped:
return False
first = stripped.split()[0].lower().rstrip(";")
return first in _ALLOWED
def _fmt(columns: list[str], rows: list[tuple]) -> str:
if not rows:
return f"({len(columns)} column{'s' if len(columns) != 1 else ''}, 0 rows)"
str_rows = [
[("NULL" if v is None else str(v))[:_MAX_CELL] for v in row]
for row in rows
]
widths = [
max([len(col)] + [len(r[i]) for r in str_rows])
for i, col in enumerate(columns)
]
sep = "+" + "+".join("-" * (w + 2) for w in widths) + "+"
header = "|" + "|".join(f" {c:<{w}} " for c, w in zip(columns, widths)) + "|"
lines = [sep, header, sep]
for row in str_rows:
lines.append("|" + "|".join(f" {v:<{w}} " for v, w in zip(row, widths)) + "|")
lines.append(sep)
note = " — results truncated at limit" if len(rows) == _MAX_ROWS else ""
lines.append(f"({len(rows)} row{'s' if len(rows) != 1 else ''}{note})")
return "\n".join(lines)
def _connect(cfg: dict):
import pymysql
import pymysql.cursors
return pymysql.connect(
host=cfg["host"],
port=int(cfg.get("port", 3306)),
user=cfg["user"],
password=cfg.get("password", ""),
database=cfg.get("name", "aether_dev"),
cursorclass=pymysql.cursors.Cursor,
connect_timeout=10,
)
async def ae_db_query(sql: str) -> str:
"""Run a read-only SQL query against the Aether MariaDB and return formatted results."""
cfg, err = _get_db_cfg()
if err:
return err
if not _is_read_only(sql):
first = sql.strip().split()[0] if sql.strip() else "(empty)"
return f"Only SELECT, SHOW, DESCRIBE, and EXPLAIN are permitted. Got: {first!r}"
def _run() -> tuple[list[str], list[tuple]]:
conn = _connect(cfg)
try:
with conn.cursor() as cur:
cur.execute(sql)
columns = [d[0] for d in cur.description] if cur.description else []
rows = list(cur.fetchmany(_MAX_ROWS))
return columns, rows
finally:
conn.close()
try:
columns, rows = await asyncio.to_thread(_run)
return _fmt(columns, rows)
except Exception as e:
logger.warning("ae_db_query error: %s", e)
return f"Query error: {e}"
async def ae_db_describe(table: str, detailed: bool = False) -> str:
"""Describe the columns of an Aether DB table or view."""
cfg, err = _get_db_cfg()
if err:
return err
if not _SAFE_ID.match(table):
return f"Invalid table name: {table!r}. Only letters, digits, and underscores allowed."
def _run():
conn = _connect(cfg)
try:
with conn.cursor() as cur:
cur.execute(f"DESCRIBE `{table}`")
columns = [d[0] for d in cur.description] if cur.description else []
rows = list(cur.fetchall())
return columns, rows
finally:
conn.close()
try:
columns, rows = await asyncio.to_thread(_run)
if not detailed:
fields = [row[0] for row in rows]
return f"{table}: " + ", ".join(fields)
return _fmt(columns, rows)
except Exception as e:
logger.warning("ae_db_describe error: %s", e)
return f"Describe error: {e}"
async def ae_db_show_view(view_name: str) -> str:
"""Return the CREATE VIEW SQL for an Aether DB view."""
cfg, err = _get_db_cfg()
if err:
return err
if not _SAFE_ID.match(view_name):
return f"Invalid view name: {view_name!r}. Only letters, digits, and underscores allowed."
def _run():
conn = _connect(cfg)
try:
with conn.cursor() as cur:
cur.execute(f"SHOW CREATE VIEW `{view_name}`")
return cur.fetchone()
finally:
conn.close()
try:
row = await asyncio.to_thread(_run)
if not row:
return f"View not found: {view_name}"
return str(row[1]) if len(row) > 1 else str(row[0])
except Exception as e:
logger.warning("ae_db_show_view error: %s", e)
return f"Show view error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="ae_db_describe",
description=(
"Describe the columns of an Aether Platform table or view. "
"Returns a compact field list by default; pass detailed=true for full schema "
"(type, nullability, default, key). Use to understand data structure before "
"writing a SELECT query, or to answer 'what fields does X have?'. "
"Examples: table='ae_journals'; table='clients'; table='time_entries'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"table": types.Schema(
type=types.Type.STRING,
description="Table or view name (letters, digits, underscores only)",
),
"detailed": types.Schema(
type=types.Type.BOOLEAN,
description="Return full schema (type, nullability, key, default) instead of just field names",
),
},
required=["table"],
),
),
types.FunctionDeclaration(
name="ae_db_show_view",
description=(
"Return the CREATE VIEW SQL for an Aether Platform database view. "
"Use to understand how a view is constructed before querying it, "
"or to debug unexpected results from a view. "
"Example: view_name='v_active_journals'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"view_name": types.Schema(
type=types.Type.STRING,
description="View name (letters, digits, underscores only)",
),
},
required=["view_name"],
),
),
types.FunctionDeclaration(
name="ae_db_query",
description=(
"Run a read-only SQL query against the Aether Platform MariaDB. "
"Permitted statements: SELECT, SHOW, DESCRIBE, EXPLAIN. No writes are possible. "
"Use for debugging: bad data, missing records, broken foreign keys, schema questions. "
"Results capped at 200 rows; cells truncated at 120 chars. "
"Examples: SELECT * FROM clients WHERE email = 'x@y.com'; "
"SELECT COUNT(*) FROM time_entries WHERE billed = 0 AND deleted_at IS NULL; "
"SHOW TABLES; DESCRIBE ae_journals; "
"SELECT id_random, enable, deleted_at FROM ae_journals WHERE id_random = 'abc123'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"sql": types.Schema(
type=types.Type.STRING,
description=(
"SQL query to run — SELECT, SHOW, DESCRIBE, or EXPLAIN only. "
"No semicolons required but harmless if present."
),
),
},
required=["sql"],
),
),
]

View File

@@ -1,15 +1,17 @@
"""
Aether Platform knowledge tools — journal search and entry creation.
Aether Platform knowledge tools — journal search, listing, and entry management.
These tools give the orchestrator read/write access to the AE Journals module,
which serves as the primary long-term knowledge base.
Auth: x-aether-api-key + x-account-id headers (same pattern as agents_sync scripts).
API: V3 CRUD — POST /v3/crud/journal_entry/search, POST /v3/crud/journal/{id}/journal_entry/
PATCH /v3/crud/journal_entry/{entry_id}, GET /v3/crud/journal_entry/{entry_id}
"""
import asyncio
import logging
from google.genai import types
from config import settings
logger = logging.getLogger(__name__)
@@ -40,36 +42,98 @@ def _check_config() -> str | None:
# Tool: ae_journal_search
# ---------------------------------------------------------------------------
async def journal_search(query: str, journal_id: str | None = None, max_results: int = 10) -> str:
"""Search AE Journal entries by keyword.
async def journal_search(
query: str = "",
journal_id: str = "",
tags: str = "",
type_code: str = "",
topic_code: str = "",
date_from: str = "",
date_to: str = "",
sort_by: str = "updated",
sort_order: str = "desc",
status: int | None = None,
priority: int | None = None,
max_results: int = 10,
page: int = 1,
) -> str:
"""Search AE Journal entries.
Searches across the default_qry_str field (title + content excerpt).
Optionally scoped to a specific journal by journal_id (id_random).
Returns a markdown-formatted list of matching entries.
At least one of query, tags, type_code, topic_code, date_from, or journal_id
should be provided. All filters combine with AND.
"""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_search, query, journal_id, max_results)
return await asyncio.to_thread(
_sync_journal_search,
query, journal_id, tags, type_code, topic_code,
date_from, date_to, sort_by, sort_order,
status, priority, max_results, page,
)
def _sync_journal_search(query: str, journal_id: str | None, max_results: int) -> str:
def _sync_journal_search(
query: str,
journal_id: str,
tags: str,
type_code: str,
topic_code: str,
date_from: str,
date_to: str,
sort_by: str,
sort_order: str,
status: int | None,
priority: int | None,
max_results: int,
page: int,
) -> str:
import requests
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
search_body = {
"and_filters": [
{"field": "default_qry_str", "op": "icontains", "value": query}
],
"page_size": max_results,
# Build sort field
sort_field_map = {
"updated": "updated_on",
"created": "created_on",
"name": "name",
"priority": "priority",
}
sort_field = sort_field_map.get(sort_by, "updated_on")
order_by = f"{'-' if sort_order == 'desc' else ''}{sort_field}"
params = {}
search_body: dict = {"page_size": max_results, "page": page, "order_by": order_by}
# Fulltext keyword — uses MATCH/AGAINST index
if query:
search_body["query_string"] = query
# Additional AND filters
and_filters: list[dict] = []
if tags:
and_filters.append({"field": "tags", "op": "icontains", "value": tags})
if type_code:
and_filters.append({"field": "type_code", "op": "eq", "value": type_code})
if topic_code:
and_filters.append({"field": "topic_code", "op": "eq", "value": topic_code})
if date_from:
and_filters.append({"field": "created_on", "op": "gte", "value": date_from})
if date_to:
and_filters.append({"field": "created_on", "op": "lte", "value": date_to})
if status is not None:
and_filters.append({"field": "status", "op": "eq", "value": status})
if priority is not None:
and_filters.append({"field": "priority", "op": "eq", "value": priority})
if and_filters:
search_body["and"] = and_filters
# query_string must be present for `and` filters to apply
if "query_string" not in search_body:
search_body["query_string"] = "%"
params: dict = {}
if journal_id:
params["for_obj_type"] = "journal"
params["for_obj_id"] = journal_id
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
try:
resp = requests.post(
url,
@@ -85,33 +149,92 @@ def _sync_journal_search(query: str, journal_id: str | None, max_results: int) -
return f"Journal search error: {e}"
entries = data.get("data", [])
if not entries:
return f"No journal entries found matching: {query}"
total = (data.get("meta") or {}).get("data_list_count") or len(entries)
if not entries:
desc = query or tags or type_code or topic_code or f"journal {journal_id}"
return f"No journal entries found for: {desc}"
label = query or tags or f"{len(entries)} entries"
lines = [f"Journal entries — **{label}** ({total} total, page {page}):\n"]
lines = [f"Journal entries matching **{query}** ({len(entries)} result(s)):\n"]
for entry in entries:
title = entry.get("name") or "(untitled)"
entry_id = entry.get("id_random", "")
entry_id = entry.get("journal_entry_id") or entry.get("id") or ""
journal_name = entry.get("journal_name") or entry.get("parent_name") or ""
summary = entry.get("summary") or ""
content_preview = (entry.get("content") or "")[:200].replace("\n", " ")
entry_tags = entry.get("tags") or []
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:10]
content_preview = (entry.get("content") or "")[:400].replace("\n", " ")
header = f"**{title}**"
if journal_name:
header += f" ({journal_name})"
if entry_id:
header += f" — id: `{entry_id}`"
if updated:
header += f" [{updated}]"
lines.append(header)
if entry_tags:
tag_list = entry_tags if isinstance(entry_tags, list) else [t.strip() for t in str(entry_tags).split(",")]
lines.append(f" Tags: {', '.join(tag_list)}")
if summary:
lines.append(f" Summary: {summary}")
if content_preview:
lines.append(f" {content_preview}")
lines.append(f" {summary}")
elif content_preview:
lines.append(f" {content_preview}{'' if len(entry.get('content', '')) > 400 else ''}")
lines.append("")
if total > page * max_results:
lines.append(f"(More results — call again with page={page + 1})")
return "\n".join(lines).strip()
# ---------------------------------------------------------------------------
# Tool: ae_journal_list
# ---------------------------------------------------------------------------
async def journal_list() -> str:
"""List all journals accessible to the configured AE account."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_list)
def _sync_journal_list() -> str:
import requests
url = f"{settings.ae_api_url}/v3/crud/journal/search"
try:
resp = requests.post(
url,
headers=_headers(),
json={"page_size": 100},
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
data = resp.json()
except Exception as e:
logger.warning("ae_journal_list failed: %s", e)
return f"Journal list error: {e}"
journals = data.get("data", [])
if not journals:
return "No journals found for this account."
lines = [f"Journals ({len(journals)}):\n"]
for j in journals:
jid = j.get("journal_id") or j.get("id_random") or j.get("id") or "?"
name = j.get("name") or "(untitled)"
desc = j.get("description") or ""
line = f"- **{name}** — id: `{jid}`"
if desc:
line += f"\n {desc}"
lines.append(line)
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_create
# ---------------------------------------------------------------------------
@@ -170,8 +293,455 @@ def _sync_journal_entry_create(
return f"Journal entry creation error: {e}"
entry_id = (
result.get("data", {}).get("id_random")
result.get("data", {}).get("journal_entry_id")
or result.get("data", {}).get("id_random")
or result.get("id_random")
or "unknown"
)
return f"Journal entry created. id: `{entry_id}`, title: \"{title}\", journal: `{journal_id}`"
# ---------------------------------------------------------------------------
# Shared helper: fetch a single journal entry by id
# ---------------------------------------------------------------------------
def _get_entry(entry_id: str) -> dict | str:
"""Return the entry dict, or an error string on failure."""
import requests
url = f"{settings.ae_api_url}/v3/crud/journal_entry/{entry_id}"
try:
resp = requests.get(url, headers=_headers(), timeout=settings.ae_api_timeout)
resp.raise_for_status()
data = resp.json()
entry = data.get("data") or data
if not isinstance(entry, dict):
return f"Unexpected response shape for entry {entry_id}"
return entry
except Exception as e:
logger.warning("_get_entry %s failed: %s", entry_id, e)
return f"Error fetching entry {entry_id}: {e}"
def _patch_entry(entry_id: str, payload: dict) -> str:
"""PATCH a journal entry. Returns a success/error string."""
import requests
url = f"{settings.ae_api_url}/v3/crud/journal_entry/{entry_id}"
try:
resp = requests.patch(
url,
headers=_headers(),
json=payload,
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
return "ok"
except Exception as e:
logger.warning("_patch_entry %s failed: %s", entry_id, e)
return f"Error updating entry {entry_id}: {e}"
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_read
# ---------------------------------------------------------------------------
async def journal_entry_read(entry_id: str, max_content_chars: int = 4000) -> str:
"""Return the full content of a single journal entry by its id_random."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_read, entry_id, max_content_chars)
def _sync_journal_entry_read(entry_id: str, max_content_chars: int) -> str:
entry = _get_entry(entry_id)
if isinstance(entry, str):
return entry
title = entry.get("name") or "(untitled)"
journal = entry.get("journal_name") or entry.get("parent_name") or ""
summary = entry.get("summary") or ""
raw_tags = entry.get("tags") or []
tags = raw_tags if isinstance(raw_tags, list) else [t.strip() for t in str(raw_tags).split(",") if t.strip()]
content = entry.get("content") or ""
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:19].replace("T", " ")
enabled = entry.get("enable", True)
lines = [f"# {title}"]
meta: list[str] = [f"id: `{entry_id}`"]
if journal:
meta.append(f"journal: {journal}")
if updated:
meta.append(f"updated: {updated}")
if not enabled:
meta.append("**DISABLED**")
lines.append(" ".join(meta))
if tags:
lines.append(f"Tags: {', '.join(tags)}")
if summary:
lines.append(f"\nSummary: {summary}")
lines.append("\n---\n")
truncated = len(content) > max_content_chars
lines.append(content[:max_content_chars])
if truncated:
lines.append(
f"\n\n[Content truncated at {max_content_chars} chars — "
f"{len(content)} total. Call again with a higher max_content_chars to read more.]"
)
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Tool: ae_journal_entries_list
# ---------------------------------------------------------------------------
async def journal_entries_list(journal_id: str, max_results: int = 20, page: int = 1) -> str:
"""List entries in a specific journal, newest first."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entries_list, journal_id, max_results, page)
def _sync_journal_entries_list(journal_id: str, max_results: int, page: int) -> str:
import requests
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
search_body: dict = {
"page_size": max_results,
"page": page,
"order_by": "-updated_on",
}
params = {"for_obj_type": "journal", "for_obj_id": journal_id}
try:
resp = requests.post(
url,
headers=_headers(),
params=params,
json=search_body,
timeout=settings.ae_api_timeout,
)
resp.raise_for_status()
data = resp.json()
except Exception as e:
logger.warning("ae_journal_entries_list failed: %s", e)
return f"Journal entries list error: {e}"
entries = data.get("data", [])
total = (data.get("meta") or {}).get("data_list_count") or len(entries)
if not entries:
return f"No entries found in journal `{journal_id}`."
offset = (page - 1) * max_results + 1
lines = [f"Entries in journal `{journal_id}` — showing {offset}{offset + len(entries) - 1} of {total}:\n"]
for i, entry in enumerate(entries, offset):
title = entry.get("name") or "(untitled)"
entry_id = entry.get("journal_entry_id") or entry.get("id") or ""
raw_tags = entry.get("tags") or []
tags = raw_tags if isinstance(raw_tags, list) else [t.strip() for t in str(raw_tags).split(",") if t.strip()]
summary = entry.get("summary") or ""
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:10]
enabled = entry.get("enable", True)
status = "" if enabled else " [disabled]"
date_str = f" [{updated}]" if updated else ""
lines.append(f"{i}. **{title}**{status} — id: `{entry_id}`{date_str}")
if tags:
lines.append(f" Tags: {', '.join(tags)}")
if summary:
lines.append(f" {summary[:150]}{'' if len(summary) > 150 else ''}")
lines.append("")
if total > offset + len(entries) - 1:
lines.append(f"(More entries available — call again with page={page + 1})")
return "\n".join(lines).rstrip()
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_update
# ---------------------------------------------------------------------------
async def journal_entry_update(
entry_id: str,
title: str = "",
content: str = "",
summary: str = "",
tags: str = "",
enable: bool | None = None,
) -> str:
"""Update fields on an existing journal entry. Only provided fields are changed."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_update, entry_id, title, content, summary, tags, enable)
def _sync_journal_entry_update(
entry_id: str,
title: str,
content: str,
summary: str,
tags: str,
enable: bool | None,
) -> str:
payload: dict = {}
if title:
payload["name"] = title
if content:
payload["content"] = content
if summary:
payload["summary"] = summary
if tags:
payload["tags"] = [t.strip() for t in tags.split(",") if t.strip()]
if enable is not None:
payload["enable"] = enable
if not payload:
return "Nothing to update — no fields provided."
result = _patch_entry(entry_id, payload)
if result != "ok":
return result
updated = ", ".join(payload.keys())
return f"Journal entry `{entry_id}` updated. Fields changed: {updated}"
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_disable
# ---------------------------------------------------------------------------
async def journal_entry_disable(entry_id: str) -> str:
"""Soft-delete a journal entry by setting enable=false."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_patch_entry, entry_id, {"enable": False})
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_append
# ---------------------------------------------------------------------------
async def journal_entry_append(entry_id: str, content: str, heading: str = "") -> str:
"""Append a timestamped section to the bottom of a journal entry's content."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_append, entry_id, content, heading)
def _sync_journal_entry_append(entry_id: str, content: str, heading: str) -> str:
from datetime import datetime, timezone
entry = _get_entry(entry_id)
if isinstance(entry, str):
return entry
existing = (entry.get("content") or "").rstrip()
ts = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
section_heading = heading or ts
new_content = f"{existing}\n\n### {section_heading}\n{content.strip()}"
result = _patch_entry(entry_id, {"content": new_content})
if result != "ok":
return result
return f"Appended to journal entry `{entry_id}` under heading \"{section_heading}\"."
# ---------------------------------------------------------------------------
# Tool: ae_journal_entry_prepend
# ---------------------------------------------------------------------------
async def journal_entry_prepend(entry_id: str, content: str, heading: str = "") -> str:
"""Prepend a timestamped section to the top of a journal entry's content."""
err = _check_config()
if err:
return err
return await asyncio.to_thread(_sync_journal_entry_prepend, entry_id, content, heading)
def _sync_journal_entry_prepend(entry_id: str, content: str, heading: str) -> str:
from datetime import datetime, timezone
entry = _get_entry(entry_id)
if isinstance(entry, str):
return entry
existing = (entry.get("content") or "").lstrip()
ts = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
section_heading = heading or ts
new_content = f"### {section_heading}\n{content.strip()}\n\n{existing}"
result = _patch_entry(entry_id, {"content": new_content})
if result != "ok":
return result
return f"Prepended to journal entry `{entry_id}` under heading \"{section_heading}\"."
DECLARATIONS = [
types.FunctionDeclaration(
name="ae_journal_list",
description=(
"List all Aether Journals available for this account. "
"Returns each journal's name and id_random. "
"Call this first when you need to write a new entry or scope a search to a specific journal "
"and don't already know the journal's id."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="ae_journal_search",
description=(
"Search Aether Journal entries. All parameters are optional — combine freely. "
"Use 'query' for fulltext keyword search (supports boolean: +required -excluded \"phrase\"). "
"Use 'tags' to filter by tag substring. Use 'date_from'/'date_to' for date ranges (YYYY-MM-DD). "
"Always search before creating a new entry to avoid duplicates."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"query": types.Schema(type=types.Type.STRING, description="Fulltext keyword search. Supports boolean mode: +required -excluded \"exact phrase\"."),
"journal_id": types.Schema(type=types.Type.STRING, description="Scope results to a specific journal by its id_random. Omit to search all journals."),
"tags": types.Schema(type=types.Type.STRING, description="Filter by tag substring (e.g. 'networking' matches entries tagged 'networking' or 'home-networking')."),
"type_code": types.Schema(type=types.Type.STRING, description="Filter by exact type_code (e.g. 'note', 'meeting', 'log')."),
"topic_code": types.Schema(type=types.Type.STRING, description="Filter by exact topic_code."),
"date_from": types.Schema(type=types.Type.STRING, description="Return entries created on or after this date (YYYY-MM-DD)."),
"date_to": types.Schema(type=types.Type.STRING, description="Return entries created on or before this date (YYYY-MM-DD)."),
"sort_by": types.Schema(type=types.Type.STRING, description="Sort field: 'updated' (default), 'created', 'name', or 'priority'."),
"sort_order": types.Schema(type=types.Type.STRING, description="Sort direction: 'desc' (default, newest first) or 'asc'."),
"status": types.Schema(type=types.Type.INTEGER, description="Filter by exact status code."),
"priority": types.Schema(type=types.Type.INTEGER, description="Filter by exact priority (1=low, 5=high)."),
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of results per page (default 10)."),
"page": types.Schema(type=types.Type.INTEGER, description="Page number for pagination (default 1)."),
},
required=[],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_read",
description=(
"Fetch the full content of a single journal entry by its id_random. "
"Use this when you need to read an entry before editing it, or when search results "
"don't show enough content. Returns title, journal, tags, summary, and full content."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="The id_random of the journal entry to read."),
"max_content_chars": types.Schema(type=types.Type.INTEGER, description="Maximum characters of content to return (default 4000). Increase for long entries."),
},
required=["entry_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entries_list",
description=(
"List entries in a specific journal, newest first. "
"Use this to browse what's in a journal when you don't have a search keyword, "
"or to find entries by browsing rather than searching. "
"Returns numbered entries with id, title, tags, summary, and date."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"journal_id": types.Schema(type=types.Type.STRING, description="The id_random of the journal to list entries from."),
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of entries to return (default 20, max 50)."),
"page": types.Schema(type=types.Type.INTEGER, description="Page number for pagination (default 1)."),
},
required=["journal_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_create",
description=(
"Create a new entry in an Aether Journal. "
"Use this to save notes, summaries, or any content the user wants to store. "
"Always call ae_journal_search first to check for existing entries on the same topic."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"journal_id": types.Schema(type=types.Type.STRING, description="The id_random of the target journal. Ask the user which journal to write to if not specified."),
"title": types.Schema(type=types.Type.STRING, description="Entry title"),
"content": types.Schema(type=types.Type.STRING, description="Full entry content (markdown supported)"),
"summary": types.Schema(type=types.Type.STRING, description="Optional short summary (1-2 sentences)"),
"tags": types.Schema(type=types.Type.STRING, description="Optional comma-separated tags (e.g. 'wireguard, networking, homelab')"),
},
required=["journal_id", "title", "content"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_update",
description=(
"Update fields on an existing journal entry. Only the fields you provide are changed — "
"omitted fields are left as-is. Use ae_journal_search to find the entry_id first. "
"To soft-delete, use ae_journal_entry_disable instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
"title": types.Schema(type=types.Type.STRING, description="New title"),
"content": types.Schema(type=types.Type.STRING, description="Replacement content (full, markdown supported)"),
"summary": types.Schema(type=types.Type.STRING, description="New summary"),
"tags": types.Schema(type=types.Type.STRING, description="Replacement comma-separated tags"),
"enable": types.Schema(type=types.Type.BOOLEAN, description="Set false to hide/disable the entry"),
},
required=["entry_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_disable",
description=(
"Soft-delete a journal entry by setting enable=false. "
"The entry is hidden but not permanently removed. "
"Use ae_journal_search to find the entry_id first."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
},
required=["entry_id"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_append",
description=(
"Append a new section to the bottom of a journal entry's content. "
"Each section gets a UTC timestamp heading unless you provide one. "
"Ideal for timestamped logs, running notes, or data logs."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
"content": types.Schema(type=types.Type.STRING, description="The text to append (markdown supported)"),
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading (defaults to current UTC timestamp)"),
},
required=["entry_id", "content"],
),
),
types.FunctionDeclaration(
name="ae_journal_entry_prepend",
description=(
"Prepend a new section to the top of a journal entry's content. "
"Each section gets a UTC timestamp heading unless you provide one. "
"Useful for most-recent-first logs."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
"content": types.Schema(type=types.Type.STRING, description="The text to prepend (markdown supported)"),
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading (defaults to current UTC timestamp)"),
},
required=["entry_id", "content"],
),
),
]

View File

@@ -16,6 +16,8 @@ import json
import logging
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
# Resolved at import time — agents_sync is always at ~/agents_sync on this machine.
@@ -98,3 +100,20 @@ def _read_bucket(bucket_dir: Path) -> list[dict]:
except Exception as e:
logger.warning("Failed to read task file %s: %s", path, e)
return tasks
DECLARATIONS = [
types.FunctionDeclaration(
name="ae_task_list",
description=(
"List tasks from the agents_sync Kanban board (todo and in-progress). "
"Use this when asked about current work, pending tasks, or project status."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"include_done": types.Schema(type=types.Type.BOOLEAN, description="If true, also include completed tasks (default false)"),
},
),
),
]

155
cortex/tools/agent_notes.py Normal file
View File

@@ -0,0 +1,155 @@
"""
Agent private notes — AGENT_NOTES.md.
A persistent notepad only the orchestrator can write to. The file itself is
never exposed in the Files panel or loaded into user-facing context tiers.
Up to 3 rolling backups are kept automatically before each write so past
versions can be reviewed.
Use for: observations about the user's patterns, working hypotheses,
long-running goals, things to remember across sessions that shouldn't
be part of the distilled memory visible to the user.
"""
import asyncio
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path
_FILENAME = "AGENT_NOTES.md"
_N_BACKUPS = 3
def _notes_path() -> Path:
return persona_path() / _FILENAME
def _now_label() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
def _rotate(path: Path) -> None:
"""Rotate up to _N_BACKUPS rolling backups before a write."""
if not path.exists():
return
for i in range(_N_BACKUPS, 1, -1):
older = path.parent / f"{path.stem}.bak{i}.md"
newer = path.parent / f"{path.stem}.bak{i - 1}.md"
if newer.exists():
older.write_text(newer.read_text())
bak1 = path.parent / f"{path.stem}.bak1.md"
bak1.write_text(path.read_text())
# ── Sync implementations ────────────────────────────────────────────────────
def _agent_notes_read() -> str:
p = _notes_path()
if not p.exists() or not p.read_text().strip():
return "Agent notes are empty."
return p.read_text()
def _agent_notes_write(content: str) -> str:
p = _notes_path()
_rotate(p)
p.write_text(content.rstrip() + "\n")
return "Agent notes updated."
def _agent_notes_append(content: str, heading: str | None = None) -> str:
p = _notes_path()
_rotate(p)
existing = p.read_text() if p.exists() else ""
label = heading or _now_label()
section = f"\n## {label}\n\n{content.strip()}\n"
p.write_text(existing.rstrip() + "\n" + section)
return f"Appended to agent notes: {label}"
def _agent_notes_clear() -> str:
p = _notes_path()
_rotate(p)
p.write_text("")
return "Agent notes cleared."
# ── Async wrappers ───────────────────────────────────────────────────────────
async def agent_notes_read() -> str:
return await asyncio.to_thread(_agent_notes_read)
async def agent_notes_write(content: str) -> str:
return await asyncio.to_thread(_agent_notes_write, content)
async def agent_notes_append(content: str, heading: str | None = None) -> str:
return await asyncio.to_thread(_agent_notes_append, content, heading)
async def agent_notes_clear() -> str:
return await asyncio.to_thread(_agent_notes_clear)
# ── Gemini FunctionDeclarations ──────────────────────────────────────────────
DECLARATIONS = [
types.FunctionDeclaration(
name="agent_notes_read",
description=(
"Read your private agent notes — a persistent notepad only you can write to. "
"Use this to recall observations, working hypotheses, long-running goals, or "
"anything you want to remember across sessions without surfacing it to the user. "
"This file is never shown in the user's Files panel."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="agent_notes_write",
description=(
"Replace your private agent notes with new content. "
"A backup is saved automatically before writing. "
"Use agent_notes_append to add without replacing."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(
type=types.Type.STRING,
description="The new notes content (markdown supported).",
),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="agent_notes_append",
description=(
"Add a new section to your private agent notes without replacing existing content. "
"A backup is saved automatically before writing. "
"Each section gets a UTC timestamp heading unless you supply one."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(
type=types.Type.STRING,
description="The content to append (markdown supported).",
),
"heading": types.Schema(
type=types.Type.STRING,
description="Optional section heading. Defaults to current UTC timestamp.",
),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="agent_notes_clear",
description=(
"Erase all private agent notes. A backup is saved automatically before clearing."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

446
cortex/tools/agents.py Normal file
View File

@@ -0,0 +1,446 @@
"""
Agent spawning and lifecycle tools.
spawn_agent — synchronous or background sub-agent via any configured role model.
agent_status / agent_list / agent_cancel — lifecycle management for background agents.
Sub-agents run using the model and tools assigned to the given role. The three-level
hierarchy (Persona → Specialized → Support) is enforced by denying spawn_agent and
aider_run at the L2→L3 boundary — Level 3 agents cannot delegate further.
Supported model types for sub-agents: local_openai, gemini_api.
claude_cli / gemini_cli are chat-only and do not support tool-enabled sub-agents.
"""
import asyncio
import logging
from datetime import datetime
from google.genai import types
import agent_manager
logger = logging.getLogger(__name__)
# Per-host semaphores — keyed by "host:<host_id>" or "type:<model_type>"
# Created lazily on first use; never deleted (module-level singletons)
_semaphores: dict[str, asyncio.Semaphore] = {}
_sem_lock = asyncio.Lock()
# Tools denied at the L2→L3 boundary so Level 3 agents cannot delegate further.
_L3_DENY_TOOLS = ["spawn_agent", "aider_run"]
async def _get_semaphore(key: str, max_concurrent: int) -> asyncio.Semaphore:
"""Return (or create) the semaphore for a given host/type key."""
async with _sem_lock:
if key not in _semaphores:
_semaphores[key] = asyncio.Semaphore(max_concurrent)
return _semaphores[key]
async def spawn_agent(
task: str,
role: str = "chat",
tier: int = 1,
timeout: int = 120,
max_rounds: int | None = None,
allow_tools: list[str] | None = None,
deny_tools: list[str] | None = None,
background: bool = False,
notify: bool = False,
_agent_level: int = 2,
) -> str:
"""
Spawn a sub-agent to complete a task.
In synchronous mode (background=False, the default): blocks until done and returns
the result string.
In background mode (background=True): registers the agent, fires it as an asyncio
background task, and returns an agent_id string immediately. Use agent_status() to
poll, or set notify=True to receive a push notification on completion.
Level enforcement: this agent (level _agent_level) spawns children at level+1.
Children at level 3 automatically have spawn_agent and aider_run denied so they
cannot delegate further.
"""
import model_registry
from context_loader import load_context
from auth_utils import get_user_role, get_tool_policy
from persona import get_user
user = get_user() or "scott"
role_cfg = model_registry.get_role_config(user, role)
model_cfg = model_registry.get_model_for_role(user, role)
if not model_cfg:
return f"spawn_agent: no model configured for role '{role}'"
model_type = model_cfg.get("type", "unknown")
if model_type not in ("local_openai", "gemini_api"):
return (
f"spawn_agent: model type '{model_type}' does not support tool-enabled sub-agents. "
f"Assign a local_openai or gemini_api model to role '{role}'."
)
# Determine concurrency key and semaphore limit
host_id = model_cfg.get("host_id")
if host_id:
registry = model_registry.get_registry(user)
host = next((h for h in registry.get("hosts", []) if h["id"] == host_id), None)
max_concurrent = (host or {}).get("max_concurrent", 3)
sem_key = f"host:{host_id}"
else:
max_concurrent = 5 if model_type == "gemini_api" else 3
sem_key = f"type:{model_type}"
sem = await _get_semaphore(sem_key, max_concurrent)
system_prompt = load_context(
tier=tier,
include_long=(tier >= 2),
include_mid=(tier >= 2),
include_short=(tier >= 2),
role_append=role_cfg.get("system_append", ""),
inject_datetime=role_cfg.get("inject_datetime", True),
)
user_role = get_user_role(user)
tool_list = role_cfg.get("tools")
policy = get_tool_policy(user)
confirm_allow = set(policy.get("allow", []))
confirm_deny = set(policy.get("deny", []))
# Per-call tool restrictions — role config remains the authoritative ceiling
if allow_tools is not None:
if tool_list is not None:
tool_list = [t for t in tool_list if t in allow_tools]
else:
tool_list = list(allow_tools)
if deny_tools is not None:
deny_set = set(deny_tools)
if tool_list is not None:
tool_list = [t for t in tool_list if t not in deny_set]
else:
confirm_deny = confirm_deny | deny_set
# Level enforcement: children of this agent are at level _agent_level + 1.
# Level 3 children cannot delegate — auto-deny the spawning tools.
child_level = _agent_level + 1
if child_level >= 3:
l3_deny = set(_L3_DENY_TOOLS)
if tool_list is not None:
tool_list = [t for t in tool_list if t not in l3_deny]
else:
confirm_deny = confirm_deny | l3_deny
if max_rounds is not None:
model_cfg = dict(model_cfg)
model_cfg["max_rounds"] = max_rounds
async def _run() -> str:
if model_type == "local_openai":
import openai_orchestrator
result = await openai_orchestrator.run(
task=task,
system_prompt=system_prompt,
model_cfg=model_cfg,
respond_with_final=True,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
)
if result.checkpoint:
return (
"Sub-agent requires user confirmation — "
"confirmation gates are not supported inside spawn_agent. "
"Pre-allow the tool in the user's tool policy or use a different role."
)
return result.response or "(sub-agent returned no output)"
# gemini_api
import orchestrator_engine
from auth_utils import get_user_gemini_key
gemini_key = model_cfg.get("api_key") or get_user_gemini_key(user)
result = await orchestrator_engine.run(
task=task,
system_prompt=system_prompt,
session_messages=None,
respond_with_claude=True,
gemini_api_key=gemini_key,
model_name=model_cfg.get("model_name"),
response_role=role,
user_role=user_role,
tool_list=tool_list,
confirm_allow=confirm_allow,
confirm_deny=confirm_deny,
max_rounds=model_cfg.get("max_rounds"),
)
if result.checkpoint:
return (
"Sub-agent requires user confirmation — "
"confirmation gates are not supported inside spawn_agent."
)
return result.response or "(sub-agent returned no output)"
if background:
rec = await agent_manager.register(
user=user,
role=role,
task=task,
level=_agent_level,
notify=notify,
)
async def _bg_task() -> None:
async with sem:
try:
logger.info(
"spawn_agent [bg]: %s role=%s level=%d timeout=%ds",
rec.agent_id[:8], role, _agent_level, timeout,
)
result = await asyncio.wait_for(_run(), timeout=float(timeout))
await agent_manager.finish(rec.agent_id, result, "done")
logger.info("spawn_agent [bg]: done %s", rec.agent_id[:8])
except asyncio.CancelledError:
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
raise
except asyncio.TimeoutError:
msg = f"Sub-agent timed out after {timeout}s (role={role})"
logger.warning("spawn_agent [bg]: timeout %s", rec.agent_id[:8])
await agent_manager.finish(rec.agent_id, msg, "timeout")
except Exception as e:
logger.exception("spawn_agent [bg]: failed %s", rec.agent_id[:8])
await agent_manager.finish(rec.agent_id, str(e), "failed")
bg = asyncio.create_task(_bg_task())
agent_manager.set_task_ref(rec.agent_id, bg)
return f"Agent started in background. ID: {rec.agent_id}\nUse agent_status('{rec.agent_id}') to check progress."
# Synchronous path — unchanged behaviour
async with sem:
try:
logger.info(
"spawn_agent: role=%s tier=%d timeout=%ds task=%.80s",
role, tier, timeout, task,
)
response = await asyncio.wait_for(_run(), timeout=float(timeout))
logger.info("spawn_agent: done role=%s response=%d chars", role, len(response))
return response
except asyncio.TimeoutError:
logger.warning("spawn_agent: timed out after %ds role=%s", timeout, role)
return f"Sub-agent timed out after {timeout}s (role={role})"
except Exception as e:
logger.exception("spawn_agent: failed role=%s", role)
return f"Sub-agent error ({role}): {e}"
# ── Agent lifecycle tools ─────────────────────────────────────────────────────
async def agent_status(agent_id: str) -> str:
"""Return the status and result preview of a background agent."""
from persona import get_user
user = get_user() or "unknown"
rec = agent_manager.get(agent_id)
if not rec:
return f"No agent found with ID: {agent_id}"
if rec.user != user:
return "Access denied."
now = datetime.now()
end = rec.finished or now
elapsed = int((end - rec.started).total_seconds())
lines = [
f"Agent {rec.agent_id[:8]}",
f" Status: {rec.status}",
f" Role: {rec.role} (Level {rec.level})",
f" Elapsed: {elapsed}s",
f" Started: {rec.started.strftime('%Y-%m-%d %H:%M:%S')}",
f" Task: {rec.task}",
]
if rec.parent_id:
lines.append(f" Parent: {rec.parent_id[:8]}")
if rec.result is not None:
lines.append(f" Result: {rec.result[:300]}")
return "\n".join(lines)
async def agent_list(status: str | None = None, limit: int = 10) -> str:
"""List background agents for the current user."""
from persona import get_user
user = get_user() or "unknown"
limit = min(max(int(limit), 1), 50)
records = agent_manager.list_agents(user, status=status, limit=limit)
if not records:
suffix = f" (filter: status={status})" if status else ""
return f"No agents found.{suffix}"
now = datetime.now()
lines = []
for rec in records:
end = rec.finished or now
elapsed = int((end - rec.started).total_seconds())
preview = rec.task[:60].replace("\n", " ")
result_hint = f"{rec.result[:50]}" if rec.result else ""
lines.append(
f"[{rec.agent_id[:8]}] {rec.status:<10s} L{rec.level} "
f"{rec.role:<12s} {elapsed:>5}s {preview}{result_hint}"
)
header = f"{len(records)} agent(s)" + (f" (status={status})" if status else "") + ":"
return header + "\n" + "\n".join(lines)
async def agent_cancel(agent_id: str) -> str:
"""Cancel a running background agent."""
from persona import get_user
user = get_user() or "unknown"
return await agent_manager.cancel_agent(agent_id, user)
# ── Declarations ──────────────────────────────────────────────────────────────
DECLARATIONS = [
types.FunctionDeclaration(
name="spawn_agent",
description=(
"Spawn a sub-agent to complete a task. "
"In synchronous mode (default): blocks until the sub-agent finishes and returns its response. "
"In background mode (background=True): fires the agent asynchronously and returns an agent_id "
"immediately — use agent_status() to check progress or set notify=True for a completion alert. "
"The sub-agent uses the model and tool set assigned to the given role. "
"Use for processing pipelines, parallel analysis, or delegating specialized work "
"(research, coding, data migration, etc.)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"task": types.Schema(
type=types.Type.STRING,
description="The complete task description for the sub-agent.",
),
"role": types.Schema(
type=types.Type.STRING,
description=(
"Role determining the model and tools. "
"E.g. 'research' for web lookups, 'coder' for code tasks, "
"'distill' for summarization. Defaults to 'chat'."
),
),
"tier": types.Schema(
type=types.Type.INTEGER,
description=(
"Context tier: 1 = minimal (fast, identity only), "
"2 = standard (+ memory), 3 = + last 2 session logs. "
"Use 1 for pure processing tasks."
),
),
"timeout": types.Schema(
type=types.Type.INTEGER,
description="Max seconds to wait (default 120). Applies in both sync and background mode.",
),
"max_rounds": types.Schema(
type=types.Type.INTEGER,
description="Override max tool-loop iterations for this call.",
),
"allow_tools": types.Schema(
type=types.Type.ARRAY,
items=types.Schema(type=types.Type.STRING),
description=(
"Restrict the sub-agent to only these tools. "
"Intersected with the role's tool set — cannot grant more than the role allows. "
"Example: ['web_search', 'web_read'] for a pure research agent."
),
),
"deny_tools": types.Schema(
type=types.Type.ARRAY,
items=types.Schema(type=types.Type.STRING),
description=(
"Block these tools from the sub-agent regardless of role config. "
"Example: ['shell_exec', 'file_write', 'cortex_restart']."
),
),
"background": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Run asynchronously in the background (default: false). "
"When true, returns an agent_id immediately instead of blocking for the result. "
"Use agent_status(agent_id) to check progress. "
"Best for tasks that take more than ~30 seconds."
),
),
"notify": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Send a push/Talk notification when the background agent completes (default: false). "
"Only meaningful when background=true."
),
),
},
required=["task"],
),
),
types.FunctionDeclaration(
name="agent_status",
description=(
"Get the current status of a background agent by ID. "
"Returns status (running/done/failed/cancelled/timeout), role, elapsed time, "
"task description, and result preview."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"agent_id": types.Schema(
type=types.Type.STRING,
description="The agent ID returned by spawn_agent(background=True) or aider_run(background=True).",
),
},
required=["agent_id"],
),
),
types.FunctionDeclaration(
name="agent_list",
description=(
"List background agents for the current user. "
"Returns recent agents with ID, status, role, level, elapsed time, and task preview. "
"Use to survey what's running or recently completed."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"status": types.Schema(
type=types.Type.STRING,
description="Filter by status: 'running', 'done', 'failed', 'cancelled', 'timeout'. Omit for all.",
),
"limit": types.Schema(
type=types.Type.INTEGER,
description="Max agents to return (default 10, max 50).",
),
},
),
),
types.FunctionDeclaration(
name="agent_cancel",
description=(
"Cancel a running background agent. ADMIN ONLY. Requires confirmation. "
"Use agent_list() to find the agent ID first."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"agent_id": types.Schema(
type=types.Type.STRING,
description="The agent ID to cancel.",
),
},
required=["agent_id"],
),
),
]

406
cortex/tools/aider.py Normal file
View File

@@ -0,0 +1,406 @@
"""
Aider coding agent tool — invokes Aider AI pair programming as a subprocess.
Aider handles repo-map generation, file editing, git commits, and linting automatically.
It works with any OpenAI-compatible model — point it at DeepSeek, Ollama, OpenRouter, etc.
via AIDER_MODEL / AIDER_OPENAI_API_BASE env vars or the project's .aider.conf.yml.
Credentials are pulled automatically from the Cortex model registry:
- Named cloud providers (OpenRouter, OpenAI, Groq, Anthropic, …) → --api-key slug=key
- Generic OpenAI-compatible hosts (Open WebUI, Ollama, local) → --openai-api-base + key
- Anthropic from providers.anthropic.credentials → --api-key anthropic=key
background=True runs the subprocess asynchronously and returns an agent_id immediately.
"""
import asyncio
import logging
import os
from pathlib import Path
from google.genai import types
import agent_manager
logger = logging.getLogger(__name__)
_CORTEX_DIR = Path(__file__).parent # .../Cortex_and_Inara_dev/cortex/
_PROJECT_ROOT = _CORTEX_DIR.parent # .../Cortex_and_Inara_dev/
# Known project aliases — expand before passing to subprocess
_PROJECT_ALIASES: dict[str, str] = {
"cortex": str(_PROJECT_ROOT),
"aether_api": "~/OSIT_dev/aether_api_fastapi",
"aether_frontend": "~/OSIT_dev/aether_app_sveltekit",
"aether_container": "~/OSIT_dev/aether_container_env",
}
_MAX_OUTPUT_CHARS = 12_000
# Maps URL fragments → Aider --api-key provider slug.
# Order matters: more specific patterns first.
_CLOUD_PROVIDER_URL_MAP: list[tuple[str, str]] = [
("openrouter.ai", "openrouter"),
("api.openai.com", "openai"),
("groq.com", "groq"),
("api.together.xyz", "togetherai"),
("fireworks.ai", "fireworks"),
("api.x.ai", "xai"),
("api.deepseek.com", "deepseek"),
("api.mistral.ai", "mistral"),
]
def _provider_slug(api_url: str) -> str | None:
"""Return the Aider --api-key provider slug for a known cloud URL, None for generic."""
url_lower = api_url.lower()
for fragment, slug in _CLOUD_PROVIDER_URL_MAP:
if fragment in url_lower:
return slug
return None
def _host_flags(host: dict, model: str | None) -> tuple[list[str], str | None]:
"""Build Aider credential flags for a specific host entry.
Returns (extra_args, adjusted_model). For generic (local) endpoints the model
name may be prefixed with 'openai/' so Aider routes through the OpenAI client.
"""
api_url = (host.get("api_url") or "").rstrip("/")
api_key = host.get("api_key") or "none"
host_type = host.get("host_type", "openai")
slug = _provider_slug(api_url)
if slug:
# Named cloud provider — Aider maps --api-key slug=key → SLUG_API_KEY env var
flags = ["--api-key", f"{slug}={api_key}"] if api_key and api_key != "none" else []
return flags, model
# Generic OpenAI-compatible (local Open WebUI, Ollama, custom)
base_url = api_url
if host_type == "openwebui":
# Open WebUI serves the chat endpoint at /api/chat/completions
base_url = base_url + "/api"
flags = ["--openai-api-base", base_url, "--openai-api-key", api_key]
# Prefix model with 'openai/' for generic endpoints when no provider prefix is set
adj_model = model
if model and "/" not in model:
adj_model = f"openai/{model}"
return flags, adj_model
def _resolve_credentials(
registry: dict,
model: str | None,
host_label: str | None,
) -> tuple[list[str], str | None]:
"""Determine Aider credential flags and (possibly adjusted) model name.
Resolution order:
1. Anthropic model hint (claude-* / anthropic/*) → Anthropic API key
2. Explicit host_label → that host's credentials
3. Model prefix hint (openrouter/*, groq/*, …) → matching host
4. Default priority: OpenRouter → Anthropic → any keyed cloud host → local host
Returns (extra_args, adjusted_model).
"""
hosts = registry.get("hosts", [])
# Extract Anthropic key from providers.anthropic.credentials (not a host entry)
anthropic_key = None
for cred in registry.get("providers", {}).get("anthropic", {}).get("credentials", []):
if cred.get("api_key"):
anthropic_key = cred["api_key"]
break
# ── 1. Anthropic model hint ────────────────────────────────────────────────
if model and any(h in model.lower() for h in ("claude-", "anthropic/")):
if anthropic_key:
logger.debug("aider: Anthropic model detected — using Anthropic API key")
return ["--api-key", f"anthropic={anthropic_key}"], model
# ── 2. Explicit host_label override ───────────────────────────────────────
if host_label:
ll = host_label.lower()
host = next((h for h in hosts if ll in h.get("label", "").lower()), None)
if host:
logger.debug("aider: using explicitly requested host '%s'", host.get("label"))
return _host_flags(host, model)
# ── 3. Model prefix hints ─────────────────────────────────────────────────
if model:
ml = model.lower()
for fragment, slug in _CLOUD_PROVIDER_URL_MAP:
if ml.startswith(slug + "/") or ml.startswith(fragment):
host = next(
(h for h in hosts if fragment in h.get("api_url", "").lower()), None
)
if host:
logger.debug("aider: model prefix '%s' → host '%s'", slug, host.get("label"))
return _host_flags(host, model)
# ── 4. Default priority ───────────────────────────────────────────────────
# OpenRouter first (most model coverage)
or_host = next((h for h in hosts if "openrouter.ai" in h.get("api_url", "")), None)
if or_host and or_host.get("api_key"):
logger.debug("aider: defaulting to OpenRouter")
return _host_flags(or_host, model)
# Anthropic API key (no model hint but it's configured)
if anthropic_key:
logger.debug("aider: defaulting to Anthropic API key")
return ["--api-key", f"anthropic={anthropic_key}"], model
# Any other keyed cloud host
for host in hosts:
slug = _provider_slug(host.get("api_url", ""))
if slug and host.get("api_key"):
logger.debug("aider: using keyed cloud host '%s'", host.get("label"))
return _host_flags(host, model)
# Generic / local host (no key or unknown provider)
for host in hosts:
flags, adj_model = _host_flags(host, model)
if flags:
logger.debug("aider: using local host '%s'", host.get("label"))
return flags, adj_model
logger.debug("aider: no credentials found in registry — relying on env vars / .aider.conf.yml")
return [], model
async def aider_run(
project: str,
task: str,
files: list[str] | None = None,
model: str | None = None,
host_label: str | None = None,
auto_commit: bool = True,
timeout: int = 300,
background: bool = False,
notify: bool = False,
) -> str:
"""Run Aider with a single task in a project directory, then exit.
Credentials are resolved automatically from the Cortex model registry. Use
host_label to pick a specific configured host (e.g. 'OpenRouter', 'Local').
When background=True, fires the subprocess asynchronously and returns an agent_id
immediately. Use agent_status(agent_id) to check progress; set notify=True to
receive a push/Talk notification on completion.
"""
resolved = _PROJECT_ALIASES.get(project, project)
cwd = Path(os.path.expanduser(resolved))
if not cwd.is_dir():
return f"Error: project directory '{resolved}' does not exist."
timeout = min(max(int(timeout), 10), 600)
# Resolve credentials before building the command (model name may be adjusted)
user = "scott"
extra_cred_flags: list[str] = []
try:
import model_registry
from persona import get_user
user = get_user() or "scott"
registry = model_registry.get_registry(user)
extra_cred_flags, model = _resolve_credentials(registry, model, host_label)
except Exception as e:
logger.debug("aider: credential resolution failed (%s) — relying on env", e)
cmd: list[str] = [
"aider",
"--message", task,
"--yes-always",
"--no-pretty",
"--no-stream",
"--no-check-update",
"--no-detect-urls",
"--auto-commits" if auto_commit else "--no-auto-commits",
]
cmd += extra_cred_flags
if model:
cmd += ["--model", model]
for f in (files or []):
cmd += ["--file", f]
logger.info(
"aider_run: project=%s model=%s host_label=%s auto_commit=%s background=%s task=%.120s",
project, model, host_label, auto_commit, background, task,
)
async def _run() -> str:
proc = await asyncio.create_subprocess_exec(
*cmd,
cwd=str(cwd),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=float(timeout))
out = stdout.decode(errors="replace").strip()
err = stderr.decode(errors="replace").strip()
parts = []
if out:
parts.append(out)
if err:
parts.append(f"[stderr]\n{err}")
combined = "\n".join(parts) if parts else "(no output)"
if len(combined) > _MAX_OUTPUT_CHARS:
half = _MAX_OUTPUT_CHARS // 2
combined = (
combined[:half]
+ f"\n\n[... {len(combined) - _MAX_OUTPUT_CHARS} chars trimmed ...]\n\n"
+ combined[-half:]
)
if proc.returncode not in (0, 1):
return f"[exit {proc.returncode}]\n{combined}"
return combined
if background:
rec = await agent_manager.register(
user=user,
role="aider",
task=task,
level=2,
notify=notify,
)
async def _bg_task() -> None:
try:
result = await _run()
await agent_manager.finish(rec.agent_id, result, "done")
logger.info("aider_run [bg]: done %s", rec.agent_id[:8])
except asyncio.CancelledError:
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
raise
except asyncio.TimeoutError:
msg = f"Aider timed out after {timeout}s"
logger.warning("aider_run [bg]: timeout %s", rec.agent_id[:8])
await agent_manager.finish(rec.agent_id, msg, "timeout")
except FileNotFoundError:
msg = "Error: 'aider' not found in PATH — run: pip install aider-chat"
await agent_manager.finish(rec.agent_id, msg, "failed")
except Exception as e:
logger.error("aider_run [bg]: failed %s: %s", rec.agent_id[:8], e)
await agent_manager.finish(rec.agent_id, str(e), "failed")
bg = asyncio.create_task(_bg_task())
agent_manager.set_task_ref(rec.agent_id, bg)
return (
f"Aider task started in background. ID: {rec.agent_id}\n"
f"Use agent_status('{rec.agent_id}') to monitor progress."
)
# Synchronous path
try:
return await _run()
except asyncio.TimeoutError:
return f"Error: aider timed out after {timeout}s"
except FileNotFoundError:
return "Error: 'aider' not found in PATH — run: pip install aider-chat"
except Exception as e:
logger.error("aider_run error: %s", e)
return f"Error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="aider_run",
description=(
"Run the Aider AI coding agent on a project with a single task, then exit. "
"Aider maps the repo, edits files, runs lint checks, and optionally commits. "
"Credentials are resolved automatically from the Cortex model registry — "
"OpenRouter, local Open WebUI/Ollama, Anthropic API, and other configured hosts "
"are all supported. Use host_label to pick a specific host. "
"Set background=True for long tasks — returns an agent_id immediately and sends "
"a notification when done. ADMIN ONLY. Requires confirmation."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"project": types.Schema(
type=types.Type.STRING,
description=(
"Project alias or absolute path. Known aliases: "
"'cortex' (this project), 'aether_api', 'aether_frontend', "
"'aether_container'. Or provide an absolute path."
),
),
"task": types.Schema(
type=types.Type.STRING,
description=(
"Full task description sent to Aider as --message. "
"Be specific — include file names, what to change, and why."
),
),
"files": types.Schema(
type=types.Type.ARRAY,
items=types.Schema(type=types.Type.STRING),
description=(
"Optional files to add explicitly to the editing context "
"(paths relative to project root). Aider builds a repo map "
"automatically — these get priority."
),
),
"model": types.Schema(
type=types.Type.STRING,
description=(
"Optional model override. Format depends on the provider: "
"'openrouter/anthropic/claude-3-5-haiku-20241022' (OpenRouter), "
"'claude-3-5-sonnet-20241022' (Anthropic direct), "
"'gemma-4-27b-it' or 'openai/gemma-4-27b-it' (local Open WebUI), "
"'deepseek/deepseek-chat' (DeepSeek via OpenRouter). "
"Defaults to the project's .aider.conf.yml model or AIDER_MODEL env var."
),
),
"host_label": types.Schema(
type=types.Type.STRING,
description=(
"Pick a specific configured host by label (partial match, case-insensitive). "
"Examples: 'OpenRouter', 'Local', 'scott-lt-i7-rtx'. "
"Overrides automatic credential resolution. "
"Omit to let credentials be chosen automatically."
),
),
"auto_commit": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Auto-commit changes after edits (default: true). "
"Set to false to review diffs before committing manually."
),
),
"timeout": types.Schema(
type=types.Type.INTEGER,
description="Max seconds to wait for Aider to finish (default 300, max 600).",
),
"background": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Run asynchronously in the background (default: false). "
"Returns an agent_id immediately; use agent_status(agent_id) to monitor. "
"Recommended for tasks expected to take more than ~60 seconds."
),
),
"notify": types.Schema(
type=types.Type.BOOLEAN,
description=(
"Send a push/Talk notification when the background task completes "
"(default: false). Only applies when background=true."
),
),
},
required=["project", "task"],
),
)
]

View File

@@ -17,6 +17,7 @@ import secrets
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path, get_user, get_persona
from cron_runner import load_crons, save_crons, parse_schedule
@@ -57,8 +58,9 @@ def _cron_add(label: str, schedule: str, job_type: str, payload: str) -> str:
except ValueError as e:
return f"Bad schedule: {e}"
if job_type not in ("remind", "note"):
return "Bad type: must be 'remind' or 'note'."
_VALID_TYPES = ("remind", "note", "message", "brief", "task")
if job_type not in _VALID_TYPES:
return f"Bad type: must be one of {', '.join(_VALID_TYPES)}."
current_user = get_user()
current_persona = get_persona()
@@ -194,3 +196,73 @@ async def cron_toggle(cron_id: str) -> str:
async def reminders_clear() -> str:
return await asyncio.to_thread(_reminders_clear)
DECLARATIONS = [
types.FunctionDeclaration(
name="cron_list",
description=(
"List all scheduled cron jobs — their ID, label, schedule, type, and last run time. "
"Use this to see what's scheduled before adding or removing jobs."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="cron_add",
description=(
"Create a new scheduled cron job and register it immediately (no restart needed). "
"Job types: "
"'remind' — appends to REMINDERS.md, auto-surfaced in chat context at tier 2+; "
"'note' — appends to SCRATCH.md, read on demand; "
"'message' — sends payload text directly to the user's notification channel; "
"'brief' — calls the LLM (no tools) with payload as the prompt, sends the response; "
"'task' — runs the full orchestrator tool loop with payload as the request, sends "
"Claude's response to the notification channel (use for agentic scheduled work: "
"research, checks, file updates, summaries that need tool access). "
"Schedule formats: 'hourly' | 'daily' | 'daily:HH:MM' | 'weekly:DOW' | 'weekly:DOW:HH:MM' | "
"'monthly' | 'monthly:DD' | 'monthly:DD:HH:MM' | 'yearly:MM:DD' | 'yearly:MM:DD:HH:MM'. "
"Examples: schedule='weekly:mon:08:00' for Monday briefings; "
"schedule='monthly:1:09:00' for a first-of-month review; "
"schedule='yearly:03:15' for a March 15 birthday reminder."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"label": types.Schema(type=types.Type.STRING, description="Short human-readable name for this job (e.g. 'Monday task summary')"),
"schedule": types.Schema(type=types.Type.STRING, description="When to run: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM | monthly | monthly:DD | monthly:DD:HH:MM | yearly:MM:DD | yearly:MM:DD:HH:MM"),
"job_type": types.Schema(type=types.Type.STRING, description="remind | note | message | brief | task"),
"payload": types.Schema(type=types.Type.STRING, description="The text/prompt to use when the job fires"),
},
required=["label", "schedule", "job_type", "payload"],
),
),
types.FunctionDeclaration(
name="cron_remove",
description=(
"Permanently delete a scheduled cron job. Use cron_list first to get the ID. "
"To temporarily disable without deleting, use cron_toggle instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"cron_id": types.Schema(type=types.Type.STRING, description="Job ID (e.g. c_abc123) — get from cron_list"),
},
required=["cron_id"],
),
),
types.FunctionDeclaration(
name="cron_toggle",
description=(
"Pause a running cron job, or resume a paused one. "
"The job stays in the list and can be re-enabled later. "
"Use cron_list to see current enabled/paused state."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"cron_id": types.Schema(type=types.Type.STRING, description="Job ID (e.g. c_abc123) — get from cron_list"),
},
required=["cron_id"],
),
),
]

View File

@@ -1,108 +1,78 @@
"""
File read tool — restricted to known-safe directory roots.
File read/write/search toolstwo access scopes.
Lets the orchestrator read local files (documentation, notes, config references)
without exposing arbitrary filesystem access. All paths are resolved and checked
against an allowlist of roots before any read is performed.
Project scope (no admin required):
project_file_read — read a file with optional line-range (offset)
project_file_list — list a directory with sizes + timestamps
file_stat — size, modified time, line count for a path
file_grep — regex search with context lines; up to 50 matches
file_syntax_check — py_compile (.py) or json.loads (.json) check
System scope (admin-only):
file_read — read a file from ~/agents_sync/, ~/OSIT_dev/, etc.
file_list — list a directory (same roots)
file_write — write/append (~/agents_sync/ + Cortex home/)
Session tools (user-level, persona-isolated):
session_read — read a session log by date
session_search — keyword search across session logs
All project-scope tools are restricted to the Cortex project root:
~/agents_sync/projects/Cortex_and_Inara_dev/
"""
import asyncio
import json
import logging
import re
import subprocess
from datetime import datetime
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
# Directories the orchestrator is allowed to read from.
# Paths are resolved (symlinks followed, ~ expanded) at import time.
_ALLOWED_ROOTS: list[Path] = [
# ── Access roots ──────────────────────────────────────────────────────────────
# Project root: two levels up from cortex/tools/files.py → Cortex_and_Inara_dev/
_PROJECT_ROOT: Path = Path(__file__).parent.parent.parent.resolve()
# System-wide read roots
def _build_allowed_roots() -> list[Path]:
roots = [
Path.home() / "agents_sync",
Path.home() / "OSIT_dev",
Path.home() / "DgrZone_Nextcloud",
Path.home() / "OSIT_Nextcloud",
]
]
try:
from config import settings
roots.append(settings.home_root())
except Exception:
pass
return roots
# Hard cap on file size to prevent accidental context blowout
_MAX_BYTES = 50_000 # ~50 KB
_ALLOWED_ROOTS: list[Path] = _build_allowed_roots()
# Write is tighter
_WRITE_ROOTS: list[Path] = [Path.home() / "agents_sync"]
# Size limits
_MAX_BYTES = 50_000
_MAX_LINES = 500
_MAX_GREP_MATCHES = 50
async def file_read(path: str, max_lines: int | None = None) -> str:
"""Read a local file and return its contents as a string.
Only files within allowed directories can be read:
~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, ~/OSIT_Nextcloud/
Args:
path: Absolute or home-relative path to the file (e.g. ~/agents_sync/CLAUDE.md).
max_lines: Optional line limit (default 500, hard cap). Use for large files.
Returns the file contents (truncated if over the size limit), or an error message.
"""
return await asyncio.to_thread(_sync_file_read, path, max_lines)
def _sync_file_read(path: str, max_lines: int | None) -> str:
# Expand ~ and resolve to absolute path
def _is_project_allowed(resolved: Path) -> bool:
try:
resolved = Path(path).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
# Security check — must be under an allowed root
if not _is_allowed(resolved):
allowed_str = ", ".join(str(r) for r in _ALLOWED_ROOTS)
return (
f"Access denied: {resolved}\n"
f"Allowed directories: {allowed_str}"
)
if not resolved.exists():
return f"File not found: {resolved}"
if not resolved.is_file():
# If it's a directory, list its contents instead
try:
entries = sorted(resolved.iterdir())
names = [e.name + ("/" if e.is_dir() else "") for e in entries[:100]]
return f"Directory listing for {resolved}:\n" + "\n".join(names)
except Exception as e:
return f"Cannot list directory: {e}"
# Read the file
try:
raw = resolved.read_bytes()
except Exception as e:
return f"Read error: {e}"
# Binary files
try:
text = raw.decode("utf-8")
except UnicodeDecodeError:
return f"Binary file (not readable as text): {resolved} [{len(raw)} bytes]"
# Apply line limit
limit = min(max_lines or _MAX_LINES, _MAX_LINES)
lines = text.splitlines()
truncated = False
if len(lines) > limit:
lines = lines[:limit]
truncated = True
# Apply byte cap as a final safety net
result = "\n".join(lines)
if len(result) > _MAX_BYTES:
result = result[:_MAX_BYTES]
truncated = True
if truncated:
result += f"\n\n… [truncated — file has {len(text.splitlines())} lines total]"
return result
resolved.relative_to(_PROJECT_ROOT)
return True
except ValueError:
return False
def _is_allowed(resolved: Path) -> bool:
"""Check that resolved path is under one of the allowed roots."""
for root in _ALLOWED_ROOTS:
try:
resolved.relative_to(root)
@@ -110,3 +80,725 @@ def _is_allowed(resolved: Path) -> bool:
except ValueError:
continue
return False
def _is_write_allowed(resolved: Path) -> bool:
for root in _WRITE_ROOTS:
try:
resolved.relative_to(root)
return True
except ValueError:
continue
try:
from config import settings
resolved.relative_to(settings.home_root())
return True
except (ValueError, Exception):
pass
return False
# ── Shared implementations ────────────────────────────────────────────────────
def _read_impl(path_str: str, offset: int | None, max_lines: int | None, is_allowed_fn) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not is_allowed_fn(resolved):
return f"Access denied: {resolved}"
if not resolved.exists():
return f"File not found: {resolved}"
if not resolved.is_file():
try:
entries = sorted(resolved.iterdir())
names = [e.name + ("/" if e.is_dir() else "") for e in entries[:100]]
return f"Directory listing for {resolved}:\n" + "\n".join(names)
except Exception as e:
return f"Cannot list directory: {e}"
try:
raw = resolved.read_bytes()
except Exception as e:
return f"Read error: {e}"
try:
text = raw.decode("utf-8")
except UnicodeDecodeError:
return f"Binary file (not readable as text): {resolved} [{len(raw)} bytes]"
all_lines = text.splitlines()
total = len(all_lines)
# offset is 1-based; default = start of file
start = max(0, (offset or 1) - 1)
working = all_lines[start:]
limit = min(max_lines or _MAX_LINES, _MAX_LINES)
truncated = False
if len(working) > limit:
working = working[:limit]
truncated = True
result = "\n".join(working)
if len(result) > _MAX_BYTES:
result = result[:_MAX_BYTES]
truncated = True
end_line = start + len(working)
header = f"[Lines {start + 1}{end_line} of {total}]\n" if (start > 0 or truncated) else ""
trailer = f"\n\n… [truncated — file has {total} lines; use offset={end_line + 1} to read more]" if truncated else ""
return header + result + trailer
def _list_impl(path_str: str, is_allowed_fn) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not is_allowed_fn(resolved):
return f"Access denied: {resolved}"
if not resolved.exists():
return f"Path not found: {resolved}"
if resolved.is_file():
return f"{resolved} is a file. Use file_read / project_file_read to read it."
try:
entries = sorted(resolved.iterdir(), key=lambda e: (e.is_file(), e.name.lower()))
lines = []
for e in entries[:200]:
if e.is_dir():
suffix = "/"
else:
try:
st = e.stat()
mtime = datetime.fromtimestamp(st.st_mtime).strftime("%Y-%m-%d %H:%M")
suffix = f" ({st.st_size:,} B, {mtime})"
except Exception:
suffix = ""
lines.append(f"{e.name}{suffix}")
result = "\n".join(lines)
if len(entries) > 200:
result += f"\n… ({len(entries) - 200} more not shown)"
return f"Contents of {resolved}:\n\n{result}"
except Exception as e:
return f"Cannot list directory: {e}"
# ── Project-scoped tools ──────────────────────────────────────────────────────
async def project_file_read(path: str, offset: int | None = None, max_lines: int | None = None) -> str:
"""Read a file within the Cortex project directory, with optional line range."""
return await asyncio.to_thread(_read_impl, path, offset, max_lines, _is_project_allowed)
async def project_file_list(path: str) -> str:
"""List directory contents within the Cortex project directory, with sizes and timestamps."""
return await asyncio.to_thread(_list_impl, path, _is_project_allowed)
async def file_stat(path: str) -> str:
"""Return metadata for a file or directory: type, size, modified time, line count."""
return await asyncio.to_thread(_sync_file_stat, path)
def _sync_file_stat(path_str: str) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
if not resolved.exists():
return f"Path not found: {resolved}"
try:
st = resolved.stat()
except Exception as e:
return f"Cannot stat: {e}"
modified = datetime.fromtimestamp(st.st_mtime).strftime("%Y-%m-%d %H:%M:%S")
lines = [
f"Path: {resolved}",
f"Type: {'directory' if resolved.is_dir() else 'file'}",
f"Size: {st.st_size:,} bytes",
f"Modified: {modified}",
]
if resolved.is_file():
try:
raw = resolved.read_bytes()
if b'\x00' not in raw[:1024]:
lines.append(f"Lines: {len(raw.decode('utf-8', errors='replace').splitlines())}")
except Exception:
pass
elif resolved.is_dir():
try:
entries = list(resolved.iterdir())
n_files = sum(1 for e in entries if e.is_file())
n_dirs = sum(1 for e in entries if e.is_dir())
lines.append(f"Contents: {n_files} file(s), {n_dirs} subdirector{'y' if n_dirs == 1 else 'ies'}")
except Exception:
pass
return "\n".join(lines)
async def file_grep(path: str, pattern: str, context_lines: int = 2, recursive: bool = True) -> str:
"""Search for a regex pattern in a file or directory, returning matching lines with context."""
return await asyncio.to_thread(_sync_file_grep, path, pattern, context_lines, recursive)
def _sync_file_grep(path_str: str, pattern: str, context_lines: int, recursive: bool) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
if not resolved.exists():
return f"Path not found: {resolved}"
try:
regex = re.compile(pattern, re.IGNORECASE)
except re.error as e:
return f"Invalid regex pattern: {e}"
ctx = max(0, min(context_lines, 5))
if resolved.is_file():
files_to_search = [resolved]
elif recursive:
files_to_search = sorted(f for f in resolved.rglob("*") if f.is_file())
else:
files_to_search = sorted(f for f in resolved.iterdir() if f.is_file())
total_matches = 0
sections: list[str] = []
capped = False
for fp in files_to_search:
if total_matches >= _MAX_GREP_MATCHES:
capped = True
break
try:
raw = fp.read_bytes()
except OSError:
continue
if b'\x00' in raw[:1024]:
continue # skip binary
try:
text = raw.decode("utf-8", errors="replace")
except Exception:
continue
file_lines = text.splitlines()
match_indices = [i for i, line in enumerate(file_lines) if regex.search(line)]
if not match_indices:
continue
total_matches += len(match_indices)
try:
label = str(fp.relative_to(_PROJECT_ROOT))
except ValueError:
label = str(fp)
file_output = [f"── {label} ──"]
printed: set[int] = set()
for mi in match_indices:
start = max(0, mi - ctx)
end = min(len(file_lines), mi + ctx + 1)
if printed and start > max(printed) + 1:
file_output.append(" ···")
for j in range(start, end):
if j not in printed:
marker = "" if j == mi else " "
file_output.append(f" {j + 1:4d}{marker} {file_lines[j]}")
printed.add(j)
sections.append("\n".join(file_output))
if not sections:
return f"No matches for '{pattern}' in {resolved}"
cap_note = f" (capped at {_MAX_GREP_MATCHES})" if capped else ""
header = f"grep '{pattern}'{total_matches} match(es){cap_note}:"
return header + "\n\n" + "\n\n".join(sections)
async def file_diff(path_a: str, path_b: str) -> str:
"""Compare two files and return a unified diff."""
return await asyncio.to_thread(_sync_file_diff, path_a, path_b)
def _sync_file_diff(path_a: str, path_b: str) -> str:
try:
resolved_a = Path(path_a).expanduser().resolve()
resolved_b = Path(path_b).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
for resolved in (resolved_a, resolved_b):
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}"
if not resolved.exists():
return f"File not found: {resolved}"
if not resolved.is_file():
return f"Not a file: {resolved}"
try:
result = subprocess.run(
["diff", "-u", str(resolved_a), str(resolved_b)],
capture_output=True, text=True, timeout=15,
)
if result.returncode == 0:
return f"Files are identical: {resolved_a.name} vs {resolved_b.name}"
output = result.stdout
if not output:
return f"diff returned no output (exit {result.returncode}): {result.stderr}"
if len(output) > _MAX_BYTES:
output = output[:_MAX_BYTES] + "\n… [truncated]"
return output
except subprocess.TimeoutExpired:
return "Timeout running diff"
except Exception as e:
return f"Error: {e}"
async def file_syntax_check(path: str) -> str:
"""Check syntax of a Python (.py) or JSON (.json) file."""
return await asyncio.to_thread(_sync_file_syntax_check, path)
def _sync_file_syntax_check(path_str: str) -> str:
try:
resolved = Path(path_str).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_project_allowed(resolved):
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
if not resolved.exists():
return f"File not found: {resolved}"
if not resolved.is_file():
return f"Not a file: {resolved}"
suffix = resolved.suffix.lower()
if suffix == ".py":
try:
result = subprocess.run(
["python3", "-m", "py_compile", str(resolved)],
capture_output=True, text=True, timeout=15,
)
if result.returncode == 0:
return f"OK — {resolved.name}: syntax valid"
err = (result.stderr or result.stdout).strip()
return f"Syntax error in {resolved.name}:\n{err}"
except subprocess.TimeoutExpired:
return f"Timeout running py_compile on {resolved.name}"
except Exception as e:
return f"Error: {e}"
elif suffix == ".json":
try:
text = resolved.read_text(encoding="utf-8")
json.loads(text)
return f"OK — {resolved.name}: valid JSON"
except json.JSONDecodeError as e:
return f"JSON error in {resolved.name}: {e}"
except Exception as e:
return f"Error reading {resolved.name}: {e}"
else:
return f"Syntax check not supported for '{suffix}' files. Supported: .py, .json"
# ── System-scoped tools ───────────────────────────────────────────────────────
async def file_read(path: str, offset: int | None = None, max_lines: int | None = None) -> str:
"""Read a local file from the broader system. Allowed: ~/agents_sync/, ~/OSIT_dev/, etc. ADMIN ONLY."""
return await asyncio.to_thread(_read_impl, path, offset, max_lines, _is_allowed)
async def file_list(path: str) -> str:
"""List directory contents from the broader system. ADMIN ONLY."""
return await asyncio.to_thread(_list_impl, path, _is_allowed)
async def file_write(path: str, content: str, mode: str = "overwrite") -> str:
"""Write or append content to a file. Write roots: ~/agents_sync/ and Cortex home/. ADMIN ONLY."""
return await asyncio.to_thread(_sync_file_write, path, content, mode)
def _sync_file_write(path: str, content: str, mode: str) -> str:
try:
resolved = Path(path).expanduser().resolve()
except Exception as e:
return f"Invalid path: {e}"
if not _is_write_allowed(resolved):
return (
f"Write access denied: {resolved}\n"
f"Allowed write roots: ~/agents_sync/ and the Cortex home/ directory."
)
if mode not in ("overwrite", "append"):
return f"Invalid mode '{mode}' — use 'overwrite' or 'append'."
try:
resolved.parent.mkdir(parents=True, exist_ok=True)
if mode == "append":
with resolved.open("a", encoding="utf-8") as f:
f.write(content)
return f"Appended {len(content)} chars to {resolved}"
else:
resolved.write_text(content, encoding="utf-8")
return f"Wrote {len(content)} chars to {resolved}"
except Exception as e:
logger.error("file_write error for %s: %s", resolved, e)
return f"Write error: {e}"
# ── Session tools ─────────────────────────────────────────────────────────────
_SEARCH_EXCERPT_CHARS = 150
async def session_read(date: str) -> str:
"""Read a full session log by date (YYYY-MM-DD)."""
return await asyncio.to_thread(_sync_session_read, date.strip())
def _sync_session_read(date: str) -> str:
from persona import persona_path
sessions_dir = persona_path() / "sessions"
if not sessions_dir.exists():
return "No session logs found."
target = sessions_dir / f"{date}.md"
if target.exists():
content = target.read_text()
return f"Session log for {date} ({len(content)} chars):\n\n{content}"
available = sorted([f.stem for f in sessions_dir.glob("*.md")], reverse=True)
if not available:
return "No session logs found."
recent = "\n".join(f" {d}" for d in available[:15])
return f"No session log found for '{date}'. Available dates (most recent first):\n{recent}"
async def session_search(query: str, limit: int = 5) -> str:
"""Search past session logs for a keyword or phrase."""
return await asyncio.to_thread(_sync_session_search, query, limit)
def _sync_session_search(query: str, limit: int) -> str:
from persona import persona_path
sessions_dir = persona_path() / "sessions"
if not sessions_dir.exists():
return "No session logs found."
limit = max(1, min(limit, 20))
pattern = re.compile(re.escape(query), re.IGNORECASE)
session_files = sorted(sessions_dir.glob("*.md"), reverse=True)
matches = []
for sf in session_files:
if len(matches) >= limit:
break
try:
text = sf.read_text()
except OSError:
continue
for m in pattern.finditer(text):
if len(matches) >= limit:
break
start = max(0, m.start() - _SEARCH_EXCERPT_CHARS)
end = min(len(text), m.end() + _SEARCH_EXCERPT_CHARS)
excerpt = text[start:end].strip()
if start > 0:
excerpt = "" + excerpt
if end < len(text):
excerpt = excerpt + ""
matches.append(f"[{sf.stem}] {excerpt}")
if not matches:
return f"No matches for '{query}' across {len(session_files)} session logs."
header = f"Session search: '{query}'{len(matches)} match(es) across {len(session_files)} logs\n"
return header + "\n\n".join(matches)
# ── Declarations ──────────────────────────────────────────────────────────────
DECLARATIONS = [
# Project-scoped
types.FunctionDeclaration(
name="project_file_read",
description=(
"Read a file within the Cortex project directory (source code, docs, config, persona files). "
"Supports reading a specific line range via offset — use to page through large files "
"without re-reading from the top. If given a directory path, returns a listing instead. "
"Project root: ~/agents_sync/projects/Cortex_and_Inara_dev/"
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the file",
),
"offset": types.Schema(
type=types.Type.INTEGER,
description="Start reading from this line number (1-based). Omit to read from the top.",
),
"max_lines": types.Schema(
type=types.Type.INTEGER,
description="Maximum lines to return (default 500)",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="project_file_list",
description=(
"List files and subdirectories within the Cortex project directory. "
"Shows file sizes and modified timestamps. "
"Project root: ~/agents_sync/projects/Cortex_and_Inara_dev/"
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the directory",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_stat",
description=(
"Get metadata for a file or directory: type, size, modified timestamp, line count (for text files) "
"or entry counts (for directories). Use before reading to check recency or size. "
"Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the file or directory",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_grep",
description=(
"Search for a regex pattern in a file or directory, returning matching lines with surrounding "
"context. Much more efficient than reading an entire source file — use this to find function "
"definitions, variable names, TODO comments, imports, error strings, etc. "
"Searches recursively by default. Capped at 50 matches. Skips binary files. "
"Case-insensitive. Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="File or directory to search (e.g. ~/agents_sync/projects/Cortex_and_Inara_dev/cortex/)",
),
"pattern": types.Schema(
type=types.Type.STRING,
description="Regex pattern to search for (case-insensitive). Examples: 'def ha_', 'import httpx', 'TODO'",
),
"context_lines": types.Schema(
type=types.Type.INTEGER,
description="Lines of context before/after each match (default 2, max 5)",
),
"recursive": types.Schema(
type=types.Type.BOOLEAN,
description="Search subdirectories recursively (default true)",
),
},
required=["path", "pattern"],
),
),
types.FunctionDeclaration(
name="file_diff",
description=(
"Compare two files and return a unified diff (diff -u). "
"Use for code review, verifying what changed between two versions of a file, "
"or comparing config files side-by-side. "
"Returns 'Files are identical' if there are no differences. "
"Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path_a": types.Schema(
type=types.Type.STRING,
description="Path to the first file (the 'before' or reference file)",
),
"path_b": types.Schema(
type=types.Type.STRING,
description="Path to the second file (the 'after' or comparison file)",
),
},
required=["path_a", "path_b"],
),
),
types.FunctionDeclaration(
name="file_syntax_check",
description=(
"Check the syntax of a Python (.py) or JSON (.json) file without executing it. "
"Returns OK or the error with line number. "
"Use after editing a file before restarting Cortex. "
"Restricted to the Cortex project directory."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Path to the .py or .json file to check",
),
},
required=["path"],
),
),
# System-scoped
types.FunctionDeclaration(
name="file_read",
description=(
"Read a local file from the broader system (~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, "
"~/OSIT_Nextcloud/, Cortex home/). Supports offset for reading specific line ranges. "
"For files within the Cortex project, prefer project_file_read instead. "
"ADMIN ONLY."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the file",
),
"offset": types.Schema(
type=types.Type.INTEGER,
description="Start reading from this line number (1-based)",
),
"max_lines": types.Schema(
type=types.Type.INTEGER,
description="Maximum lines to return (default 500)",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_list",
description=(
"List files and subdirectories from the broader system. "
"Shows sizes and modified timestamps. "
"Allowed: ~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, ~/OSIT_Nextcloud/. "
"ADMIN ONLY."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to the directory",
),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="file_write",
description=(
"Write or append content to a file. "
"Write-allowed paths: ~/agents_sync/ and the Cortex home/ directory. "
"Creates parent directories if needed. "
"ADMIN ONLY. Requires user confirmation before executing."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(
type=types.Type.STRING,
description="Absolute or ~/... path to write to",
),
"content": types.Schema(
type=types.Type.STRING,
description="Content to write",
),
"mode": types.Schema(
type=types.Type.STRING,
description="'overwrite' (default, replaces file) or 'append' (adds to end)",
),
},
required=["path", "content"],
),
),
types.FunctionDeclaration(
name="session_read",
description=(
"Read a full conversation session log by date (YYYY-MM-DD). "
"Useful for continuity and recalling past decisions. "
"If the date is not found, lists available dates. "
"Only reads this user's own sessions."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"date": types.Schema(
type=types.Type.STRING,
description="Date in YYYY-MM-DD format (e.g. '2026-05-08')",
),
},
required=["date"],
),
),
types.FunctionDeclaration(
name="session_search",
description=(
"Search past conversation session logs for a keyword or phrase. "
"Returns matching excerpts with session dates, newest first. "
"Only searches this user's own sessions."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"query": types.Schema(
type=types.Type.STRING,
description="Keyword or phrase to search for",
),
"limit": types.Schema(
type=types.Type.INTEGER,
description="Max results to return (default 5, max 20)",
),
},
required=["query"],
),
),
]

158
cortex/tools/git.py Normal file
View File

@@ -0,0 +1,158 @@
"""
Git inspection tools — project-scoped, read-only.
git_status — working tree status (staged, unstaged, untracked changes)
git_log — recent commit history with optional path filter
git_diff — diff between commits, branches, or working tree vs HEAD
"""
import asyncio
import logging
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
_PROJECT_ROOT: Path = Path(__file__).parent.parent.parent.resolve()
_MAX_OUTPUT = 50_000
async def _git(*args: str, timeout: int = 15) -> tuple[int, str]:
"""Run a git command in the project root. Returns (returncode, output)."""
proc = await asyncio.create_subprocess_exec(
"git", "-C", str(_PROJECT_ROOT), *args,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
except asyncio.TimeoutError:
proc.kill()
return 1, "git command timed out"
out = (stdout or b"").decode(errors="replace").strip()
err = (stderr or b"").decode(errors="replace").strip()
combined = out if out else err
return proc.returncode, combined
def _cap(text: str) -> str:
if len(text) > _MAX_OUTPUT:
return text[:_MAX_OUTPUT] + "\n… [truncated]"
return text
async def git_status() -> str:
"""Return the current git working tree status."""
rc, out = await _git("status")
if rc != 0:
return f"git status failed: {out}"
return out or "Working tree clean — nothing to report."
async def git_log(n: int = 20, path: str = "", oneline: bool = True) -> str:
"""Return recent git commit history."""
args = ["log"]
if oneline:
args += ["--oneline"]
else:
args += ["--format=%H %as %an%n %s", "--date=short"]
args += [f"-{max(1, min(n, 200))}"]
if path:
args += ["--", path]
rc, out = await _git(*args)
if rc != 0:
return f"git log failed: {out}"
return _cap(out) or "No commits found."
async def git_diff(ref_a: str = "", ref_b: str = "", path: str = "", stat_only: bool = False) -> str:
"""Show a git diff. Defaults to working tree vs HEAD (unstaged changes)."""
args = ["diff"]
if stat_only:
args += ["--stat"]
if ref_a and ref_b:
args += [f"{ref_a}..{ref_b}"]
elif ref_a:
args += [ref_a]
if path:
args += ["--", path]
rc, out = await _git(*args)
# diff exits 1 when there are differences — that's normal
if rc not in (0, 1):
return f"git diff failed: {out}"
return _cap(out) or "No differences found."
# ── Declarations ──────────────────────────────────────────────────────────────
DECLARATIONS = [
types.FunctionDeclaration(
name="git_status",
description=(
"Show the current git working tree status for the Cortex project: "
"staged changes, unstaged modifications, and untracked files. "
"Use to check whether there are uncommitted changes before restarting or deploying."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={},
),
),
types.FunctionDeclaration(
name="git_log",
description=(
"Show recent git commit history for the Cortex project. "
"Returns commit hashes, dates, and messages. "
"Optionally filter to a specific file or directory path."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"n": types.Schema(
type=types.Type.INTEGER,
description="Number of commits to return (default 20, max 200)",
),
"path": types.Schema(
type=types.Type.STRING,
description="Optional file or directory path to filter commits by",
),
"oneline": types.Schema(
type=types.Type.BOOLEAN,
description="Use compact one-line format (default true). Set false for more detail.",
),
},
),
),
types.FunctionDeclaration(
name="git_diff",
description=(
"Show a git diff for the Cortex project. "
"With no arguments: shows unstaged working tree changes vs HEAD. "
"With ref_a only: shows changes between that ref and HEAD. "
"With ref_a and ref_b: shows changes between the two refs (commits, branches, or tags). "
"Use stat_only to get a summary of changed files instead of full patch output."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"ref_a": types.Schema(
type=types.Type.STRING,
description="First ref (commit hash, branch name, or tag). Omit for working tree diff.",
),
"ref_b": types.Schema(
type=types.Type.STRING,
description="Second ref. When provided with ref_a, shows diff between the two.",
),
"path": types.Schema(
type=types.Type.STRING,
description="Optional file or directory path to restrict the diff to",
),
"stat_only": types.Schema(
type=types.Type.BOOLEAN,
description="Return only a file-change summary (--stat) instead of the full diff",
),
},
),
),
]

View File

@@ -0,0 +1,277 @@
"""
Home Assistant tools — read device states and call services.
Credentials are read automatically from the current user's channels.json:
"homeassistant": {"url": "https://ha.example.com", "token": "<long-lived-token>"}
Configure in Settings → Notifications → Home Assistant.
"""
import json
import logging
import httpx
from google.genai import types
from auth_utils import get_user_channels
from persona import get_user
logger = logging.getLogger(__name__)
_TIMEOUT = 10
# Attributes that are internal/noisy and not useful to show
_SKIP_ATTRS = {
"friendly_name", "icon", "entity_picture", "supported_features",
"supported_color_modes", "color_mode", "min_color_temp_kelvin",
"max_color_temp_kelvin", "min_mireds", "max_mireds",
"assumed_state", "attribution",
}
def _get_ha_cfg() -> tuple[str, str]:
"""Return (base_url, token) from the current user's channels.json."""
channels = get_user_channels(get_user())
ha = channels.get("homeassistant") or {}
url = (ha.get("url") or "").rstrip("/")
token = ha.get("token") or ""
if not url or not token:
raise ValueError(
"Home Assistant not configured — add URL and token in Settings → Notifications."
)
return url, token
def _auth(token: str) -> dict:
return {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
def _fmt_state(s: dict) -> str:
"""Format a single HA state dict as a compact readable line."""
entity_id = s.get("entity_id", "")
state = s.get("state", "unknown")
attrs = s.get("attributes", {})
name = attrs.get("friendly_name", entity_id)
label = f"{name} ({entity_id})" if name != entity_id else entity_id
useful = {k: v for k, v in attrs.items() if k not in _SKIP_ATTRS}
extra = ""
if useful:
parts = []
for k, v in list(useful.items())[:6]: # cap at 6 attrs per entity
parts.append(f"{k}: {v}")
extra = " [" + ", ".join(parts) + "]"
return f"{label}: {state}{extra}"
async def ha_get_state(entity_id: str) -> str:
"""Return the current state and attributes of a single Home Assistant entity."""
try:
url, token = _get_ha_cfg()
except ValueError as e:
return str(e)
try:
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
resp = await client.get(f"{url}/api/states/{entity_id}", headers=_auth(token))
if resp.status_code == 404:
return f"Entity not found: {entity_id}"
if resp.status_code != 200:
return f"HA API error {resp.status_code}: {resp.text[:400]}"
s = resp.json()
attrs = s.get("attributes", {})
lines = [
f"**{attrs.get('friendly_name', entity_id)}** (`{entity_id}`)",
f"State: **{s.get('state', 'unknown')}**",
]
changed = (s.get("last_changed") or "")[:19].replace("T", " ")
if changed:
lines.append(f"Last changed: {changed} UTC")
useful = {k: v for k, v in attrs.items() if k not in _SKIP_ATTRS}
if useful:
lines.append("Attributes:")
for k, v in useful.items():
lines.append(f" {k}: {v}")
return "\n".join(lines)
except httpx.HTTPError as e:
return f"Connection error: {e}"
except Exception as e:
logger.warning("ha_get_state error: %s", e)
return f"Error: {e}"
async def ha_get_states(domain: str = "", area: str = "") -> str:
"""List HA entity states, optionally filtered by domain (e.g. 'light') or area name."""
try:
url, token = _get_ha_cfg()
except ValueError as e:
return str(e)
try:
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
resp = await client.get(f"{url}/api/states", headers=_auth(token))
if resp.status_code != 200:
return f"HA API error {resp.status_code}: {resp.text[:400]}"
states = resp.json()
if domain:
states = [s for s in states if s.get("entity_id", "").startswith(f"{domain}.")]
if area:
al = area.lower()
states = [s for s in states
if al in (s.get("attributes", {}).get("friendly_name") or "").lower()]
if not states:
filters = [f"domain={domain}"] * bool(domain) + [f"area={area}"] * bool(area)
return "No entities found" + (f" ({', '.join(filters)})" if filters else "")
lines = [f"{len(states)} entit{'y' if len(states) == 1 else 'ies'}:"]
for s in sorted(states, key=lambda x: x.get("entity_id", "")):
lines.append(_fmt_state(s))
return "\n".join(lines)
except httpx.HTTPError as e:
return f"Connection error: {e}"
except Exception as e:
logger.warning("ha_get_states error: %s", e)
return f"Error: {e}"
async def ha_call_service(
domain: str,
service: str,
entity_id: str = "",
data: str = "",
) -> str:
"""Call a Home Assistant service (turn on/off lights, set thermostat, lock doors, etc.)."""
try:
url, token = _get_ha_cfg()
except ValueError as e:
return str(e)
payload: dict = {}
if entity_id:
payload["entity_id"] = entity_id
if data:
try:
extra = json.loads(data)
if isinstance(extra, dict):
payload.update(extra)
except json.JSONDecodeError:
return f"Invalid JSON in data: {data}"
try:
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
resp = await client.post(
f"{url}/api/services/{domain}/{service}",
headers=_auth(token),
json=payload,
)
if resp.status_code not in (200, 201):
return f"HA API error {resp.status_code}: {resp.text[:400]}"
changed = resp.json()
if not changed:
return f"{domain}.{service} called (no state changes reported)."
lines = [f"{domain}.{service}{len(changed)} entity state(s) updated:"]
for s in changed:
lines.append(f" {s.get('entity_id', '')}: {s.get('state', '')}")
return "\n".join(lines)
except httpx.HTTPError as e:
return f"Connection error: {e}"
except Exception as e:
logger.warning("ha_call_service error: %s", e)
return f"Error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="ha_get_state",
description=(
"Get the current state and attributes of a single Home Assistant entity. "
"Use to check if a light is on, read a thermostat temperature, check a "
"door/window sensor, battery level, HVAC mode, etc. "
"entity_id format: domain.name — e.g. light.living_room, switch.garage, "
"climate.ecobee, binary_sensor.front_door, sensor.outdoor_temp."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"entity_id": types.Schema(
type=types.Type.STRING,
description="Full entity ID, e.g. light.living_room or climate.ecobee_main",
),
},
required=["entity_id"],
),
),
types.FunctionDeclaration(
name="ha_get_states",
description=(
"List Home Assistant entity states, optionally filtered by domain or area. "
"Use to survey what devices exist or check multiple entities at once. "
"Domain examples: light, switch, sensor, climate, binary_sensor, lock, cover, "
"media_player, input_boolean. Leave both blank to list everything (can be large)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"domain": types.Schema(
type=types.Type.STRING,
description="Filter to this domain, e.g. 'light' or 'switch' (optional)",
),
"area": types.Schema(
type=types.Type.STRING,
description="Filter by area name substring match on friendly name (optional)",
),
},
),
),
types.FunctionDeclaration(
name="ha_call_service",
description=(
"Call a Home Assistant service to control a device or trigger an automation. "
"Requires user confirmation before executing. Common examples: "
"domain=light service=turn_on entity_id=light.living_room; "
"domain=light service=turn_off entity_id=light.all; "
"domain=switch service=toggle entity_id=switch.garage; "
"domain=climate service=set_temperature data={\"temperature\":72}; "
"domain=lock service=lock entity_id=lock.front_door; "
"domain=script service=turn_on entity_id=script.goodnight."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"domain": types.Schema(
type=types.Type.STRING,
description="Service domain: light, switch, climate, lock, cover, script, automation, etc.",
),
"service": types.Schema(
type=types.Type.STRING,
description="Service name: turn_on, turn_off, toggle, set_temperature, lock, unlock, open, close, etc.",
),
"entity_id": types.Schema(
type=types.Type.STRING,
description="Target entity ID — omit for services that don't target a specific entity",
),
"data": types.Schema(
type=types.Type.STRING,
description='Extra service data as JSON string, e.g. {"temperature": 72, "hvac_mode": "heat"}',
),
},
required=["domain", "service"],
),
),
]

234
cortex/tools/notify.py Normal file
View File

@@ -0,0 +1,234 @@
"""
Notification tools — proactively send messages to user channels.
nc_talk_send routes through notification.py → channels.json.
email_send uses the server SMTP config from .env (smtp_server, smtp_from_*).
"""
import asyncio
import json
import logging
import re
import httpx
from google.genai import types
from config import settings
from persona import get_user
logger = logging.getLogger(__name__)
def _load_allowlist(username: str) -> list[str]:
"""Load the per-user email allowlist. Returns empty list if not configured."""
path = settings.home_root() / username / "email_allowlist.json"
try:
return [str(p).strip() for p in json.loads(path.read_text()) if str(p).strip()]
except FileNotFoundError:
return []
except Exception as e:
logger.warning("failed to read email_allowlist.json for %s: %s", username, e)
return []
def _email_allowed(address: str, patterns: list[str]) -> bool:
"""Return True if address matches any pattern (regex, case-insensitive full match)."""
addr = address.strip()
for pattern in patterns:
try:
if re.fullmatch(pattern, addr, re.IGNORECASE):
return True
except re.error:
logger.warning("invalid regex in email allowlist: %r", pattern)
return False
async def email_send(to: str, subject: str, body: str) -> str:
"""Send an email via the server's configured SMTP account."""
username = get_user()
allowlist = _load_allowlist(username)
if not allowlist:
return (
"Email blocked — no allowlist configured. "
f"Add allowed patterns to home/{username}/email_allowlist.json as a JSON array."
)
if not _email_allowed(to, allowlist):
return f"Email blocked — {to} does not match any allowed pattern for {username}."
from email_utils import send_email
ok = await asyncio.to_thread(
send_email,
to_email=to,
subject=subject,
body_text=body,
body_html=body.replace("\n", "<br>"),
)
if ok:
return f"Email sent to {to}."
return "Failed to send email — check SMTP configuration in .env."
async def web_push(title: str, body: str, url: str = "") -> str:
"""Send a browser push notification to the current user's registered devices."""
import push_utils
username = get_user()
result = await push_utils.send_push(username, title, body, url)
if "error" in result:
return f"Push failed: {result['error']}"
return f"Push sent to {result['sent']} device(s) for {username} (pruned {result['pruned']} stale)."
async def nc_talk_history(conversation_token: str = "", limit: int = 20) -> str:
"""Read recent messages from a Nextcloud Talk conversation.
Requires nc_username and nc_app_password in channels.json under 'nextcloud'.
conversation_token defaults to notification_room if not specified.
"""
from auth_utils import get_user_channels
username = get_user()
channels = get_user_channels(username)
nct = channels.get("nextcloud", {})
url = nct.get("url", "").rstrip("/")
nc_username = nct.get("nc_username", "").strip()
nc_app_password = nct.get("nc_app_password", "").strip()
token = conversation_token.strip() or nct.get("notification_room", "").strip()
if not url or not nc_username or not nc_app_password:
return (
"nc_talk_history requires nc_username and nc_app_password in channels.json "
f"(under 'nextcloud'). Add these to home/{username}/channels.json to enable message reading."
)
if not token:
return "No conversation token provided and no notification_room set in channels.json."
limit = min(max(int(limit), 1), 200)
return await asyncio.to_thread(_sync_nc_talk_history, url, nc_username, nc_app_password, token, limit)
def _sync_nc_talk_history(url: str, nc_user: str, nc_pass: str, token: str, limit: int) -> str:
from datetime import datetime, timezone
endpoint = f"{url}/ocs/v2.php/apps/spreed/api/v4/chat/{token}"
try:
resp = httpx.get(
endpoint,
params={"limit": limit, "lookIntoFuture": 0, "setReadMarker": 0, "noStatusUpdate": 1},
auth=(nc_user, nc_pass),
headers={"OCS-APIRequest": "true", "Accept": "application/json"},
timeout=15,
)
except Exception as e:
return f"NC Talk API error: {e}"
if resp.status_code != 200:
return f"NC Talk API returned HTTP {resp.status_code}: {resp.text[:200]}"
try:
messages = resp.json().get("ocs", {}).get("data", [])
except Exception as e:
return f"Failed to parse NC Talk response: {e}"
if not messages:
return "No messages found in this conversation."
# NC Talk returns newest-first; reverse to chronological order
lines = [f"Last {len(messages)} messages from {token}:\n"]
for msg in reversed(messages):
sender = msg.get("actorDisplayName") or msg.get("actorId") or "Unknown"
ts = msg.get("timestamp", 0)
time_str = datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
text = msg.get("message", "")
if msg.get("messageType") == "system":
lines.append(f"[system {time_str}] {text}")
else:
lines.append(f"{sender} ({time_str}): {text}")
return "\n".join(lines)
async def nc_talk_send(message: str) -> str:
"""Send a message to the user via their configured notification channel.
Channel is resolved from the user's channels.json (notification_channel key).
Falls back to Nextcloud Talk if configured. No-op if no channel is set.
"""
from notification import notify
username = get_user()
try:
await notify(username, message)
return f"Message sent to {username}'s notification channel."
except Exception as e:
logger.warning("nc_talk_send error for %s: %s", username, e)
return f"Failed to send notification: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="web_push",
description=(
"Send a browser push notification to the current user. Works even when the "
"Cortex tab is not open. Use for completing long tasks, reminders that fire "
"in the background, or anything the user should see immediately. "
"url is optional — if set, clicking the notification opens that URL."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"title": types.Schema(type=types.Type.STRING, description="Notification title (short)"),
"body": types.Schema(type=types.Type.STRING, description="Notification body text"),
"url": types.Schema(type=types.Type.STRING, description="Optional URL to open on click"),
},
required=["title", "body"],
),
),
types.FunctionDeclaration(
name="email_send",
description=(
"Send an email from the server's configured SMTP account. Use for delivering "
"summaries, reports, reminders, or any content the user wants emailed. "
"body is plain text; newlines are preserved."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"to": types.Schema(type=types.Type.STRING, description="Recipient email address"),
"subject": types.Schema(type=types.Type.STRING, description="Email subject line"),
"body": types.Schema(type=types.Type.STRING, description="Plain-text email body"),
},
required=["to", "subject", "body"],
),
),
types.FunctionDeclaration(
name="nc_talk_send",
description=(
"Send a proactive message to the user via their configured notification channel "
"(Nextcloud Talk by default). Use this to notify the user of completed background "
"tasks, important events, or anything they should know between sessions. "
"Requires notification_channel and notification_room set in channels.json."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"message": types.Schema(type=types.Type.STRING, description="The message to send to the user"),
},
required=["message"],
),
),
types.FunctionDeclaration(
name="nc_talk_history",
description=(
"Read recent messages from a Nextcloud Talk conversation. Useful for checking "
"what was said in a room before composing a reply, or reviewing recent context. "
"Requires nc_username and nc_app_password in channels.json under 'nextcloud'. "
"conversation_token defaults to notification_room if not provided."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"conversation_token": types.Schema(type=types.Type.STRING, description="NC Talk room token (defaults to notification_room from channels.json)"),
"limit": types.Schema(type=types.Type.INTEGER, description="Number of messages to return (default 20, max 200)"),
},
required=[],
),
),
]

255
cortex/tools/reminders.py Normal file
View File

@@ -0,0 +1,255 @@
"""
Reminders tools.
Reminders are stored in persona/REMINDERS.md and automatically surfaced
in the system prompt at Tier 2+. Each reminder can have an optional due date —
only due or undated reminders surface in context; future-dated ones are stored
but invisible until their date arrives.
Operations:
reminders_add — append a new reminder, optional due date (YYYY-MM-DD)
reminders_list — return all reminders with due status (including future)
reminders_remove — remove a single reminder by number
reminders_clear — erase all reminders
"""
import asyncio
import re
from datetime import datetime, timezone, date as _date
from pathlib import Path
from google.genai import types
from persona import persona_path
def _reminders_path() -> Path:
return persona_path() / "REMINDERS.md"
def _now_label() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
def _parse_sections(text: str) -> list[tuple[str, str]]:
"""Split REMINDERS.md into (heading, body) tuples, one per ## section."""
sections: list[tuple[str, str]] = []
heading: str | None = None
body_lines: list[str] = []
for line in text.splitlines():
if line.startswith("## "):
if heading is not None:
sections.append((heading, "\n".join(body_lines).strip()))
heading = line[3:].strip()
body_lines = []
elif heading is not None:
body_lines.append(line)
if heading is not None:
sections.append((heading, "\n".join(body_lines).strip()))
return sections
def _sections_to_text(sections: list[tuple[str, str]]) -> str:
return "".join(f"\n## {h}\n\n{b}\n" for h, b in sections)
def _parse_due(body: str) -> _date | None:
"""Extract due date from a 'due: YYYY-MM-DD' line in the body, if present."""
m = re.search(r'^due:\s*(\d{4}-\d{2}-\d{2})', body, re.MULTILINE | re.IGNORECASE)
if not m:
return None
try:
return _date.fromisoformat(m.group(1))
except ValueError:
return None
def _today() -> _date:
return datetime.now().astimezone().date()
def _is_due_or_undated(body: str) -> bool:
"""Return True if this reminder has no due date or its due date is today or past."""
due = _parse_due(body)
return due is None or due <= _today()
def _due_label(body: str) -> str:
"""Return a human-readable due status string for reminders_list output."""
due = _parse_due(body)
if due is None:
return ""
today = _today()
if due < today:
days = (today - due).days
return f" [OVERDUE by {days} day{'s' if days != 1 else ''} — due {due}]"
if due == today:
return " [due TODAY]"
return f" [due: {due}]"
def _body_without_due(body: str) -> str:
"""Strip the due: line from body for display (due status shown in heading line)."""
return re.sub(r'^due:\s*\S+\s*\n?', '', body, count=1, flags=re.MULTILINE | re.IGNORECASE).strip()
# ---------------------------------------------------------------------------
# Sync implementations
# ---------------------------------------------------------------------------
def _reminders_list() -> str:
p = _reminders_path()
if not p.exists() or not p.read_text().strip():
return "No pending reminders."
sections = _parse_sections(p.read_text())
if not sections:
return "No pending reminders."
lines = []
for i, (heading, body) in enumerate(sections, 1):
status = _due_label(body)
lines.append(f"{i}. {heading}{status}")
display_body = _body_without_due(body)
if display_body:
for bline in display_body.splitlines()[:4]:
lines.append(f" {bline}")
lines.append("")
return "\n".join(lines).rstrip()
def _reminders_add(text: str, label: str | None = None, due: str | None = None) -> str:
p = _reminders_path()
existing = p.read_text() if p.exists() else ""
heading = label or _now_label()
body = text.strip()
if due:
body = f"due: {due}\n{body}"
section = f"\n## {heading}\n\n{body}\n"
p.write_text(existing.rstrip() + "\n" + section)
msg = f"Reminder added: {heading}"
if due:
msg += f" (due: {due})"
return msg
def _reminders_remove(index: int) -> str:
p = _reminders_path()
if not p.exists() or not p.read_text().strip():
return "No reminders to remove."
sections = _parse_sections(p.read_text())
if not sections:
return "No reminders to remove."
if index < 1 or index > len(sections):
return (
f"Index {index} is out of range. "
f"There {'is' if len(sections) == 1 else 'are'} {len(sections)} "
f"reminder{'s' if len(sections) != 1 else ''} (1{len(sections)}). "
"Call reminders_list to see them."
)
removed_heading = sections[index - 1][0]
sections.pop(index - 1)
p.write_text(_sections_to_text(sections))
return f"Removed reminder {index}: {removed_heading}"
def _reminders_clear() -> str:
p = _reminders_path()
p.write_text("")
return "All reminders cleared."
# ---------------------------------------------------------------------------
# Public helper for context_loader
# ---------------------------------------------------------------------------
def load_due_reminders() -> str:
"""Return REMINDERS.md content filtered to only due and undated sections.
Called by context_loader at Tier 2+. Future-dated reminders are excluded
from the system prompt until their due date arrives.
"""
p = _reminders_path()
if not p.exists():
return ""
text = p.read_text()
if not text.strip():
return ""
sections = _parse_sections(text)
due_sections = [(h, b) for h, b in sections if _is_due_or_undated(b)]
if not due_sections:
return ""
# Strip the raw due: line from body — the date is already part of the heading context
cleaned = [(h, _body_without_due(b)) for h, b in due_sections]
return _sections_to_text(cleaned).strip()
# ---------------------------------------------------------------------------
# Async wrappers
# ---------------------------------------------------------------------------
async def reminders_list() -> str:
return await asyncio.to_thread(_reminders_list)
async def reminders_add(text: str, label: str | None = None, due: str | None = None) -> str:
return await asyncio.to_thread(_reminders_add, text, label, due)
async def reminders_remove(index: int) -> str:
return await asyncio.to_thread(_reminders_remove, index)
async def reminders_clear() -> str:
return await asyncio.to_thread(_reminders_clear)
DECLARATIONS = [
types.FunctionDeclaration(
name="reminders_add",
description=(
"Add a new reminder to REMINDERS.md. Reminders are automatically surfaced "
"in context at the start of each session (Tier 2+). "
"Use this when the user asks you to remember something or follow up on something. "
"Set a due date to suppress the reminder until that date — useful for future tasks "
"that would be noise today."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"text": types.Schema(type=types.Type.STRING, description="The reminder text"),
"label": types.Schema(type=types.Type.STRING, description="Optional heading (e.g. 'Follow up on NC Talk'). Defaults to current timestamp."),
"due": types.Schema(type=types.Type.STRING, description="Optional due date in YYYY-MM-DD format. Reminder is hidden from context until this date arrives. Omit for an always-visible reminder."),
},
required=["text"],
),
),
types.FunctionDeclaration(
name="reminders_list",
description=(
"Read all pending reminders, including future-dated ones not yet in context. "
"Shows due status for each (due today, overdue, or future date). "
"Use this before adding to avoid duplicates, or to show the user what's queued."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="reminders_remove",
description=(
"Remove a single reminder by its number. "
"Call reminders_list first to get the numbered list, then pass the number to remove."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"index": types.Schema(type=types.Type.INTEGER, description="The number of the reminder to remove (1 = first in reminders_list output)."),
},
required=["index"],
),
),
types.FunctionDeclaration(
name="reminders_clear",
description=(
"Erase all pending reminders from REMINDERS.md. "
"Use this after you have acknowledged and acted on the reminders shown in your context."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

View File

@@ -17,6 +17,7 @@ import asyncio
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path
@@ -77,3 +78,51 @@ async def scratch_append(content: str, heading: str | None = None) -> str:
async def scratch_clear() -> str:
return await asyncio.to_thread(_scratch_clear)
DECLARATIONS = [
types.FunctionDeclaration(
name="scratch_read",
description=(
"Read the full contents of the scratchpad. "
"Use this to recall working notes, mid-task context, or anything previously jotted down. "
"The scratchpad is transient — nothing here is distilled or archived."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="scratch_write",
description=(
"Replace the entire scratchpad with new content. "
"Use this to set a clean working note, replacing whatever was there before. "
"For adding without replacing, use scratch_append instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(type=types.Type.STRING, description="The new scratchpad content (markdown supported)"),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="scratch_append",
description=(
"Add a new section to the bottom of the scratchpad without replacing existing content. "
"Each section gets a timestamp heading unless you supply one."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"content": types.Schema(type=types.Type.STRING, description="The content to append (markdown supported)"),
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading. Defaults to current UTC timestamp."),
},
required=["content"],
),
),
types.FunctionDeclaration(
name="scratch_clear",
description="Erase everything in the scratchpad. Use when the working notes are no longer needed.",
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

View File

@@ -2,13 +2,23 @@
System tools — local machine operations.
These tools affect the host system directly. Use with care.
All tools in this module require the admin role.
"""
import asyncio
import logging
import os
import subprocess
from pathlib import Path
from google.genai import types
logger = logging.getLogger(__name__)
# Absolute paths — resolved relative to this file so they work regardless of cwd
_CORTEX_DIR = Path(__file__).parent # .../Cortex_and_Inara_dev/cortex/
_PROJECT_ROOT = _CORTEX_DIR.parent # .../Cortex_and_Inara_dev/
ALLOW_SCRIPT = "/home/scott/.local/bin/claude-allow-dir"
@@ -42,3 +52,283 @@ async def claude_allow_dir(path: str, mode: str = "rw") -> str:
except Exception as e:
logger.error("claude_allow_dir error: %s", e)
return f"Error: {e}"
async def shell_exec(command: str, working_dir: str | None = None, timeout: int = 30) -> str:
"""Execute a shell command on the Cortex host and return combined stdout/stderr."""
timeout = min(max(timeout, 1), 120)
cwd = None
if working_dir:
cwd = os.path.expanduser(working_dir)
if not os.path.isdir(cwd):
return f"Error: working_dir '{working_dir}' does not exist or is not a directory"
try:
proc = await asyncio.create_subprocess_shell(
command,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
out = stdout.decode(errors="replace").strip()
err = stderr.decode(errors="replace").strip()
parts = []
if out:
parts.append(out)
if err:
parts.append(f"[stderr]\n{err}")
combined = "\n".join(parts) if parts else "(no output)"
if proc.returncode != 0:
return f"Exit {proc.returncode}:\n{combined}"
return combined
except asyncio.TimeoutError:
return f"Error: command timed out after {timeout}s"
except Exception as e:
logger.error("shell_exec error: %s", e)
return f"Error: {e}"
async def cortex_restart() -> str:
"""Schedule a Cortex service restart 5 seconds from now.
Uses a detached subprocess so the restart survives the current process being
terminated by systemd. The calling session will drop — user should refresh.
"""
subprocess.Popen(
["bash", "-c", "sleep 5 && systemctl --user restart cortex"],
start_new_session=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
close_fds=True,
)
logger.info("cortex_restart: restart scheduled in 5 seconds")
return (
"Cortex restart scheduled in 5 seconds. "
"The current connection will drop — please refresh the page after a moment."
)
async def cortex_logs(lines: int = 50) -> str:
"""Return recent lines from the Cortex systemd journal."""
n = min(max(int(lines), 1), 200)
try:
proc = await asyncio.create_subprocess_exec(
"journalctl", "--user", "-u", "cortex", f"-n{n}", "--no-pager",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=15)
out = stdout.decode(errors="replace").strip()
return out or stderr.decode(errors="replace").strip() or "No log output."
except asyncio.TimeoutError:
return "Error: journalctl timed out"
except Exception as e:
logger.error("cortex_logs error: %s", e)
return f"Error: {e}"
async def cortex_status() -> str:
"""Return Cortex service status: git branch/commit, ahead/behind remote, and systemctl state."""
lines = []
async def _git(*args: str) -> str:
proc = await asyncio.create_subprocess_exec(
"git", "-C", str(_PROJECT_ROOT), *args,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.DEVNULL,
)
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=10)
return stdout.decode(errors="replace").strip()
try:
branch = await _git("rev-parse", "--abbrev-ref", "HEAD")
commit = await _git("log", "--oneline", "-1")
# fetch quietly so ahead/behind is current
await asyncio.create_subprocess_exec(
"git", "-C", str(_PROJECT_ROOT), "fetch", "--quiet",
stdout=asyncio.subprocess.DEVNULL, stderr=asyncio.subprocess.DEVNULL,
)
ahead_behind = await _git("rev-list", "--left-right", "--count", f"HEAD...origin/{branch}")
ahead, behind = (ahead_behind.split() + ["?", "?"])[:2]
lines.append(f"**Branch:** {branch} | ahead {ahead} / behind {behind}")
lines.append(f"**Commit:** {commit}")
except Exception as e:
lines.append(f"Git info unavailable: {e}")
lines.append("")
try:
proc = await asyncio.create_subprocess_exec(
"systemctl", "--user", "status", "cortex", "--no-pager", "-l",
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
)
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=10)
# First 15 lines of systemctl output is enough — avoids log flood
status_lines = stdout.decode(errors="replace").splitlines()[:15]
lines.extend(status_lines)
except Exception as e:
lines.append(f"systemctl status unavailable: {e}")
return "\n".join(lines)
async def cortex_update() -> str:
"""Pull the latest code from git, syntax-check all Python files, and report.
Does NOT restart automatically — call cortex_restart separately after reviewing
the output if you want to apply changes.
"""
lines = []
async def _run(*cmd: str, cwd: Path = _PROJECT_ROOT, timeout: int = 30) -> tuple[int, str]:
proc = await asyncio.create_subprocess_exec(
*cmd, cwd=str(cwd),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
)
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout)
return proc.returncode, stdout.decode(errors="replace").strip()
# 1. Check for incoming commits before pulling
try:
await _run("git", "fetch", "--quiet")
rc, incoming = await _run("git", "log", "--oneline", "HEAD..origin/HEAD")
if rc == 0 and not incoming:
# Double-check with branch name in case origin/HEAD isn't set
branch_rc, branch = await _run("git", "rev-parse", "--abbrev-ref", "HEAD")
_, incoming = await _run("git", "log", "--oneline", f"HEAD..origin/{branch.strip()}")
except asyncio.TimeoutError:
return "Error: git fetch timed out — check network connectivity."
except Exception as e:
return f"Error during git fetch: {e}"
if not incoming:
rc2, current = await _run("git", "log", "--oneline", "-1")
return f"Already up to date.\n\nCurrent commit: {current}"
lines.append(f"**Incoming commits:**\n{incoming}\n")
# 2. Pull
try:
rc, pull_out = await _run("git", "pull", "--ff-only")
except asyncio.TimeoutError:
return "Error: git pull timed out."
except Exception as e:
return f"Error during git pull: {e}"
if rc != 0:
return f"git pull failed (exit {rc}):\n{pull_out}"
lines.append(f"**git pull:**\n{pull_out}\n")
# 3. Syntax check all Python files under cortex/
py_files = sorted(_CORTEX_DIR.rglob("*.py"))
errors = []
for f in py_files:
result = subprocess.run(
["python3", "-m", "py_compile", str(f)],
capture_output=True, text=True,
)
if result.returncode != 0:
errors.append(f" {f.relative_to(_PROJECT_ROOT)}: {result.stderr.strip()}")
if errors:
lines.append(f"**Syntax errors — do NOT restart until fixed:**")
lines.extend(errors)
else:
lines.append(f"**Syntax check:** {len(py_files)} files — all OK.")
lines.append("Call `cortex_restart` to apply the update.")
return "\n".join(lines)
DECLARATIONS = [
types.FunctionDeclaration(
name="shell_exec",
description=(
"Execute a shell command on the Cortex host machine and return its output. "
"Use for system diagnostics: disk usage (df -h), process status (ps aux), "
"directory listings (ls), memory (free -h), uptime, network info, log tails, etc. "
"Commands run as the Cortex service user. Timeout enforced (default 30s, max 120s). "
"Avoid destructive commands — prefer read-only system queries."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"command": types.Schema(type=types.Type.STRING, description="Shell command to run (e.g. 'df -h', 'ls ~/agents_sync/', 'journalctl --user -u cortex -n 50')"),
"working_dir": types.Schema(type=types.Type.STRING, description="Optional working directory (e.g. '~/agents_sync/projects'). Defaults to home directory."),
"timeout": types.Schema(type=types.Type.INTEGER, description="Timeout in seconds (default 30, max 120)"),
},
required=["command"],
),
),
types.FunctionDeclaration(
name="claude_allow_dir",
description=(
"Add a directory to Claude Code's auto-allow list so Claude can read or write "
"files there without prompting. Edits ~/.claude/settings.json on the local machine. "
"Use this when Claude is silently hanging or being blocked from accessing a directory. "
"Changes take effect in the next Claude Code session."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"path": types.Schema(type=types.Type.STRING, description="Absolute or home-relative path to the directory (e.g. ~/OSIT_dev/aether_api_fastapi or /home/scott/agents_sync)"),
"mode": types.Schema(type=types.Type.STRING, description="Permission mode: 'r' (read-only), 'w' (write-only), or 'rw' (both). Default: rw"),
},
required=["path"],
),
),
types.FunctionDeclaration(
name="cortex_restart",
description=(
"Restart the Cortex service via systemd. Schedules a restart 5 seconds from now. "
"The current connection will drop — inform the user to refresh the page. "
"Use after config changes, memory edits, or when the service needs a fresh start. "
"ADMIN ONLY."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="cortex_logs",
description=(
"Fetch recent lines from the Cortex systemd service journal. "
"Use for debugging errors, checking startup status, or reviewing recent activity. "
"ADMIN ONLY."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"lines": types.Schema(type=types.Type.INTEGER, description="Number of log lines to return (default 50, max 200)"),
},
),
),
types.FunctionDeclaration(
name="cortex_status",
description=(
"Return Cortex service status: current git branch and commit, how many commits "
"ahead/behind the remote, and the systemctl service state. "
"Use to check what version is running or whether the service is healthy. "
"ADMIN ONLY."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
types.FunctionDeclaration(
name="cortex_update",
description=(
"Pull the latest code from git, run a syntax check on all Python files, and report "
"what changed. Does NOT restart automatically — call cortex_restart separately after "
"reviewing the output. Will report syntax errors if the pull introduces broken code. "
"ADMIN ONLY. Requires confirmation."
),
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
),
]

View File

@@ -20,6 +20,7 @@ import asyncio
from datetime import datetime, timezone
from pathlib import Path
from google.genai import types
from persona import persona_path
@@ -59,13 +60,15 @@ def _format_task(t: dict) -> str:
# Sync implementations — called via asyncio.to_thread
# ---------------------------------------------------------------------------
def _task_list(status: str | None) -> str:
def _task_list(status: str | None, priority: str | None) -> str:
tasks = _load()
if status:
tasks = [t for t in tasks if t["status"] == status]
if priority:
tasks = [t for t in tasks if t.get("priority") == priority]
if not tasks:
label = f"No {status} tasks." if status else "No tasks yet."
return label
filters = " ".join(f for f in [status, priority] if f)
return f"No {filters} tasks." if filters else "No tasks yet."
lines = [f"Tasks ({len(tasks)}):\n"]
for t in tasks:
lines.append(_format_task(t))
@@ -117,8 +120,8 @@ def _task_complete(task_id: str) -> str:
# Async wrappers
# ---------------------------------------------------------------------------
async def task_list(status: str | None = None) -> str:
return await asyncio.to_thread(_task_list, status)
async def task_list(status: str | None = None, priority: str | None = None) -> str:
return await asyncio.to_thread(_task_list, status, priority)
async def task_create(title: str, description: str | None = None,
@@ -133,3 +136,71 @@ async def task_update(task_id: str, status: str | None = None, title: str | None
async def task_complete(task_id: str) -> str:
return await asyncio.to_thread(_task_complete, task_id)
DECLARATIONS = [
types.FunctionDeclaration(
name="task_list",
description=(
"List personal tasks from Inara's task list. "
"Use this to check what's on the list, review pending work, or find a task ID. "
"Optionally filter by status: 'todo', 'in_progress', or 'done'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"status": types.Schema(type=types.Type.STRING, description="Filter by status: 'todo', 'in_progress', or 'done'. Omit to list all."),
"priority": types.Schema(type=types.Type.STRING, description="Filter by priority: 'low', 'normal', or 'high'. Omit to list all priorities."),
},
),
),
types.FunctionDeclaration(
name="task_create",
description=(
"Add a new task to Inara's personal task list. "
"Use this when the user asks to remember something, add a to-do, or track a follow-up."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"title": types.Schema(type=types.Type.STRING, description="Short task title"),
"description": types.Schema(type=types.Type.STRING, description="Optional longer description or context"),
"priority": types.Schema(type=types.Type.STRING, description="Priority: 'low', 'normal', or 'high'. Default: normal."),
},
required=["title"],
),
),
types.FunctionDeclaration(
name="task_update",
description=(
"Update an existing task. Use task_list first to get the task ID. "
"Can update status, title, description, or priority. "
"To just mark complete, use task_complete instead."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"task_id": types.Schema(type=types.Type.STRING, description="Task ID (e.g. t_abc123) — get from task_list"),
"status": types.Schema(type=types.Type.STRING, description="New status: 'todo', 'in_progress', or 'done'"),
"title": types.Schema(type=types.Type.STRING, description="Updated title"),
"description": types.Schema(type=types.Type.STRING, description="Updated description"),
"priority": types.Schema(type=types.Type.STRING, description="Updated priority: 'low', 'normal', or 'high'"),
},
required=["task_id"],
),
),
types.FunctionDeclaration(
name="task_complete",
description=(
"Mark a task as done. Use task_list first to get the task ID. "
"Shorthand for task_update with status='done'."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"task_id": types.Schema(type=types.Type.STRING, description="Task ID (e.g. t_abc123) — get from task_list"),
},
required=["task_id"],
),
),
]

View File

@@ -1,13 +1,17 @@
"""
Web search tool — DuckDuckGo backend.
Uses the duckduckgo-search library. Set DDG_API_KEY in .env for a paid account
(higher rate limits). The free unauthenticated tier works for moderate usage.
Web tools — search (DuckDuckGo), direct HTTP fetch, clean content extraction, and HTTP POST.
"""
import asyncio
import json
import logging
from urllib.parse import urlparse
import httpx
from google.genai import types
from config import settings
from persona import get_user
logger = logging.getLogger(__name__)
@@ -48,3 +52,216 @@ def _sync_search(query: str, max_results: int) -> list[dict]:
except Exception as e:
logger.warning("DuckDuckGo search error: %s", e)
return []
async def http_fetch(
url: str,
method: str = "GET",
body: str | None = None,
timeout: int = 15,
max_chars: int = 8192,
) -> str:
"""Fetch a URL directly and return the raw response body.
Unlike web_search, this hits a specific URL — useful for health checks,
API probing, JSON endpoints, webhook testing, or reading raw page source.
For readable article content, use web_read instead.
Response body is capped at max_chars (default 8192, max 32768).
"""
method = method.upper()
timeout = min(max(int(timeout), 1), 60)
max_chars = min(max(int(max_chars), 100), 131072)
try:
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
resp = await client.request(method, url, content=body)
body_text = resp.text[:max_chars]
truncated = len(resp.text) > max_chars
suffix = f"\n\n[… truncated at {max_chars} chars]" if truncated else ""
return f"HTTP {resp.status_code} {resp.url}\n\n{body_text}{suffix}"
except httpx.HTTPError as e:
return f"HTTP error: {e}"
except Exception as e:
logger.warning("http_fetch error for %s: %s", url, e)
return f"Error: {e}"
async def web_read(url: str, max_chars: int = 16000) -> str:
"""Fetch a URL and extract clean readable text, stripping ads, navigation, and boilerplate.
Uses trafilatura to extract the main article content — ideal for blog posts,
documentation, news articles, and any page where you want the text without
surrounding noise. Returns markdown-formatted output.
For raw responses (JSON APIs, health checks), use http_fetch instead.
"""
max_chars = min(max(int(max_chars), 1000), 131072)
return await asyncio.to_thread(_sync_web_read, url, max_chars)
def _sync_web_read(url: str, max_chars: int) -> str:
try:
import trafilatura
except ImportError:
return "web_read requires trafilatura — run: pip install trafilatura"
downloaded = trafilatura.fetch_url(url)
if downloaded is None:
return f"Failed to download content from: {url}"
text = trafilatura.extract(downloaded, output_format="markdown", include_links=True, url=url)
if not text:
text = trafilatura.extract(downloaded, url=url)
if not text:
return f"Could not extract readable content from: {url}"
if len(text) > max_chars:
text = text[:max_chars] + f"\n\n[… truncated at {max_chars} chars — pass a larger max_chars (up to 131072) to see more]"
return f"Content from {url}:\n\n{text}"
def _load_http_allowlist(username: str) -> list[str]:
"""Load per-user HTTP POST allowlist (URL prefixes). Empty list = all blocked."""
path = settings.home_root() / username / "http_allowlist.json"
try:
return [str(p).strip() for p in json.loads(path.read_text()) if str(p).strip()]
except FileNotFoundError:
return []
except Exception as e:
logger.warning("failed to read http_allowlist.json for %s: %s", username, e)
return []
def _http_post_allowed(url: str, allowlist: list[str]) -> bool:
"""Return True if url starts with any allowlist entry (prefix match)."""
for prefix in allowlist:
if url.startswith(prefix):
return True
return False
async def http_post(
url: str,
body: str = "",
headers: dict | None = None,
max_chars: int = 4096,
) -> str:
"""POST to an external URL. Requires the URL to match home/{user}/http_allowlist.json.
body may be a JSON string or plain text. If body is valid JSON, Content-Type is set
to application/json; otherwise text/plain. Override via the headers param.
Response is capped at max_chars (default 4096, max 131072).
"""
username = get_user()
allowlist = _load_http_allowlist(username)
if not allowlist:
return (
f"http_post blocked — no allowlist configured. "
f"Add allowed URL prefixes to home/{username}/http_allowlist.json as a JSON array. "
f"Example: [\"https://api.example.com\"]"
)
if not _http_post_allowed(url, allowlist):
return (
f"http_post blocked — {url} does not match any allowlist entry for {username}. "
f"Add the URL prefix to home/{username}/http_allowlist.json."
)
max_chars = min(max(int(max_chars), 100), 131072)
# Auto-detect content type from body
body_str = body if isinstance(body, str) else json.dumps(body)
try:
json.loads(body_str)
content_type = "application/json"
except (json.JSONDecodeError, ValueError):
content_type = "text/plain"
req_headers = {"Content-Type": content_type}
if headers:
req_headers.update(headers)
try:
async with httpx.AsyncClient(timeout=30, follow_redirects=True) as client:
resp = await client.post(url, content=body_str.encode(), headers=req_headers)
body_text = resp.text[:max_chars]
truncated = len(resp.text) > max_chars
suffix = f"\n\n[… truncated at {max_chars} chars]" if truncated else ""
return f"HTTP {resp.status_code} {resp.url}\n\n{body_text}{suffix}"
except httpx.HTTPError as e:
return f"HTTP error: {e}"
except Exception as e:
logger.warning("http_post error for %s: %s", url, e)
return f"Error: {e}"
DECLARATIONS = [
types.FunctionDeclaration(
name="web_search",
description=(
"Search the web for current information. Use this when you need up-to-date "
"facts, news, documentation, or anything not in your training data."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"query": types.Schema(type=types.Type.STRING, description="The search query string"),
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of results to return (default 5, max 10)"),
},
required=["query"],
),
),
types.FunctionDeclaration(
name="http_fetch",
description=(
"Fetch a specific URL and return the raw response body. Unlike web_search, this hits "
"a direct URL — useful for health checks, JSON API endpoints, webhook testing, "
"or inspecting raw page source. For readable article/doc content, use web_read instead. "
"Response body is capped at max_chars (default 8192, max 32768)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"url": types.Schema(type=types.Type.STRING, description="Full URL to fetch"),
"method": types.Schema(type=types.Type.STRING, description="HTTP method: GET (default), POST, HEAD"),
"body": types.Schema(type=types.Type.STRING, description="Optional request body (for POST requests)"),
"timeout": types.Schema(type=types.Type.INTEGER, description="Request timeout in seconds (default 15, max 60)"),
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max characters to return (default 8192, max 131072)"),
},
required=["url"],
),
),
types.FunctionDeclaration(
name="web_read",
description=(
"Fetch a URL and extract clean readable text, stripping ads, navigation, sidebars, "
"and other boilerplate. Returns the main article/document content as markdown. "
"Use this for blog posts, documentation, news articles, GitHub READMEs, or any page "
"where you want the content without surrounding noise. "
"For raw HTTP responses (JSON APIs, health checks, source inspection), use http_fetch."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"url": types.Schema(type=types.Type.STRING, description="Full URL to fetch and extract"),
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max characters to return (default 16000, max 131072)"),
},
required=["url"],
),
),
types.FunctionDeclaration(
name="http_post",
description=(
"POST to an external URL. Requires the URL to match the user's http_allowlist.json. "
"Use for calling webhooks, triggering automations, posting to APIs, or any HTTP action. "
"body is a string — JSON or plain text are both accepted (Content-Type auto-detected). "
"Override headers as needed. Response capped at max_chars (default 4096, max 131072)."
),
parameters=types.Schema(
type=types.Type.OBJECT,
properties={
"url": types.Schema(type=types.Type.STRING, description="Full URL to POST to"),
"body": types.Schema(type=types.Type.STRING, description="Request body — JSON string or plain text"),
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max response chars (default 4096, max 131072)"),
},
required=["url"],
),
),
]

75
cortex/usage_tracker.py Normal file
View File

@@ -0,0 +1,75 @@
"""
API usage and token tracking.
Writes daily buckets to home/{username}/usage.json:
{
"2026-05-01": {
"gemini_api/gemini-2.0-flash": {"calls": 3, "prompt_tokens": 8400, "completion_tokens": 520},
"local/llama3.2:latest": {"calls": 2, "prompt_tokens": 1200, "completion_tokens": 310}
}
}
Claude CLI and Gemini CLI backends produce no structured token data and are not tracked.
"""
import asyncio
import json
import logging
from datetime import date
from pathlib import Path
from config import settings
logger = logging.getLogger(__name__)
_LOCK = asyncio.Lock()
def _usage_path(username: str) -> Path:
return settings.home_root() / username / "usage.json"
async def record(
username: str,
backend: str,
model_name: str,
prompt_tokens: int,
completion_tokens: int,
) -> None:
"""Append one call's token counts to the daily usage log for this user.
backend — "gemini_api" | "local"
model_name — the exact model string (e.g. "gemini-2.0-flash", "llama3.2:latest")
"""
path = _usage_path(username)
today = date.today().isoformat()
key = f"{backend}/{model_name}"
async with _LOCK:
try:
data: dict = json.loads(path.read_text()) if path.exists() else {}
except Exception:
data = {}
entry = data.setdefault(today, {}).setdefault(
key, {"calls": 0, "prompt_tokens": 0, "completion_tokens": 0}
)
entry["calls"] += 1
entry["prompt_tokens"] += prompt_tokens
entry["completion_tokens"] += completion_tokens
try:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, indent=2))
except Exception as e:
logger.warning("Failed to write usage data to %s: %s", path, e)
def read_usage(username: str) -> dict:
"""Return the full usage dict for this user. Empty dict if no file yet."""
path = _usage_path(username)
try:
return json.loads(path.read_text()) if path.exists() else {}
except Exception:
return {}

194
cortex/user_settings.py Normal file
View File

@@ -0,0 +1,194 @@
"""
Per-user settings stored in home/{user}/local_llm.json.
Structure:
{
"hosts": [{"id", "label", "api_url", "api_key"}, ...],
"models": [{"id", "host_id", "label", "model_name"}, ...],
"active_model_id": "<model id>" | null
}
Values not configured here fall back to .env server defaults.
"""
import json
import logging
import secrets
from pathlib import Path
from config import settings as app_settings
logger = logging.getLogger(__name__)
def _llm_path(username: str) -> Path:
return app_settings.home_root() / username / "local_llm.json"
def _empty() -> dict:
return {"hosts": [], "models": [], "active_model_id": None}
def _load(username: str) -> dict:
path = _llm_path(username)
if not path.exists():
return _empty()
try:
data = json.loads(path.read_text())
except (json.JSONDecodeError, OSError):
logger.warning("local_llm.json for %s is unreadable — starting fresh", username)
return _empty()
# Migrate old single-model format {api_url, api_key, model} → new format
if "hosts" not in data:
return _migrate_v0(data)
return data
def _migrate_v0(old: dict) -> dict:
"""Migrate flat {api_url, api_key, model} → hosts/models structure."""
data = _empty()
api_url = old.get("api_url") or app_settings.local_api_url
api_key = old.get("api_key") or app_settings.local_api_key
model_name = old.get("model") or app_settings.local_model
if not api_url:
return data
host_id = secrets.token_hex(4)
data["hosts"].append({
"id": host_id,
"label": "Local Model Server",
"api_url": api_url,
"api_key": api_key,
})
if model_name:
model_id = secrets.token_hex(4)
data["models"].append({
"id": model_id,
"host_id": host_id,
"label": model_name,
"model_name": model_name,
})
data["active_model_id"] = model_id
logger.info("migrated local_llm.json v0 → v1 for user (host=%s)", host_id)
return data
def _save(username: str, data: dict) -> None:
_llm_path(username).write_text(json.dumps(data, indent=2))
# ── Public read API ───────────────────────────────────────────────────────────
def get_config(username: str) -> dict:
"""Return the full local LLM config for the user."""
return _load(username)
def get_active_local_model(username: str) -> dict | None:
"""Return effective {api_url, api_key, model_name, label} for the active model.
Resolution order:
1. User's active model + its host config
2. .env server defaults (LOCAL_API_URL / LOCAL_API_KEY / LOCAL_MODEL)
3. None — caller should raise a helpful error
"""
data = _load(username)
active_id = data.get("active_model_id")
model = next((m for m in data["models"] if m["id"] == active_id), None)
if model:
host = next((h for h in data["hosts"] if h["id"] == model["host_id"]), None)
if host:
return {
"api_url": host.get("api_url", ""),
"api_key": host.get("api_key", ""),
"model_name": model["model_name"],
"label": model.get("label") or model["model_name"],
}
# Fall back to .env defaults
if app_settings.local_api_url and app_settings.local_model:
return {
"api_url": app_settings.local_api_url,
"api_key": app_settings.local_api_key,
"model_name": app_settings.local_model,
"label": app_settings.local_model,
}
return None
# ── Host management ───────────────────────────────────────────────────────────
def save_host(username: str, host_id: str | None,
label: str, api_url: str, api_key: str) -> str:
"""Create or update a host. Returns the host ID.
api_key is only written when non-empty, so submitting a masked placeholder
with a blank key field leaves the stored key unchanged.
"""
data = _load(username)
if host_id:
for h in data["hosts"]:
if h["id"] == host_id:
h["label"] = label.strip()
h["api_url"] = api_url.strip()
if api_key.strip():
h["api_key"] = api_key.strip()
break
else:
host_id = None # ID not found — fall through to create
if not host_id:
host_id = secrets.token_hex(4)
data["hosts"].append({
"id": host_id,
"label": label.strip(),
"api_url": api_url.strip(),
"api_key": api_key.strip(),
})
_save(username, data)
return host_id
# ── Model management ──────────────────────────────────────────────────────────
def add_model(username: str, host_id: str, label: str, model_name: str) -> str:
"""Add a model entry. Auto-activates if it is the first model. Returns the model ID."""
data = _load(username)
model_id = secrets.token_hex(4)
data["models"].append({
"id": model_id,
"host_id": host_id,
"label": label.strip() or model_name.strip(),
"model_name": model_name.strip(),
})
if not data.get("active_model_id"):
data["active_model_id"] = model_id
_save(username, data)
return model_id
def remove_model(username: str, model_id: str) -> None:
data = _load(username)
data["models"] = [m for m in data["models"] if m["id"] != model_id]
if data.get("active_model_id") == model_id:
data["active_model_id"] = data["models"][0]["id"] if data["models"] else None
_save(username, data)
def set_active_model(username: str, model_id: str) -> bool:
"""Set the active model. Returns False if the model ID is not found."""
data = _load(username)
if not any(m["id"] == model_id for m in data["models"]):
return False
data["active_model_id"] = model_id
_save(username, data)
return True

26
dev-restart.sh Executable file
View File

@@ -0,0 +1,26 @@
#!/usr/bin/env bash
# dev-restart.sh — restart Cortex on the gaming laptop and tail logs
# Usage:
# ./dev-restart.sh restart and show last 30 log lines
# ./dev-restart.sh logs tail live logs (ctrl-c to stop)
# ./dev-restart.sh status show service status only
# "scott-lt-i7-rtx" or "192.168.32.19"
CORTEX_HOST="scott-lt-i7-rtx" # hostname or IP of the machine running Cortex
SERVICE="cortex"
case "${1:-restart}" in
logs)
echo "→ Tailing $SERVICE logs on $CORTEX_HOST (ctrl-c to stop)"
ssh "$CORTEX_HOST" "journalctl --user -u $SERVICE -f --no-pager"
;;
status)
ssh "$CORTEX_HOST" "systemctl --user status $SERVICE --no-pager -l"
;;
restart|*)
echo "→ Restarting $SERVICE on $CORTEX_HOST"
ssh "$CORTEX_HOST" "systemctl --user restart $SERVICE"
echo "→ Last 30 log lines:"
ssh "$CORTEX_HOST" "journalctl --user -u $SERVICE --no-pager -n 30"
;;
esac

100
docs/GOOGLE_CHAT_BOT.md Normal file
View File

@@ -0,0 +1,100 @@
# Google Chat Bot Integration
Cortex connects to Google Chat as a **Workspace Add-on** — each Cortex user gets their own webhook endpoint routed to their chosen persona.
**Status:** Live and confirmed working (2026-03-27)
---
## Prerequisites
- A Google Cloud project with **Google Chat API** enabled
- The Cortex server reachable at a public HTTPS URL
- The user pre-registered in Cortex (`manage_passwords.py invite` or `google-add`)
---
## Per-User Setup
### 1. Create the user's `channels.json`
Create `home/{username}/channels.json` on the Cortex server:
```json
{
"google_chat": {
"persona": "inara",
"audience": "https://cortex.dgrzone.com/channels/google-chat/{username}",
"backend": "claude",
"timeout": 25
}
}
```
- **`persona`** — which persona responds (must exist under `home/{username}/persona/`)
- **`audience`** — must exactly match the HTTP endpoint URL you set in Google Cloud Console (Google uses this as the JWT `aud` claim)
- **`backend`** — `"claude"` recommended; Google Chat requires a response within 30s
- **`timeout`** — keep at 25 (Google's hard limit is 30s; this leaves a 5s buffer)
### 2. Configure Google Chat API in Google Cloud Console
1. Go to [console.cloud.google.com](https://console.cloud.google.com) and select the project
2. **APIs & Services → Enabled APIs & services → Google Chat API**
3. Click the **Configuration** tab
4. Fill in **Application info:**
- App name: `Cortex` (or your persona name)
- Avatar URL: optional
- Description: optional
5. Under **Interactive features:**
- Enable **"Join spaces and group conversations"** if you want the bot in group chats, or leave it off for DM-only
6. Under **Connection settings:**
- Select **HTTP endpoint URL**
- Enter: `https://cortex.dgrzone.com/channels/google-chat/{username}`
7. Under **Visibility:**
- Add the specific Google accounts that should be able to use this bot
- For One Sky IT Workspace users: add individuals or the whole domain
8. Click **Save**
> **Important:** The URL in step 6 must exactly match the `audience` value in `channels.json`. Google includes this URL as the JWT `aud` claim on every request, and Cortex rejects any request where they don't match.
---
## How It Works
1. User sends a message in Google Chat → Google POSTs a signed JSON payload to `/channels/google-chat/{username}`
2. Cortex reads the user's `channels.json`, verifies the JWT `systemIdToken` from `authorizationEventObject`
3. Sets the persona context, builds the system prompt, calls the LLM
4. Returns the response wrapped in `hostAppDataAction → chatDataAction → createMessageAction`
The response must be returned synchronously (Google Chat does not support async/background replies like NC Talk does). The 25s timeout is a hard constraint.
---
## JWT Verification
Google Chat Workspace Add-ons send a `systemIdToken` in the request body at:
`body["authorizationEventObject"]["systemIdToken"]`
Claims verified by Cortex:
- `iss` = `https://accounts.google.com`
- `aud` = the value of `audience` in `channels.json`
If `audience` is empty, verification is skipped (useful for local testing, never in production).
---
## Nginx
The `/channels/` prefix is already public in `auth_middleware.py` — no Nginx changes needed if you're already proxying all traffic to Cortex. Verify the path isn't blocked by basic auth or IP restrictions.
---
## Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| 404 on the webhook | `channels.json` missing or no `google_chat` key | Create/check `home/{username}/channels.json` |
| 401 Invalid token | `audience` in `channels.json` doesn't match the endpoint URL | Make them identical — copy the URL exactly |
| 401 Missing token | No `systemIdToken` in request | Bot may not be a Workspace Add-on; check connection settings type |
| Timeout / no response | LLM too slow | `backend: "claude"` recommended; reduce context tier if needed |
| Bot not receiving messages | Visibility not configured | Add the user's Google account under Visibility in Cloud Console |

View File

@@ -1,69 +1,78 @@
# Nextcloud Talk Bot Integration
Inara is registered as a bot in Nextcloud Talk, receiving messages via webhook and replying through the bot API.
Cortex connects to Nextcloud Talk as a bot — each Cortex user gets their own webhook endpoint routed to their chosen persona.
**Status:** Live and confirmed working (2026-03-20)
**Status:** Live and confirmed working (2026-03-20); per-user routing added 2026-03-27
---
## Installation
## Prerequisites
Run on the Nextcloud server (inside the Docker container):
```bash
docker exec -it --user www-data <nc-app-container> php /var/www/html/occ talk:bot:install \
"Inara" \
"<secret from cortex .env NEXTCLOUD_TALK_BOT_SECRET>" \
"https://cortex.dgrzone.com/inara-nextcloud-talk-webhook" \
--feature webhook --feature response --feature reaction
```
After installing, enable the bot in each Talk conversation via the conversation settings UI (three-dot menu → Bots).
To list installed bots and verify registration:
```bash
docker exec -it --user www-data <nc-app-container> php /var/www/html/occ talk:bot:list
```
To uninstall (if re-registering with a new secret):
```bash
docker exec -it --user www-data <nc-app-container> php /var/www/html/occ talk:bot:remove <bot-id>
```
- Access to the Nextcloud server (Docker exec or SSH)
- The Cortex server reachable at a public HTTPS URL
- The user pre-registered in Cortex (`manage_passwords.py invite`)
---
## Configuration
## Per-User Setup
**`cortex/.env`:**
```
NEXTCLOUD_URL=https://cloud.dgrzone.com
NEXTCLOUD_TALK_BOT_SECRET=<shared secret — must match occ install command>
```
### 1. Create the user's `channels.json`
`NEXTCLOUD_URL` defaults to `https://cloud.dgrzone.com` in `config.py`.
Create `home/{username}/channels.json` on the Cortex server:
**Nginx:** The `/inara-nextcloud-talk-webhook` endpoint must be reachable by Nextcloud without basic auth. Add a location block before the default `auth_basic` block:
```nginx
location = /inara-nextcloud-talk-webhook {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
```json
{
"nextcloud": {
"persona": "inara",
"url": "https://cloud.dgrzone.com",
"bot_secret": "<a secret you choose — must match the occ install command>",
"timeout": 55
}
}
```
(The `/channels/` prefix is already bypassed for Google Chat — consider moving the webhook path to `/channels/nextcloud` in a future cleanup to unify the nginx config.)
- **`persona`** — which persona responds (must exist under `home/{username}/persona/`)
- **`url`** — base URL of the Nextcloud instance
- **`bot_secret`** — a shared HMAC secret; you choose this value and use it in both `channels.json` and the `occ` install command
- **`timeout`** — seconds to wait for the LLM before sending a timeout message (NC Talk is async, so 55s is safe)
### 2. Register the bot in Nextcloud
The Nextcloud container for DgrZone is `dgr_zone_nextcloud-app-1`. Substitute your own container name if different.
First, list existing bots to check if one is already registered (note the bot ID):
```bash
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ talk:bot:list
```
If re-registering (new URL or new secret), uninstall the old bot first:
```bash
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ talk:bot:uninstall <bot-id>
```
Install the bot:
```bash
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ talk:bot:install \
"Inara" \
"<bot_secret from channels.json>" \
"https://cortex.dgrzone.com/webhook/nextcloud/{username}" \
--feature webhook --feature response --feature reaction
```
After installing, enable the bot in each Talk conversation: open the conversation → three-dot menu → **Bots** → enable the bot by name.
---
## How It Works
1. User sends a message in Talk → Nextcloud POSTs a signed webhook to `/inara-nextcloud-talk-webhook`
2. Cortex verifies the incoming HMAC signature, extracts the message text, runs it through the LLM
3. Cortex POSTs the reply to `/ocs/v2.php/apps/spreed/api/v1/bot/{token}/message` with its own HMAC signature
4. The webhook handler returns HTTP 200 immediately; the LLM call happens in a `BackgroundTask` (prevents Nextcloud from disabling the bot due to slow response)
1. User sends a message in Talk → Nextcloud POSTs a signed webhook to `/webhook/nextcloud/{username}`
2. Cortex reads the user's `channels.json`, verifies the incoming HMAC signature
3. Sets the persona context, builds the system prompt, runs the LLM in a `BackgroundTask`
4. Returns HTTP 200 immediately (prevents Nextcloud from disabling the bot due to slow response)
5. Cortex POSTs the reply to `/ocs/v2.php/apps/spreed/api/v1/bot/{token}/message` with its own HMAC signature
---
@@ -76,7 +85,6 @@ location = /inara-nextcloud-talk-webhook {
Nextcloud signs its outgoing webhook with `HMAC-SHA256(secret, random + raw_body)`:
```python
# _verify_signature in nextcloud_talk.py
expected = hmac.new(
secret.encode(),
(random_header + body.decode("utf-8")).encode(),
@@ -89,7 +97,6 @@ expected = hmac.new(
When Cortex posts a reply, Nextcloud verifies the signature against the *parsed message string*, not the raw body. This is because `BotController::sendMessage` passes the parsed `$message` parameter to `checksumVerificationService::validateRequest`, not `$request->getContent()`.
```python
# _send_reply in nextcloud_talk.py
sig = hmac.new(
secret.encode(),
(random_str + message).encode("utf-8"), # message text only, NOT json.dumps({"message": ...})
@@ -105,35 +112,50 @@ sig = hmac.new(secret.encode(), (random_str + '{"message": "..."}').encode(), ha
---
## Multi-User Note
## Nginx
NC Talk currently uses the **default user and persona** (`settings.default_tier`, `load_context()`). All Talk conversations go to Inara regardless of who is messaging. Per-conversation persona routing (e.g., Holly gets Tina) is a future enhancement — would require mapping Nextcloud user IDs or conversation tokens to Cortex users.
The `/webhook/` prefix is already public in `auth_middleware.py`. If Nginx applies basic auth or IP restrictions, add a `location` block before the default auth block:
---
## Claude CLI Auth in systemd
The `CLAUDE_CODE_OAUTH_TOKEN` in `.env` goes stale after each `claude auth login` (tokens rotate). Cortex reads the token live from `~/.claude/.credentials.json` on every Claude call (`llm_client._fresh_claude_token()`), so no manual `.env` update is needed after re-authentication.
Also: never set `ANTHROPIC_API_KEY` to an OAuth token value (`sk-ant-oat01-...`) — the Claude CLI treats it as a direct API key and fails. Only real API keys (`sk-ant-api03-...`) belong in `ANTHROPIC_API_KEY`.
```nginx
location ^~ /webhook/ {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
```
---
## Triggering the Bot
- **@mention** — prefix the message with `@inara` (or whatever `AGENT_NAME` is set to); the prefix is stripped before sending to the LLM
- **@mention** — prefix the message with `@{persona_name}`; the prefix is stripped before sending to the LLM
- **Any message** in a conversation where the bot is enabled — all messages are forwarded, not just @mentions
---
## Logs
Two log streams are useful when debugging:
```bash
# Nextcloud server logs (bot registration errors, webhook rejections)
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ log:tail
# Cortex service logs (LLM errors, signature failures, timeouts)
journalctl --user -u cortex -f
```
---
## Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| 404 on the webhook | `channels.json` missing or no `nextcloud` key | Create/check `home/{username}/channels.json` |
| Webhook not received | Bot not enabled for conversation | Enable in Talk conversation settings (Bots) |
| Incoming 401 | Wrong secret in `.env` | Match secret to `occ talk:bot:install` value |
| Incoming 401 | `bot_secret` in `channels.json` doesn't match `occ install` secret | Re-register with matching secret |
| Reply POST returns 401 (first try) | HMAC computed over wrong data | Sign `random + message_text` only (not raw JSON body) |
| Reply POST returns 401 (persistent) | Brute force protection triggered | `occ security:bruteforce:reset <cortex-IP>` |
| Bot auto-disabled by Nextcloud | Webhook held open too long | Verify `BackgroundTasks` is used — return 200 immediately |
| Claude falls back to Gemini | Stale/wrong auth token | Token is auto-refreshed from `~/.claude/.credentials.json`; run `claude auth login` if expired |
| No response at all | Nginx blocking the path with basic auth | Add a `location =` block before the auth block (see Nginx section above) |
| Reply POST returns 401 (persistent) | Brute force protection triggered | `docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ security:bruteforce:reset <cortex-IP>` |
| Bot auto-disabled by Nextcloud | Webhook held open too long | Verify `BackgroundTasks` is used — Cortex returns 200 immediately |
| Claude falls back to Gemini | Stale/expired auth token | Run `claude auth login`; token is auto-refreshed from `~/.claude/.credentials.json` |
| No response at all | Nginx blocking the path | Add a `location ^~ /webhook/` block before any auth block |

276
docs/OPEN_WEBUI_API.md Normal file
View File

@@ -0,0 +1,276 @@
# Open WebUI API Reference for Cortex
> Last updated: 2026-04-03
> Source: https://docs.openwebui.com/reference/api-endpoints/
> Host in use: `http://192.168.32.19:3000` (scott_gaming — 8 GB VRAM)
## Local Model Performance (scott_gaming, 8 GB VRAM)
| Model | Alias | Speed | Practical Context | Spec Context |
|---|---|---|---|---|
| Gemma 4 E4B | `agent-support-gemma-small` | ~25 t/s | **72k tokens** | 128k |
| Gemma 4 26B A4B (MoE) | `agent-support-gemma-medium` | ~9 t/s | **50k tokens** | 256k |
Context is VRAM-constrained — spec limits are higher but KV cache fills available VRAM first.
Techniques to improve: lower KV cache quantization, flash attention, context length tuning in Ollama.
**Practical implications for the local orchestrator:**
- System prompt + memory (T2) + tool results + history: budget ~40-50k for small, ~35-40k for medium
- Medium at 9 t/s is fine for background/async tasks; small at 25 t/s is responsive enough for interactive use
- Both are well above what's needed for most tool loop iterations (~2-5k tokens per round)
---
## Authentication
All API calls use a bearer token:
```
Authorization: Bearer sk-<api-key>
```
API keys are managed in Open WebUI → Settings → Account → API Keys.
Cortex stores these per-user in `home/{username}/local_llm.json``hosts[].api_key`.
---
## Core Endpoints Used by Cortex
### List Available Models
```
GET /api/models
Authorization: Bearer sk-...
```
Returns all models (Ollama, OpenAI-proxied, custom functions).
Used by `/api/local-llm/fetch-models` in `routers/local_llm.py`.
Response shape:
```json
{
"data": [
{ "id": "gemma4-e4b", "name": "Gemma 4 E4B" },
...
]
}
```
### Chat Completions (OpenAI-compatible)
```
POST /api/chat/completions
Authorization: Bearer sk-...
Content-Type: application/json
```
Standard OpenAI chat format. Supports:
- `messages` — standard role/content array
- `model` — model ID or workspace alias
- `tools` + `tool_choice` — function calling (see Tool Loop below)
- `stream: true/false`
This is the endpoint used by `_local()` in `llm_client.py`.
### Anthropic Messages API Compatibility
```
POST /api/v1/messages
Authorization: Bearer sk-...
```
Open WebUI also accepts Anthropic-format requests and auto-converts them.
Could be used to route Claude SDK calls through Open WebUI.
Base URL for this mode: `http://192.168.32.19:3000/api`
### Direct Ollama Proxy
```
GET /ollama/api/tags — list models
POST /ollama/api/generate — streaming completions
POST /ollama/api/embed — generate embeddings
```
Use these if you need to bypass Open WebUI's filter layer and hit Ollama directly.
Ollama is also accessible directly at `http://192.168.32.19:11434`.
---
## Tool / Function Calling
Both Gemma 4 models (E4B and 26B A4B) support function calling via the standard
OpenAI `tools` parameter. Open WebUI passes this through to the underlying model.
### Request Format
```json
POST /api/chat/completions
{
"model": "gemma4-26b-a4b",
"messages": [
{ "role": "system", "content": "..." },
{ "role": "user", "content": "What's the weather?" }
],
"tools": [
{
"type": "function",
"function": {
"name": "web_search",
"description": "Search the web for current information",
"parameters": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "Search query" }
},
"required": ["query"]
}
}
}
],
"tool_choice": "auto"
}
```
### Tool Call Response
When the model wants to call a tool, it returns `finish_reason: "tool_calls"`:
```json
{
"choices": [{
"finish_reason": "tool_calls",
"message": {
"role": "assistant",
"content": null,
"tool_calls": [{
"id": "call_abc123",
"type": "function",
"function": {
"name": "web_search",
"arguments": "{\"query\": \"current weather NYC\"}"
}
}]
}
}]
}
```
### Sending Tool Results Back
Append the assistant's tool_call message and a tool result message, then re-submit:
```json
{
"messages": [
{ "role": "user", "content": "What's the weather?" },
{ "role": "assistant", "content": null,
"tool_calls": [{ "id": "call_abc123", "function": { "name": "web_search", "arguments": "..." } }] },
{ "role": "tool", "tool_call_id": "call_abc123",
"content": "Current weather in NYC: 62°F, partly cloudy." }
],
"tools": [...],
"tool_choice": "auto"
}
```
Repeat until `finish_reason: "stop"`.
---
## RAG (Retrieval Augmented Generation)
### Upload a File
```
POST /api/v1/files/
Authorization: Bearer sk-...
Content-Type: multipart/form-data
file=@/path/to/document.pdf
```
Returns a file ID. Poll `/api/v1/files/{id}/process/status` until `completed`.
### Knowledge Collections
```
POST /api/v1/knowledge/{collection_id}/file/add
{ "file_id": "..." }
```
### Use in Chat
Reference files or knowledge collections in any chat request:
```json
{
"model": "gemma4-26b-a4b",
"messages": [...],
"files": [
{ "type": "file", "id": "file-id" },
{ "type": "collection", "id": "collection-id" }
]
}
```
### Process a Web URL into a Collection
```
POST /api/v1/retrieval/process/web
{ "url": "https://example.com/article", "collection_id": "..." }
```
---
## Filter Behavior with Direct API Calls
Open WebUI supports inlet/outlet filter pipelines. With direct API access:
| Filter | Runs automatically? |
|-----------|---------------------|
| `inlet()` | ✅ Yes |
| `stream()`| ✅ Yes |
| `outlet()`| ❌ Manual only — call `POST /api/chat/completed` after receiving response |
For Cortex's use case (tool loop orchestration), this is not a concern — we're
driving the loop ourselves and don't rely on Open WebUI's filter pipeline.
---
## Relevant Cortex Files
| File | Purpose |
|---|---|
| `cortex/llm_client.py``_local()` | Current local backend (direct chat only) |
| `cortex/routers/local_llm.py` | Local model settings page + fetch-models endpoint |
| `cortex/user_settings.py` | Per-user host + model config (`local_llm.json`) |
| `cortex/orchestrator_engine.py` | Gemini API tool loop — reference for local version |
| `home/{user}/local_llm.json` | Stored host/model config |
---
## Planned: Local Orchestrator (`local_orchestrator_engine.py`)
A local equivalent of `orchestrator_engine.py` that:
1. Takes the same tool definitions already registered in `cortex/tools/`
2. Converts them to OpenAI `tools` format (already close — minor schema diff from Gemini)
3. Runs a ReAct loop against the local model via `/api/chat/completions`
4. Falls back gracefully if the model doesn't return a valid tool call
See `documentation/TODO__Agents.md``[Local] Tool-capable local orchestrator`.
Model recommendation:
- **Gemma 4 26B A4B** (256k ctx, MoE — fast for its size) for complex tool tasks
- **Gemma 4 E4B** (128k ctx) for lightweight/fast tasks
---
## Notes
- Open WebUI workspace aliases (e.g. `agent-support-gemma-small`) resolve to the
underlying Ollama model — use aliases in Cortex for human-friendly model names.
- `tool_choice: "auto"` lets the model decide; `"none"` forces plain text response;
`{"type": "function", "function": {"name": "..."}}` forces a specific tool.
- Gemma 4 models support combined tool use + reasoning (thinking tokens) — useful
for complex multi-step tasks.
- For embeddings (future RAG work), use `/ollama/api/embed` directly.

View File

@@ -0,0 +1,226 @@
# Aether Platform Integration — Cortex Tool Layer
> Last updated: 2026-04-30
> Status: Journal toolset complete — broader AE integration planned
This doc covers how Cortex/Inara integrates with the Aether Platform API, what's
implemented, what the data model looks like, and what's planned next.
---
## Overview
Cortex connects to the Aether Platform V3 API to give the orchestrator read/write
access to the user's knowledge base (Journals) and task data. Auth uses the same
`x-aether-api-key` + `x-account-id` headers as every other Aether client.
Config lives in `.env`:
```
AE_API_URL=https://dev-api.oneskyit.com
AE_API_KEY=...
AE_ACCOUNT_ID=...
AE_API_TIMEOUT=15
```
Tool implementation: `cortex/tools/ae_knowledge.py`
Tool registrations: `cortex/tools/__init__.py`
---
## V3 Search Engine
### Endpoint
```
POST /v3/crud/{obj_type}/search
```
For nested objects (journal_entry scoped to a journal):
```
POST /v3/crud/journal_entry/search
?for_obj_type=journal&for_obj_id={journal_id}
```
### Search body
```json
{
"query_string": "fulltext search term",
"and": [
{ "field": "tags", "op": "icontains", "value": "networking" },
{ "field": "created_on", "op": "gte", "value": "2026-01-01" }
],
"or": [...],
"page_size": 20,
"page": 1,
"order_by": "-updated_on"
}
```
**`query_string` vs `and` filters on `default_qry_str`:**
- `query_string` → triggers `MATCH(default_qry_str) AGAINST(... IN BOOLEAN MODE)` — uses the
FULLTEXT index. Faster and supports boolean operators (`+word`, `-word`, `"phrase"`).
- `and` with `icontains` on `default_qry_str` → plain `LIKE '%term%'`. Slower, no index.
**Important:** `query_string` must be present for `and`/`or` filters to apply. When using
filters without a keyword query, pass `query_string: "%"` as a wildcard to activate the
filter path without restricting by keyword.
### Supported operators
| Operator | SQL | Notes |
|---|---|---|
| `eq` | `=` | exact match |
| `ne` | `!=` | not equal |
| `gt` / `gte` | `>` / `>=` | numeric, dates |
| `lt` / `lte` | `<` / `<=` | numeric, dates |
| `contains` / `icontains` | `LIKE '%v%'` | substring; both case-insensitive on MariaDB |
| `startswith` / `istartswith` | `LIKE 'v%'` | |
| `endswith` / `iendswith` | `LIKE '%v'` | |
| `like` | `LIKE` | raw LIKE pattern |
| `in` | `IN (...)` | value is a list |
| `is_null` / `is_not_null` | `IS NULL` / `IS NOT NULL` | no value needed |
### Sorting
`order_by` accepts any indexed field name. Prefix with `-` for descending:
- `-updated_on` (default for listing)
- `-created_on`
- `name`
- `-priority`
### Pagination
`page_size` (default 10, max ~100) + `page` (1-based).
Total count is in `response["meta"]["data_list_count"]` — not a top-level key.
---
## journal_entry Schema
Full table schema from `ae_describe journal_entry --detailed`:
| Field | Type | Indexed | Notes |
|---|---|---|---|
| `id_random` | varchar(22) | UNI | DB public ID field — but API responses return this as `journal_entry_id` (the Vision ID convention: `{obj_type}_id`). `id_random` key is `None` in responses. |
| `journal_id` | int | MUL | FK — use `for_obj_id` param in search |
| `name` | varchar(250) | MUL | Entry title |
| `short_name` | varchar(25) | | |
| `summary` | text | | Short summary (12 sentences) |
| `content` | text | | Full markdown content |
| `content_html` | text | | HTML version |
| `content_json` | longtext | | Structured content (editor format) |
| `content_encrypted` | longtext | | Optional encrypted content |
| `tags` | varchar(255) | MUL | Comma-separated string — filter with `icontains` |
| `type` / `type_code` | varchar | | Classification: type |
| `topic` / `topic_code` | varchar | | Classification: topic |
| `activity` / `activity_code` | varchar | | Classification: activity |
| `category_code` | varchar(25) | | Classification: category |
| `code` | varchar(20) | | Short entry code |
| `start_datetime` | datetime | MUL | Optional event start |
| `end_datetime` | datetime | | Optional event end |
| `seconds` / `hours` | int/decimal | | Duration |
| `priority` | tinyint | MUL | 1=low → 5=high |
| `status` | int | MUL | Status code (domain-specific) |
| `private` / `public` / `personal` / `professional` | tinyint | MUL | Visibility flags |
| `billable` | tinyint | | Billing flag |
| `enable` | tinyint NOT NULL | MUL | Soft-delete flag (default 1) |
| `hide` | tinyint | MUL | UI hide flag |
| `archive` | tinyint | MUL | Archived flag |
| `default_qry_str` | text | FULLTEXT | Auto-generated search target (name + content) |
| `data_json` | longtext | | Arbitrary structured data |
| `notes` | text | | Internal notes |
| `created_on` | timestamp NOT NULL | MUL | Auto-set on create |
| `updated_on` | timestamp | MUL | Auto-updated on change |
### journal Schema (top-level)
| Field | Type | Notes |
|---|---|---|
| `id_random` | varchar(22) | DB field — returned in API as `journal_id` |
| `name` | varchar(250) | Journal name |
| `summary` / `description` | text | |
| `type_code` | varchar(25) | Journal type |
| `enable` | tinyint | |
| `created_on` / `updated_on` | timestamp | |
---
## Current Tool Inventory
| Tool | Status | Notes |
|---|---|---|
| `ae_journal_list` | ✅ | Lists journals with id + name |
| `ae_journal_search` | ✅ | Fulltext + tag/date/type/status/priority filters; paginated |
| `ae_journal_entry_read` | ✅ | Full content by entry_id; configurable truncation |
| `ae_journal_entries_list` | ✅ | Browse a journal newest-first; paginated |
| `ae_journal_entry_create` | ✅ | Create with title, content, tags, summary |
| `ae_journal_entry_update` | ✅ | Patch any fields (title, content, tags, summary, enable) |
| `ae_journal_entry_disable` | ✅ | Soft-delete (enable=false) |
| `ae_journal_entry_append` | ✅ | Timestamped append to bottom |
| `ae_journal_entry_prepend` | ✅ | Timestamped prepend to top |
| `ae_task_list` | ✅ | agents_sync Kanban (admin only) |
---
## ae_journal_search — Current Signature
All filters are optional and combine with AND. At least one should be provided.
```python
ae_journal_search(
query: str = "", # fulltext via query_string (MATCH/AGAINST)
journal_id: str = "", # scope to a specific journal
tags: str = "", # icontains on tags field
type_code: str = "", # eq on type_code
topic_code: str = "", # eq on topic_code
date_from: str = "", # created_on gte (YYYY-MM-DD)
date_to: str = "", # created_on lte (YYYY-MM-DD, exclusive of time — use next day to include full day)
sort_by: str = "updated", # updated | created | name | priority
sort_order: str = "desc",
status: int | None = None,
priority: int | None = None,
max_results: int = 10,
page: int = 1,
)
```
**date_to boundary note:** `date_to='2026-01-17'` means `<= 2026-01-17 00:00:00`, which
excludes entries created later that day. Use `date_to='2026-01-18'` to include all of Jan 17.
---
## Planned: Broader AE Platform Integration
### Phase 1 — Journal Toolset (current)
Complete read/write/search for Journals and Journal Entries.
### Phase 2 — Tasks & Projects
- `ae_task_create` / `ae_task_update` / `ae_task_complete` on Aether tasks (not just agents_sync Kanban)
- Read project/task hierarchy
### Phase 3 — Knowledge Import Pipeline
- Script to walk markdown dirs, chunk by H2, create Journal entries
- Dedup via search-before-create pattern
- Tag and classify entries automatically via orchestrator
### Phase 4 — People & Contacts
- Read contact records (person, organization)
- Link journal entries to contacts
### Phase 5 — Calendar / Events
- `start_datetime` / `end_datetime` already on journal_entry
- Could expose time-scoped journal queries as a calendar view
---
## Notes on `tags` field
`tags` is stored as a raw comma-separated varchar(255), not a JSON array.
The API accepts a Python list on write (the `tags` PATCH key takes a list and the backend joins it).
On read, it comes back as a **string** (e.g. `"shelterluv, api"`), not a list — normalize before
displaying: `[t.strip() for t in tags_str.split(",") if t.strip()]`.
For filtering: use `icontains` on `tags` inside the `"and"` list, e.g.:
`{"field": "tags", "op": "icontains", "value": "networking"}`.
A tag search for "net" matches "networking" AND "subnet" — acceptable for now.
True per-tag filtering would require a tags junction table.
## Notes on `default_qry_str`
Auto-populated by the backend from `name` + content fields. Do not write to it directly.
FULLTEXT index supports boolean mode: `+required -excluded "exact phrase"`.
The `query_string` key in the search body triggers this path automatically.

Some files were not shown because too many files have changed in this diff Show More