Commit Graph

178 Commits

Author SHA1 Message Date
Scott Idem
ac06b3bc7b feat: project-scoped file tools — grep, stat, syntax_check, offset reads
Add five project-scoped tools (user-level, no admin required):
  project_file_read — read with 1-based offset for paging large files
  project_file_list — list with sizes + timestamps
  file_stat         — size, modified time, line count / entry count
  file_grep         — regex search with context lines, up to 50 matches
  file_syntax_check — py_compile (.py) or json.loads (.json)

Also add offset support to existing file_read (system scope).
Rename "Files" tool category to "System Files"; add "Project Files" category.
Project scope restricted to Cortex_and_Inara_dev/ project root.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 22:23:50 -04:00
Scott Idem
fc6600c33e feat: Home Assistant API tools (ha_get_state, ha_get_states, ha_call_service)
Register three HA orchestrator tools so Inara can read device states and
control devices via the HA REST API. ha_call_service requires admin role
and user confirmation. Also includes accumulated UI fixes (setProcessing
helper, wasNewSession flag cleanup).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:39:35 -04:00
Scott Idem
ba91de37c5 feat: Home Assistant settings UI + fix channels.json
notifications.html: add Home Assistant section with two collapsible
blocks — Connection (HA URL + Long-Lived Access Token) and Inbound
webhook (webhook ID with endpoint URL hint showing the username).
Token field uses keep-existing pattern (blank = no change).

settings.py: wire ha_url, ha_token, ha_webhook_id through
_notifications_page() template substitution and save_notifications()
POST handler. Preserves existing HA config fields (persona, tier,
role, tools) on save.

TODO__Agents.md: add Home Assistant integration planning section
(event design, richer payload template, HA API tools).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 21:18:45 -04:00
Scott Idem
1d361fe809 feat: NCT orchestrator support + Home Assistant webhook
nextcloud_talk.py:
- Fix missing import hmac / import hashlib (NameError bug in _verify_signature)
- Add orchestrator routing when channels.json "tools": true — sends
  " Working on it…" immediately, then runs the full tool loop and
  replies with the result; checkpoint case gets a web UI confirmation note
- Read tier and role from channel config (defaults: default_tier / "chat")
- Pass cfg through to _process_message

homeassistant.py (new):
- POST /webhook/ha/{username}/{webhook_id}
- Auth: webhook_id path segment matched against channels.json
- Accepts JSON or form-encoded body from HA automations
- Builds natural-language task from payload (uses "message" key if present,
  otherwise serialises full body as context)
- Same orchestrator/direct dispatch as NCT
- Delivers response via notify() — NC Talk, web push, or configured channel
- Session key: ha_{username} for continuity across HA events
- Registered in main.py; /webhook/ prefix already public in auth_middleware

channels.json schema addition:
  "homeassistant": {
    "webhook_id": "your-secret-id",
    "persona": "inara",
    "tier": 2,
    "role": "chat",
    "tools": false
  }

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 19:45:59 -04:00
Scott Idem
19d6f004ed feat: reasoning level select (Off/Light/Moderate/High/Max)
Replace free-form reasoning_budget_tokens number input with a 5-level
select in both the edit form (local_llm.py) and add-model form
(local_llm.html). Values: 0 / 1024 / 4096 / 8192 / 32768 tokens.
Edit form pre-selects the stored value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:44:20 -04:00
Scott Idem
a66c5a7f84 feat: reasoning token budget + session name in header
- reasoning_budget_tokens: optional int field on local_openai models;
  when set, injects {"reasoning": {"budget_tokens": N}} via extra_body
  into every OpenRouter API call (both tool-loop and confirmation-gate
  rounds). Field exposed in the model edit form in Settings.

- session name moved from standalone full-row div between #messages
  and #input-area into the persona-switcher block in the header, as a
  third dim line under "Cortex · Local". Collapses when empty via
  :empty CSS. No JS changes required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 21:35:23 -04:00
Scott Idem
85792a7bcf feat: per-role inject_mode, OTR fixes, hover metadata, send/stop tooltip
- inject_mode: per-role toggle (parallel to inject_datetime) gates the
  "Current mode: Off The Record" line in the system prompt; wired through
  model_registry, context_loader, chat router, orchestrator router, and
  local_llm settings UI

- OTR orchestrator fix: OrchestrateRequest now carries off_record;
  _finalize_job stores it per message and gates log_turn on it; JS
  orchestrate payload sends off_record correctly

- Per-message hover metadata: removed always-visible .model-tag; replaced
  with .msg-meta strip in the action bar (hover-only); shows model label,
  host, fallback indicator, and OTR badge; stored in session JSON

- Send/stop button tooltip: shows role + model and (when tools on)
  separate orchestrator model + engine label; live elapsed timer on stop
  button via startRunTimer/stopRunTimer

- OrchestratorResult.backend_label: new field; openai_orchestrator fills
  it; finalize_job propagates it to job dict and session messages

- GET /backend: exposes orchestrator_model label so the frontend tooltip
  can show both models separately

- TODO: session delete confirmation added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 16:12:03 -04:00
Scott Idem
0afa135ce9 docs: document System block and OTR mode in ARCH__PERSONA.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:40:21 -04:00
Scott Idem
128d8a7c1e feat: inject session mode into persona system prompt
context_loader.load_context() now accepts a mode param ("chat"|"otr").
In OTR mode, the --- System --- block gains a second line:

  Current mode: Off The Record — this conversation is private
  and will not be logged or included in memory distillation

routers/chat.py passes mode="otr" when req.off_record is True.
Normal chat and all orchestrator calls stay at mode="chat" (no change
to the System block). The System block consolidates date/time and mode
in one place, matching the existing timestamp pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:35:09 -04:00
Scott Idem
3a4f518300 docs: add SELF_UPDATE.md — agent self-maintenance bootstrap
Short reference covering: git repo location, Syncthing fleet sync,
ignore files (.gitignore / .stignore), helper scripts (install.py,
dev-restart.sh, backup.sh), standard change workflow, doc update
checklist, pip dependency process, and key paths on the service host.
Linked from MASTER.md document map.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 14:00:28 -04:00
Scott Idem
348ca120c1 feat: full channels.json UI + http_allowlist settings
Notifications page:
- NC Talk section expanded: url, bot_secret, notification_room,
  nc_username, nc_app_password — all fields from channels.json now editable
- Per-channel sections use <details>/<summary> collapsibles; auto-open
  when values are present
- Secrets use type=password with "leave blank to keep" semantics
- Google Chat outbound webhook in its own collapsible section

Account settings:
- HTTP POST Allowlist section added (same textarea pattern as email allowlist)
- POST /settings/http-allowlist route saves home/{user}/http_allowlist.json
- Example placeholder shows ha.dgrzone.com and n8n patterns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:57:18 -04:00
Scott Idem
7b443b40a4 feat: http_post tool, nc_talk_history tool, local orchestrator retry
- http_post: POST to external URLs with per-user URL prefix allowlist
  (home/{user}/http_allowlist.json); admin-only, confirm-required
- nc_talk_history: read recent NC Talk messages via Basic Auth (requires
  nc_username + nc_app_password in channels.json under nextcloud)
- openai_orchestrator: _chat_with_retry() wraps both API calls with
  exponential backoff (3 attempts, 1s/2s) on connection errors and
  transient status codes (429, 500, 502, 503, 504)
- Docs updated: CLAUDE.md, HELP.md, TODO, MASTER, ROADMAP (50 tools)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:38:38 -04:00
Scott Idem
b9a78819ac docs: add LLM wiki concept (Karpathy pattern) to ARCH__FUTURE.md
Inara's exploration of a living-wiki knowledge compilation architecture
as an alternative to RAG — three-layer model, ingest/query/lint ops,
and a mapping to existing Cortex concepts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:22:55 -04:00
Scott Idem
3672fa1506 docs: comprehensive doc audit — sync all docs to current state
- MASTER.md: tool count 40→47, add proactive notifications + spawn_agent rows, date bump
- ROADMAP.md: mark local orchestrator/web push/proactive notifs/spawn_agent/web_read/session_read as done, date bump
- ARCH__CHANNELS.md: rewrite notification channel config section — all 4 channels, all triggers, on-demand endpoints
- ARCH__SYSTEM.md: update tools/ module list to include files, agents
- README.md: update LLM backends in architecture diagram, add browser push to channels table
- CLAUDE.md: add doc update checklist to Documentation Philosophy section

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:13:45 -04:00
Scott Idem
52c19afbcc fix: raise web_read and http_fetch max_chars cap to 128K
Both tools now accept max_chars up to 131072 to accommodate long
documentation pages and large API responses.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:08:17 -04:00
Scott Idem
17e8869d12 docs: update tool count (45→47), HELP.md, and TODO for new web/file tools
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:05:04 -04:00
Scott Idem
7c3291960a feat: web_read (trafilatura), session_read, http_fetch max_chars
web_read(url, max_chars=16000) — fetches a URL and extracts clean article
text via trafilatura, stripping ads/nav/boilerplate. Returns markdown.

session_read(date) — reads a full session log by YYYY-MM-DD date; lists
available dates if the requested one is not found.

http_fetch gains a max_chars param (default 8192, max 32768) so the cap
is configurable instead of hardcoded.

Tool count: 45 → 47.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 13:04:24 -04:00
Scott Idem
a99ebb8c30 feat: retry button for orchestrator errors + explicit client timeout
Extract orchestrator inner loop into _doOrchestrate() so the retry button
can re-run without re-adding the user message to DOM or history — same
pattern as the existing chat retry.

Also set AsyncOpenAI(timeout=settings.timeout_local) so slow remote models
(OpenRouter/DeepSeek) get the same 300s budget as local chat calls instead
of the SDK default which varies by connection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 12:39:34 -04:00
Scott Idem
ff154b1ec0 docs: update CLAUDE.md, HELP.md, and TODO for notifications page + push fix
- CLAUDE.md: date → 2026-05-08, add Proactive notifications row to channel table
- HELP.md: update Notifications settings entry, expand Push Notifications section
  with channel config link, add test API endpoints to reference table
- TODO__Agents.md: mark notifications dedicated page and pywebpush fix as done

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:58:47 -04:00
Scott Idem
c21f9a23ec fix: use Vapid.from_pem() instead of passing PEM string to webpush()
pywebpush 2.x routes string keys through Vapid.from_string() which only
handles raw/DER base64 — not PEM. Pre-build the Vapid object so the key
deserializes correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:56:17 -04:00
Scott Idem
19475610be feat: move Notifications to its own settings sub-page
Adds GET /settings/notifications (dedicated page with channel form + two
test buttons) and updates POST /settings/notifications to render that page.
Settings page now shows a compact link card instead of the full form.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:43:52 -04:00
Scott Idem
3c7ecf4e4f feat: notification test endpoints — POST /api/push/test and /api/push/reminders/check
- POST /api/push/test: sends "Test notification from Cortex" via the
  user's configured notification channel (web_push / NCT / email / etc.)
- POST /api/push/reminders/check: runs the daily reminder check immediately
  for the current user, returns reminders_found count

Both require an active session cookie. Useful for verifying channel setup
without waiting for the 09:00 scheduler job.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:34:58 -04:00
Scott Idem
64020ad982 feat: proactive notifications — web_push channel + daily reminder check
Routes web_push through notification.py alongside NCT/email/Google Chat,
and fires daily reminder summaries via the scheduler.

- notification.py: _notify_web_push() + "web_push" case in notify();
  all four channels (web_push/email/nextcloud/google_chat) now routable
- scheduler.py: _run_reminder_check() daily at 09:00 — reads due reminders
  per persona via set_context(), formats up to 3 entries, calls notify()
- routers/settings.py: "web_push" added to valid notification_channel values
- static/settings.html: "Browser Push Notification" option in channel selector
- TODO__Agents.md: proactive notifications section marked complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 23:28:49 -04:00
Scott Idem
47d23a7b2f feat: per-model max_rounds for Gemini orchestrator engine
Mirrors the pattern already in openai_orchestrator.py. The Gemini engine
was still hardcoded to the global orchestrator_max_rounds setting.

- orchestrator_engine.py: max_rounds param on run() and _run_from_contents();
  effective_limit = min(per_model_limit, global_limit); stored in checkpoint
  so resume() respects it across confirmation gates
- routers/orchestrator.py: passes orch_model.get("max_rounds") to run()
- tools/agents.py: passes model_cfg.get("max_rounds") for gemini_api spawns

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:54:37 -04:00
Scott Idem
09d775b47b feat: spawn_agent tool + host max_concurrent + docs
Adds a synchronous sub-agent spawning tool that lets the orchestrator
delegate tasks to a specific role's model and tool set.

- cortex/tools/agents.py: spawn_agent(task, role, tier, timeout, max_rounds)
  - Supports local_openai and gemini_api model types
  - Per-host asyncio semaphore (keyed by host_id or model type)
  - asyncio.wait_for() enforces timeout; admin-only tool
- cortex/model_registry.py: max_concurrent field in host schema (default 3,
  clamped 1-20); backfilled on _normalize() for existing hosts
- cortex/routers/local_llm.py + local_llm.html: "Max parallel" number input
  in host add/edit forms
- cortex/tools/__init__.py: spawn_agent registered in TOOL_CATEGORIES["Agents"],
  _CALLABLES, TOOL_ROLES (admin), and _ALL_DECLARATIONS
- Docs: TOOLS.md count 44→45, spawn_agent section; HELP.md tool table updated;
  ARCH__FUTURE.md Round 2 completed items; TODO__Agents.md spawn_agent checked;
  CLAUDE.md tool count and list updated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 22:48:21 -04:00
Scott Idem
6ad7597db8 feat: per-role inject_datetime toggle for system prompt
Each role can now disable the current date/time header injected into the
system prompt. Default is true (all existing roles unchanged). Useful for
pure processing roles (summarizer, classifier, translator) where temporal
context is irrelevant or could cause unexpected model behavior.

Changes:
- model_registry: set_role_config/get_role_config gain inject_datetime field
- context_loader: load_context gains inject_datetime param (default True)
- orchestrator router: passes inject_datetime from role_cfg to load_context
- local_llm router: reads inject_datetime from POST body, passes to registry;
  role_config_data_js includes the field
- local_llm.html: checkbox in role config panel; populate on open, save on submit

Session logs still timestamp every turn (HH:MM header in YYYY-MM-DD.md files)
regardless of this setting — the toggle only affects the system prompt header.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:53:35 -04:00
Scott Idem
8e512d4e11 feat: reminders due-date support + context filtering
reminders_add now accepts optional due: YYYY-MM-DD parameter.
Due date stored as first line of section body in REMINDERS.md.

context_loader.py calls load_due_reminders() instead of loading REMINDERS.md
wholesale — future-dated reminders are suppressed in the system prompt until
their date arrives. Undated reminders always surface (backward compatible).

reminders_list shows due status per entry: [OVERDUE by N days], [due TODAY],
or [due: YYYY-MM-DD] for future items. All reminders visible via the tool
regardless of date; only context surfacing is filtered.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:46:45 -04:00
Scott Idem
750cde489d feat: session_search tool + tool expansion docs update
session_search (tools/files.py):
- Full-text search across past session logs, exposed to the orchestrator
- Params: query (required), limit (default 5, max 20)
- Returns dated excerpts, newest first; own sessions only via ContextVars
- User-level — no TOOL_ROLES gating needed
- Registered in __init__.py callables + TOOL_CATEGORIES["Files"]

ARCH__FUTURE.md §2: updated tool count to 44, marked prior tools complete,
added Round 2 planned tools table (session_search now done, reminders due dates,
http_post, nc_talk_history, task_list priority filter, http_fetch max_chars),
noted datetime_now is not needed (already in system prompt via context_loader)

TODO__Agents.md: session_search checked off, Round 2 task list added

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:41:26 -04:00
Scott Idem
f8f7cd75da feat: audit log, usage tracking UI, OpenAI orchestrator compaction, onboarding + docs
Tool audit log:
- Every orchestrator tool call logged to home/{user}/tool_audit/YYYY-MM-DD.jsonl
- Files panel sidebar: audit log group (collapsed), date-linked read-only table
- Admin endpoints: /api/audit/files, /api/audit/day, /api/audit/recent, /api/audit/stats
- Engine and model name recorded per entry

OpenAI orchestrator improvements:
- Context budget enforcement: 75% of model context_k (min 16k)
- Message compaction: truncates old tool results when approaching budget
- max_rounds respected per model config (intersected with server cap)

OpenRouter onboarding (setup.html, onboarding.py, app.js, settings.html):
- Step 3 of 3: /setup/model with curated model picker
- Chat banner for users on server-default model (informational, not alarmist)
- Settings quick-link card; /setup/model works standalone for existing users

Model registry + session store:
- set_role_config / get_role_config for per-role tool lists and system_append
- session_store: session rename, session name backfill endpoint

UI updates (app.js, index.html, style.css, local_llm.html):
- Role toggle in context panel
- Off-the-record mode
- Agent notes read-only viewer
- OPERATIONS.md loaded at T2+ in context

Documentation:
- HELP.md: full tool table, per-role tool sets, Agent Notes, usage tracking
- TOOLS.md: Agent Notes section, count corrected to 44
- ARCH__SYSTEM.md, ARCH__BACKENDS.md, MASTER.md updated to match reality
- CLAUDE.md: onboarding flow, documentation philosophy sections
- README.md: stack in practice, DeepSeek TUI mention, architecture diagram updated
- TODO__Agents.md: onboarding task completed with deviation notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:26:43 -04:00
Scott Idem
c02d2462b0 feat: agent notes, OpenRouter onboarding, usage tracking, per-role tools docs
Agent notes tool (cortex/tools/agent_notes.py):
- Private durable notepad for the orchestrator — not user-visible
- agent_notes_read/write/append/clear with 3 rolling backups
- Per-persona isolation via ContextVars; no TOOL_ROLES gating needed
- PROTOCOLS.md updated to make this a core proactive tool

OpenRouter guided onboarding:
- Setup Step 3 (/setup/model) — OpenRouter quick-connect with curated model list
- Amber banner in chat for users on server-default model
- Settings quick-link card (/settings/models OpenRouter section)
- POST /setup/model/skip for users who want to bypass Step 3
- Holly pre-configured: DeepSeek V4 Flash (OpenRouter) → Gemma Medium (local) → claude_cli

Usage tracking:
- cortex/routers/usage.py — GET /api/usage, /api/usage/summary, /api/usage/all (admin)

Documentation:
- HELP.md: Tools section rewritten — full tool table by category, per-role tool sets explained
- TOOLS.md: Agent Notes section added; count corrected to 44
- ARCH__SYSTEM.md, ARCH__BACKENDS.md, MASTER.md, CLAUDE.md, README.md updated
- TODO__Agents.md: onboarding task checked off with deviation notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:25:31 -04:00
Scott Idem
5d4f5ee598 feat: OPERATIONS.md bootstrap doc + load at T2+; patch stale persona files
- New home/scott/persona/inara/OPERATIONS.md: self-maintenance workflow
  (cortex_update → review → cortex_restart), access control table, key
  paths, memory file map, distillation cadence, channel/architecture notes
- context_loader.py: load OPERATIONS.md at Tier 2+ after PROTOCOLS.md
- TOOLS.md: count 39→40, add web_push to Notifications section
- PROTOCOLS.md: replace stale 10-tool list with reference to TOOLS.md
- CONTEXT_TIERS.md: fix memory file names (MEMORY.md → LONG/MID/SHORT),
  update Tier 2 load list, fix Hard Rules credentials note

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 22:21:03 -04:00
Scott Idem
a75546485b feat: context budget enforcement + compaction in OpenAI orchestrator
Protects all models in the Primary/Backup chain regardless of context window:
- _context_budget(): 75% of model_cfg["context_k"] * 1000 (default 32k if unset)
- _estimate_tokens(): char count / 4 + 3k overhead for tool schemas
- _compact_messages(): truncates old tool results to 400 chars, keeps last 6
  intact (~2 recent rounds), logs chars saved per compaction pass
- Compaction runs before every API call; log line now shows estimated token count
- Malformed tool call args logged with model/args detail instead of silent {}
- finish_reason check accepts "stop" and None alongside "tool_calls" (some
  models return wrong reason even when tool_calls are present)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 22:01:54 -04:00
Scott Idem
7d221863dc feat: engine/model in audit log + docs update
- tool_audit: ContextVars (engine, model) set at orchestrator run start; fields added to every entry
- orchestrator_engine: tool_audit.set_context("gemini", model_name) at run() start
- openai_orchestrator: tool_audit.set_context("openai", model label) at run() start
- audit table: Model column between Status and Args
- HELP.md: push notifications section, audit log in Files section, tool count 30→40, new API endpoints
- TODO__Agents.md: web_push and audit log marked complete with full detail

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:42:32 -04:00
Scott Idem
02accefe8f feat: audit log in Files panel sidebar
Adds an "Audit Log" section (collapsed by default) at the bottom of the Files
panel showing tool_audit/YYYY-MM-DD.jsonl files for the current user.

- GET /api/audit/files  — lists available dates (newest first, any auth user)
- GET /api/audit/day    — returns entries for one date as JSON (any auth user)
- tool_audit.read_day() — reads a single day's JSONL file chronologically
- Clicking a date renders a read-only table: time / tool / status / args / result
- Status cells are colour-coded (green ok, red error, amber denied)
- Edit/Raw/Preview/Save buttons are hidden in audit view, restored on file switch
- Audit group starts collapsed; expands on click like other file groups

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 20:36:08 -04:00
Scott Idem
584ae679a6 feat: tool call audit log
Every orchestrator tool invocation is recorded to home/{user}/tool_audit/YYYY-MM-DD.jsonl.
Each entry captures: timestamp, user, tool, args (truncated), status (ok/error/denied),
result length, and a 300-char result snippet.

- tool_audit.py: JSONL writer with per-file asyncio locks; read_recent / read_recent_all_users helpers
- tools/__init__.py: hook in call_tool() — fire-and-forget record on every dispatch
- routers/audit.py: GET /api/audit/recent and /api/audit/stats (admin-only)
- tools/files.py: add home_root() to file_read allowed roots so agents can read audit JSONL

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:55:59 -04:00
Scott Idem
ddf44a2aee feat: web push notifications (VAPID)
- push_utils.py: subscription storage + send helper (auto-prunes 410 endpoints)
- routers/push.py: GET /api/push/vapid-key (public), POST/DELETE /api/push/subscribe
- sw.js: push event listener shows notification; notificationclick focuses/opens tab
- app.js: subscribe/unsubscribe flow + "Enable notifications" toggle in settings dropdown
- tools/notify.py: web_push orchestrator tool (user-level, no admin required)
- VAPID keys in .env; pywebpush added to requirements.txt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:38:58 -04:00
Scott Idem
0b96772fa6 fix: show session friendly name in resume message and status bar
/history/{session_id} now returns a 'name' field alongside messages.
resumeSession() uses data.name first, then the sessionNames map, then
raw ID as fallback — so named sessions display correctly even on page
load before the sessions panel has been opened.

'Resumed session X' message also now shows the friendly name.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:14:59 -04:00
Scott Idem
5d23d04e7e fix: session panel wider + two-line layout for session names
Root cause: 300px panel minus edit btn (28px) + meta (~130px) + delete
btn (28px) + gaps/padding left only ~70px (~7 chars) for the session name.

- Panel: 300px → 420px desktop, 300px → 380px mobile drawer
- Max-height: 340px → 400px
- Session item: name and meta now in a .session-body flex column, so the
  name gets full body width (panel minus two buttons) — meta lives below
- Edit mode: hides .session-body + delete, input takes the full body slot

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:07:33 -04:00
Scott Idem
7a0fbdb659 feat: session rename UX overhaul
- Edit button (✎) moved to left of row, separated from delete (×)
- Clicking ✎ hides name/meta/delete and expands input to full row width
- Button changes to ✓ (accent color) while editing
- Enter or ✓ click = save; Escape = cancel without saving
- Removed accidental-save-on-blur behavior
- Edit button: 30% opacity at rest, 75% on row hover, 100% on direct hover
- Touch devices: edit button always at 60% opacity (no hover to reveal it)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 19:00:39 -04:00
Scott Idem
508fb638ad feat: distill safeguards — rolling backups + sanity checks
Before any memory file is overwritten, _rotate_backup() keeps 2 rolling
backups: MEMORY_*.bak1.md (most recent) and MEMORY_*.bak2.md (older).

_sanity_check() now also guards against size anomalies: the new content
must be between 40% and 250% of the old file size — anything outside that
range looks like truncation or runaway output and aborts the write.
Existing checks (min length, refusal phrases) still apply.

Backup files exposed in the Files panel (ALLOWED set) so they can be
reviewed and manually restored if needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:54:27 -04:00
Scott Idem
0ffcd57c95 fix: multi-user distillation + datetime in context + session log labels
Distillation was silently operating on scott/inara for all users due to
ContextVar defaults. All three distill endpoints now require ?user=&persona=
query params and validate them via persona.validate(). Memory distiller
signatures changed from Optional to required positional args — no more
global settings fallback. Scheduler now iterates all users/personas instead
of hardcoding the primary user.

- context_loader: inject current date/time as first system prompt section
- session_logger: use get_user()/get_persona() from context instead of
  settings globals so Holly/Brian sessions show correct speaker labels
- memory_distiller: system prompts now reference u.title()/p.title()
  instead of settings.user_name/settings.agent_name
- distill router: Query(...) enforces params; _resolve() validates persona
- scheduler: _all_personas() helper iterates every user/persona for distill
- app.js: runDistill() now appends ?user=&persona= via _fileParams

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 18:44:51 -04:00
Scott Idem
8d4aa4094c feat: usage tracking + knowledge import script
- usage_tracker.py: daily token/call buckets per user (home/{user}/usage.json)
- Hook into local backend (OpenAI usage field) and Gemini API (usage_metadata)
- Claude/Gemini CLI backends produce no structured token data and are not tracked
- Fix CLAUDE.md stale tool count (27 → 39) and refresh tool list
- scripts/import_knowledge.py: walk markdown dirs, chunk by H2, call local LLM
  for summaries, create AE journal entries with path-derived tags; resumable via
  state file; --dry-run and --limit flags for safe testing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 20:38:31 -04:00
Scott Idem
eab92d876d refactor: split tool declarations into domain files + role config UI
tools/__init__.py shrinks from 1,137 → 250 lines. Each domain file now
owns both its callables and its FunctionDeclarations (DECLARATIONS list),
so adding a new tool only touches one file.

New TOOL_CATEGORIES dict exported from __init__ — used by the UI for
grouped tool checkboxes.

Role config UI (Settings → Model Registry → Role Assignments):
- ⚙ button per role expands an inline configure panel
- Textarea for system_append (injected into system prompt for this role)
- Grouped checkboxes for tool allow-list (all checked = no restriction)
- POST /api/models/role-config saves both fields; updates ROLE_CONFIG_DATA
  in-page so re-open reflects current state without a page reload

Backend:
- model_registry.set_role_config() writes system_append + tools to registry
- TOOL_CATEGORIES exported from tools/__init__ for UI rendering
- TOOLS.md header updated: 30 → 39 tools (ae_journal_* and cortex_* additions)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 20:40:50 -04:00
Scott Idem
49123cdd5c feat: per-role tool lists and system prompt overlays
Each role in model_registry.json can now carry two optional keys:
  system_append — injected into the system prompt at position 7 (after
                  memory, closest to the turn) for the active chat_role
  tools         — explicit tool allow-list; intersected with the user's
                  access-level filter so it can only restrict, never elevate

No changes needed for existing users — missing keys fall back to current
behavior. Add keys to a role to give it a specialty focus:

  "coder": {
    "primary": "claude_cli",
    "system_append": "You are in code-specialist mode...",
    "tools": ["web_search", "file_read", "shell_exec", "scratch_write"]
  }

Changes:
- model_registry.py: get_role_config() returns system_append + tools
- context_loader.py: role_append param appended as "--- Role Context ---"
- tools/__init__.py: get_tools_for_role/get_openai_tools_for_role accept
  optional tool_list and intersect with access-level filter
- orchestrator_engine.py: tool_list threaded through run/resume/checkpoint
- openai_orchestrator.py: tool_list threaded through run/resume/checkpoint;
  _build_client now calls get_openai_tools_for_role instead of returning
  unfiltered OPENAI_TOOL_SCHEMAS
- routers/orchestrator.py: pulls role_cfg for chat_role, passes both
  role_append and tool_list to context loader and engine

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 20:00:38 -04:00
Scott Idem
5ad2e50d69 feat: split help into tabbed UI Guide / Tools / Persona pages
- cortex/static/TOOLS.md — tool reference extracted from HELP.md; uses ##
  headers so each category is collapsible. All 30 tools with descriptions.
- cortex/static/HELP.md — UI guide only; tools section replaced with a
  one-line pointer to the Tools tab.
- help.html — three tabs (UI Guide / Tools / Persona); tab choice persists
  in localStorage. Tools tab defaults all sections open. Persona tab shows
  home/{user}/persona/{name}/HELP.md with an empty-state message if unset.
- context_loader.py — loads cortex/static/TOOLS.md into context at tier 2+
  (replaces the previously empty persona HELP.md load). Persona HELP.md
  still loaded if non-empty, as persona-specific additions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:25:47 -04:00
Scott Idem
552fd56abb docs: expand tools section in HELP.md with per-tool descriptions
Replaces single flat table with category sections, each with a description
column. Footnotes moved to the top of the section for clarity. Covers all
30 tools including the new cortex_status, cortex_update, reminders_remove,
ae_journal_entry_read, ae_journal_entries_list, and email_send.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:11:22 -04:00
Scott Idem
77997bc4ae feat: add cortex_status and cortex_update tools
cortex_status: git branch/commit/ahead-behind + systemctl state — read-only
cortex_update: git pull + syntax check all .py files + report; does NOT auto-restart.
  If syntax errors are found after pull, warns and blocks restart suggestion.
  Call cortex_restart separately to apply a clean update.

Both are admin-only. cortex_update is confirm-required (modifies files on disk).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:01:42 -04:00
Scott Idem
1ffa846edd docs: sync HELP.md tools table and files list with current implementation
- Add reminders_remove (targeted single-reminder removal, no confirm needed)
- Add ae_journal_entry_read, ae_journal_entries_list to AE Journals row
- Add email_send (admin-only) to Notifications row
- Remove TASKS.json from Files table (not in the Files panel)
- Add email_allowlist.json to Files table (Settings group in Files panel)
- Update last-updated date

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:54:50 -04:00
Scott Idem
98546abe21 docs: update ARCH__AE_INTEGRATION with verified API behavior
- query_string required for and/or filters to apply; use "%" as wildcard
- Total count is in meta.data_list_count, not top-level
- id_random is None in responses; Vision ID convention uses {obj_type}_id
- tags comes back as string on read, not list — normalize before joining
- Replace stale "Planned: Search Improvements" with current signature + notes
- Clarify date_to boundary (lte midnight, use next day to include full day)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:17:19 -04:00
Scott Idem
1fa5151d8a fix: correct V3 search filter key and response field names in ae_knowledge
- Filter key is "and" not "and_filters" (V3 API format)
- Entry IDs use journal_entry_id/id, not id_random (id_random is None)
- Dates use updated_on/created_on, not updated_at/created_at
- Total count lives in meta.data_list_count, not top-level total/count
- Inject query_string="%" when and filters present but no query, since
  the V3 search engine requires query_string for filters to apply
- Normalize tags from string to list in both entry_read and entries_list
- Fix order_by to use updated_on (not updated_at) in entries_list
- Correct ARCH__AE_INTEGRATION.md: and_filters → and, or_filters → or

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:12:44 -04:00