Compare commits
163 Commits
c01ef663f5
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e8819773ee | ||
|
|
0c1cf3989a | ||
|
|
658c508925 | ||
|
|
29d8aa4aae | ||
|
|
29940c299b | ||
|
|
105ff8507f | ||
|
|
c2a12a895a | ||
|
|
df1f358912 | ||
|
|
7a27190ffe | ||
|
|
070f1ce156 | ||
|
|
a92fd90f0d | ||
|
|
70665fadff | ||
|
|
96b3c796c5 | ||
|
|
50c1997e91 | ||
|
|
3716e5974f | ||
|
|
85e13314a2 | ||
|
|
20f3fe4f71 | ||
|
|
f336ae9687 | ||
|
|
76fef827c5 | ||
|
|
b7144d5903 | ||
|
|
3c9b8f5909 | ||
|
|
54eef73b74 | ||
|
|
4c3d9a7a65 | ||
|
|
8ab1942514 | ||
|
|
69ec2f667d | ||
|
|
c9c1ca7de6 | ||
|
|
ac06b3bc7b | ||
|
|
fc6600c33e | ||
|
|
ba91de37c5 | ||
|
|
1d361fe809 | ||
|
|
19d6f004ed | ||
|
|
a66c5a7f84 | ||
|
|
85792a7bcf | ||
|
|
0afa135ce9 | ||
|
|
128d8a7c1e | ||
|
|
3a4f518300 | ||
|
|
348ca120c1 | ||
|
|
7b443b40a4 | ||
|
|
b9a78819ac | ||
|
|
3672fa1506 | ||
|
|
52c19afbcc | ||
|
|
17e8869d12 | ||
|
|
7c3291960a | ||
|
|
a99ebb8c30 | ||
|
|
ff154b1ec0 | ||
|
|
c21f9a23ec | ||
|
|
19475610be | ||
|
|
3c7ecf4e4f | ||
|
|
64020ad982 | ||
|
|
47d23a7b2f | ||
|
|
09d775b47b | ||
|
|
6ad7597db8 | ||
|
|
8e512d4e11 | ||
|
|
750cde489d | ||
|
|
f8f7cd75da | ||
|
|
c02d2462b0 | ||
|
|
5d4f5ee598 | ||
|
|
a75546485b | ||
|
|
7d221863dc | ||
|
|
02accefe8f | ||
|
|
584ae679a6 | ||
|
|
ddf44a2aee | ||
|
|
0b96772fa6 | ||
|
|
5d23d04e7e | ||
|
|
7a0fbdb659 | ||
|
|
508fb638ad | ||
|
|
0ffcd57c95 | ||
|
|
8d4aa4094c | ||
|
|
eab92d876d | ||
|
|
49123cdd5c | ||
|
|
5ad2e50d69 | ||
|
|
552fd56abb | ||
|
|
77997bc4ae | ||
|
|
1ffa846edd | ||
|
|
98546abe21 | ||
|
|
1fa5151d8a | ||
|
|
71e472bebe | ||
|
|
77327d97ad | ||
|
|
36fdda6728 | ||
|
|
6405dd338d | ||
|
|
bce7de647c | ||
|
|
165cf3552d | ||
|
|
db3dd465b2 | ||
|
|
e0e3170de3 | ||
|
|
b8bc4ea21f | ||
|
|
fd0fb76c08 | ||
|
|
a5658eb3c4 | ||
|
|
334e7f0dea | ||
|
|
1603ad5124 | ||
|
|
25182a1765 | ||
|
|
f726d78979 | ||
|
|
217c7c3d6a | ||
|
|
66cb197de0 | ||
|
|
1222f806ce | ||
|
|
ed191cf0b4 | ||
|
|
44f215c764 | ||
|
|
d61e39d614 | ||
|
|
93a692f3f0 | ||
|
|
af4d78136a | ||
|
|
af7d8b40e2 | ||
|
|
4159f470d6 | ||
|
|
e2a61bb78d | ||
|
|
80702a21e2 | ||
|
|
2b9dd53566 | ||
|
|
1cc7988953 | ||
|
|
8baab874f1 | ||
|
|
962d58d2e2 | ||
|
|
3bc6b45f9f | ||
|
|
ef07596955 | ||
|
|
6e56024815 | ||
|
|
9f6b162fbd | ||
|
|
f08b033d6c | ||
|
|
45c95d20ba | ||
|
|
27ca7c7efd | ||
|
|
869a596f4b | ||
|
|
3b3456600a | ||
|
|
6c84d6ae72 | ||
|
|
2629983452 | ||
|
|
f668826f79 | ||
|
|
576f22216a | ||
|
|
bf800acca8 | ||
|
|
d9a322164a | ||
|
|
8ba5247ef5 | ||
|
|
a6e404c143 | ||
|
|
2dd94696d5 | ||
|
|
8570e8d852 | ||
|
|
9299ce5ba6 | ||
|
|
608e1de246 | ||
|
|
6a1a1c2686 | ||
|
|
a4daebdc9b | ||
|
|
bd6532e93a | ||
|
|
a94fdc869d | ||
|
|
1fefd42e19 | ||
|
|
0c17b4b1ab | ||
|
|
cec6d3e23a | ||
|
|
2d3a380d6b | ||
|
|
662924c6a1 | ||
|
|
e5b6d58889 | ||
|
|
6b725afc3e | ||
|
|
ddf5dd6338 | ||
|
|
93f7f44e51 | ||
|
|
496da58f58 | ||
|
|
8e20bfbea8 | ||
|
|
3a94df1eaf | ||
|
|
ce806e52ed | ||
|
|
7438031797 | ||
|
|
8aec6aafcc | ||
|
|
62fde62653 | ||
|
|
f8d89bc272 | ||
|
|
92350f7a7b | ||
|
|
4f09823afe | ||
|
|
826bd6cfe3 | ||
|
|
c3507f8e11 | ||
|
|
65548ebf36 | ||
|
|
24c9f52b49 | ||
|
|
3bada5f311 | ||
|
|
cf277f822e | ||
|
|
6cf10d2755 | ||
|
|
fa04b5e6b0 | ||
|
|
b3f40cf437 | ||
|
|
8487645224 | ||
|
|
0cf0d65e9e | ||
|
|
1b425a539f |
88
.env.default
88
.env.default
@@ -1,88 +0,0 @@
|
||||
# Cortex .env reference — copy to .env and fill in values
|
||||
# DO NOT commit .env — it contains secrets
|
||||
|
||||
# ── Agent identity ───────────────────────────────────────────────────────────
|
||||
# Global display names used in distillation prompts and session logs.
|
||||
# Individual persona identities live in home/{username}/persona/{name}/IDENTITY.md
|
||||
AGENT_NAME=Inara
|
||||
USER_NAME=Scott
|
||||
|
||||
# ── Home directory ────────────────────────────────────────────────────────────
|
||||
# Root for all user/persona data. Layout: home/{username}/persona/{name}/
|
||||
# Relative paths are resolved from the cortex/ directory.
|
||||
# Default: ../home (i.e. Cortex_and_Inara_dev/home/)
|
||||
# HOME_DIR=../home
|
||||
|
||||
# ── Session auth ─────────────────────────────────────────────────────────────
|
||||
# Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
|
||||
JWT_SECRET=change-me-in-dotenv
|
||||
JWT_EXPIRE_DAYS=30
|
||||
|
||||
# ── SMTP (invite emails + future notifications) ───────────────────────────────
|
||||
SMTP_SERVER=linode.oneskyit.com
|
||||
SMTP_PORT=465
|
||||
SMTP_USERNAME=send_mail
|
||||
SMTP_PASSWORD=
|
||||
SMTP_FROM_EMAIL=noreply@oneskyit.com
|
||||
SMTP_FROM_NAME=Cortex
|
||||
# Base URL included in invite links
|
||||
CORTEX_BASE_URL=https://cortex.dgrzone.com
|
||||
|
||||
# ── Server ──────────────────────────────────────────────────────────────────
|
||||
HOST=0.0.0.0
|
||||
PORT=8000
|
||||
|
||||
# ── Google Chat bot ──────────────────────────────────────────────────────────
|
||||
# JWT audience for verifying inbound Workspace Add-on Chat webhook requests.
|
||||
# For Workspace Add-on Chat apps, the aud claim = the endpoint URL.
|
||||
# Leave blank to disable verification (dev/testing only).
|
||||
GOOGLE_CHAT_AUDIENCE=https://cortex.dgrzone.com/channels/google-chat
|
||||
|
||||
# ── Nextcloud Talk bot ───────────────────────────────────────────────────────
|
||||
NEXTCLOUD_URL=https://cloud.dgrzone.com
|
||||
NEXTCLOUD_TALK_BOT_SECRET=
|
||||
|
||||
# ── LLM backends ────────────────────────────────────────────────────────────
|
||||
# Primary backend: "claude" or "gemini" (other is always fallback)
|
||||
PRIMARY_BACKEND=claude
|
||||
|
||||
# Timeouts in seconds
|
||||
TIMEOUT_CLAUDE=60
|
||||
TIMEOUT_GEMINI=120
|
||||
|
||||
# ── Orchestrator (Gemini API — not Gemini CLI) ───────────────────────────────
|
||||
# Required for /orchestrate endpoint and tool use
|
||||
# Free tier key: https://aistudio.google.com/apikey
|
||||
GEMINI_API_KEY=
|
||||
|
||||
# Model for the orchestration tool loop (not the user-facing response)
|
||||
ORCHESTRATOR_MODEL=gemini-2.5-flash
|
||||
|
||||
# Safety cap on tool loop iterations
|
||||
ORCHESTRATOR_MAX_ROUNDS=10
|
||||
|
||||
# ── DuckDuckGo search ────────────────────────────────────────────────────────
|
||||
# Leave blank for free unauthenticated tier
|
||||
# Set to your API key for higher rate limits (paid DuckDuckGo account)
|
||||
DDG_API_KEY=
|
||||
DDG_MAX_RESULTS=5
|
||||
|
||||
# ── Aether Platform API ───────────────────────────────────────────────────────
|
||||
# Used by orchestrator tools: ae_journal_search, ae_journal_entry_create, ae_task_list
|
||||
# Same values as agents_sync/mcp/.env — copy from there
|
||||
AE_API_URL=https://dev-api.oneskyit.com
|
||||
AE_API_KEY=
|
||||
AE_ACCOUNT_ID=
|
||||
AE_API_TIMEOUT=15
|
||||
|
||||
# ── Distillation schedule ────────────────────────────────────────────────────
|
||||
SCHEDULER_TIMEZONE=America/New_York
|
||||
AUTO_DISTILL=true
|
||||
AUTO_DISTILL_SHORT=true
|
||||
AUTO_DISTILL_MID=true
|
||||
AUTO_DISTILL_LONG=false # manual review recommended before enabling
|
||||
|
||||
# Memory tier token budgets (soft caps)
|
||||
MEMORY_BUDGET_SHORT=3000
|
||||
MEMORY_BUDGET_MID=2000
|
||||
MEMORY_BUDGET_LONG=2000
|
||||
28
.gitignore
vendored
28
.gitignore
vendored
@@ -3,25 +3,33 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# Secrets — keep .env.example, never commit real .env
|
||||
# Secrets — keep .env.default, never commit real .env
|
||||
.env
|
||||
cortex/.env*.bak
|
||||
|
||||
# Session data (runtime state, not source)
|
||||
# Pip install artifacts
|
||||
cortex/=*
|
||||
|
||||
# Runtime data
|
||||
cortex/data/
|
||||
home/**/session_data/
|
||||
|
||||
# User credentials and tokens — never commit
|
||||
home/**/auth.json
|
||||
home/**/invite.json
|
||||
home/**/profile.json
|
||||
# User home directory — all persona data, memory, tasks, and credentials
|
||||
# are personal and must never be committed. Back up via encrypted means.
|
||||
home/
|
||||
|
||||
# Syncthing Metadata
|
||||
# Syncthing metadata
|
||||
.stfolder/
|
||||
|
||||
# Temporary Files
|
||||
# Temporary files
|
||||
tmp/
|
||||
*.tmp
|
||||
*.log
|
||||
|
||||
# System Files
|
||||
# Aider — history files are personal/ephemeral; .aider.conf.yml is project config and IS tracked
|
||||
.aider.chat.history.md
|
||||
.aider.input.history
|
||||
.aider.llm.history
|
||||
|
||||
# System files
|
||||
.DS_Store
|
||||
.aider*
|
||||
|
||||
5
.stignore
Normal file
5
.stignore
Normal file
@@ -0,0 +1,5 @@
|
||||
// Machine-local — never sync across hosts
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
cortex/data/
|
||||
128
CLAUDE.md
128
CLAUDE.md
@@ -22,7 +22,7 @@ Cortex_and_Inara_dev/
|
||||
main.py ← App entry point, router registration
|
||||
config.py ← All settings (pydantic-settings, reads .env)
|
||||
persona.py ← Two-level identity: user + persona, path resolution, ContextVars
|
||||
llm_client.py ← Claude CLI + Gemini CLI subprocess backends
|
||||
llm_client.py ← Claude CLI + Gemini CLI subprocess backends + Anthropic SDK direct
|
||||
orchestrator_engine.py ← Gemini API ReAct tool loop → Claude handoff
|
||||
context_loader.py ← Builds system prompt from persona files (tier 1–4)
|
||||
session_store.py ← In-memory + file session persistence
|
||||
@@ -45,12 +45,15 @@ Cortex_and_Inara_dev/
|
||||
google_chat.py ← POST /webhook/google (Google Chat Add-on)
|
||||
ui.py ← Login/logout, /{user}/{persona} UI route, /api/personas
|
||||
onboarding.py ← /setup/{token} password step + /setup/persona creation
|
||||
settings.py ← /settings, /settings/notifications, /settings/integrations (admin)
|
||||
tools_settings.py ← /settings/tools
|
||||
crons.py ← /settings/crons — Schedules web UI (list/add/edit/toggle/remove)
|
||||
tools/
|
||||
__init__.py ← Tool registry (Gemini FunctionDeclarations + dispatcher)
|
||||
web.py ← DuckDuckGo web_search tool
|
||||
scratch.py ← Scratchpad tools (scratch_read/write/append/clear)
|
||||
tasks.py ← Personal task management (task_create/list/update/complete)
|
||||
cron.py ← Scheduled job tools (cron_list/add/remove/toggle)
|
||||
cron.py ← Scheduled job tools (cron_list/add/remove/toggle); 5 types; hourly/daily/weekly/monthly/yearly schedules
|
||||
system.py ← Local machine tools (claude_allow_dir)
|
||||
tests/ ← pytest test suite (80 tests)
|
||||
static/ ← Single-page web UI (index.html, style.css, app.js)
|
||||
@@ -82,6 +85,7 @@ Cortex_and_Inara_dev/
|
||||
|
||||
docs/ ← Integration reference docs
|
||||
NEXTCLOUD_TALK_BOT.md
|
||||
OPEN_WEBUI_API.md ← Open WebUI API: tool calling, RAG, model management
|
||||
|
||||
documentation/ ← Architecture decisions and agent task list
|
||||
TODO__Agents.md ← READ THIS FIRST — active task list
|
||||
@@ -97,8 +101,8 @@ Cortex_and_Inara_dev/
|
||||
## Run Commands
|
||||
|
||||
```bash
|
||||
# Start (Docker)
|
||||
docker compose up -d
|
||||
# First-time setup or update on any machine
|
||||
python3 install.py
|
||||
|
||||
# Restart service (after any Python change)
|
||||
systemctl --user restart cortex
|
||||
@@ -135,9 +139,10 @@ http://localhost:8000/docs
|
||||
- **Orchestrated tasks** go to `POST /orchestrate` — returns a job_id, result is polled
|
||||
|
||||
### LLM Backends
|
||||
- `llm_client.py` manages Claude CLI (`claude --print`) and Gemini CLI (`gemini -p`) subprocesses
|
||||
- `llm_client.py` manages Claude CLI (`claude --print`), Gemini CLI (`gemini -p`), and Anthropic SDK (`anthropic_api` type) subprocesses/calls
|
||||
- `orchestrator_engine.py` uses the Gemini **API** (google-genai SDK) — completely separate from the Gemini CLI
|
||||
- Claude OAuth token is read live from `~/.claude/.credentials.json` (never rely on stale env var)
|
||||
- `anthropic_api` backend: user-configured API key from `providers.anthropic.credentials` in `model_registry.json` — uses `anthropic.AsyncAnthropic`
|
||||
|
||||
### Tool Strategy
|
||||
- Orchestrator tools live in `cortex/tools/` — separate from the `ae_*` MCP tools
|
||||
@@ -145,8 +150,8 @@ http://localhost:8000/docs
|
||||
- Tools are registered in `cortex/tools/__init__.py` as both Gemini FunctionDeclarations and Python callables
|
||||
|
||||
### Context / Memory
|
||||
- `context_loader.py` assembles Inara's system prompt from `inara/` files based on tier (1–3)
|
||||
- Tier 1 = minimal (identity only); Tier 2 = standard (+ memory + user profile); Tier 3 = full
|
||||
- `context_loader.py` assembles Inara's system prompt from `inara/` files based on tier (1–4)
|
||||
- Tier 1 = minimal (identity only); Tier 2 = standard (+ memory + user profile); Tier 3 = + last 2 sessions; Tier 4 = + last 7 sessions
|
||||
- Memory files are written by the distiller or manually — do not delete them
|
||||
|
||||
### Security / Safety
|
||||
@@ -159,6 +164,44 @@ http://localhost:8000/docs
|
||||
- Passwords are bcrypt-hashed and stored in `home/{username}/auth.json` — never in `.env` or the DB
|
||||
- Invite tokens are one-time-use, 72-hour expiry, stored in `home/{username}/invite.json`
|
||||
|
||||
### Onboarding Flow
|
||||
New users follow a three-step setup before reaching the chat:
|
||||
1. `GET /setup/{token}` → password form → `POST /setup/{token}` sets password + session cookie
|
||||
2. `GET /setup/persona` → persona creation form → `POST /setup/persona` bootstraps persona directory
|
||||
3. `GET /setup/model` → OpenRouter quick-connect → `POST /setup/model` saves host + model + role assignment
|
||||
|
||||
Step 3 is optional (skip link goes straight to `/{user}/{persona}`). `/setup/model` also works
|
||||
standalone (accessible from Settings) for existing users who haven't configured a model.
|
||||
|
||||
All in `cortex/routers/onboarding.py`. Model writes use `model_registry.py`: `save_host()`,
|
||||
`save_model()`, `set_role(username, "chat", "primary", model_id)`.
|
||||
|
||||
### Documentation Philosophy
|
||||
Cortex is a no-black-box system. Docs must match reality — at all times.
|
||||
|
||||
- **Docs first:** When planning significant changes, update `TODO__Agents.md` and the relevant
|
||||
`ARCH__*.md` to describe the intended design *before* implementing. This creates a spec to
|
||||
implement against.
|
||||
- **Verify after:** Once implementation is complete, re-read the pre-written docs and confirm
|
||||
they match what was actually built. Update anything that drifted.
|
||||
- **HELP.md is a user contract:** It describes what users can do. Never let it describe
|
||||
features that don't exist or omit features that do.
|
||||
- **CLAUDE.md + ARCH__*.md are the developer contract:** Update them as the architecture evolves.
|
||||
- **Stale docs are bugs.** If you notice drift, fix it before moving on.
|
||||
|
||||
### Doc update checklist (run after any significant change)
|
||||
|
||||
| Doc | Update when |
|
||||
|---|---|
|
||||
| `CLAUDE.md` | New tool, channel, router, major design change, tool count |
|
||||
| `cortex/static/HELP.md` | Any user-visible feature — tools, settings, UI, API endpoints |
|
||||
| `documentation/TODO__Agents.md` | Mark completed items; add new planned work |
|
||||
| `documentation/MASTER.md` | New capability goes live; tool count changes |
|
||||
| `documentation/ROADMAP.md` | Phase items completed or added |
|
||||
| `documentation/ARCH__CHANNELS.md` | New channel, notification trigger, or scheduler job |
|
||||
| `documentation/ARCH__SYSTEM.md` | New module, router, or tools/ file |
|
||||
| `README.md` | Architecture diagram, channels table, or setup steps change |
|
||||
|
||||
---
|
||||
|
||||
## Adding a New Tool
|
||||
@@ -166,7 +209,13 @@ http://localhost:8000/docs
|
||||
1. Implement the tool function in `cortex/tools/<domain>.py`
|
||||
- Must be `async def`; use `asyncio.to_thread` for blocking calls
|
||||
- Return a plain string result
|
||||
2. Add a `FunctionDeclaration` and register it in `cortex/tools/__init__.py`
|
||||
2. Add a `FunctionDeclaration` and register it in `cortex/tools/__init__.py`:
|
||||
- Import the callable
|
||||
- Add to `TOOL_CATEGORIES` (pick an existing category or create one)
|
||||
- Add to `_CALLABLES`
|
||||
- Add a `TOOL_RISK` rating (low/medium/high)
|
||||
- Add to `TOOL_ROLES` if admin-only; add to `CONFIRM_REQUIRED` if destructive
|
||||
- Add module to `_ALL_DECLARATIONS`
|
||||
3. Syntax check: `python3 -m py_compile cortex/tools/<domain>.py`
|
||||
4. Restart Cortex
|
||||
|
||||
@@ -211,37 +260,48 @@ clearly asked for a directory to be unblocked.
|
||||
|
||||
---
|
||||
|
||||
## Current State (2026-03-20)
|
||||
## Current State (2026-05-12)
|
||||
|
||||
Cortex is running and stable. All three primary channels are live:
|
||||
Cortex is running and stable. All channels are live:
|
||||
|
||||
| Channel | Status | Notes |
|
||||
|---|---|---|
|
||||
| Web UI | ✅ Live | `https://cortex.dgrzone.com` (basic auth) |
|
||||
| Web UI | ✅ Live | `https://cortex.dgrzone.com` — PWA-installable |
|
||||
| Nextcloud Talk | ✅ Live | HMAC-signed webhook, async reply |
|
||||
| Google Chat | ✅ Live | Workspace Add-on, `hostAppDataAction` response format |
|
||||
| Local backend | ✅ Live | Open WebUI/Ollama on scott_gaming, per-user multi-model config |
|
||||
| Gemini orchestrator | ✅ Live | Gemini API tool loop → Claude response; ⚡ toggle in UI |
|
||||
| Local orchestrator | ✅ Live | OpenAI-compatible ReAct loop; fires when orchestrator role → local model |
|
||||
| Tool audit log | ✅ Live | Every tool call logged to `home/{user}/tool_audit/YYYY-MM-DD.jsonl` |
|
||||
| Token usage tracking | ✅ Live | Per-user `home/{user}/usage.json`; summary in Settings |
|
||||
| Web push | ✅ Live | VAPID push notifications; `web_push` tool; subscribe via ☰ menu |
|
||||
| Proactive notifications | ✅ Live | Daily reminder check (09:00); distill/cron completions; `GET /settings/notifications` dedicated page |
|
||||
| Proactive cron | ✅ Live | 5 job types: `remind`, `note`, `message`, `brief`, `task` (full orchestrator loop); monthly/yearly schedule formats; HA inbound webhook tools toggle |
|
||||
| Schedules web UI | ✅ Live | `/settings/crons` — list, add, edit, pause/resume, delete scheduled jobs |
|
||||
|
||||
### Active Tasks
|
||||
Active users: scott (inara), holly (tina), brian (wintermute)
|
||||
|
||||
See `documentation/TODO__Agents.md` for the full list. Current priorities:
|
||||
**69 orchestrator tools** across 17 domain modules:
|
||||
web_search/http_fetch/web_read/http_post,
|
||||
project_file_read/list + file_stat/grep/diff/syntax_check (project-scoped),
|
||||
file_read/list/write/session_read/session_search (system-scoped, admin),
|
||||
git_status/git_log/git_diff (read-only git inspection, project-scoped),
|
||||
shell_exec/claude_allow_dir,
|
||||
cortex_restart/logs/status/update,
|
||||
task_list/create/update/complete, cron_list/add/remove/toggle,
|
||||
reminders_add/list/remove/clear, scratch_read/write/append/clear,
|
||||
web_push/email_send/nc_talk_send/nc_talk_history,
|
||||
ae_journal_list/search/entries_list/entry_read/create/update/disable/append/prepend,
|
||||
ae_task_list, ae_db_query/describe/show_view (SELECT-only MariaDB access, admin; disable requires confirm),
|
||||
agent_notes_read/write/append/clear, spawn_agent/aider_run (admin; aider_run requires confirm),
|
||||
agent_status/agent_list (user-level)/agent_cancel (admin, confirm-required),
|
||||
ha_get_state/ha_get_states/ha_call_service.
|
||||
|
||||
- **[High]** Ollama backend — local LLM via `scott_gaming` over WireGuard
|
||||
- **[Medium]** NC Talk — complete bot registration docs (`docs/NEXTCLOUD_TALK_BOT.md`)
|
||||
- **[Medium]** Knowledge consolidation — markdown → AE Journals
|
||||
Each tool has a `TOOL_RISK` rating (low/medium/high). Configure access at `/settings/tools`
|
||||
(max_risk threshold + per-tool whitelist/blacklist). Risk policy stored in `home/{user}/tool_policy.json`.
|
||||
|
||||
### Recently Completed
|
||||
|
||||
- ✅ Session auth — bcrypt passwords, JWT cookies, login/logout, `SessionAuthMiddleware` — 2026-03-20
|
||||
- ✅ Persona onboarding — invite tokens, self-service password setup, persona creation form — 2026-03-20
|
||||
- ✅ Multi-persona switcher — dropdown in UI header, `/api/personas` endpoint — 2026-03-20
|
||||
- ✅ SMTP invite email — `noreply@oneskyit.com`, HTML + plain text, `manage_passwords.py invite` — 2026-03-20
|
||||
- ✅ CSS routing fix — `/static/*` mount must precede wildcard `/{user}/{persona}` route — 2026-03-20
|
||||
- ✅ Multi-user/multi-persona support (`home/{username}/persona/{name}/` two-level layout) — 2026-03-20
|
||||
- ✅ Scratchpad, task management, and cron/scheduled job tools — 2026-03-20
|
||||
- ✅ Test suite (80 tests) covering API, persona routing, tools, security — 2026-03-20
|
||||
- ✅ Google Chat bot (Workspace Add-on, JWT auth, `hostAppDataAction` format) — 2026-03-20
|
||||
- ✅ Orchestrator Agent mode UI + session persistence — 2026-03-18
|
||||
- ✅ Memory distiller (APScheduler, short/mid/long) — 2026-03
|
||||
See `documentation/TODO__Agents.md` for the active task list.
|
||||
See `documentation/ROADMAP.md` for phases and what's next.
|
||||
|
||||
---
|
||||
|
||||
@@ -249,8 +309,14 @@ See `documentation/TODO__Agents.md` for the full list. Current priorities:
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `documentation/MASTER.md` | **Start here** — index, current state, all doc links |
|
||||
| `documentation/TODO__Agents.md` | Active task list — read before starting work |
|
||||
| `documentation/ARCH__Intelligence_Layer.md` | Full architecture design |
|
||||
| `~/agents_sync/projects/CORTEX.md` | High-level project vision and phases |
|
||||
| `documentation/ROADMAP.md` | Phases — what's done, what's next |
|
||||
| `documentation/ARCH__SYSTEM.md` | System architecture and component map |
|
||||
| `documentation/ARCH__BACKENDS.md` | LLM backends, routing, per-user config |
|
||||
| `documentation/ARCH__PERSONA.md` | Persona system, context tiers, memory distillation |
|
||||
| `documentation/ARCH__CHANNELS.md` | Input channels — web, NC Talk, Google Chat, cron |
|
||||
| `documentation/ARCH__FUTURE.md` | Planned: local orchestrator, dev agents, knowledge layer |
|
||||
| `~/agents_sync/projects/CORTEX.md` | Project vision and philosophy |
|
||||
| `~/agents_sync/CLAUDE.md` | Fleet coordination rules |
|
||||
| `~/CLAUDE.md` | Machine identity (`scott_lpt`) |
|
||||
|
||||
75
Cortex_and_Inara.code-workspace
Normal file
75
Cortex_and_Inara.code-workspace
Normal file
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"folders": [
|
||||
{
|
||||
"name": "cortex (service)",
|
||||
"path": "cortex"
|
||||
},
|
||||
{
|
||||
"name": "home (personas)",
|
||||
"path": "home"
|
||||
},
|
||||
{
|
||||
"name": "documentation",
|
||||
"path": "documentation"
|
||||
},
|
||||
{
|
||||
"name": "docs (integrations)",
|
||||
"path": "docs"
|
||||
},
|
||||
{
|
||||
"name": "project root",
|
||||
"path": "."
|
||||
}
|
||||
],
|
||||
"settings": {
|
||||
"files.exclude": {
|
||||
"**/__pycache__": true,
|
||||
"**/*.pyc": true,
|
||||
"cortex/.venv": true,
|
||||
"cortex/data": true
|
||||
},
|
||||
"search.exclude": {
|
||||
"**/__pycache__": true,
|
||||
"cortex/.venv": true,
|
||||
"cortex/data": true,
|
||||
"home/**/sessions": true,
|
||||
"home/**/session_data": true
|
||||
},
|
||||
"[python]": {
|
||||
"editor.formatOnSave": false
|
||||
},
|
||||
"editor.rulers": [100],
|
||||
"files.associations": {
|
||||
"*.env": "dotenv",
|
||||
"*.env.default": "dotenv"
|
||||
}
|
||||
},
|
||||
"extensions": {
|
||||
"recommendations": [
|
||||
"ms-python.python",
|
||||
"ms-python.vscode-pylance",
|
||||
"humao.rest-client",
|
||||
"tamasfe.even-better-toml"
|
||||
]
|
||||
},
|
||||
"launch": {
|
||||
"version": "0.2.0",
|
||||
"configurations": [
|
||||
{
|
||||
"name": "Cortex (uvicorn dev)",
|
||||
"type": "python",
|
||||
"request": "launch",
|
||||
"module": "uvicorn",
|
||||
"args": [
|
||||
"main:app",
|
||||
"--host", "0.0.0.0",
|
||||
"--port", "8000",
|
||||
"--reload"
|
||||
],
|
||||
"cwd": "${workspaceFolder:cortex (service)}",
|
||||
"envFile": "${workspaceFolder:cortex (service)}/.env",
|
||||
"justMyCode": false
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
147
README.md
147
README.md
@@ -6,7 +6,44 @@
|
||||
|
||||
> *"You can't stop the signal."*
|
||||
|
||||
Cortex is a self-hosted multi-agent AI platform. It supports multiple users, each with their own named AI persona. Inara (Scott's persona) and Tina (Holly's persona) are the initial instances.
|
||||
Cortex is a self-hosted multi-agent AI platform. It supports multiple users, each with their own named AI persona.
|
||||
|
||||
---
|
||||
|
||||
## Where Cortex Fits
|
||||
|
||||
AI tools aren't one-size-fits-all. Cortex exists in a specific niche — it's not trying to be everything.
|
||||
|
||||
**Cortex is a self-hosted persona platform.** It gives you a persistent AI companion with its own
|
||||
identity, memory, and voice — reachable through your chat apps, not just a browser tab. It remembers
|
||||
who you are across days and weeks. It can proactively message you on a schedule. It runs on your
|
||||
own hardware, behind your own auth.
|
||||
|
||||
### What Cortex is good at
|
||||
- **Being a consistent AI presence** — same persona, same memory, day after day
|
||||
- **Multi-channel access** — web, Nextcloud Talk, Google Chat, all routed to the same brain
|
||||
- **Proactive work** — scheduled messages, reminders, cron jobs that reach out to you
|
||||
- **Multi-user households** — each person gets their own persona (Scott → Inara, Holly → Tina)
|
||||
- **Private, offline-capable** — local models via Ollama when you don't want anything leaving the LAN
|
||||
|
||||
### What Cortex is not
|
||||
- **Not a coding assistant.** Cortex lives in chat apps, not in your terminal or IDE.
|
||||
Use Claude Code, DeepSeek TUI, Gemini CLI, or Copilot for code-level work — they specialize in reading and
|
||||
editing project files. Cortex can't open a codebase.
|
||||
- **Not a generic LLM chat UI.** Open WebUI and LibreChat are excellent model-switching frontends.
|
||||
Cortex isn't a frontend — it's a platform with its own identity system, orchestrator, and memory
|
||||
pipeline. Two different jobs.
|
||||
- **Not a SaaS product.** Nobody else hosts your Cortex instance. Nobody else sees your conversations.
|
||||
The trade-off is you manage the service yourself — `systemctl --user restart cortex`.
|
||||
- **Not an agent framework.** LangChain, CrewAI, and similar are libraries for building AI pipelines.
|
||||
Cortex is a running service with concrete personas, not an abstraction layer to build on top of.
|
||||
|
||||
### The stack in practice
|
||||
- Use **Cortex** to talk to Inara — daily assistant, memory keeper, scheduled check-ins
|
||||
- Use **Claude Code / DeepSeek TUI** to work *on* Cortex — code edits, architecture, debugging
|
||||
- Use **Open WebUI** when you want to test a new model or run a quick prompt without persona context
|
||||
|
||||
Same AI, different interfaces for different jobs.
|
||||
|
||||
---
|
||||
|
||||
@@ -16,9 +53,7 @@ Cortex is a self-hosted multi-agent AI platform. It supports multiple users, eac
|
||||
|---|---|
|
||||
| `cortex/` | FastAPI service — dispatcher, routing, LLM backends, session management |
|
||||
| `home/` | User and persona data (`home/{username}/persona/{name}/`) |
|
||||
| `home/scott/persona/inara/` | Inara identity, memory, and context files |
|
||||
| `home/holly/persona/tina/` | Tina identity, memory, and context files |
|
||||
| `docs/` | Integration reference docs (NC Talk bot, etc.) |
|
||||
| `docs/` | Integration reference docs (NC Talk bot, Google Chat bot) |
|
||||
| `documentation/` | Architecture decisions, project plans, agent task lists |
|
||||
|
||||
---
|
||||
@@ -48,6 +83,20 @@ with a letter or underscore; max 32 characters. Example: `scott`, `holly`, `my_a
|
||||
|
||||
---
|
||||
|
||||
## Setup / Install
|
||||
|
||||
Run `install.py` on any machine to set up or update Cortex. It is idempotent — safe to re-run.
|
||||
|
||||
```bash
|
||||
python3 install.py # install / update everything
|
||||
python3 install.py --check # status check only, no changes
|
||||
```
|
||||
|
||||
What it does: creates the Python venv, installs dependencies, writes the systemd user service,
|
||||
enables linger, starts/restarts the service, checks LLM CLI auth, and sets up the daily backup timer.
|
||||
|
||||
Config: copy `cortex/.env.default` to `cortex/.env` and fill in secrets before first run.
|
||||
|
||||
## Running Cortex
|
||||
|
||||
Cortex runs as a **systemd user service** (no sudo required).
|
||||
@@ -69,49 +118,83 @@ http://localhost:8000 (or cortex.dgrzone.com on WireGuard)
|
||||
The service starts automatically at boot via `loginctl enable-linger`.
|
||||
Service file: `~/.config/systemd/user/cortex.service`
|
||||
|
||||
Config lives in `cortex/config.py` and a `.env` file at the project root (not tracked — see `.env.default`).
|
||||
Config lives in `cortex/config.py` and `cortex/.env` (not tracked — see `cortex/.env.default`).
|
||||
|
||||
## Development Workflow
|
||||
|
||||
The codebase lives in `agents_sync/` and syncs to all fleet machines via Syncthing.
|
||||
Edit code on any machine; use `dev-restart.sh` to apply changes on the host running the service.
|
||||
|
||||
```bash
|
||||
./dev-restart.sh # restart service, show last 30 log lines
|
||||
./dev-restart.sh logs # tail live logs (ctrl-c to stop)
|
||||
./dev-restart.sh status # show service status only
|
||||
```
|
||||
|
||||
## Backup
|
||||
|
||||
Persona data (`home/`) is excluded from git and backed up with restic.
|
||||
`install.py` sets up a systemd timer that runs `backup.sh` daily at 03:00.
|
||||
|
||||
```bash
|
||||
./backup.sh # run a backup manually
|
||||
|
||||
# Inspect snapshots (set env vars or export them)
|
||||
RESTIC_REPOSITORY=~/backups/cortex-home-restic \
|
||||
RESTIC_PASSWORD_FILE=~/.config/cortex/restic-password \
|
||||
restic snapshots
|
||||
```
|
||||
|
||||
The restic password is generated at `~/.config/cortex/restic-password` on first install.
|
||||
Back it up separately — it is required to restore from any snapshot.
|
||||
|
||||
---
|
||||
|
||||
## Key Documentation
|
||||
|
||||
**Start here for a full picture:** [`documentation/MASTER.md`](documentation/MASTER.md)
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `documentation/TODO__Agents.md` | Active task list — read first |
|
||||
| `documentation/ARCH__Intelligence_Layer.md` | Intelligence layer architecture (orchestrator, dev agents, knowledge) |
|
||||
| `docs/NEXTCLOUD_TALK_BOT.md` | NC Talk bot setup |
|
||||
| `home/scott/persona/inara/IDENTITY.md` | Inara persona and identity |
|
||||
| `home/scott/persona/inara/HELP.md` | In-app help content (rendered in UI) |
|
||||
| `home/scott/persona/inara/PROTOCOLS.md` | Inara behavioral protocols |
|
||||
| `~/agents_sync/projects/CORTEX.md` | High-level project vision and phases |
|
||||
| `documentation/MASTER.md` | Index — current state, all doc links, quick reference |
|
||||
| `documentation/ROADMAP.md` | Phases — what's done, what's next |
|
||||
| `documentation/TODO__Agents.md` | Active task list |
|
||||
| `documentation/ARCH__SYSTEM.md` | System architecture and component map |
|
||||
| `documentation/ARCH__BACKENDS.md` | LLM backends, routing, fallback |
|
||||
| `documentation/ARCH__PERSONA.md` | Persona system, context tiers, memory distillation |
|
||||
| `documentation/ARCH__CHANNELS.md` | Input channels — web, NC Talk, Google Chat, cron |
|
||||
| `documentation/ARCH__FUTURE.md` | Planned features — local orchestrator, dev agents, knowledge layer |
|
||||
| `docs/NEXTCLOUD_TALK_BOT.md` | NC Talk bot setup and troubleshooting |
|
||||
| `docs/GOOGLE_CHAT_BOT.md` | Google Chat Add-on setup |
|
||||
| `docs/OPEN_WEBUI_API.md` | Open WebUI/Ollama API reference |
|
||||
|
||||
---
|
||||
|
||||
## Architecture at a Glance
|
||||
|
||||
```
|
||||
[User / Cron / Webhook]
|
||||
[Web UI / NC Talk / Google Chat / Cron / Webhooks]
|
||||
↓
|
||||
Cortex Dispatcher (FastAPI, cortex/)
|
||||
├─ POST /chat — direct to LLM (streaming SSE)
|
||||
├─ POST /orchestrate — Gemini tool loop → Claude response
|
||||
├─ POST /webhook/nextcloud — Nextcloud Talk bot
|
||||
└─ POST /webhook/google — Google Chat Add-on
|
||||
├─ POST /webhook/nextcloud/{username} — Nextcloud Talk bot (per-user)
|
||||
└─ POST /channels/google-chat/{username} — Google Chat Add-on (per-user)
|
||||
↓
|
||||
LLM Backend(s)
|
||||
• Claude CLI — primary reasoning, coding, long-context
|
||||
• Gemini CLI — secondary / cost routing
|
||||
• Gemini API — orchestrator tool loop (separate from Gemini CLI)
|
||||
• Ollama — offline/private (scott_gaming, future)
|
||||
LLM Backends
|
||||
• Claude CLI — primary, all user-facing responses
|
||||
• Gemini CLI — fallback
|
||||
• Gemini API — orchestrator tool loop (two-brain: Gemini plans, Claude responds)
|
||||
• Local OpenAI — Open WebUI/Ollama on scott_gaming; also runs local orchestrator loop
|
||||
↓
|
||||
Persona context loaded from home/{user}/persona/{name}/
|
||||
```
|
||||
|
||||
See `documentation/ARCH__Intelligence_Layer.md` for the orchestrator/responder and dev-agent architecture.
|
||||
See `documentation/ARCH__SYSTEM.md` for the full architecture breakdown.
|
||||
|
||||
---
|
||||
|
||||
## Inara / Tina
|
||||
## Personas
|
||||
|
||||
Each persona has its own identity, memory, and session history.
|
||||
They are not tied to a specific LLM model — the name is fixed, the backend varies.
|
||||
@@ -120,17 +203,24 @@ Context is loaded at request time from `home/{user}/persona/{name}/` via `cortex
|
||||
| User | Persona | Description |
|
||||
|---|---|---|
|
||||
| scott | inara | Scott's primary AI assistant |
|
||||
| scott | developer | Scott's dev-focused persona |
|
||||
| holly | tina | Holly's primary AI assistant |
|
||||
| brian | wintermute | Brian's primary AI assistant |
|
||||
|
||||
---
|
||||
|
||||
## Channels
|
||||
|
||||
| Channel | Status | Notes |
|
||||
Webhook endpoints are per-user — each user configures their own secrets in `home/{username}/channels.json`.
|
||||
|
||||
| Channel | Status | Endpoint / Notes |
|
||||
|---|---|---|
|
||||
| Web UI | Live | `https://cortex.dgrzone.com` — session auth (login form + JWT cookie) |
|
||||
| Nextcloud Talk | Live | HMAC-signed webhook, async reply |
|
||||
| Google Chat | Live | Workspace Add-on, JWT auth |
|
||||
| Nextcloud Talk | Live | `POST /webhook/nextcloud/{username}` — HMAC-signed, async reply |
|
||||
| Google Chat | Live | `POST /channels/google-chat/{username}` — Workspace Add-on, JWT auth |
|
||||
| Browser Push | Live | VAPID push notifications — subscribe via ☰ menu; proactive reminders + distill alerts |
|
||||
|
||||
See `docs/NEXTCLOUD_TALK_BOT.md` and `docs/GOOGLE_CHAT_BOT.md` for setup instructions.
|
||||
|
||||
---
|
||||
|
||||
@@ -142,7 +232,10 @@ cd cortex
|
||||
# Create a user directory and send an invite email
|
||||
.venv/bin/python manage_passwords.py invite <username> <email>
|
||||
|
||||
# List users with password and email status
|
||||
# Register a Google account for sign-in (run after user completes onboarding)
|
||||
.venv/bin/python manage_passwords.py google-add <username> <email>
|
||||
|
||||
# List users with password, Google, and email status
|
||||
.venv/bin/python manage_passwords.py list
|
||||
|
||||
# Set/check a password directly
|
||||
@@ -152,6 +245,8 @@ cd cortex
|
||||
|
||||
New users receive a link to `/setup/{token}` where they set their own password and create their first persona. Invite tokens expire in 72 hours and are one-time-use.
|
||||
|
||||
To enable a channel for a user, create `home/{username}/channels.json` — see the relevant doc in `docs/`.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
70
backup.sh
Executable file
70
backup.sh
Executable file
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env bash
|
||||
# backup.sh — restic backup of Cortex persona/home data
|
||||
#
|
||||
# Backs up the home/ directory (all user persona files, memory, tasks, crons).
|
||||
# Code is in git; this covers everything git intentionally excludes.
|
||||
#
|
||||
# Config — override via environment or edit here:
|
||||
REPO_DIR="${RESTIC_REPOSITORY:-$HOME/backups/cortex-home-restic}"
|
||||
PASSWORD_FILE="${RESTIC_PASSWORD_FILE:-$HOME/.config/cortex/restic-password}"
|
||||
SOURCE="$(cd "$(dirname "$0")" && pwd)/home"
|
||||
|
||||
# Retention policy
|
||||
KEEP_DAILY=7
|
||||
KEEP_WEEKLY=4
|
||||
KEEP_MONTHLY=6
|
||||
|
||||
# ── Preflight ─────────────────────────────────────────────────────────────────
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
if ! command -v restic &>/dev/null; then
|
||||
echo "ERROR: restic not found. Install with: sudo pacman -S restic" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ ! -d "$SOURCE" ]]; then
|
||||
echo "ERROR: source directory not found: $SOURCE" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ── Password setup ────────────────────────────────────────────────────────────
|
||||
|
||||
if [[ ! -f "$PASSWORD_FILE" ]]; then
|
||||
mkdir -p "$(dirname "$PASSWORD_FILE")"
|
||||
chmod 700 "$(dirname "$PASSWORD_FILE")"
|
||||
python3 -c "import secrets; print(secrets.token_urlsafe(32))" > "$PASSWORD_FILE"
|
||||
chmod 600 "$PASSWORD_FILE"
|
||||
echo "Generated new restic password: $PASSWORD_FILE"
|
||||
echo "IMPORTANT: back this file up separately — you need it to restore."
|
||||
fi
|
||||
|
||||
export RESTIC_REPOSITORY="$REPO_DIR"
|
||||
export RESTIC_PASSWORD_FILE="$PASSWORD_FILE"
|
||||
|
||||
# ── Init repo if needed ───────────────────────────────────────────────────────
|
||||
|
||||
if [[ ! -d "$REPO_DIR" ]]; then
|
||||
echo "Initializing restic repository at $REPO_DIR …"
|
||||
restic init
|
||||
fi
|
||||
|
||||
# ── Backup ────────────────────────────────────────────────────────────────────
|
||||
|
||||
echo "Backing up $SOURCE → $REPO_DIR"
|
||||
restic backup "$SOURCE" \
|
||||
--exclude="**/sessions" \
|
||||
--exclude="**/session_data" \
|
||||
--tag "cortex-home"
|
||||
|
||||
# ── Prune ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
restic forget \
|
||||
--keep-daily "$KEEP_DAILY" \
|
||||
--keep-weekly "$KEEP_WEEKLY" \
|
||||
--keep-monthly "$KEEP_MONTHLY" \
|
||||
--tag "cortex-home" \
|
||||
--prune
|
||||
|
||||
echo "Backup complete."
|
||||
restic snapshots --tag cortex-home --last 3
|
||||
@@ -1,15 +0,0 @@
|
||||
[Unit]
|
||||
Description=Cortex / Holly LLM Gateway
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=scott
|
||||
WorkingDirectory=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/cortex
|
||||
EnvironmentFile=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/cortex/.env.holly
|
||||
ExecStart=/home/scott/agents_sync/projects/Cortex_and_Inara_dev/cortex/.venv/bin/uvicorn main:app --host 0.0.0.0 --port 8001
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,33 +1,118 @@
|
||||
# Auth is handled by the claude CLI (claude setup-token) — no API key needed here.
|
||||
# ANTHROPIC_API_KEY=only_needed_if_switching_to_sdk
|
||||
# Cortex .env reference — copy to .env and fill in values
|
||||
# DO NOT commit .env — it contains secrets
|
||||
|
||||
# Path to the inara/ identity directory — relative to cortex/ or absolute
|
||||
INARA_DIR=../inara
|
||||
# ── Agent identity ───────────────────────────────────────────────────────────
|
||||
# Global display names used in distillation prompts and session logs.
|
||||
# Individual persona identities live in home/{username}/persona/{name}/IDENTITY.md
|
||||
AGENT_NAME=Inara
|
||||
USER_NAME=Scott
|
||||
|
||||
# Path for persistent JSON session files
|
||||
SESSIONS_DIR=./data/sessions
|
||||
# ── Home directory ────────────────────────────────────────────────────────────
|
||||
# Root for all user/persona data. Layout: home/{username}/persona/{name}/
|
||||
# Relative paths are resolved from the cortex/ directory.
|
||||
# Default: ../home (i.e. Cortex_and_Inara_dev/home/)
|
||||
# HOME_DIR=../home
|
||||
|
||||
# LLM defaults
|
||||
DEFAULT_MODEL=claude-sonnet-4-6
|
||||
DEFAULT_TIER=2
|
||||
# ── Google OAuth — "Sign in with Google" ────────────────────────────────────
|
||||
# Create credentials at console.cloud.google.com → APIs & Services → Credentials
|
||||
# Application type: Web Application
|
||||
# Authorised redirect URI: https://cortex.dgrzone.com/auth/google/callback
|
||||
# Pre-register users: cd cortex && .venv/bin/python manage_passwords.py google-add <user> <email>
|
||||
# Per-user Gemini key: add "gemini_api_key": "AIza..." to home/{username}/auth.json
|
||||
GOOGLE_CLIENT_ID=
|
||||
GOOGLE_CLIENT_SECRET=
|
||||
|
||||
# Session rolling window — number of messages to keep (user + assistant pairs)
|
||||
# 40 = 20 turns
|
||||
MAX_HISTORY_MESSAGES=40
|
||||
# ── Session auth ─────────────────────────────────────────────────────────────
|
||||
# Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
|
||||
JWT_SECRET=change-me-in-dotenv
|
||||
JWT_EXPIRE_DAYS=30
|
||||
|
||||
# Per-backend timeouts (seconds)
|
||||
# Gemini is generous — it frequently takes 30-60s under load
|
||||
# Local models may need time to load into VRAM before first response
|
||||
# ── SMTP (invite emails + future notifications) ───────────────────────────────
|
||||
SMTP_SERVER=linode.oneskyit.com
|
||||
SMTP_PORT=465
|
||||
SMTP_USERNAME=send_mail
|
||||
SMTP_PASSWORD=
|
||||
SMTP_FROM_EMAIL=noreply@oneskyit.com
|
||||
SMTP_FROM_NAME=Cortex
|
||||
# Base URL included in invite links
|
||||
CORTEX_BASE_URL=https://cortex.dgrzone.com
|
||||
|
||||
# ── Server ──────────────────────────────────────────────────────────────────
|
||||
HOST=0.0.0.0
|
||||
PORT=8000
|
||||
|
||||
# ── Google Chat bot ──────────────────────────────────────────────────────────
|
||||
# JWT audience for verifying inbound Workspace Add-on Chat webhook requests.
|
||||
# For Workspace Add-on Chat apps, the aud claim = the endpoint URL.
|
||||
# Leave blank to disable verification (dev/testing only).
|
||||
GOOGLE_CHAT_AUDIENCE=https://cortex.dgrzone.com/channels/google-chat
|
||||
|
||||
# ── Nextcloud Talk bot ───────────────────────────────────────────────────────
|
||||
NEXTCLOUD_URL=https://cloud.dgrzone.com
|
||||
NEXTCLOUD_TALK_BOT_SECRET=
|
||||
|
||||
# ── LLM backends ────────────────────────────────────────────────────────────
|
||||
# Primary backend: "claude", "gemini", or "local" (switchable at runtime via UI)
|
||||
PRIMARY_BACKEND=claude
|
||||
|
||||
# Timeouts in seconds
|
||||
TIMEOUT_CLAUDE=60
|
||||
TIMEOUT_GEMINI=120
|
||||
TIMEOUT_LOCAL=300
|
||||
TIMEOUT_LOCAL=300 # local models may need time to load
|
||||
|
||||
# Google Chat — must respond within 30s or Chat shows an error to the user
|
||||
GOOGLE_CHAT_TIMEOUT=25
|
||||
# Backend pinned for Google Chat (claude recommended — more reliable within 25s)
|
||||
GOOGLE_CHAT_BACKEND=claude
|
||||
# TODO: add GOOGLE_CHAT_TOKEN for request verification once endpoint is public
|
||||
# ── Local model (Open WebUI / Ollama — OpenAI-compatible API) ────────────────
|
||||
# Leave LOCAL_API_URL blank to disable. When set, "local" appears as a backend option.
|
||||
# API key: Open WebUI → Settings → Account → API Keys
|
||||
# Model: workspace alias or full Ollama model name
|
||||
LOCAL_API_URL=http://192.168.32.19:3000
|
||||
LOCAL_API_KEY=
|
||||
LOCAL_MODEL=test-agent-simple
|
||||
|
||||
# Server
|
||||
PORT=8000
|
||||
HOST=0.0.0.0
|
||||
# ── Orchestrator (Gemini API — not Gemini CLI) ───────────────────────────────
|
||||
# Required for /orchestrate endpoint and tool use
|
||||
# Free tier key: https://aistudio.google.com/apikey
|
||||
GEMINI_API_KEY=
|
||||
|
||||
# Model for the orchestration tool loop (not the user-facing response)
|
||||
ORCHESTRATOR_MODEL=gemini-2.5-flash
|
||||
|
||||
# Safety cap on tool loop iterations
|
||||
ORCHESTRATOR_MAX_ROUNDS=10
|
||||
|
||||
# ── DuckDuckGo search ────────────────────────────────────────────────────────
|
||||
# Leave blank for free unauthenticated tier
|
||||
# Set to your API key for higher rate limits (paid DuckDuckGo account)
|
||||
DDG_API_KEY=
|
||||
DDG_MAX_RESULTS=5
|
||||
|
||||
# ── Aether Platform API ───────────────────────────────────────────────────────
|
||||
# Used by orchestrator tools: ae_journal_search, ae_journal_entry_create, ae_task_list
|
||||
# Same values as agents_sync/mcp/.env — copy from there
|
||||
AE_API_URL=https://dev-api.oneskyit.com
|
||||
AE_API_KEY=
|
||||
AE_ACCOUNT_ID=
|
||||
AE_API_TIMEOUT=15
|
||||
|
||||
# ── Aether MariaDB (direct — SELECT-only via ae_db_query/describe/show_view tools) ─
|
||||
# Configured per-user in home/{username}/channels.json — NOT in .env.
|
||||
# Add this block to the user's channels.json to enable the tools:
|
||||
#
|
||||
# "aether_db": {
|
||||
# "host": "192.168.64.5",
|
||||
# "port": 3306,
|
||||
# "name": "aether_dev",
|
||||
# "user": "aether_dev",
|
||||
# "password": "..."
|
||||
# }
|
||||
|
||||
# ── Distillation schedule ────────────────────────────────────────────────────
|
||||
SCHEDULER_TIMEZONE=America/New_York
|
||||
AUTO_DISTILL=true
|
||||
AUTO_DISTILL_SHORT=true
|
||||
AUTO_DISTILL_MID=true
|
||||
AUTO_DISTILL_LONG=false # manual review recommended before enabling
|
||||
|
||||
# Memory tier token budgets (soft caps)
|
||||
MEMORY_BUDGET_SHORT=3000
|
||||
MEMORY_BUDGET_MID=2000
|
||||
MEMORY_BUDGET_LONG=2000
|
||||
|
||||
158
cortex/agent_manager.py
Normal file
158
cortex/agent_manager.py
Normal file
@@ -0,0 +1,158 @@
|
||||
"""
|
||||
Agent lifecycle manager — registry for background spawn_agent and aider_run tasks.
|
||||
|
||||
Tracks running and recently completed agents in-process. On completion, fires
|
||||
notification.notify() if notify=True (same channel used by reminders and cron jobs).
|
||||
|
||||
Records are kept for 24 hours after completion, then pruned on next registration.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_PRUNE_AFTER = timedelta(hours=24)
|
||||
_RESULT_PREVIEW_CHARS = 500
|
||||
_TASK_PREVIEW_CHARS = 200
|
||||
|
||||
|
||||
@dataclass
|
||||
class AgentRecord:
|
||||
agent_id: str
|
||||
level: int # 1 = persona, 2 = specialized sub-agent, 3 = support agent
|
||||
role: str # e.g. "coder", "research", "chat"
|
||||
task: str # first _TASK_PREVIEW_CHARS of the task
|
||||
status: str # running / done / failed / cancelled / timeout
|
||||
started: datetime
|
||||
user: str
|
||||
parent_id: str | None = None # agent_id of the spawner (lineage tracking)
|
||||
finished: datetime | None = None
|
||||
result: str | None = None # first _RESULT_PREVIEW_CHARS on completion
|
||||
notify: bool = False # push notification on completion
|
||||
_task_ref: "asyncio.Task | None" = field(default=None, repr=False)
|
||||
|
||||
|
||||
# Module-level registry — in-process only, not persisted across restarts.
|
||||
_agents: dict[str, AgentRecord] = {}
|
||||
_lock = asyncio.Lock()
|
||||
|
||||
|
||||
async def register(
|
||||
user: str,
|
||||
role: str,
|
||||
task: str,
|
||||
level: int = 2,
|
||||
parent_id: str | None = None,
|
||||
notify: bool = False,
|
||||
) -> AgentRecord:
|
||||
"""Create and register a new running agent. Returns the record (agent_id is set)."""
|
||||
agent_id = str(uuid.uuid4())
|
||||
rec = AgentRecord(
|
||||
agent_id=agent_id,
|
||||
level=level,
|
||||
role=role,
|
||||
task=task[:_TASK_PREVIEW_CHARS],
|
||||
status="running",
|
||||
started=datetime.now(),
|
||||
user=user,
|
||||
parent_id=parent_id,
|
||||
notify=notify,
|
||||
)
|
||||
async with _lock:
|
||||
_prune_locked()
|
||||
_agents[agent_id] = rec
|
||||
logger.info(
|
||||
"agent_manager: registered %s role=%s level=%d user=%s task=%.60s",
|
||||
agent_id[:8], role, level, user, task,
|
||||
)
|
||||
return rec
|
||||
|
||||
|
||||
def set_task_ref(agent_id: str, task_ref: "asyncio.Task") -> None:
|
||||
"""Store the asyncio.Task reference so it can be cancelled later.
|
||||
|
||||
Call immediately after asyncio.create_task() — before the event loop yields.
|
||||
"""
|
||||
rec = _agents.get(agent_id)
|
||||
if rec:
|
||||
rec._task_ref = task_ref
|
||||
|
||||
|
||||
async def finish(agent_id: str, result: str, status: str = "done") -> None:
|
||||
"""Mark an agent complete, store the result, and notify the user if requested."""
|
||||
async with _lock:
|
||||
rec = _agents.get(agent_id)
|
||||
if not rec:
|
||||
return
|
||||
rec.status = status
|
||||
rec.finished = datetime.now()
|
||||
rec.result = (result or "")[:_RESULT_PREVIEW_CHARS]
|
||||
|
||||
logger.info("agent_manager: finished %s status=%s", agent_id[:8], status)
|
||||
|
||||
if rec.notify and status != "cancelled":
|
||||
try:
|
||||
from notification import notify as _notify
|
||||
elapsed = int((rec.finished - rec.started).total_seconds())
|
||||
emoji = "✅" if status == "done" else "⚠️"
|
||||
preview = (rec.result or "(no output)")[:200]
|
||||
msg = f"{emoji} Agent done [{rec.role}, {elapsed}s]: {preview}"
|
||||
await _notify(rec.user, msg)
|
||||
except Exception as e:
|
||||
logger.warning("agent_manager: notification failed for %s: %s", agent_id[:8], e)
|
||||
|
||||
|
||||
async def cancel_agent(agent_id: str, user: str) -> str:
|
||||
"""Cancel a running background agent. Returns a human-readable status message."""
|
||||
async with _lock:
|
||||
rec = _agents.get(agent_id)
|
||||
if not rec:
|
||||
return f"No agent found: {agent_id}"
|
||||
if rec.user != user:
|
||||
return "Access denied."
|
||||
if rec.status != "running":
|
||||
return f"Agent {agent_id[:8]}… is already {rec.status}."
|
||||
task_ref = rec._task_ref
|
||||
rec.status = "cancelled"
|
||||
rec.finished = datetime.now()
|
||||
|
||||
if task_ref and not task_ref.done():
|
||||
task_ref.cancel()
|
||||
|
||||
logger.info("agent_manager: cancelled %s by user=%s", agent_id[:8], user)
|
||||
return f"Agent {agent_id[:8]}… cancelled."
|
||||
|
||||
|
||||
def get(agent_id: str) -> AgentRecord | None:
|
||||
"""Look up an agent record by ID."""
|
||||
return _agents.get(agent_id)
|
||||
|
||||
|
||||
def list_agents(user: str, status: str | None = None, limit: int = 10) -> list[AgentRecord]:
|
||||
"""Return recent agents for a user, newest first.
|
||||
|
||||
Does not acquire the lock — safe for read-only listing (Python dict iteration is
|
||||
thread-safe for reads; we don't care about racing with a concurrent registration).
|
||||
"""
|
||||
records = [r for r in _agents.values() if r.user == user]
|
||||
if status:
|
||||
records = [r for r in records if r.status == status]
|
||||
records.sort(key=lambda r: r.started, reverse=True)
|
||||
return records[:limit]
|
||||
|
||||
|
||||
def _prune_locked() -> None:
|
||||
"""Remove completed agents older than _PRUNE_AFTER. Must be called inside _lock."""
|
||||
cutoff = datetime.now() - _PRUNE_AFTER
|
||||
stale = [
|
||||
aid for aid, r in _agents.items()
|
||||
if r.status != "running" and r.finished and r.finished < cutoff
|
||||
]
|
||||
for aid in stale:
|
||||
del _agents[aid]
|
||||
if stale:
|
||||
logger.debug("agent_manager: pruned %d stale records", len(stale))
|
||||
@@ -17,10 +17,11 @@ from starlette.responses import RedirectResponse, JSONResponse
|
||||
from auth_utils import COOKIE_NAME, decode_token
|
||||
|
||||
# Paths that don't require a session cookie
|
||||
_PUBLIC = {"/login", "/logout", "/health"}
|
||||
_PUBLIC = {"/login", "/logout", "/health", "/manifest.json", "/sw.js", "/favicon.ico",
|
||||
"/api/push/vapid-key"}
|
||||
|
||||
# Path prefixes that are always public (setup flow + webhooks)
|
||||
_PUBLIC_PREFIXES = ("/setup/", "/channels/", "/webhook/")
|
||||
# Path prefixes that are always public (setup flow + webhooks + Google OAuth)
|
||||
_PUBLIC_PREFIXES = ("/setup/", "/channels/", "/webhook/", "/auth/google")
|
||||
|
||||
|
||||
class SessionAuthMiddleware(BaseHTTPMiddleware):
|
||||
|
||||
@@ -29,33 +29,102 @@ ALGORITHM = "HS256"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Password helpers
|
||||
# auth.json helpers — read/write without clobbering unrelated fields
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _auth_path(username: str) -> Path:
|
||||
return settings.home_root() / username / "auth.json"
|
||||
|
||||
|
||||
def _read_auth(username: str) -> dict:
|
||||
path = _auth_path(username)
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
return json.loads(path.read_text())
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def _write_auth(username: str, data: dict) -> None:
|
||||
path = _auth_path(username)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(data, indent=2) + "\n")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Password helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def set_password(username: str, password: str) -> None:
|
||||
"""Hash and store a password for a user. Creates auth.json if needed."""
|
||||
hashed = bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
|
||||
_auth_path(username).write_text(json.dumps({"password_hash": hashed}) + "\n")
|
||||
"""Hash and store a password. Preserves any existing fields in auth.json."""
|
||||
data = _read_auth(username)
|
||||
data["password_hash"] = bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
|
||||
_write_auth(username, data)
|
||||
logger.info("password set for user: %s", username)
|
||||
|
||||
|
||||
def check_credentials(username: str, password: str) -> bool:
|
||||
"""Return True if username+password are valid, False otherwise."""
|
||||
path = _auth_path(username)
|
||||
if not path.exists():
|
||||
return False
|
||||
try:
|
||||
data = json.loads(path.read_text())
|
||||
stored = data.get("password_hash", "").encode()
|
||||
stored = _read_auth(username).get("password_hash", "").encode()
|
||||
if not stored:
|
||||
return False
|
||||
return bcrypt.checkpw(password.encode(), stored)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Google OAuth helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def find_user_by_google(sub: str, email: str) -> str | None:
|
||||
"""
|
||||
Scan all users for one whose auth.json matches the given Google sub or email.
|
||||
Sub match takes priority (stable); email match is a fallback for first sign-in.
|
||||
Returns the username, or None if no match.
|
||||
"""
|
||||
root = settings.home_root()
|
||||
if not root.exists():
|
||||
return None
|
||||
for user_dir in sorted(root.iterdir()):
|
||||
if not user_dir.is_dir():
|
||||
continue
|
||||
data = _read_auth(user_dir.name)
|
||||
if not data:
|
||||
continue
|
||||
if sub and data.get("google_sub") == sub:
|
||||
return user_dir.name
|
||||
if email and data.get("google_email", "").lower() == email.lower():
|
||||
return user_dir.name
|
||||
return None
|
||||
|
||||
|
||||
def link_google(username: str, sub: str, email: str) -> None:
|
||||
"""Store / update Google sub and email in a user's auth.json."""
|
||||
data = _read_auth(username)
|
||||
data["google_sub"] = sub
|
||||
data["google_email"] = email
|
||||
_write_auth(username, data)
|
||||
logger.info("Google account linked for user: %s (%s)", username, email)
|
||||
|
||||
|
||||
def get_user_gemini_key(username: str) -> str | None:
|
||||
"""Return the user's personal Gemini API key, or None to use the server key."""
|
||||
return _read_auth(username).get("gemini_api_key") or None
|
||||
|
||||
|
||||
def get_user_role(username: str) -> str:
|
||||
"""Return the user's role: 'admin' or 'user' (default).
|
||||
|
||||
Role is stored as auth.json["role"]. Any unrecognised value falls back to 'user'.
|
||||
Set via: manage_passwords.py role <username> admin|user
|
||||
"""
|
||||
role = _read_auth(username).get("role", "user")
|
||||
return role if role in ("admin", "user") else "user"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# JWT helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -136,3 +205,56 @@ def consume_invite(username: str) -> None:
|
||||
path.write_text(json.dumps(data) + "\n")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Per-user channel config
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _channels_path(username: str) -> Path:
|
||||
return settings.home_root() / username / "channels.json"
|
||||
|
||||
|
||||
def get_user_channels(username: str) -> dict:
|
||||
"""Return the parsed channels.json for a user, or {} if not found."""
|
||||
path = _channels_path(username)
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
return json.loads(path.read_text())
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def get_tool_policy(username: str) -> dict:
|
||||
"""Return the parsed tool_policy.json for a user.
|
||||
|
||||
Confirmation-gate keys (existing):
|
||||
allow — tools in CONFIRM_REQUIRED that this user has pre-approved (skip gate)
|
||||
deny — tools always blocked for this user regardless of global CONFIRM_REQUIRED
|
||||
|
||||
Risk-policy keys (new):
|
||||
max_risk — auto-include all tools at/below this level ("low"|"medium"|"high")
|
||||
whitelist — force-include specific tools above max_risk
|
||||
blacklist — force-exclude specific tools regardless of max_risk
|
||||
"""
|
||||
path = settings.home_root() / username / "tool_policy.json"
|
||||
try:
|
||||
return json.loads(path.read_text())
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def get_risk_policy(username: str) -> tuple[str | None, list[str], list[str]]:
|
||||
"""Return (max_risk, whitelist, blacklist) from the user's tool policy."""
|
||||
policy = get_tool_policy(username)
|
||||
return (
|
||||
policy.get("max_risk") or None,
|
||||
policy.get("whitelist") or [],
|
||||
policy.get("blacklist") or [],
|
||||
)
|
||||
|
||||
|
||||
def save_tool_policy(username: str, data: dict) -> None:
|
||||
path = settings.home_root() / username / "tool_policy.json"
|
||||
path.write_text(json.dumps(data, indent=2) + "\n")
|
||||
|
||||
@@ -5,6 +5,12 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
class Settings(BaseSettings):
|
||||
anthropic_api_key: str | None = None # not used — claude CLI handles auth
|
||||
|
||||
# Google OAuth — "Sign in with Google" for all users
|
||||
# Create credentials at console.cloud.google.com → APIs & Services → Credentials
|
||||
# Add https://<your-domain>/auth/google/callback as an authorised redirect URI
|
||||
google_client_id: str | None = None
|
||||
google_client_secret: str | None = None
|
||||
|
||||
# Orchestrator (Gemini API — separate from Gemini CLI)
|
||||
# Get a key at: https://aistudio.google.com/apikey (free tier is sufficient)
|
||||
gemini_api_key: str | None = None
|
||||
@@ -34,26 +40,17 @@ class Settings(BaseSettings):
|
||||
max_history_messages: int = 40 # rolling window — 20 turns (user + assistant)
|
||||
primary_backend: str = "claude" # "claude" or "gemini" — other is always fallback
|
||||
|
||||
# Local model backend — OpenAI-compatible API (Open WebUI / Ollama)
|
||||
# Set LOCAL_API_URL in .env to enable; leave blank to disable
|
||||
local_api_url: str = "" # e.g. http://192.168.32.19:3000
|
||||
local_api_key: str = "" # sk-... from Open WebUI → Settings → Account → API Keys
|
||||
local_model: str = "" # workspace or model name, e.g. test-agent-simple
|
||||
|
||||
# Per-backend timeouts in seconds
|
||||
timeout_claude: int = 60
|
||||
timeout_gemini: int = 120 # frequently slow under load
|
||||
timeout_local: int = 300 # local models may need to load first
|
||||
|
||||
# Google Chat
|
||||
# JWT audience (aud) claim to verify on inbound webhook requests.
|
||||
# Google Chat sets aud = the Google Cloud project number (e.g. "741112865538").
|
||||
# Set to "" to disable verification (dev/testing only).
|
||||
google_chat_audience: str = ""
|
||||
# Google Chat must receive a response within 30s or shows an error to the user
|
||||
google_chat_timeout: int = 25
|
||||
# Backend forced for Google Chat — Claude is more reliable within the 25s deadline
|
||||
google_chat_backend: str = "claude"
|
||||
|
||||
# Nextcloud Talk bot
|
||||
nextcloud_url: str = "https://cloud.dgrzone.com"
|
||||
nextcloud_talk_bot_secret: str = "" # set in .env
|
||||
nextcloud_talk_timeout: int = 55
|
||||
|
||||
# Auto-distillation schedule — override in .env
|
||||
# AUTO_DISTILL=false disables entirely
|
||||
scheduler_timezone: str = "America/New_York" # IANA tz — override in .env if needed
|
||||
@@ -62,6 +59,26 @@ class Settings(BaseSettings):
|
||||
auto_distill_mid: bool = True # weekly Sunday at 03:30 — LLM summarizes short → mid
|
||||
auto_distill_long: bool = False # monthly 1st at 04:00 — off by default (manual review recommended)
|
||||
|
||||
# Which backend to use for distillation LLM calls.
|
||||
# "" = use primary_backend (default); "local" = use local model (saves API credits).
|
||||
# "long" stays on default (claude/gemini) for best quality.
|
||||
distill_backend_mid: str = ""
|
||||
distill_backend_long: str = ""
|
||||
|
||||
# Model registry: default backend type per role when user registry has no entry.
|
||||
# Values: "claude_cli" | "gemini_cli" | "gemini_api" (builtin IDs)
|
||||
# Override in .env: ROLE_CHAT=claude_cli ROLE_DISTILL=gemini_api etc.
|
||||
role_chat: str = "claude_cli"
|
||||
role_orchestrator: str = "gemini_api"
|
||||
role_distill: str = "claude_cli"
|
||||
role_coder: str = "claude_cli"
|
||||
role_research: str = "gemini_api"
|
||||
|
||||
# Comma-separated list of standard roles shown in the model settings UI.
|
||||
# Add custom roles here to extend the UI without code changes.
|
||||
# Example: DEFINED_ROLES=chat,orchestrator,distill,coder,research,medical
|
||||
defined_roles: str = "chat,orchestrator,distill,coder,research"
|
||||
|
||||
# Memory tier token budgets — soft caps used during distillation
|
||||
# Override in .env: MEMORY_BUDGET_LONG=4000 etc.
|
||||
memory_budget_long: int = 2000
|
||||
@@ -72,6 +89,12 @@ class Settings(BaseSettings):
|
||||
jwt_secret: str = "change-me-in-dotenv" # override in .env: JWT_SECRET=<random>
|
||||
jwt_expire_days: int = 30
|
||||
|
||||
# Web Push (VAPID) — for browser push notifications
|
||||
# Generate once with py_vapid; see push_utils.py for key format details
|
||||
vapid_public_key: str = "" # base64url-encoded uncompressed EC point (for browser)
|
||||
vapid_private_key_b64: str = "" # base64-encoded PEM private key (single-line .env storage)
|
||||
vapid_contact: str = "mailto:admin@example.com"
|
||||
|
||||
# SMTP — for sending invite emails
|
||||
smtp_server: str = ""
|
||||
smtp_port: int = 465
|
||||
@@ -87,6 +110,14 @@ class Settings(BaseSettings):
|
||||
|
||||
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
|
||||
def get_defined_roles(self) -> list[str]:
|
||||
"""Return the ordered list of standard roles from the defined_roles setting."""
|
||||
return [r.strip() for r in self.defined_roles.split(",") if r.strip()]
|
||||
|
||||
def get_role_default(self, role: str) -> str:
|
||||
"""Return the .env default backend type for a role (e.g. 'claude_cli')."""
|
||||
return getattr(self, f"role_{role.replace('-', '_')}", "claude_cli")
|
||||
|
||||
def home_root(self) -> Path:
|
||||
"""Resolve home_dir relative to this file's location if not absolute."""
|
||||
if self.home_dir.is_absolute():
|
||||
|
||||
@@ -1,4 +1,10 @@
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from persona import persona_path
|
||||
from tools.reminders import load_due_reminders
|
||||
|
||||
_STATIC_DIR = Path(__file__).parent / "static"
|
||||
|
||||
|
||||
# Core identity files — always loaded regardless of tier
|
||||
@@ -13,6 +19,10 @@ def load_context(
|
||||
include_long: bool = True,
|
||||
include_mid: bool = True,
|
||||
include_short: bool = True,
|
||||
role_append: str = "",
|
||||
inject_datetime: bool = True,
|
||||
inject_mode: bool = True,
|
||||
mode: str = "chat",
|
||||
) -> str:
|
||||
"""
|
||||
Build the system-prompt context block for a given tier and memory toggles.
|
||||
@@ -24,10 +34,26 @@ def load_context(
|
||||
Tier 2 — + USER full + PROTOCOLS + memory (~5,000 tokens)
|
||||
Tier 3 — + last 2 raw session logs (~15,000 tokens)
|
||||
Tier 4 — + last 7 raw session logs (~50,000 tokens)
|
||||
|
||||
role_append — optional text injected last (closest to the turn),
|
||||
sourced from the active role's system_append config.
|
||||
"""
|
||||
inara_dir = persona_path()
|
||||
parts = []
|
||||
|
||||
# ── 0. System block — date/time and session mode (injected first so it's prominent) ──
|
||||
system_lines = []
|
||||
if inject_datetime:
|
||||
now = datetime.now().astimezone()
|
||||
system_lines.append(f"Current date and time: {now.strftime('%A, %Y-%m-%d at %I:%M %p %Z')}")
|
||||
if mode == "otr" and inject_mode:
|
||||
system_lines.append(
|
||||
"Current mode: Off The Record — "
|
||||
"this conversation is private and will not be logged or included in memory distillation"
|
||||
)
|
||||
if system_lines:
|
||||
parts.append("--- System ---\n" + "\n".join(system_lines))
|
||||
|
||||
# ── 1. Core identity (always) ──────────────────────────────────
|
||||
for filename in _CORE:
|
||||
path = inara_dir / filename
|
||||
@@ -52,15 +78,24 @@ def load_context(
|
||||
if proto_path.exists():
|
||||
parts.append(f"--- PROTOCOLS.md ---\n{proto_path.read_text()}")
|
||||
|
||||
ops_path = inara_dir / "OPERATIONS.md"
|
||||
if ops_path.exists():
|
||||
parts.append(f"--- OPERATIONS.md ---\n{ops_path.read_text()}")
|
||||
|
||||
# Global tool reference (same for all personas)
|
||||
tools_path = _STATIC_DIR / "TOOLS.md"
|
||||
if tools_path.exists():
|
||||
parts.append(f"--- TOOLS.md ---\n{tools_path.read_text()}")
|
||||
|
||||
# Persona-specific help additions (optional)
|
||||
help_path = inara_dir / "HELP.md"
|
||||
if help_path.exists():
|
||||
if help_path.exists() and help_path.stat().st_size > 10:
|
||||
parts.append(f"--- HELP.md ---\n{help_path.read_text()}")
|
||||
|
||||
# ── 4. Pending reminders (tier 2+) ────────────────────────────
|
||||
# Written by cron jobs; cleared by Inara after acting on them.
|
||||
reminders_path = inara_dir / "REMINDERS.md"
|
||||
if reminders_path.exists() and reminders_path.stat().st_size > 10:
|
||||
content = reminders_path.read_text().strip()
|
||||
# Only due and undated reminders are surfaced — future-dated ones
|
||||
# are stored in REMINDERS.md but suppressed until their date arrives.
|
||||
content = load_due_reminders()
|
||||
if content:
|
||||
parts.append(f"--- REMINDERS.md ---\n{content}")
|
||||
|
||||
@@ -97,4 +132,8 @@ def load_context(
|
||||
for sf in session_files:
|
||||
parts.append(f"--- Session: {sf.name} ---\n{sf.read_text()}")
|
||||
|
||||
# ── 7. Role-specific instructions (always last — closest to the turn) ──
|
||||
if role_append and role_append.strip():
|
||||
parts.append(f"--- Role Context ---\n{role_append.strip()}")
|
||||
|
||||
return "\n\n".join(parts)
|
||||
|
||||
@@ -10,16 +10,25 @@ Job schema:
|
||||
"id": "c_abc123",
|
||||
"label": "Human-readable name",
|
||||
"schedule": "daily:09:00", # see parse_schedule() for all formats
|
||||
"type": "remind" | "note",
|
||||
"payload": "Text to write when the job fires",
|
||||
"type": "remind" | "note" | "message" | "brief" | "task",
|
||||
"payload": "Text or prompt when the job fires",
|
||||
"channel": null | "nextcloud" | "google_chat", # for message/brief/task types
|
||||
"enabled": true,
|
||||
"created_at": "ISO 8601",
|
||||
"last_run": null | "ISO 8601"
|
||||
}
|
||||
|
||||
Job types:
|
||||
remind → appends to inara/REMINDERS.md (auto-loaded into Inara's context)
|
||||
note → appends to inara/SCRATCH.md (read on demand via scratch_read)
|
||||
remind → appends to REMINDERS.md (auto-loaded into context at tier 2+)
|
||||
note → appends to SCRATCH.md (read on demand via scratch_read)
|
||||
message → sends payload as-is to notification channel
|
||||
brief → calls LLM (no tools) with payload as prompt, sends response
|
||||
(good for morning briefings, summaries, proactive check-ins)
|
||||
task → runs full orchestrator tool loop with payload as the user request,
|
||||
sends Claude's response to notification channel
|
||||
(good for agentic scheduled work: research, file updates, checks)
|
||||
Tools that require confirmation are skipped — pre-approve them
|
||||
in Settings → Tools to allow them in scheduled tasks.
|
||||
"""
|
||||
|
||||
import logging
|
||||
@@ -81,6 +90,11 @@ def parse_schedule(schedule: str) -> dict:
|
||||
"daily:HH:MM" → every day at HH:MM
|
||||
"weekly:DOW" → every DOW at 09:00
|
||||
"weekly:DOW:HH:MM" → every DOW at HH:MM
|
||||
"monthly" → 1st of every month at 09:00
|
||||
"monthly:DD" → day DD of every month at 09:00
|
||||
"monthly:DD:HH:MM" → day DD of every month at HH:MM
|
||||
"yearly:MM:DD" → every year on MM/DD at 09:00 (birthdays, anniversaries)
|
||||
"yearly:MM:DD:HH:MM" → every year on MM/DD at HH:MM
|
||||
"""
|
||||
s = schedule.strip().lower()
|
||||
|
||||
@@ -108,9 +122,37 @@ def parse_schedule(schedule: str) -> dict:
|
||||
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
|
||||
return {"day_of_week": dow, "hour": h, "minute": m}
|
||||
|
||||
if s.startswith("monthly"):
|
||||
rest = s[7:].lstrip(":")
|
||||
if not rest:
|
||||
return {"day": 1, "hour": _DEFAULT_HOUR, "minute": _DEFAULT_MINUTE}
|
||||
parts = rest.split(":")
|
||||
day = _parse_day(parts[0], schedule)
|
||||
if len(parts) == 3:
|
||||
h, m = _parse_hhmm(f"{parts[1]}:{parts[2]}", schedule)
|
||||
else:
|
||||
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
|
||||
return {"day": day, "hour": h, "minute": m}
|
||||
|
||||
if s.startswith("yearly:"):
|
||||
rest = s[7:].split(":")
|
||||
if len(rest) < 2:
|
||||
raise ValueError(
|
||||
f"yearly requires at least MM:DD in {schedule!r}. "
|
||||
f"Example: yearly:03:15 or yearly:03:15:09:00"
|
||||
)
|
||||
month = _parse_month(rest[0], schedule)
|
||||
day = _parse_day(rest[1], schedule)
|
||||
if len(rest) == 4:
|
||||
h, m = _parse_hhmm(f"{rest[2]}:{rest[3]}", schedule)
|
||||
else:
|
||||
h, m = _DEFAULT_HOUR, _DEFAULT_MINUTE
|
||||
return {"month": month, "day": day, "hour": h, "minute": m}
|
||||
|
||||
raise ValueError(
|
||||
f"Unrecognised schedule {schedule!r}. "
|
||||
f"Valid formats: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM"
|
||||
f"Valid formats: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM | "
|
||||
f"monthly | monthly:DD | monthly:DD:HH:MM | yearly:MM:DD | yearly:MM:DD:HH:MM"
|
||||
)
|
||||
|
||||
|
||||
@@ -121,6 +163,26 @@ def _parse_hhmm(s: str, original: str) -> tuple[int, int]:
|
||||
return int(parts[0]), int(parts[1])
|
||||
|
||||
|
||||
def _parse_day(s: str, original: str) -> int:
|
||||
try:
|
||||
d = int(s)
|
||||
except ValueError:
|
||||
raise ValueError(f"Expected day number (1–31) in {original!r}, got {s!r}")
|
||||
if not 1 <= d <= 31:
|
||||
raise ValueError(f"Day must be 1–31 in {original!r}, got {d}")
|
||||
return d
|
||||
|
||||
|
||||
def _parse_month(s: str, original: str) -> int:
|
||||
try:
|
||||
m = int(s)
|
||||
except ValueError:
|
||||
raise ValueError(f"Expected month number (1–12) in {original!r}, got {s!r}")
|
||||
if not 1 <= m <= 12:
|
||||
raise ValueError(f"Month must be 1–12 in {original!r}, got {m}")
|
||||
return m
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Execution
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -150,6 +212,89 @@ async def run_job(job: dict) -> None:
|
||||
p.write_text(existing.rstrip() + "\n" + section)
|
||||
logger.info("cron [note] fired: %s", label)
|
||||
|
||||
elif job_type == "message":
|
||||
# Send payload text directly to the user's notification channel
|
||||
from notification import notify
|
||||
username = job.get("user") or "scott"
|
||||
channel = job.get("channel") or None
|
||||
await notify(username, payload, channel=channel)
|
||||
logger.info("cron [message] sent: %s", label)
|
||||
|
||||
elif job_type == "brief":
|
||||
# Run LLM with payload as the prompt, send response to notification channel.
|
||||
# Great for morning briefings, reminders, proactive check-ins.
|
||||
from context_loader import load_context
|
||||
from llm_client import complete
|
||||
from notification import notify
|
||||
from persona import set_context
|
||||
from config import settings as _s
|
||||
|
||||
username = job.get("user") or _s.user_name.lower()
|
||||
persona_nm = job.get("persona") or _s.agent_name.lower()
|
||||
channel = job.get("channel") or None
|
||||
set_context(username, persona_nm)
|
||||
|
||||
system_prompt = load_context(2) # tier 2: identity + memory + user profile
|
||||
try:
|
||||
response_text, backend = await complete(
|
||||
system_prompt=system_prompt,
|
||||
messages=[{"role": "user", "content": payload}],
|
||||
role="chat",
|
||||
)
|
||||
await notify(username, response_text, channel=channel)
|
||||
logger.info("cron [brief] sent via %s: %s", backend, label)
|
||||
except Exception as e:
|
||||
logger.error("cron [brief] LLM error for %s: %s", label, e)
|
||||
|
||||
elif job_type == "task":
|
||||
# Run the full orchestrator tool loop, send Claude's response to the
|
||||
# notification channel. Tools that require confirmation are skipped in
|
||||
# cron context — the user is notified to pre-approve them.
|
||||
from orchestrator_engine import run as _orch_run
|
||||
from context_loader import load_context
|
||||
from notification import notify
|
||||
from persona import set_context
|
||||
from auth_utils import get_user_gemini_key, get_tool_policy, get_risk_policy
|
||||
from config import settings as _s
|
||||
|
||||
username = job.get("user") or _s.user_name.lower()
|
||||
persona_nm = job.get("persona") or _s.agent_name.lower()
|
||||
channel = job.get("channel") or None
|
||||
set_context(username, persona_nm)
|
||||
|
||||
system_prompt = load_context(2)
|
||||
policy = get_tool_policy(username)
|
||||
max_risk, whitelist, blacklist = get_risk_policy(username)
|
||||
gemini_key = get_user_gemini_key(username)
|
||||
|
||||
try:
|
||||
result = await _orch_run(
|
||||
task=payload,
|
||||
system_prompt=system_prompt,
|
||||
gemini_api_key=gemini_key,
|
||||
respond_with_claude=True,
|
||||
confirm_allow=set(policy.get("allow") or []),
|
||||
confirm_deny=set(policy.get("deny") or []),
|
||||
max_risk=max_risk,
|
||||
risk_whitelist=whitelist,
|
||||
risk_blacklist=blacklist,
|
||||
)
|
||||
if result.checkpoint:
|
||||
tool_name = (result.checkpoint.pending_calls[0].name
|
||||
if result.checkpoint.pending_calls else "unknown tool")
|
||||
msg = (
|
||||
f"Scheduled task '{label}' paused — "
|
||||
f"'{tool_name}' requires confirmation. "
|
||||
"Pre-approve it in Settings → Tools to allow it in scheduled tasks."
|
||||
)
|
||||
await notify(username, msg, channel=channel)
|
||||
logger.warning("cron [task] %s: confirmation required for %s", label, tool_name)
|
||||
else:
|
||||
await notify(username, result.response, channel=channel)
|
||||
logger.info("cron [task] completed via %s: %s", result.backend, label)
|
||||
except Exception as e:
|
||||
logger.error("cron [task] error for %s: %s", label, e)
|
||||
|
||||
else:
|
||||
logger.warning("cron: unknown type %r (job %s)", job_type, job.get("id"))
|
||||
return
|
||||
|
||||
@@ -4,6 +4,7 @@ import os
|
||||
import signal
|
||||
import subprocess
|
||||
from config import settings
|
||||
import event_bus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -30,24 +31,83 @@ async def cleanup() -> None:
|
||||
_active_pgroups.clear()
|
||||
|
||||
|
||||
# Map from registry model type → dispatch function key
|
||||
_TYPE_TO_BACKEND = {
|
||||
"claude_cli": "claude",
|
||||
"gemini_cli": "gemini",
|
||||
"gemini_api": "gemini", # gemini_api falls back to CLI in this context
|
||||
"local_openai": "local",
|
||||
"anthropic_api": "anthropic_api",
|
||||
}
|
||||
|
||||
# Explicit UI toggle values (kept for backward compat)
|
||||
_EXPLICIT_BACKENDS = ("claude", "gemini", "local")
|
||||
_FALLBACK = {"claude": "gemini", "gemini": "claude", "local": "claude", "anthropic_api": "claude"}
|
||||
|
||||
|
||||
async def complete(
|
||||
system_prompt: str,
|
||||
messages: list[dict],
|
||||
model: str | None = None,
|
||||
role: str = "chat",
|
||||
slot: str | None = None,
|
||||
max_tokens: int = 2048,
|
||||
attachment: dict | None = None,
|
||||
) -> tuple[str, str]:
|
||||
"""Returns (response_text, actual_backend_used)."""
|
||||
if model in ("claude", "gemini"):
|
||||
"""
|
||||
Returns (response_text, actual_backend_used).
|
||||
|
||||
slot: Phase 3 — specific role slot ("primary" | "backup_1" | "backup_2").
|
||||
Resolves that exact slot, no fallback chain. Takes priority over model.
|
||||
model: legacy backend override ("claude" | "gemini" | "local") from old toggle.
|
||||
None = resolve via model registry for the given role.
|
||||
role: registry role used for slot/auto routing (default: "chat").
|
||||
"""
|
||||
import model_registry as _reg
|
||||
from persona import _user
|
||||
|
||||
username = _user.get()
|
||||
resolved_cfg: dict | None = None
|
||||
|
||||
if slot is not None:
|
||||
# Phase 3: explicit slot selection — no fallback within the role
|
||||
resolved_cfg = _reg.get_model_for_slot(username, role, slot)
|
||||
if resolved_cfg:
|
||||
primary = _TYPE_TO_BACKEND.get(resolved_cfg["type"], "claude")
|
||||
else:
|
||||
# Slot not configured — fall through to auto routing
|
||||
slot = None
|
||||
|
||||
if slot is None:
|
||||
if model in _EXPLICIT_BACKENDS:
|
||||
# Legacy: explicit backend override from old UI toggle
|
||||
if model == "local":
|
||||
resolved_cfg = _reg.get_best_local_model(username, role)
|
||||
if not resolved_cfg:
|
||||
raise RuntimeError("No local model configured — add one at /settings/models")
|
||||
primary = model
|
||||
else:
|
||||
# Auto: role-based routing via model registry
|
||||
resolved = _reg.get_model_for_role(username, role)
|
||||
if resolved:
|
||||
resolved_cfg = resolved
|
||||
primary = _TYPE_TO_BACKEND.get(resolved["type"], "claude")
|
||||
else:
|
||||
primary = settings.primary_backend
|
||||
|
||||
fallback = "gemini" if primary == "claude" else "claude"
|
||||
fallback = _FALLBACK.get(primary, "claude")
|
||||
|
||||
try:
|
||||
response = await _dispatch(primary, system_prompt, messages, model)
|
||||
response = await _dispatch(primary, system_prompt, messages, resolved_cfg, attachment=attachment)
|
||||
return response, primary
|
||||
except Exception as e:
|
||||
err_str = str(e)
|
||||
if primary == "claude" and any(k in err_str for k in ("401", "authenticate", "expired", "OAuth")):
|
||||
await event_bus.publish({"type": "claude_auth_expired"})
|
||||
# Surface errors when a model is explicitly configured or a specific slot was pinned.
|
||||
if resolved_cfg is not None:
|
||||
logger.error("%s failed (no fallback — model explicitly configured): %s", primary, e)
|
||||
raise
|
||||
logger.warning("%s failed (%s) — falling back to %s", primary, e, fallback)
|
||||
response = await _dispatch(fallback, system_prompt, messages, None)
|
||||
return response, fallback
|
||||
@@ -57,11 +117,16 @@ async def _dispatch(
|
||||
backend: str,
|
||||
system_prompt: str,
|
||||
messages: list[dict],
|
||||
model: str | None,
|
||||
model_cfg: dict | None,
|
||||
attachment: dict | None = None,
|
||||
) -> str:
|
||||
if backend == "gemini":
|
||||
return await _gemini(system_prompt, messages)
|
||||
return await _claude(system_prompt, messages, model)
|
||||
if backend == "local":
|
||||
return await _local(system_prompt, messages, model_cfg, attachment=attachment)
|
||||
if backend == "anthropic_api":
|
||||
return await _anthropic_api(system_prompt, messages, model_cfg)
|
||||
return await _claude(system_prompt, messages, model_cfg)
|
||||
|
||||
|
||||
def _fresh_claude_token() -> str | None:
|
||||
@@ -81,14 +146,16 @@ def _fresh_claude_token() -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
async def _claude(system_prompt: str, messages: list[dict], model: str | None) -> str:
|
||||
async def _claude(system_prompt: str, messages: list[dict], model_cfg: dict | None) -> str:
|
||||
model_name = (model_cfg or {}).get("model_name") if model_cfg else None
|
||||
cmd = [
|
||||
"claude", "--print",
|
||||
"--no-session-persistence",
|
||||
"--output-format", "text",
|
||||
]
|
||||
if model and model not in ("claude", "gemini"):
|
||||
cmd.extend(["--model", model])
|
||||
# Only pass --model if it's a real model name (not a backend type string)
|
||||
if model_name and model_name not in ("claude", "gemini", "local", ""):
|
||||
cmd.extend(["--model", model_name])
|
||||
if system_prompt:
|
||||
cmd.extend(["--system-prompt", system_prompt])
|
||||
cmd.append(_build_conversation(messages))
|
||||
@@ -104,6 +171,137 @@ async def _claude(system_prompt: str, messages: list[dict], model: str | None) -
|
||||
return await _run(cmd, timeout=settings.timeout_claude, env=env)
|
||||
|
||||
|
||||
async def _local(
|
||||
system_prompt: str,
|
||||
messages: list[dict],
|
||||
model_cfg: dict | None = None,
|
||||
attachment: dict | None = None,
|
||||
) -> str:
|
||||
"""OpenAI-compatible backend — Open WebUI / Ollama.
|
||||
|
||||
model_cfg is pre-resolved by complete() via model_registry.
|
||||
Falls back to registry lookup if not provided.
|
||||
attachment: optional image dict {filename, mime_type, data} for vision calls.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
cfg = model_cfg
|
||||
if not cfg:
|
||||
# Fallback: resolve directly from registry
|
||||
import model_registry as _reg
|
||||
from persona import _user
|
||||
cfg = _reg.get_best_local_model(_user.get())
|
||||
if not cfg:
|
||||
raise RuntimeError("No local model configured — add one at /settings/models")
|
||||
|
||||
api_url = cfg["api_url"]
|
||||
api_key = cfg["api_key"]
|
||||
model = cfg["model_name"]
|
||||
|
||||
if not api_url:
|
||||
raise RuntimeError("local_api_url not configured — set LOCAL_API_URL in .env or add a host at /settings/models")
|
||||
if not model:
|
||||
raise RuntimeError("local_model not configured — add a model at /settings/models")
|
||||
|
||||
host_type = cfg.get("host_type", "openwebui")
|
||||
# "openwebui" uses Open WebUI/Ollama path layout; "openai" uses standard OpenAI layout
|
||||
chat_path = "/chat/completions" if host_type == "openai" else "/api/chat/completions"
|
||||
logger.info("local backend (%s): %s @ %s", host_type, model, api_url)
|
||||
|
||||
msgs: list[dict] = []
|
||||
if system_prompt:
|
||||
msgs.append({"role": "system", "content": system_prompt})
|
||||
|
||||
# Build message list; inject image into the last user message when present.
|
||||
for i, m in enumerate(messages):
|
||||
is_last = (i == len(messages) - 1)
|
||||
if is_last and m["role"] == "user" and attachment:
|
||||
content: list[dict] = [{"type": "text", "text": m["content"]}]
|
||||
content.append({
|
||||
"type": "image_url",
|
||||
"image_url": {"url": attachment["data"]},
|
||||
})
|
||||
msgs.append({"role": "user", "content": content})
|
||||
else:
|
||||
# Strip non-standard metadata fields before sending to the API
|
||||
msgs.append({"role": m["role"], "content": m["content"]})
|
||||
|
||||
url = api_url.rstrip("/") + chat_path
|
||||
headers: dict[str, str] = {}
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
|
||||
payload = {"model": model, "messages": msgs}
|
||||
|
||||
async with httpx.AsyncClient(timeout=settings.timeout_local) as client:
|
||||
resp = await client.post(url, json=payload, headers=headers)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
text = data["choices"][0]["message"]["content"]
|
||||
if not text or not text.strip():
|
||||
raise RuntimeError("Local model returned an empty response")
|
||||
|
||||
usage = data.get("usage") or {}
|
||||
if usage.get("prompt_tokens") is not None:
|
||||
import usage_tracker
|
||||
from persona import _user
|
||||
asyncio.create_task(usage_tracker.record(
|
||||
username=_user.get(),
|
||||
backend="local",
|
||||
model_name=model,
|
||||
prompt_tokens=usage.get("prompt_tokens", 0),
|
||||
completion_tokens=usage.get("completion_tokens", 0),
|
||||
))
|
||||
|
||||
return text.strip()
|
||||
|
||||
|
||||
async def _anthropic_api(system_prompt: str, messages: list[dict], model_cfg: dict | None) -> str:
|
||||
"""Direct Anthropic API backend using the anthropic SDK."""
|
||||
try:
|
||||
import anthropic
|
||||
except ImportError:
|
||||
raise RuntimeError("anthropic SDK not installed — run: pip install 'anthropic>=0.40.0'")
|
||||
|
||||
cfg = model_cfg or {}
|
||||
api_key = cfg.get("api_key", "")
|
||||
model_name = cfg.get("model_name") or settings.default_model
|
||||
|
||||
if not api_key:
|
||||
raise RuntimeError("No Anthropic API key — add one at /settings/models")
|
||||
|
||||
client = anthropic.AsyncAnthropic(api_key=api_key)
|
||||
|
||||
msgs = [{"role": m["role"], "content": m["content"]} for m in messages]
|
||||
kwargs: dict = {
|
||||
"model": model_name,
|
||||
"max_tokens": 4096,
|
||||
"messages": msgs,
|
||||
}
|
||||
if system_prompt:
|
||||
kwargs["system"] = system_prompt
|
||||
|
||||
resp = await client.messages.create(**kwargs)
|
||||
|
||||
text = resp.content[0].text if resp.content else ""
|
||||
if not text.strip():
|
||||
raise RuntimeError("Anthropic API returned an empty response")
|
||||
|
||||
if resp.usage:
|
||||
import usage_tracker
|
||||
from persona import _user
|
||||
asyncio.create_task(usage_tracker.record(
|
||||
username=_user.get(),
|
||||
backend="anthropic_api",
|
||||
model_name=model_name,
|
||||
prompt_tokens=resp.usage.input_tokens,
|
||||
completion_tokens=resp.usage.output_tokens,
|
||||
))
|
||||
|
||||
return text.strip()
|
||||
|
||||
|
||||
async def _gemini(system_prompt: str, messages: list[dict]) -> str:
|
||||
# Gemini CLI spawns MCP child processes that keep stdout pipes open after responding.
|
||||
# start_new_session=True puts the whole tree in its own process group so
|
||||
|
||||
@@ -8,8 +8,8 @@ logging.basicConfig(level=logging.INFO, format="%(levelname)s:%(name)s: %(messag
|
||||
|
||||
from config import settings
|
||||
from auth_middleware import SessionAuthMiddleware
|
||||
from routers import chat, google_chat, nextcloud_talk, files, distill, auth, orchestrator
|
||||
from routers import ui, onboarding
|
||||
from routers import chat, google_chat, nextcloud_talk, homeassistant, files, distill, auth, orchestrator
|
||||
from routers import ui, onboarding, settings, tools_settings, help, auth_google, local_llm, push, audit, usage, crons
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
@@ -30,27 +30,44 @@ app.add_middleware(SessionAuthMiddleware)
|
||||
app.include_router(chat.router)
|
||||
app.include_router(google_chat.router)
|
||||
app.include_router(nextcloud_talk.router)
|
||||
app.include_router(homeassistant.router)
|
||||
app.include_router(files.router)
|
||||
app.include_router(distill.router)
|
||||
app.include_router(auth.router)
|
||||
app.include_router(orchestrator.router)
|
||||
app.include_router(push.router)
|
||||
app.include_router(audit.router)
|
||||
app.include_router(usage.router)
|
||||
|
||||
# Static files — must be mounted BEFORE ui.router so /static/* is matched first.
|
||||
# ui.router has a wildcard /{username}/{persona} that would otherwise catch /static/style.css etc.
|
||||
app.mount("/static", StaticFiles(directory="static"), name="static")
|
||||
|
||||
# Google OAuth — must be before ui.router (wildcard /{user}/{persona} would swallow it)
|
||||
app.include_router(auth_google.router)
|
||||
|
||||
# Onboarding (invite tokens + persona creation — before ui.router)
|
||||
app.include_router(onboarding.router)
|
||||
|
||||
# UI router (login + /{user}/{persona} — must be last to avoid swallowing API paths)
|
||||
app.include_router(ui.router)
|
||||
# Account settings
|
||||
app.include_router(settings.router)
|
||||
app.include_router(tools_settings.router)
|
||||
app.include_router(local_llm.router)
|
||||
app.include_router(crons.router)
|
||||
|
||||
# Help page
|
||||
app.include_router(help.router)
|
||||
|
||||
# Health check — must be before ui.router so /{username} catch-all doesn't swallow it.
|
||||
@app.get("/health")
|
||||
async def health() -> dict:
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
# UI router (login + /{user}/{persona} — must be last to avoid swallowing API paths)
|
||||
app.include_router(ui.router)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
uvicorn.run(
|
||||
"main:app",
|
||||
|
||||
@@ -6,9 +6,10 @@ Usage:
|
||||
python manage_passwords.py set <username> # prompt for password
|
||||
python manage_passwords.py set <username> <pass> # set directly (avoid in shell history)
|
||||
python manage_passwords.py check <username> # test a password interactively
|
||||
python manage_passwords.py list # show users, passwords, and emails
|
||||
python manage_passwords.py list # show users, auth methods, and emails
|
||||
python manage_passwords.py invite <username> [email] # generate + optionally email invite link
|
||||
python manage_passwords.py email <username> <email> # store/update an email address
|
||||
python manage_passwords.py google-add <username> <email> # register a user for Google sign-in
|
||||
"""
|
||||
|
||||
import json
|
||||
@@ -18,7 +19,7 @@ import getpass
|
||||
# Add cortex/ to path so we can import config and auth_utils
|
||||
sys.path.insert(0, str(__import__('pathlib').Path(__file__).parent))
|
||||
|
||||
from auth_utils import set_password, check_credentials, _auth_path, create_invite
|
||||
from auth_utils import set_password, check_credentials, _auth_path, create_invite, link_google, _read_auth
|
||||
from persona import list_users
|
||||
from config import settings
|
||||
|
||||
@@ -96,10 +97,14 @@ def cmd_list(_args):
|
||||
if not users:
|
||||
print(" No users found in home/")
|
||||
return
|
||||
print(f" {'USER':<18} {'PW':<6} {'GOOGLE':<8} {'EMAIL'}")
|
||||
print(f" {'-'*18} {'-'*6} {'-'*8} {'-'*30}")
|
||||
for user in users:
|
||||
has_pw = "✓ pw" if _auth_path(user).exists() else "✗ pw"
|
||||
auth = _read_auth(user)
|
||||
has_pw = "✓" if auth.get("password_hash") else "—"
|
||||
google = auth.get("google_email") or "—"
|
||||
email = get_email(user) or "—"
|
||||
print(f" {user:<20} {has_pw} {email}")
|
||||
print(f" {user:<18} {has_pw:<6} {google:<36} {email}")
|
||||
|
||||
|
||||
def cmd_email(args):
|
||||
@@ -149,6 +154,41 @@ def cmd_invite(args):
|
||||
print("Tip: python manage_passwords.py invite <username> <email> to email it next time.\n")
|
||||
|
||||
|
||||
def cmd_google_add(args):
|
||||
if len(args) < 2:
|
||||
print("Usage: manage_passwords.py google-add <username> <google_email>")
|
||||
sys.exit(1)
|
||||
username, email = args[0], args[1].lower().strip()
|
||||
|
||||
# Ensure the user directory exists
|
||||
(settings.home_root() / username).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Store in auth.json (google_sub filled in on first sign-in) + profile.json (for invites)
|
||||
link_google(username, sub="", email=email)
|
||||
set_email(username, email)
|
||||
print(f"Google sign-in registered for {username!r}: {email}")
|
||||
print(f"They can now sign in at {settings.cortex_base_url}/login using that Google account.")
|
||||
|
||||
|
||||
def cmd_role(args):
|
||||
if len(args) < 2:
|
||||
print("Usage: manage_passwords.py role <username> admin|user")
|
||||
sys.exit(1)
|
||||
username, role = args[0], args[1].lower().strip()
|
||||
if role not in ("admin", "user"):
|
||||
print("Role must be 'admin' or 'user'.")
|
||||
sys.exit(1)
|
||||
from auth_utils import _read_auth, _write_auth
|
||||
data = _read_auth(username)
|
||||
if not data:
|
||||
print(f"User '{username}' not found — no auth.json.")
|
||||
sys.exit(1)
|
||||
old_role = data.get("role", "user")
|
||||
data["role"] = role
|
||||
_write_auth(username, data)
|
||||
print(f"Role for '{username}': {old_role} → {role}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print(__doc__)
|
||||
@@ -167,6 +207,10 @@ if __name__ == "__main__":
|
||||
cmd_email(rest)
|
||||
elif command == "invite":
|
||||
cmd_invite(rest)
|
||||
elif command == "google-add":
|
||||
cmd_google_add(rest)
|
||||
elif command == "role":
|
||||
cmd_role(rest)
|
||||
else:
|
||||
print(f"Unknown command: {command}")
|
||||
print(__doc__)
|
||||
|
||||
@@ -1,9 +1,17 @@
|
||||
"""
|
||||
Inara tiered memory distillation.
|
||||
Tiered memory distillation.
|
||||
|
||||
distill_short() — roll recent session logs → MEMORY_SHORT.md (no LLM)
|
||||
distill_mid() — summarize MEMORY_SHORT → MEMORY_MID.md (LLM)
|
||||
distill_long() — integrate MEMORY_MID → MEMORY_LONG.md (LLM)
|
||||
|
||||
Before any file is overwritten, two rolling backups are kept:
|
||||
MEMORY_*.bak1.md — most recent backup (created just before last write)
|
||||
MEMORY_*.bak2.md — backup before that
|
||||
|
||||
LLM responses are sanity-checked before writing. If the response looks like
|
||||
a refusal, is too short, or is obviously not memory content, the distill is
|
||||
aborted and the original file is left untouched.
|
||||
"""
|
||||
import logging
|
||||
from datetime import datetime
|
||||
@@ -16,6 +24,25 @@ logger = logging.getLogger(__name__)
|
||||
# Rough chars-per-token estimate for budget enforcement
|
||||
_CHARS_PER_TOKEN = 4
|
||||
|
||||
# Phrases that indicate the LLM refused or misunderstood the task
|
||||
_REFUSAL_PREFIXES = (
|
||||
"i'm sorry",
|
||||
"i am sorry",
|
||||
"i can't",
|
||||
"i cannot",
|
||||
"i'm unable",
|
||||
"i am unable",
|
||||
"as an ai",
|
||||
"as a language model",
|
||||
"i don't have access",
|
||||
"i do not have access",
|
||||
"i'm not able",
|
||||
"i am not able",
|
||||
)
|
||||
|
||||
# Minimum characters for a valid mid/long distill response
|
||||
_MIN_RESPONSE_CHARS = 80
|
||||
|
||||
|
||||
def _budget_chars(tokens: int) -> int:
|
||||
return tokens * _CHARS_PER_TOKEN
|
||||
@@ -25,7 +52,62 @@ def _read(path: Path) -> str:
|
||||
return path.read_text() if path.exists() else ""
|
||||
|
||||
|
||||
def distill_short(username: str | None = None, persona: str | None = None) -> dict:
|
||||
def _rotate_backup(path: Path, n: int = 2) -> None:
|
||||
"""Rotate up to n rolling backups of path before a write.
|
||||
|
||||
MEMORY_LONG.md → MEMORY_LONG.bak1.md (most recent), MEMORY_LONG.bak2.md (older)
|
||||
"""
|
||||
if not path.exists():
|
||||
return
|
||||
# Shift older backups down: bak(n-1) → bak(n), …, bak1 stays as bak1 source
|
||||
for i in range(n, 1, -1):
|
||||
older = path.parent / f"{path.stem}.bak{i}.md"
|
||||
newer = path.parent / f"{path.stem}.bak{i - 1}.md"
|
||||
if newer.exists():
|
||||
older.write_text(newer.read_text())
|
||||
# Current file → bak1
|
||||
bak1 = path.parent / f"{path.stem}.bak1.md"
|
||||
bak1.write_text(path.read_text())
|
||||
|
||||
|
||||
def _sanity_check(response_text: str, context: str, existing_content: str = "") -> str | None:
|
||||
"""Return an error string if the LLM response looks invalid, else None.
|
||||
|
||||
Checks:
|
||||
- Minimum absolute length
|
||||
- Refusal / AI preamble phrases
|
||||
- Size shrinkage: new content must be at least 40% of the old (catches truncation)
|
||||
- Size explosion: new content must not exceed 250% of the old (catches runaway output)
|
||||
(Both bounds only apply when an existing file is present and reasonably sized.)
|
||||
"""
|
||||
stripped = response_text.strip()
|
||||
if len(stripped) < _MIN_RESPONSE_CHARS:
|
||||
return f"{context}: response too short ({len(stripped)} chars) — not writing"
|
||||
|
||||
first_line = stripped.lower().splitlines()[0]
|
||||
if any(first_line.startswith(p) for p in _REFUSAL_PREFIXES):
|
||||
return f"{context}: response looks like a refusal — not writing"
|
||||
|
||||
if existing_content:
|
||||
old_len = len(existing_content.strip())
|
||||
new_len = len(stripped)
|
||||
if old_len >= _MIN_RESPONSE_CHARS * 4: # only compare when old file has real content
|
||||
ratio = new_len / old_len
|
||||
if ratio < 0.40:
|
||||
return (
|
||||
f"{context}: new content is only {ratio:.0%} of the old "
|
||||
f"({new_len} vs {old_len} chars) — looks truncated, not writing"
|
||||
)
|
||||
if ratio > 2.50:
|
||||
return (
|
||||
f"{context}: new content is {ratio:.0%} of the old "
|
||||
f"({new_len} vs {old_len} chars) — looks like runaway output, not writing"
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def distill_short(username: str, persona: str) -> dict:
|
||||
"""
|
||||
Roll the most recent session log files into MEMORY_SHORT.md.
|
||||
No LLM involved — pure aggregation with budget truncation.
|
||||
@@ -64,8 +146,9 @@ def distill_short(username: str | None = None, persona: str | None = None) -> di
|
||||
)
|
||||
|
||||
out_path = inara_dir / "MEMORY_SHORT.md"
|
||||
_rotate_backup(out_path)
|
||||
out_path.write_text(header + body)
|
||||
logger.info("distill_short: wrote %d chars from %d files", len(header) + len(body), len(parts))
|
||||
logger.info("distill_short [%s/%s]: wrote %d chars from %d files", username, persona, len(header) + len(body), len(parts))
|
||||
|
||||
return {
|
||||
"files_included": len(parts),
|
||||
@@ -74,57 +157,78 @@ def distill_short(username: str | None = None, persona: str | None = None) -> di
|
||||
}
|
||||
|
||||
|
||||
async def distill_mid(username: str | None = None, persona: str | None = None) -> dict:
|
||||
async def distill_mid(username: str, persona: str) -> dict:
|
||||
"""
|
||||
Ask the LLM to summarize MEMORY_SHORT.md → MEMORY_MID.md.
|
||||
Backs up the current MEMORY_MID.md before overwriting.
|
||||
"""
|
||||
from llm_client import complete
|
||||
from persona import set_context
|
||||
|
||||
inara_dir = _persona_path(username, persona)
|
||||
u, p = username, persona
|
||||
set_context(u, p)
|
||||
|
||||
inara_dir = _persona_path(u, p)
|
||||
short_content = _read(inara_dir / "MEMORY_SHORT.md")
|
||||
existing_mid = _read(inara_dir / "MEMORY_MID.md")
|
||||
|
||||
if not short_content.strip() or "Not yet populated" in short_content:
|
||||
return {"error": "MEMORY_SHORT.md is empty — run distill/short first"}
|
||||
|
||||
budget_tokens = settings.memory_budget_mid
|
||||
persona_name = p.title()
|
||||
user_name = u.title()
|
||||
system_prompt = (
|
||||
f"You are {settings.agent_name}'s memory distillation system. "
|
||||
f"You are {persona_name}'s memory distillation system. "
|
||||
"Summarize the following recent session logs into a concise mid-term memory digest. "
|
||||
f"Target length: under {budget_tokens} tokens. "
|
||||
"Focus on: recurring themes, important decisions made, ongoing projects, "
|
||||
f"{settings.user_name}'s current state and priorities, and anything that should persist into future sessions. "
|
||||
f"Write in first person as {settings.agent_name} (e.g. '{settings.user_name} and I worked on...'). "
|
||||
f"{user_name}'s current state and priorities, and anything that should persist into future sessions. "
|
||||
f"Write in first person as {persona_name} (e.g. '{user_name} and I worked on...'). "
|
||||
"Use markdown headings. Be specific and concrete — no filler."
|
||||
)
|
||||
|
||||
response_text, backend = await complete(
|
||||
system_prompt=system_prompt,
|
||||
messages=[{"role": "user", "content": short_content}],
|
||||
role="distill",
|
||||
)
|
||||
|
||||
err = _sanity_check(response_text, "distill_mid", existing_mid)
|
||||
if err:
|
||||
logger.warning(err)
|
||||
return {"error": err}
|
||||
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M")
|
||||
header = (
|
||||
f"# MEMORY_MID.md — Mid-Term Memory Digest\n\n"
|
||||
f"*Auto-distilled: {now} via {backend}.*\n\n---\n\n"
|
||||
)
|
||||
out_path = inara_dir / "MEMORY_MID.md"
|
||||
_rotate_backup(out_path)
|
||||
out_path.write_text(header + response_text)
|
||||
logger.info("distill_mid: wrote %d chars via %s", len(header) + len(response_text), backend)
|
||||
logger.info("distill_mid [%s/%s]: wrote %d chars via %s", u, p, len(header) + len(response_text), backend)
|
||||
|
||||
return {
|
||||
"username": u,
|
||||
"backend": backend,
|
||||
"chars_written": len(header) + len(response_text),
|
||||
"budget_tokens": budget_tokens,
|
||||
}
|
||||
|
||||
|
||||
async def distill_long(username: str | None = None, persona: str | None = None) -> dict:
|
||||
async def distill_long(username: str, persona: str) -> dict:
|
||||
"""
|
||||
Ask the LLM to integrate MEMORY_MID.md into MEMORY_LONG.md.
|
||||
Backs up the current MEMORY_LONG.md before overwriting.
|
||||
"""
|
||||
from llm_client import complete
|
||||
from persona import set_context
|
||||
|
||||
inara_dir = _persona_path(username, persona)
|
||||
u, p = username, persona
|
||||
set_context(u, p)
|
||||
|
||||
inara_dir = _persona_path(u, p)
|
||||
long_content = _read(inara_dir / "MEMORY_LONG.md")
|
||||
mid_content = _read(inara_dir / "MEMORY_MID.md")
|
||||
|
||||
@@ -132,8 +236,9 @@ async def distill_long(username: str | None = None, persona: str | None = None)
|
||||
return {"error": "MEMORY_MID.md is empty — run distill/mid first"}
|
||||
|
||||
budget_tokens = settings.memory_budget_long
|
||||
persona_name = p.title()
|
||||
system_prompt = (
|
||||
f"You are {settings.agent_name}'s long-term memory curator. "
|
||||
f"You are {persona_name}'s long-term memory curator. "
|
||||
"You will receive the current long-term memory and a recent mid-term digest. "
|
||||
f"Integrate the new information into the long-term memory. Target: under {budget_tokens} tokens. "
|
||||
"Rules: preserve important historical facts; update or replace stale information; "
|
||||
@@ -149,22 +254,30 @@ async def distill_long(username: str | None = None, persona: str | None = None)
|
||||
response_text, backend = await complete(
|
||||
system_prompt=system_prompt,
|
||||
messages=[{"role": "user", "content": user_content}],
|
||||
role="distill",
|
||||
)
|
||||
|
||||
err = _sanity_check(response_text, "distill_long", long_content)
|
||||
if err:
|
||||
logger.warning(err)
|
||||
return {"error": err}
|
||||
|
||||
# Ensure the file has the right header if the LLM dropped it
|
||||
now = datetime.now().strftime("%Y-%m-%d %H:%M")
|
||||
if not response_text.lstrip().startswith("# MEMORY_LONG"):
|
||||
response_text = (
|
||||
f"# MEMORY_LONG.md — {settings.agent_name} Long-Term Memory\n\n"
|
||||
f"# MEMORY_LONG.md — {persona_name} Long-Term Memory\n\n"
|
||||
f"*Last distilled: {now} via {backend}.*\n\n---\n\n"
|
||||
+ response_text
|
||||
)
|
||||
|
||||
out_path = inara_dir / "MEMORY_LONG.md"
|
||||
_rotate_backup(out_path)
|
||||
out_path.write_text(response_text)
|
||||
logger.info("distill_long: wrote %d chars via %s", len(response_text), backend)
|
||||
logger.info("distill_long [%s/%s]: wrote %d chars via %s", u, p, len(response_text), backend)
|
||||
|
||||
return {
|
||||
"username": u,
|
||||
"backend": backend,
|
||||
"chars_written": len(response_text),
|
||||
"budget_tokens": budget_tokens,
|
||||
|
||||
980
cortex/model_registry.py
Normal file
980
cortex/model_registry.py
Normal file
@@ -0,0 +1,980 @@
|
||||
"""
|
||||
Per-user unified model registry — V2.
|
||||
|
||||
Stored in: home/{user}/model_registry.json
|
||||
|
||||
V2 Schema:
|
||||
{
|
||||
"version": 2,
|
||||
|
||||
# Per-provider accounts / credentials (user-configured)
|
||||
"providers": {
|
||||
"anthropic": {
|
||||
"credentials": [
|
||||
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
|
||||
]
|
||||
},
|
||||
"google": {
|
||||
"accounts": [
|
||||
{"id": "<hex>", "label": "My Google account", "api_key": "AIza..."}
|
||||
]
|
||||
}
|
||||
},
|
||||
|
||||
# Local OpenAI-compatible hosts (unchanged from V1)
|
||||
"hosts": [{"id", "label", "api_url", "api_key", "host_type"}, ...],
|
||||
|
||||
# User-registered model entries (all providers)
|
||||
"models": [
|
||||
{
|
||||
"id": str, # unique within this registry
|
||||
"type": str, # see TYPES below
|
||||
"label": str, # human-readable
|
||||
"model_name": str, # identifier sent to the API / CLI
|
||||
"provider": str | null, # "anthropic" | "google" | "local" | null
|
||||
"host_id": str | null, # local_openai only — references hosts[].id
|
||||
"credential_id":str | null, # claude_cli only — references providers.anthropic.credentials
|
||||
"account_id": str | null, # gemini_api only — references providers.google.accounts
|
||||
"context_k": int, # context window in k tokens (informational)
|
||||
"max_rounds": int | null, # per-model tool-loop cap; null = use orchestrator_max_rounds global
|
||||
"tags": [str], # user-defined capability tags
|
||||
},
|
||||
],
|
||||
|
||||
# Role assignments — any model (any provider) can fill any role
|
||||
"roles": {
|
||||
"<role>": {
|
||||
"primary": "<model_id>" | null,
|
||||
"backup_1": "<model_id>" | null,
|
||||
...
|
||||
"backup_4": "<model_id>" | null,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
Types:
|
||||
"claude_cli" — Claude CLI subprocess (~/.claude/.credentials.json)
|
||||
"gemini_cli" — Gemini CLI subprocess
|
||||
"gemini_api" — Gemini API (google-genai SDK); account_id → api_key from providers.google
|
||||
"local_openai" — OpenAI-compatible endpoint; host_id → api_url/api_key from hosts[]
|
||||
"anthropic_api" — Anthropic SDK direct; credential_id → api_key from providers.anthropic.credentials
|
||||
|
||||
Built-in model IDs (always resolvable without a registry entry):
|
||||
"claude_cli" — resolves to the default Claude CLI model
|
||||
"gemini_cli" — resolves to Gemini CLI
|
||||
"gemini_api" — resolves to Gemini API using GEMINI_API_KEY from .env
|
||||
|
||||
Role resolution for get_model_for_role(username, role):
|
||||
1. User registry: roles[role].primary → backup_1 → ... → backup_4
|
||||
2. .env default: ROLE_<ROLE>=<builtin_id>
|
||||
3. Hardcoded last-resort defaults per role
|
||||
4. claude_cli (absolute fallback)
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
from pathlib import Path
|
||||
|
||||
from config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ── Role-level tool defaults ───────────────────────────────────────────────────
|
||||
# Applied when a user hasn't configured a custom tool list for a role.
|
||||
# None = no restriction (all accessible tools); [] = no tools (pure text processing).
|
||||
# "chat" is intentionally absent: the /chat endpoint never sends tool schemas anyway,
|
||||
# and the orchestrator uses chat_role="chat" as its default — restricting it here
|
||||
# would block all tools from every default orchestration request.
|
||||
# "orchestrator" is intentionally absent — Phase 2 keyword routing narrows it per message.
|
||||
ROLE_DEFAULT_TOOLS: dict[str, list[str] | None] = {
|
||||
"distill": [], # pure text processing — no tools needed
|
||||
"research": ["web_search", "web_read", "http_fetch"],
|
||||
"coder": [
|
||||
"project_file_read", "project_file_list", "file_stat", "file_grep",
|
||||
"file_diff", "file_syntax_check", "file_read", "file_list", "file_write",
|
||||
"git_status", "git_log", "git_diff", "shell_exec",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
# ── Provider model catalogs ───────────────────────────────────────────────────
|
||||
# Server-side defaults. Update here when providers release new models.
|
||||
# Users can add entries via the settings UI (Phase 2).
|
||||
|
||||
ANTHROPIC_CATALOG: list[dict] = [
|
||||
# Latest
|
||||
{"id": "claude-opus-4-7", "label": "Claude Opus 4.7", "context_k": 1000},
|
||||
{"id": "claude-sonnet-4-6", "label": "Claude Sonnet 4.6", "context_k": 1000},
|
||||
{"id": "claude-haiku-4-5-20251001", "label": "Claude Haiku 4.5", "context_k": 200},
|
||||
# Previous versions (still available, not deprecated)
|
||||
{"id": "claude-opus-4-6", "label": "Claude Opus 4.6", "context_k": 1000},
|
||||
{"id": "claude-sonnet-4-5", "label": "Claude Sonnet 4.5", "context_k": 200},
|
||||
]
|
||||
|
||||
GOOGLE_CATALOG: list[dict] = [
|
||||
# Stable / generally available
|
||||
{"id": "gemini-2.5-pro", "label": "Gemini 2.5 Pro", "context_k": 1000},
|
||||
{"id": "gemini-2.5-flash", "label": "Gemini 2.5 Flash", "context_k": 1000},
|
||||
{"id": "gemini-2.5-flash-lite", "label": "Gemini 2.5 Flash-Lite", "context_k": 1000},
|
||||
# Preview
|
||||
{"id": "gemini-3.1-pro-preview", "label": "Gemini 3.1 Pro (preview)", "context_k": 1000},
|
||||
{"id": "gemini-3-flash-preview", "label": "Gemini 3 Flash (preview)", "context_k": 1000},
|
||||
{"id": "gemini-3.1-flash-lite-preview", "label": "Gemini 3.1 Flash-Lite (preview)", "context_k": 1000},
|
||||
]
|
||||
|
||||
# Known OpenAI-compatible cloud inference services.
|
||||
# All use host_type "openai" (/chat/completions + /models paths).
|
||||
CLOUD_API_CATALOG: list[dict] = [
|
||||
{"id": "openrouter", "label": "OpenRouter", "api_url": "https://openrouter.ai/api/v1"},
|
||||
{"id": "openai", "label": "OpenAI", "api_url": "https://api.openai.com/v1"},
|
||||
{"id": "groq", "label": "Groq", "api_url": "https://api.groq.com/openai/v1"},
|
||||
{"id": "xai", "label": "X.ai / Grok", "api_url": "https://api.x.ai/v1"},
|
||||
{"id": "together", "label": "Together.ai", "api_url": "https://api.together.xyz/v1"},
|
||||
{"id": "fireworks", "label": "Fireworks.ai", "api_url": "https://api.fireworks.ai/inference/v1"},
|
||||
{"id": "custom", "label": "Custom", "api_url": ""},
|
||||
]
|
||||
|
||||
|
||||
# ── Built-in model definitions ────────────────────────────────────────────────
|
||||
|
||||
def _builtins() -> dict[str, dict]:
|
||||
return {
|
||||
"claude_cli": {
|
||||
"id": "claude_cli",
|
||||
"type": "claude_cli",
|
||||
"label": f"Claude (CLI) — {settings.default_model}",
|
||||
"model_name": settings.default_model,
|
||||
"context_k": 200,
|
||||
"tags": ["chat", "persona", "creative"],
|
||||
},
|
||||
"gemini_cli": {
|
||||
"id": "gemini_cli",
|
||||
"type": "gemini_cli",
|
||||
"label": "Gemini (CLI)",
|
||||
"model_name": "",
|
||||
"context_k": 1000,
|
||||
"tags": ["chat", "research", "long_context"],
|
||||
},
|
||||
"gemini_api": {
|
||||
"id": "gemini_api",
|
||||
"type": "gemini_api",
|
||||
"label": f"Gemini API — {settings.orchestrator_model}",
|
||||
"model_name": settings.orchestrator_model,
|
||||
"context_k": 1000,
|
||||
"tags": ["orchestrator", "research", "long_context", "tools"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
_ROLE_LAST_RESORT: dict[str, str] = {
|
||||
"chat": "claude_cli",
|
||||
"orchestrator": "gemini_api",
|
||||
"distill": "claude_cli",
|
||||
"coder": "claude_cli",
|
||||
"research": "gemini_api",
|
||||
}
|
||||
|
||||
PRIORITY_KEYS = ["primary", "backup_1", "backup_2", "backup_3", "backup_4"]
|
||||
|
||||
REQUIRED_ROLES: list[str] = ["chat", "orchestrator", "distill"]
|
||||
|
||||
|
||||
# ── Storage ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def _registry_path(username: str) -> Path:
|
||||
return settings.home_root() / username / "model_registry.json"
|
||||
|
||||
|
||||
def _local_llm_path(username: str) -> Path:
|
||||
return settings.home_root() / username / "local_llm.json"
|
||||
|
||||
|
||||
def _auth_path(username: str) -> Path:
|
||||
return settings.home_root() / username / "auth.json"
|
||||
|
||||
|
||||
def _empty() -> dict:
|
||||
return {
|
||||
"version": 2,
|
||||
"providers": _default_providers(),
|
||||
"hosts": [],
|
||||
"models": [],
|
||||
"roles": {},
|
||||
}
|
||||
|
||||
|
||||
def _default_providers() -> dict:
|
||||
return {
|
||||
"anthropic": {
|
||||
"credentials": [
|
||||
{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}
|
||||
]
|
||||
},
|
||||
"google": {
|
||||
"accounts": []
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _normalize(data: dict) -> dict:
|
||||
"""Back-fill missing fields introduced by schema additions."""
|
||||
for h in data.get("hosts", []):
|
||||
h.setdefault("host_type", "openwebui")
|
||||
h.setdefault("max_concurrent", 3)
|
||||
data.setdefault("providers", _default_providers())
|
||||
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
|
||||
data["providers"].setdefault("google", {"accounts": []})
|
||||
return data
|
||||
|
||||
|
||||
def _load(username: str) -> dict:
|
||||
path = _registry_path(username)
|
||||
if path.exists():
|
||||
try:
|
||||
data = json.loads(path.read_text())
|
||||
if isinstance(data, dict) and "version" in data:
|
||||
if data["version"] == 1:
|
||||
data = _migrate_v1_to_v2(username, data)
|
||||
_save(username, data)
|
||||
return _normalize(data)
|
||||
except (json.JSONDecodeError, OSError):
|
||||
logger.warning("model_registry.json for %s is unreadable — starting fresh", username)
|
||||
return _empty()
|
||||
|
||||
# No registry — try migrating from local_llm.json
|
||||
legacy = _local_llm_path(username)
|
||||
if legacy.exists():
|
||||
data = _migrate_from_local_llm(username, legacy)
|
||||
_save(username, data)
|
||||
logger.info("Migrated local_llm.json → model_registry.json for %s", username)
|
||||
return data
|
||||
|
||||
return _empty()
|
||||
|
||||
|
||||
def _save(username: str, data: dict) -> None:
|
||||
_registry_path(username).write_text(json.dumps(data, indent=2))
|
||||
|
||||
|
||||
# ── Migration ─────────────────────────────────────────────────────────────────
|
||||
|
||||
def _migrate_v1_to_v2(username: str, data: dict) -> dict:
|
||||
"""
|
||||
Upgrade a V1 registry to V2.
|
||||
|
||||
Changes:
|
||||
- Adds providers section with default structure
|
||||
- Migrates gemini_api_key from auth.json → first Google account entry
|
||||
- Sets version to 2
|
||||
"""
|
||||
logger.info("Migrating model_registry.json V1 → V2 for %s", username)
|
||||
|
||||
data["version"] = 2
|
||||
if "providers" not in data:
|
||||
data["providers"] = _default_providers()
|
||||
else:
|
||||
data["providers"].setdefault("anthropic", {"credentials": [{"id": "cli", "label": "Claude CLI (OAuth)", "type": "cli"}]})
|
||||
data["providers"].setdefault("google", {"accounts": []})
|
||||
|
||||
# Pull existing Gemini key from auth.json (stored there in V1) → first account entry
|
||||
accounts = data["providers"]["google"]["accounts"]
|
||||
if not accounts:
|
||||
try:
|
||||
auth = json.loads(_auth_path(username).read_text())
|
||||
existing_key = auth.get("gemini_api_key")
|
||||
if existing_key:
|
||||
accounts.append({
|
||||
"id": secrets.token_hex(4),
|
||||
"label": "Gemini API Key",
|
||||
"api_key": existing_key,
|
||||
})
|
||||
logger.info("Migrated gemini_api_key from auth.json → providers.google.accounts for %s", username)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
pass
|
||||
|
||||
return data
|
||||
|
||||
|
||||
def _migrate_from_local_llm(username: str, path: Path) -> dict:
|
||||
"""Convert local_llm.json → V2 model_registry format."""
|
||||
try:
|
||||
old = json.loads(path.read_text())
|
||||
except Exception:
|
||||
return _empty()
|
||||
|
||||
data = _empty()
|
||||
|
||||
# Handle v0 flat format
|
||||
if "hosts" not in old:
|
||||
api_url = old.get("api_url") or settings.local_api_url
|
||||
api_key = old.get("api_key") or settings.local_api_key
|
||||
model_name = old.get("model") or settings.local_model
|
||||
if not api_url:
|
||||
return data
|
||||
host_id = secrets.token_hex(4)
|
||||
old = {
|
||||
"hosts": [{"id": host_id, "label": "Local Model Server", "api_url": api_url, "api_key": api_key}],
|
||||
"models": [{"id": secrets.token_hex(4), "host_id": host_id, "label": model_name, "model_name": model_name}] if model_name else [],
|
||||
"active_model_id": None,
|
||||
}
|
||||
if old["models"]:
|
||||
old["active_model_id"] = old["models"][0]["id"]
|
||||
|
||||
data["hosts"] = old.get("hosts", [])
|
||||
|
||||
for m in old.get("models", []):
|
||||
data["models"].append({
|
||||
"id": m["id"],
|
||||
"type": "local_openai",
|
||||
"label": m.get("label") or m.get("model_name", ""),
|
||||
"model_name": m.get("model_name", ""),
|
||||
"provider": "local",
|
||||
"host_id": m.get("host_id"),
|
||||
"context_k": 0,
|
||||
"tags": [],
|
||||
})
|
||||
|
||||
active_id = old.get("active_model_id")
|
||||
if active_id and any(m["id"] == active_id for m in data["models"]):
|
||||
data["roles"]["chat"] = {"primary": active_id}
|
||||
if settings.distill_backend_mid == "local":
|
||||
data["roles"]["distill"] = {"primary": active_id}
|
||||
|
||||
# Migrate Gemini key from auth.json
|
||||
data = _migrate_v1_to_v2(username, {"version": 1, **data})
|
||||
return data
|
||||
|
||||
|
||||
# ── Model resolution ──────────────────────────────────────────────────────────
|
||||
|
||||
def _resolve_model(registry: dict, model_id: str) -> dict | None:
|
||||
"""Resolve a model_id to its full config dict (credentials merged in), or None."""
|
||||
builtins = _builtins()
|
||||
|
||||
# Built-in IDs take priority over user-defined entries with the same ID
|
||||
if model_id in builtins:
|
||||
return dict(builtins[model_id])
|
||||
|
||||
model = next((m for m in registry.get("models", []) if m["id"] == model_id), None)
|
||||
if not model:
|
||||
return None
|
||||
|
||||
model_type = model.get("type")
|
||||
|
||||
if model_type == "local_openai":
|
||||
host_id = model.get("host_id")
|
||||
host = next((h for h in registry.get("hosts", []) if h["id"] == host_id), None)
|
||||
if not host:
|
||||
logger.warning("model %s references missing host_id %s", model_id, host_id)
|
||||
return None
|
||||
return {
|
||||
**model,
|
||||
"api_url": host.get("api_url", ""),
|
||||
"api_key": host.get("api_key", ""),
|
||||
"host_type": host.get("host_type", "openwebui"),
|
||||
}
|
||||
|
||||
if model_type == "gemini_api":
|
||||
account_id = model.get("account_id")
|
||||
if account_id:
|
||||
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
|
||||
account = next((a for a in accounts if a["id"] == account_id), None)
|
||||
if account:
|
||||
return {**model, "api_key": account.get("api_key", "")}
|
||||
logger.warning("model %s references missing account_id %s", model_id, account_id)
|
||||
return dict(model)
|
||||
|
||||
if model_type == "anthropic_api":
|
||||
credential_id = model.get("credential_id")
|
||||
if credential_id:
|
||||
creds = registry.get("providers", {}).get("anthropic", {}).get("credentials", [])
|
||||
cred = next((c for c in creds if c["id"] == credential_id), None)
|
||||
if cred and cred.get("api_key"):
|
||||
return {**model, "api_key": cred["api_key"]}
|
||||
logger.warning("model %s references missing/keyless credential_id %s", model_id, credential_id)
|
||||
return dict(model)
|
||||
|
||||
if model_type == "claude_cli":
|
||||
return dict(model)
|
||||
|
||||
return dict(model)
|
||||
|
||||
|
||||
def get_model_for_role(username: str, role: str) -> dict | None:
|
||||
"""
|
||||
Return the resolved model config for the given role.
|
||||
|
||||
Resolution order:
|
||||
1. User registry: roles[role].primary → backup_1 → ... → backup_4
|
||||
2. .env: ROLE_<ROLE> = builtin model ID
|
||||
3. Hardcoded last-resort default per role
|
||||
4. claude_cli (absolute fallback)
|
||||
"""
|
||||
registry = _load(username)
|
||||
role_cfg = registry.get("roles", {}).get(role, {})
|
||||
|
||||
for key in PRIORITY_KEYS:
|
||||
model_id = role_cfg.get(key)
|
||||
if not model_id:
|
||||
continue
|
||||
resolved = _resolve_model(registry, model_id)
|
||||
if resolved:
|
||||
return resolved
|
||||
logger.debug("role %s.%s = %s but model not found", role, key, model_id)
|
||||
|
||||
# .env default
|
||||
env_type = settings.get_role_default(role)
|
||||
builtins = _builtins()
|
||||
if env_type and env_type in builtins:
|
||||
return dict(builtins[env_type])
|
||||
|
||||
# Hardcoded last resort
|
||||
fallback_id = _ROLE_LAST_RESORT.get(role, "claude_cli")
|
||||
return dict(builtins.get(fallback_id, builtins["claude_cli"]))
|
||||
|
||||
|
||||
def get_best_local_model(username: str, role: str = "chat") -> dict | None:
|
||||
"""
|
||||
Return the best available local_openai model for the given role.
|
||||
Used when the user explicitly selects "local" backend in the UI.
|
||||
"""
|
||||
registry = _load(username)
|
||||
role_cfg = registry.get("roles", {}).get(role, {})
|
||||
|
||||
for key in PRIORITY_KEYS:
|
||||
model_id = role_cfg.get(key)
|
||||
if not model_id:
|
||||
continue
|
||||
resolved = _resolve_model(registry, model_id)
|
||||
if resolved and resolved.get("type") == "local_openai":
|
||||
return resolved
|
||||
|
||||
for model in registry.get("models", []):
|
||||
if model.get("type") == "local_openai":
|
||||
resolved = _resolve_model(registry, model["id"])
|
||||
if resolved:
|
||||
return resolved
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def set_role_config(
|
||||
username: str,
|
||||
role: str,
|
||||
system_append: str,
|
||||
tools: list[str] | None,
|
||||
inject_datetime: bool = True,
|
||||
inject_mode: bool = True,
|
||||
) -> None:
|
||||
"""Save system_append, tools allow-list, and per-injection flags for a role.
|
||||
|
||||
tools=None clears the allow-list (role uses all accessible tools).
|
||||
inject_datetime=False suppresses the date/time header for pure processing roles.
|
||||
inject_mode=False suppresses the session mode (OTR) line for pure processing roles.
|
||||
"""
|
||||
data = _load(username)
|
||||
roles = data.setdefault("roles", {})
|
||||
if role not in roles:
|
||||
roles[role] = {}
|
||||
roles[role]["system_append"] = system_append.strip()
|
||||
roles[role]["inject_datetime"] = inject_datetime
|
||||
roles[role]["inject_mode"] = inject_mode
|
||||
if tools is None:
|
||||
roles[role].pop("tools", None)
|
||||
else:
|
||||
roles[role]["tools"] = [t for t in tools if t]
|
||||
_save(username, data)
|
||||
|
||||
|
||||
def get_role_config(username: str, role: str) -> dict:
|
||||
"""
|
||||
Return supplemental config for a role: system_append, tools, and injection flags.
|
||||
|
||||
All keys are optional in the registry — missing means "use defaults":
|
||||
system_append: str — appended to the system prompt for this role
|
||||
tools: list[str] | None — explicit tool allow-list (None = no restriction)
|
||||
inject_datetime: bool — whether to inject current date/time (default True)
|
||||
inject_mode: bool — whether to inject session mode (OTR) line (default True)
|
||||
"""
|
||||
registry = _load(username)
|
||||
role_cfg = registry.get("roles", {}).get(role, {})
|
||||
user_tools = role_cfg.get("tools")
|
||||
if user_tools is None:
|
||||
# No user-configured list — fall back to system defaults for this role
|
||||
effective_tools: list[str] | None = ROLE_DEFAULT_TOOLS.get(role)
|
||||
else:
|
||||
# User has configured tools; preserve their setting (empty list → no restriction)
|
||||
effective_tools = user_tools or None
|
||||
return {
|
||||
"system_append": role_cfg.get("system_append", ""),
|
||||
"tools": effective_tools,
|
||||
"inject_datetime": role_cfg.get("inject_datetime", True),
|
||||
"inject_mode": role_cfg.get("inject_mode", True),
|
||||
}
|
||||
|
||||
|
||||
def get_model_for_slot(username: str, role: str, slot: str) -> dict | None:
|
||||
"""
|
||||
Resolve a single named priority slot from a role without walking the fallback chain.
|
||||
|
||||
Used by Phase 3 explicit slot selection — the user has pinned a specific model;
|
||||
don't silently redirect to another slot if this one is empty or broken.
|
||||
Returns None if the slot is unset or the model can't be resolved.
|
||||
"""
|
||||
if slot not in PRIORITY_KEYS:
|
||||
return None
|
||||
registry = _load(username)
|
||||
model_id = registry.get("roles", {}).get(role, {}).get(slot)
|
||||
if not model_id:
|
||||
return None
|
||||
return _resolve_model(registry, model_id)
|
||||
|
||||
|
||||
def get_google_api_key(username: str, account_id: str | None = None) -> str | None:
|
||||
"""
|
||||
Return the best available Gemini API key for the user.
|
||||
|
||||
If account_id is specified, returns that account's key (or None if not found).
|
||||
Otherwise returns the first configured account key, falling back to the
|
||||
server-level GEMINI_API_KEY from .env.
|
||||
"""
|
||||
registry = _load(username)
|
||||
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
|
||||
|
||||
if account_id:
|
||||
account = next((a for a in accounts if a["id"] == account_id), None)
|
||||
return account.get("api_key") if account else None
|
||||
|
||||
# First configured account
|
||||
if accounts:
|
||||
return accounts[0].get("api_key") or None
|
||||
|
||||
# Fall back to .env server key
|
||||
return settings.gemini_api_key or None
|
||||
|
||||
|
||||
# ── Read API ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def get_registry(username: str) -> dict:
|
||||
"""Return the full registry (providers + hosts + models + roles)."""
|
||||
return _load(username)
|
||||
|
||||
|
||||
def get_all_models(username: str) -> list[dict]:
|
||||
"""Return all user-defined models (resolved — credentials/hosts merged in)."""
|
||||
registry = _load(username)
|
||||
out = []
|
||||
for m in registry.get("models", []):
|
||||
resolved = _resolve_model(registry, m["id"])
|
||||
if resolved:
|
||||
out.append(resolved)
|
||||
return out
|
||||
|
||||
|
||||
def get_defined_roles(username: str) -> dict[str, dict]:
|
||||
"""Return the roles section, filling gaps with empty dicts."""
|
||||
registry = _load(username)
|
||||
roles = registry.get("roles", {})
|
||||
return {role: roles.get(role, {}) for role in settings.get_defined_roles()}
|
||||
|
||||
|
||||
def get_google_accounts(username: str) -> list[dict]:
|
||||
"""Return Google account entries (api_key masked for display)."""
|
||||
registry = _load(username)
|
||||
accounts = registry.get("providers", {}).get("google", {}).get("accounts", [])
|
||||
return [
|
||||
{
|
||||
"id": a["id"],
|
||||
"label": a.get("label", ""),
|
||||
"hint": (a.get("api_key") or "")[:8] + "…" if a.get("api_key") else "",
|
||||
}
|
||||
for a in accounts
|
||||
]
|
||||
|
||||
|
||||
def get_catalog(provider: str, username: str | None = None) -> list[dict]:
|
||||
"""
|
||||
Return the model catalog for a provider.
|
||||
|
||||
For now returns server defaults. Phase 2 will merge in per-user additions.
|
||||
"""
|
||||
if provider == "anthropic":
|
||||
return list(ANTHROPIC_CATALOG)
|
||||
if provider == "google":
|
||||
return list(GOOGLE_CATALOG)
|
||||
if provider == "cloud":
|
||||
return list(CLOUD_API_CATALOG)
|
||||
return []
|
||||
|
||||
|
||||
# ── Write API — Google accounts ───────────────────────────────────────────────
|
||||
|
||||
def save_google_account(username: str, account_id: str | None,
|
||||
label: str, api_key: str) -> str:
|
||||
"""Create or update a Google account entry. Returns the account ID."""
|
||||
data = _load(username)
|
||||
accounts = data["providers"]["google"]["accounts"]
|
||||
|
||||
if account_id:
|
||||
for a in accounts:
|
||||
if a["id"] == account_id:
|
||||
a["label"] = label.strip()
|
||||
if api_key.strip():
|
||||
a["api_key"] = api_key.strip()
|
||||
_save(username, data)
|
||||
return account_id
|
||||
|
||||
account_id = secrets.token_hex(4)
|
||||
accounts.append({
|
||||
"id": account_id,
|
||||
"label": label.strip(),
|
||||
"api_key": api_key.strip(),
|
||||
})
|
||||
_save(username, data)
|
||||
return account_id
|
||||
|
||||
|
||||
def remove_google_account(username: str, account_id: str) -> bool:
|
||||
"""Remove a Google account. Clears any model entries that reference it."""
|
||||
data = _load(username)
|
||||
accounts = data["providers"]["google"]["accounts"]
|
||||
before = len(accounts)
|
||||
data["providers"]["google"]["accounts"] = [a for a in accounts if a["id"] != account_id]
|
||||
|
||||
# Clear role assignments for models that referenced this account
|
||||
removed_model_ids = {
|
||||
m["id"] for m in data.get("models", [])
|
||||
if m.get("account_id") == account_id
|
||||
}
|
||||
data["models"] = [m for m in data.get("models", []) if m["id"] not in removed_model_ids]
|
||||
for role_cfg in data.get("roles", {}).values():
|
||||
for key in PRIORITY_KEYS:
|
||||
if role_cfg.get(key) in removed_model_ids:
|
||||
role_cfg[key] = None
|
||||
|
||||
_save(username, data)
|
||||
return len(data["providers"]["google"]["accounts"]) < before
|
||||
|
||||
|
||||
# ── Write API — Anthropic API keys ───────────────────────────────────────────
|
||||
|
||||
def get_anthropic_api_keys(username: str) -> list[dict]:
|
||||
"""Return Anthropic API key credentials (type='api_key') with key masked for display."""
|
||||
registry = _load(username)
|
||||
creds = registry.get("providers", {}).get("anthropic", {}).get("credentials", [])
|
||||
return [
|
||||
{
|
||||
"id": c["id"],
|
||||
"label": c.get("label", ""),
|
||||
"hint": (c.get("api_key") or "")[:8] + "…" if c.get("api_key") else "no key",
|
||||
}
|
||||
for c in creds
|
||||
if c.get("type") == "api_key"
|
||||
]
|
||||
|
||||
|
||||
def save_anthropic_api_key(username: str, key_id: str | None,
|
||||
label: str, api_key: str) -> str:
|
||||
"""Create or update an Anthropic API key credential. Returns the credential ID."""
|
||||
data = _load(username)
|
||||
creds = data["providers"]["anthropic"]["credentials"]
|
||||
|
||||
if key_id:
|
||||
for c in creds:
|
||||
if c["id"] == key_id and c.get("type") == "api_key":
|
||||
c["label"] = label.strip() or c.get("label", "API Key")
|
||||
if api_key.strip():
|
||||
c["api_key"] = api_key.strip()
|
||||
_save(username, data)
|
||||
return key_id
|
||||
|
||||
key_id = secrets.token_hex(4)
|
||||
creds.append({
|
||||
"id": key_id,
|
||||
"label": label.strip() or "API Key",
|
||||
"type": "api_key",
|
||||
"api_key": api_key.strip(),
|
||||
})
|
||||
_save(username, data)
|
||||
return key_id
|
||||
|
||||
|
||||
def remove_anthropic_api_key(username: str, key_id: str) -> bool:
|
||||
"""Remove an Anthropic API key credential. Clears model entries that reference it."""
|
||||
data = _load(username)
|
||||
creds = data["providers"]["anthropic"]["credentials"]
|
||||
before = len(creds)
|
||||
data["providers"]["anthropic"]["credentials"] = [
|
||||
c for c in creds if c["id"] != key_id
|
||||
]
|
||||
|
||||
removed_model_ids = {
|
||||
m["id"] for m in data.get("models", [])
|
||||
if m.get("credential_id") == key_id
|
||||
}
|
||||
data["models"] = [m for m in data.get("models", []) if m["id"] not in removed_model_ids]
|
||||
for role_cfg in data.get("roles", {}).values():
|
||||
for key in PRIORITY_KEYS:
|
||||
if role_cfg.get(key) in removed_model_ids:
|
||||
role_cfg[key] = None
|
||||
|
||||
_save(username, data)
|
||||
return len(data["providers"]["anthropic"]["credentials"]) < before
|
||||
|
||||
|
||||
# ── Write API — Hosts ─────────────────────────────────────────────────────────
|
||||
|
||||
def save_host(username: str, host_id: str | None,
|
||||
label: str, api_url: str, api_key: str,
|
||||
host_type: str = "openwebui",
|
||||
max_concurrent: int = 3) -> str:
|
||||
"""Create or update a host. Returns the host ID."""
|
||||
data = _load(username)
|
||||
host_type = host_type if host_type in ("openwebui", "openai") else "openwebui"
|
||||
max_concurrent = max(1, min(int(max_concurrent), 20))
|
||||
|
||||
if host_id:
|
||||
for h in data["hosts"]:
|
||||
if h["id"] == host_id:
|
||||
h["label"] = label.strip()
|
||||
h["api_url"] = api_url.strip()
|
||||
h["host_type"] = host_type
|
||||
h["max_concurrent"] = max_concurrent
|
||||
if api_key.strip():
|
||||
h["api_key"] = api_key.strip()
|
||||
_save(username, data)
|
||||
return host_id
|
||||
host_id = None
|
||||
|
||||
host_id = secrets.token_hex(4)
|
||||
data["hosts"].append({
|
||||
"id": host_id,
|
||||
"label": label.strip(),
|
||||
"api_url": api_url.strip(),
|
||||
"api_key": api_key.strip(),
|
||||
"host_type": host_type,
|
||||
"max_concurrent": max_concurrent,
|
||||
})
|
||||
_save(username, data)
|
||||
return host_id
|
||||
|
||||
|
||||
def remove_host(username: str, host_id: str) -> bool:
|
||||
"""Remove a host and all models that reference it."""
|
||||
data = _load(username)
|
||||
before = len(data["hosts"])
|
||||
removed_model_ids = {m["id"] for m in data["models"] if m.get("host_id") == host_id}
|
||||
data["hosts"] = [h for h in data["hosts"] if h["id"] != host_id]
|
||||
data["models"] = [m for m in data["models"] if m.get("host_id") != host_id]
|
||||
for role_cfg in data.get("roles", {}).values():
|
||||
for key in PRIORITY_KEYS:
|
||||
if role_cfg.get(key) in removed_model_ids:
|
||||
role_cfg[key] = None
|
||||
_save(username, data)
|
||||
return len(data["hosts"]) < before
|
||||
|
||||
|
||||
# ── Write API — Models ────────────────────────────────────────────────────────
|
||||
|
||||
def save_model(username: str, model_id: str | None, host_id: str,
|
||||
label: str, model_name: str, context_k: int = 0,
|
||||
tags: list[str] | None = None,
|
||||
max_rounds: int | None = None,
|
||||
tools: bool = True,
|
||||
reasoning_budget_tokens: int | None = None) -> str:
|
||||
"""Create or update a local_openai model entry. Returns the model ID."""
|
||||
data = _load(username)
|
||||
tags = tags or []
|
||||
|
||||
if model_id:
|
||||
for m in data["models"]:
|
||||
if m["id"] == model_id:
|
||||
m["host_id"] = host_id
|
||||
m["label"] = label.strip() or model_name.strip()
|
||||
m["model_name"] = model_name.strip()
|
||||
m["context_k"] = context_k
|
||||
m["max_rounds"] = max_rounds
|
||||
m["tools"] = tools
|
||||
m["tags"] = tags
|
||||
m["reasoning_budget_tokens"] = reasoning_budget_tokens
|
||||
_save(username, data)
|
||||
return model_id
|
||||
model_id = None
|
||||
|
||||
model_id = secrets.token_hex(4)
|
||||
data["models"].append({
|
||||
"id": model_id,
|
||||
"type": "local_openai",
|
||||
"label": label.strip() or model_name.strip(),
|
||||
"model_name": model_name.strip(),
|
||||
"provider": "local",
|
||||
"host_id": host_id,
|
||||
"context_k": context_k,
|
||||
"max_rounds": max_rounds,
|
||||
"tools": tools,
|
||||
"tags": tags,
|
||||
"reasoning_budget_tokens": reasoning_budget_tokens,
|
||||
})
|
||||
_save(username, data)
|
||||
return model_id
|
||||
|
||||
|
||||
def save_cloud_model(username: str, model_id: str | None,
|
||||
provider: str, model_name: str, label: str,
|
||||
account_id: str | None = None,
|
||||
credential_id: str | None = None,
|
||||
context_k: int = 0,
|
||||
tags: list[str] | None = None,
|
||||
max_rounds: int | None = None,
|
||||
tools: bool = True) -> str:
|
||||
"""
|
||||
Create or update an Anthropic or Google model entry. Returns the model ID.
|
||||
|
||||
provider: "anthropic" | "google"
|
||||
account_id: Google only — references providers.google.accounts[].id
|
||||
credential_id: Anthropic only — "cli" for OAuth CLI, or a hex ID for an API key credential
|
||||
"""
|
||||
data = _load(username)
|
||||
|
||||
# Determine model type from credential (anthropic only)
|
||||
if provider == "anthropic":
|
||||
creds = data.get("providers", {}).get("anthropic", {}).get("credentials", [])
|
||||
cred = next((c for c in creds if c["id"] == credential_id), None) if credential_id else None
|
||||
entry_type = "anthropic_api" if (cred and cred.get("type") == "api_key") else "claude_cli"
|
||||
elif provider == "google":
|
||||
entry_type = "gemini_api"
|
||||
else:
|
||||
entry_type = "claude_cli"
|
||||
tags = tags or []
|
||||
|
||||
entry: dict = {
|
||||
"type": entry_type,
|
||||
"label": label.strip() or model_name.strip(),
|
||||
"model_name": model_name.strip(),
|
||||
"provider": provider,
|
||||
"context_k": context_k,
|
||||
"max_rounds": max_rounds,
|
||||
"tools": tools,
|
||||
"tags": tags,
|
||||
}
|
||||
if account_id:
|
||||
entry["account_id"] = account_id
|
||||
if credential_id:
|
||||
entry["credential_id"] = credential_id
|
||||
|
||||
if model_id:
|
||||
for m in data["models"]:
|
||||
if m["id"] == model_id:
|
||||
m.update(entry)
|
||||
_save(username, data)
|
||||
return model_id
|
||||
model_id = None
|
||||
|
||||
model_id = secrets.token_hex(4)
|
||||
entry["id"] = model_id
|
||||
data["models"].append(entry)
|
||||
_save(username, data)
|
||||
return model_id
|
||||
|
||||
|
||||
def remove_model(username: str, model_id: str) -> bool:
|
||||
"""Remove a model and clear any role assignments pointing to it."""
|
||||
data = _load(username)
|
||||
before = len(data["models"])
|
||||
data["models"] = [m for m in data["models"] if m["id"] != model_id]
|
||||
for role_cfg in data.get("roles", {}).values():
|
||||
for key in PRIORITY_KEYS:
|
||||
if role_cfg.get(key) == model_id:
|
||||
role_cfg[key] = None
|
||||
_save(username, data)
|
||||
return len(data["models"]) < before
|
||||
|
||||
|
||||
def get_custom_roles(username: str) -> list[str]:
|
||||
"""
|
||||
Return the user's custom (non-required) roles.
|
||||
Falls back to config-defined roles minus required ones for migration.
|
||||
"""
|
||||
registry = _load(username)
|
||||
if "custom_roles" in registry:
|
||||
return [r for r in registry["custom_roles"] if r and r not in REQUIRED_ROLES]
|
||||
from config import settings as _cfg
|
||||
return [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
|
||||
|
||||
|
||||
def get_all_roles(username: str) -> list[str]:
|
||||
"""Return required roles followed by the user's custom roles."""
|
||||
return list(REQUIRED_ROLES) + get_custom_roles(username)
|
||||
|
||||
|
||||
def add_custom_role(username: str, role_name: str) -> bool:
|
||||
"""Add a custom role. Returns False if the name is invalid or already a required role."""
|
||||
role_name = role_name.strip().lower()
|
||||
if not role_name or role_name in REQUIRED_ROLES:
|
||||
return False
|
||||
data = _load(username)
|
||||
if "custom_roles" not in data:
|
||||
from config import settings as _cfg
|
||||
data["custom_roles"] = [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
|
||||
if role_name not in data["custom_roles"]:
|
||||
data["custom_roles"].append(role_name)
|
||||
_save(username, data)
|
||||
return True
|
||||
|
||||
|
||||
def remove_custom_role(username: str, role_name: str) -> bool:
|
||||
"""Remove a custom role. Required roles cannot be removed."""
|
||||
if role_name in REQUIRED_ROLES:
|
||||
return False
|
||||
data = _load(username)
|
||||
if "custom_roles" not in data:
|
||||
from config import settings as _cfg
|
||||
data["custom_roles"] = [r for r in _cfg.get_defined_roles() if r not in REQUIRED_ROLES]
|
||||
if role_name in data["custom_roles"]:
|
||||
data["custom_roles"].remove(role_name)
|
||||
_save(username, data)
|
||||
return True
|
||||
|
||||
|
||||
def set_role(username: str, role: str, priority: str, model_id: str | None) -> bool:
|
||||
"""
|
||||
Assign a model to a role priority slot.
|
||||
|
||||
priority must be one of: primary, backup_1, backup_2, backup_3, backup_4
|
||||
model_id None clears the slot.
|
||||
Built-in IDs (claude_cli, gemini_cli, gemini_api) are always valid.
|
||||
"""
|
||||
if priority not in PRIORITY_KEYS:
|
||||
return False
|
||||
|
||||
data = _load(username)
|
||||
|
||||
if model_id and model_id not in _builtins():
|
||||
if not any(m["id"] == model_id for m in data["models"]):
|
||||
return False
|
||||
|
||||
roles = data.setdefault("roles", {})
|
||||
if role not in roles:
|
||||
roles[role] = {}
|
||||
roles[role][priority] = model_id or None
|
||||
|
||||
_save(username, data)
|
||||
return True
|
||||
|
||||
|
||||
# ── Utility ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def fetch_models_from_host(api_url: str, api_key: str,
|
||||
host_type: str = "openwebui") -> list[str]:
|
||||
"""Synchronously fetch the model list from an OpenAI-compatible host."""
|
||||
import httpx
|
||||
path = "/api/models" if host_type == "openwebui" else "/models"
|
||||
url = api_url.rstrip("/") + path
|
||||
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
|
||||
resp = httpx.get(url, headers=headers, timeout=10)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
models = data.get("data", [])
|
||||
return sorted(m.get("id", m.get("name", "")) for m in models if m.get("id") or m.get("name"))
|
||||
176
cortex/notification.py
Normal file
176
cortex/notification.py
Normal file
@@ -0,0 +1,176 @@
|
||||
"""
|
||||
Outbound notification helpers — send messages to user channels proactively.
|
||||
|
||||
Channel config lives in home/{user}/channels.json:
|
||||
{
|
||||
"notification_channel": "email" | "nextcloud" | "google_chat",
|
||||
"notification_email": "<override address — defaults to login email>",
|
||||
"nextcloud": {
|
||||
"url": "...", "bot_secret": "...", "notification_room": "<token>", ...
|
||||
},
|
||||
"google_chat": {
|
||||
"outbound_webhook": "https://chat.googleapis.com/v1/spaces/...", ...
|
||||
}
|
||||
}
|
||||
|
||||
If notification_channel is absent, defaults to "nextcloud" if configured.
|
||||
"""
|
||||
import asyncio
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def _send_nct_message(url: str, secret: str, room: str, message: str) -> None:
|
||||
"""Post a message to a Nextcloud Talk room as the bot."""
|
||||
endpoint = f"{url}/ocs/v2.php/apps/spreed/api/v1/bot/{room}/message"
|
||||
random_str = secrets.token_hex(32)
|
||||
sig = hmac.new(
|
||||
secret.encode(),
|
||||
(random_str + message).encode("utf-8"),
|
||||
hashlib.sha256,
|
||||
).hexdigest()
|
||||
body = json.dumps({"message": message}, ensure_ascii=False).encode("utf-8")
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.post(
|
||||
endpoint,
|
||||
content=body,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"OCS-APIRequest": "true",
|
||||
"X-Nextcloud-Talk-Bot-Random": random_str,
|
||||
"X-Nextcloud-Talk-Bot-Signature": sig,
|
||||
},
|
||||
timeout=15,
|
||||
)
|
||||
if resp.status_code not in (200, 201):
|
||||
logger.warning("notify NCT %s → HTTP %d: %s", room, resp.status_code, resp.text[:200])
|
||||
else:
|
||||
logger.info("notify NCT → %s (%d chars)", room, len(message))
|
||||
except Exception as e:
|
||||
logger.error("notify NCT error: %s", e)
|
||||
|
||||
|
||||
async def _notify_nct(nct: dict, message: str, username: str) -> None:
|
||||
room = nct.get("notification_room", "").strip()
|
||||
url = nct.get("url", "").rstrip("/")
|
||||
secret = nct.get("bot_secret", "")
|
||||
if not room:
|
||||
logger.debug("notify: NCT notification_room not set for %s — skipping", username)
|
||||
return
|
||||
if not url or not secret:
|
||||
logger.warning("notify: NCT config incomplete for %s (missing url or secret)", username)
|
||||
return
|
||||
await _send_nct_message(url, secret, room, message)
|
||||
|
||||
|
||||
async def _notify_email(username: str, message: str, email_override: str | None = None) -> None:
|
||||
"""Send notification via email. Address = override → google_email from auth.json."""
|
||||
from auth_utils import _read_auth
|
||||
from email_utils import send_email
|
||||
|
||||
to_addr = email_override or _read_auth(username).get("google_email", "").strip()
|
||||
if not to_addr:
|
||||
logger.warning("notify: no email address for %s — set notification_email in channels.json", username)
|
||||
return
|
||||
|
||||
ok = await asyncio.to_thread(
|
||||
send_email,
|
||||
to_email=to_addr,
|
||||
subject="Cortex",
|
||||
body_text=message,
|
||||
body_html=message.replace("\n", "<br>"),
|
||||
)
|
||||
if ok:
|
||||
logger.info("notify email → %s (%d chars)", to_addr, len(message))
|
||||
else:
|
||||
logger.warning("notify: email send failed for %s", username)
|
||||
|
||||
|
||||
async def _notify_google_chat(webhook_url: str, message: str, username: str) -> None:
|
||||
"""POST a message to a Google Chat space via incoming webhook."""
|
||||
body = json.dumps({"text": message}, ensure_ascii=False).encode("utf-8")
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.post(
|
||||
webhook_url,
|
||||
content=body,
|
||||
headers={"Content-Type": "application/json"},
|
||||
timeout=15,
|
||||
)
|
||||
if resp.status_code not in (200, 201):
|
||||
logger.warning("notify Google Chat %s → HTTP %d: %s", username, resp.status_code, resp.text[:200])
|
||||
else:
|
||||
logger.info("notify Google Chat → %s (%d chars)", username, len(message))
|
||||
except Exception as e:
|
||||
logger.error("notify Google Chat error for %s: %s", username, e)
|
||||
|
||||
|
||||
async def _notify_web_push(username: str, message: str) -> None:
|
||||
"""Send a browser push notification."""
|
||||
import push_utils
|
||||
result = await push_utils.send_push(username, "Cortex", message, "")
|
||||
if "error" in result:
|
||||
logger.warning("notify web_push error for %s: %s", username, result["error"])
|
||||
elif result.get("sent", 0) == 0:
|
||||
logger.debug("notify web_push: no subscriptions for %s", username)
|
||||
else:
|
||||
logger.info("notify web_push → %s (%d device(s))", username, result["sent"])
|
||||
|
||||
|
||||
async def notify(username: str, message: str, channel: str | None = None) -> None:
|
||||
"""Send a notification to the user's preferred outbound channel.
|
||||
|
||||
Channel resolution order:
|
||||
1. `channel` parameter if provided
|
||||
2. `notification_channel` key in channels.json
|
||||
3. "nextcloud" if notification_room is configured
|
||||
4. Silent no-op
|
||||
|
||||
Supported channels: "web_push", "email", "nextcloud", "google_chat"
|
||||
Configure via home/{user}/channels.json — see module docstring.
|
||||
"""
|
||||
from auth_utils import get_user_channels
|
||||
channels = get_user_channels(username)
|
||||
|
||||
target = channel or channels.get("notification_channel", "").strip()
|
||||
if not target:
|
||||
# Auto-detect: nextcloud if a notification_room is set
|
||||
nct = channels.get("nextcloud", {})
|
||||
if nct.get("notification_room", "").strip():
|
||||
target = "nextcloud"
|
||||
else:
|
||||
return
|
||||
|
||||
if target == "web_push":
|
||||
await _notify_web_push(username, message)
|
||||
|
||||
elif target == "email":
|
||||
email_override = channels.get("notification_email", "").strip() or None
|
||||
await _notify_email(username, message, email_override=email_override)
|
||||
|
||||
elif target == "nextcloud":
|
||||
nct = channels.get("nextcloud")
|
||||
if not nct:
|
||||
logger.debug("notify: nextcloud not configured for %s", username)
|
||||
return
|
||||
await _notify_nct(nct, message, username)
|
||||
|
||||
elif target == "google_chat":
|
||||
gc = channels.get("google_chat", {})
|
||||
webhook = gc.get("outbound_webhook", "").strip()
|
||||
if not webhook:
|
||||
logger.debug("notify: google_chat outbound_webhook not set for %s", username)
|
||||
return
|
||||
await _notify_google_chat(webhook, message, username)
|
||||
|
||||
else:
|
||||
logger.debug("notify: channel %r not supported for outbound (user %s)", target, username)
|
||||
531
cortex/openai_orchestrator.py
Normal file
531
cortex/openai_orchestrator.py
Normal file
@@ -0,0 +1,531 @@
|
||||
"""
|
||||
OpenAI-compatible orchestrator engine.
|
||||
|
||||
Implements the same ReAct tool loop as orchestrator_engine.py but uses the
|
||||
OpenAI tool calling format, which works with any OpenAI-compatible endpoint:
|
||||
OpenRouter, LiteLLM, Open WebUI, Ollama (tool-capable models), etc.
|
||||
|
||||
The model both runs the tool loop AND writes the final user-facing response —
|
||||
no separate handoff step needed when a single capable model handles everything.
|
||||
|
||||
Flow:
|
||||
1. POST to {api_url}/chat/completions with tools + user message
|
||||
2. If finish_reason == "tool_calls": execute tools, feed results back, repeat
|
||||
3. If finish_reason == "stop": final assistant message is the user-facing response
|
||||
|
||||
Used when the "orchestrator" role in the model registry resolves to a local_openai
|
||||
type model. The Gemini engine (orchestrator_engine.py) is used otherwise.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
|
||||
from openai import AsyncOpenAI, APIConnectionError, APIStatusError
|
||||
|
||||
from config import settings
|
||||
from orchestrator_engine import OrchestrateCheckpoint, OrchestratorResult
|
||||
from tools import OPENAI_TOOL_SCHEMAS, call_tool, get_openai_tools_for_role, get_tools_for_role, CONFIRM_REQUIRED, narrow_tools_by_keywords
|
||||
import tool_audit
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Appended to the persona system prompt so the model knows it has tools.
|
||||
# Kept brief — capable models handle tool use without much coaching.
|
||||
_TOOL_INSTRUCTION = (
|
||||
"\n\nYou have access to tools. Use them when you need current information, "
|
||||
"need to read files, or need to take actions on the user's behalf. "
|
||||
"Respond naturally after gathering what you need."
|
||||
)
|
||||
|
||||
|
||||
async def run(
|
||||
task: str,
|
||||
system_prompt: str = "",
|
||||
session_messages: list[dict] | None = None,
|
||||
model_cfg: dict | None = None,
|
||||
respond_with_final: bool = True,
|
||||
user_role: str = "user",
|
||||
tool_list: list[str] | None = None,
|
||||
confirm_allow: set[str] | None = None,
|
||||
confirm_deny: set[str] | None = None,
|
||||
max_risk: str | None = None,
|
||||
risk_whitelist: list[str] | None = None,
|
||||
risk_blacklist: list[str] | None = None,
|
||||
) -> OrchestratorResult:
|
||||
"""
|
||||
Run a tool-enabled task using an OpenAI-compatible API.
|
||||
|
||||
Args:
|
||||
task: The user's request (plain text)
|
||||
system_prompt: Persona system prompt from context_loader (passed through)
|
||||
session_messages: Recent conversation history for session continuity
|
||||
model_cfg: Resolved model config from model_registry (local_openai type)
|
||||
respond_with_final: If False, return just the tool-loop summary without a
|
||||
full persona-voiced response (faster; for cron/background)
|
||||
confirm_allow: Tools to bypass the confirmation gate for this user
|
||||
confirm_deny: Tools to always block for this user
|
||||
|
||||
Returns:
|
||||
OrchestratorResult — if checkpoint is set, the job is awaiting confirmation
|
||||
"""
|
||||
if not model_cfg:
|
||||
raise RuntimeError("model_cfg is required for the OpenAI orchestrator")
|
||||
|
||||
_confirm_allow = frozenset(confirm_allow or ())
|
||||
_confirm_deny = frozenset(confirm_deny or ())
|
||||
effective_confirm = (CONFIRM_REQUIRED - set(_confirm_allow)) | set(_confirm_deny)
|
||||
|
||||
# Keyword routing: narrow schemas to only what this message needs.
|
||||
# Also scans the last assistant turn so follow-ups like "yes, do that" inherit tool context.
|
||||
# Returns [] when no keywords match (zero tool overhead — model responds as plain chat).
|
||||
effective_tool_list = narrow_tools_by_keywords(task, tool_list, context_messages=session_messages)
|
||||
logger.info(
|
||||
"Keyword routing: %d tools active (role_tools=%s)",
|
||||
len(effective_tool_list),
|
||||
len(tool_list) if tool_list is not None else "all",
|
||||
)
|
||||
|
||||
client, model_name, active_tools = _build_client(
|
||||
model_cfg, user_role, effective_tool_list,
|
||||
max_risk=max_risk, risk_whitelist=risk_whitelist, risk_blacklist=risk_blacklist,
|
||||
)
|
||||
tool_audit.set_context("openai", model_cfg.get("label") or model_name)
|
||||
|
||||
sys_content = (system_prompt or "") + _TOOL_INSTRUCTION
|
||||
messages: list[dict] = [{"role": "system", "content": sys_content}]
|
||||
if session_messages:
|
||||
messages.extend(
|
||||
{"role": m["role"], "content": m["content"]}
|
||||
for m in session_messages[-6:]
|
||||
)
|
||||
messages.append({"role": "user", "content": task})
|
||||
|
||||
tool_call_log: list[dict] = []
|
||||
|
||||
final_response, checkpoint = await _run_from_messages(
|
||||
client=client,
|
||||
messages=messages,
|
||||
active_tools=active_tools,
|
||||
tool_call_log=tool_call_log,
|
||||
effective_confirm=effective_confirm,
|
||||
model_name=model_name,
|
||||
task=task,
|
||||
model_cfg=model_cfg,
|
||||
respond_with_final=respond_with_final,
|
||||
user_role=user_role,
|
||||
tool_list=effective_tool_list,
|
||||
confirm_allow=_confirm_allow,
|
||||
confirm_deny=_confirm_deny,
|
||||
starting_round=0,
|
||||
)
|
||||
|
||||
if checkpoint:
|
||||
return OrchestratorResult(
|
||||
response=final_response,
|
||||
tool_calls=list(tool_call_log),
|
||||
backend="local",
|
||||
gemini_summary=final_response,
|
||||
checkpoint=checkpoint,
|
||||
)
|
||||
|
||||
model_label = model_cfg.get("label") or model_name
|
||||
logger.info("OpenAI orchestrator complete — model=%s tools=%d", model_label, len(tool_call_log))
|
||||
return OrchestratorResult(
|
||||
response=final_response,
|
||||
tool_calls=tool_call_log,
|
||||
backend="local",
|
||||
backend_label=model_label,
|
||||
gemini_summary=final_response,
|
||||
)
|
||||
|
||||
|
||||
async def resume(checkpoint: OrchestrateCheckpoint, confirmed: bool) -> OrchestratorResult:
|
||||
"""Continue an OpenAI orchestrator job that was paused at a confirmation gate."""
|
||||
client, model_name, active_tools = _build_client(checkpoint.model_cfg, checkpoint.user_role, checkpoint.tool_list)
|
||||
|
||||
effective_confirm = (CONFIRM_REQUIRED - set(checkpoint.confirm_allow)) | set(checkpoint.confirm_deny)
|
||||
|
||||
messages = list(checkpoint.pre_fn_state)
|
||||
tool_call_log = [t for t in checkpoint.tool_call_log if t["result"] != "[awaiting confirmation]"]
|
||||
|
||||
# Build tool responses for this round
|
||||
for er in checkpoint.executed_results:
|
||||
messages.append({
|
||||
"role": "tool",
|
||||
"tool_call_id": er.get("tool_call_id", er["name"]),
|
||||
"content": er["result"],
|
||||
})
|
||||
|
||||
for pt in checkpoint.pending_tools:
|
||||
if confirmed:
|
||||
result_str = await _execute_tool_dict(pt["name"], pt["args"], checkpoint.user_role, checkpoint.tool_list)
|
||||
logger.info("Confirmed tool %s → %d chars", pt["name"], len(result_str))
|
||||
else:
|
||||
result_str = "Action denied by user."
|
||||
logger.info("Tool %s denied by user", pt["name"])
|
||||
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": result_str})
|
||||
messages.append({
|
||||
"role": "tool",
|
||||
"tool_call_id": pt.get("tool_call_id", pt["name"]),
|
||||
"content": result_str,
|
||||
})
|
||||
|
||||
final_response, new_checkpoint = await _run_from_messages(
|
||||
client=client,
|
||||
messages=messages,
|
||||
active_tools=active_tools,
|
||||
tool_call_log=tool_call_log,
|
||||
effective_confirm=effective_confirm,
|
||||
model_name=model_name,
|
||||
task=checkpoint.task,
|
||||
model_cfg=checkpoint.model_cfg,
|
||||
respond_with_final=checkpoint.respond_with_final,
|
||||
user_role=checkpoint.user_role,
|
||||
tool_list=checkpoint.tool_list,
|
||||
confirm_allow=checkpoint.confirm_allow,
|
||||
confirm_deny=checkpoint.confirm_deny,
|
||||
starting_round=checkpoint.rounds_used,
|
||||
)
|
||||
|
||||
if new_checkpoint:
|
||||
return OrchestratorResult(
|
||||
response=final_response,
|
||||
tool_calls=list(tool_call_log),
|
||||
backend="local",
|
||||
gemini_summary=final_response,
|
||||
checkpoint=new_checkpoint,
|
||||
)
|
||||
|
||||
model_label = (checkpoint.model_cfg or {}).get("label") or model_name
|
||||
logger.info("OpenAI orchestrator resumed — model=%s tools=%d", model_label, len(tool_call_log))
|
||||
return OrchestratorResult(
|
||||
response=final_response,
|
||||
tool_calls=tool_call_log,
|
||||
backend="local",
|
||||
gemini_summary=final_response,
|
||||
)
|
||||
|
||||
|
||||
_CHARS_PER_TOKEN = 4
|
||||
# Fixed token overhead budget per call (tool schemas excluded — cached separately)
|
||||
_TOOL_SCHEMA_OVERHEAD = 500
|
||||
# Chars to keep per truncated old tool result
|
||||
_TRUNC_RESULT_CHARS = 400
|
||||
# Always keep the last N tool-result messages uncompacted
|
||||
_KEEP_RECENT_TOOL_MSGS = 6 # ~2 rounds of 3 tools each
|
||||
|
||||
# Module-level schema cache: key = (user_role, sorted_tools, risk_params)
|
||||
# Bounded in practice — keyword routing produces at most ~30 distinct tool sets.
|
||||
_tool_schema_cache: dict[str, list[dict]] = {}
|
||||
|
||||
|
||||
def _get_cached_tools(
|
||||
user_role: str,
|
||||
tool_list: list[str] | None,
|
||||
max_risk: str | None = None,
|
||||
whitelist: list[str] | None = None,
|
||||
blacklist: list[str] | None = None,
|
||||
) -> list[dict]:
|
||||
key = "|".join([
|
||||
user_role,
|
||||
str(sorted(tool_list) if tool_list is not None else "all"),
|
||||
str(max_risk),
|
||||
str(sorted(whitelist) if whitelist else ""),
|
||||
str(sorted(blacklist) if blacklist else ""),
|
||||
])
|
||||
if key not in _tool_schema_cache:
|
||||
_tool_schema_cache[key] = get_openai_tools_for_role(
|
||||
user_role, tool_list,
|
||||
max_risk=max_risk, whitelist=whitelist, blacklist=blacklist,
|
||||
)
|
||||
return _tool_schema_cache[key]
|
||||
|
||||
|
||||
def _estimate_tokens(messages: list[dict]) -> int:
|
||||
total = sum(len(json.dumps(m)) for m in messages)
|
||||
return total // _CHARS_PER_TOKEN + _TOOL_SCHEMA_OVERHEAD
|
||||
|
||||
|
||||
def _compact_messages(messages: list[dict], budget_tokens: int) -> list[dict]:
|
||||
"""
|
||||
Truncate old tool result content when approaching the context budget.
|
||||
|
||||
Strategy: keep system message, recent assistant/tool rounds, and the
|
||||
original user task intact. Truncate content of old tool results in the
|
||||
middle of the conversation — the model only needs recent results to reason.
|
||||
"""
|
||||
if _estimate_tokens(messages) <= budget_tokens:
|
||||
return messages
|
||||
|
||||
tool_indices = [i for i, m in enumerate(messages) if m.get("role") == "tool"]
|
||||
n_to_compact = max(0, len(tool_indices) - _KEEP_RECENT_TOOL_MSGS)
|
||||
if n_to_compact == 0:
|
||||
return messages # nothing old enough to trim
|
||||
|
||||
compact_set = set(tool_indices[:n_to_compact])
|
||||
result = []
|
||||
chars_saved = 0
|
||||
for i, msg in enumerate(messages):
|
||||
if i in compact_set:
|
||||
content = msg.get("content", "")
|
||||
if len(content) > _TRUNC_RESULT_CHARS:
|
||||
msg = dict(msg)
|
||||
saved = len(content) - _TRUNC_RESULT_CHARS
|
||||
chars_saved += saved
|
||||
msg["content"] = (
|
||||
content[:_TRUNC_RESULT_CHARS]
|
||||
+ f" …[{len(content) - _TRUNC_RESULT_CHARS} chars omitted]"
|
||||
)
|
||||
result.append(msg)
|
||||
|
||||
new_est = _estimate_tokens(result)
|
||||
logger.info(
|
||||
"context compaction: saved ~%d tokens (%d chars), now ~%d / %d tokens",
|
||||
chars_saved // _CHARS_PER_TOKEN, chars_saved, new_est, budget_tokens,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _context_budget(model_cfg: dict | None) -> int:
|
||||
"""Return the usable token budget (75% of context window, min 16k, default 32k)."""
|
||||
context_k = (model_cfg or {}).get("context_k") or 32
|
||||
return max(16_000, int(context_k * 1000 * 0.75))
|
||||
|
||||
|
||||
async def _run_from_messages(
|
||||
client,
|
||||
messages: list[dict],
|
||||
active_tools: list,
|
||||
tool_call_log: list[dict],
|
||||
effective_confirm: set[str],
|
||||
model_name: str,
|
||||
task: str,
|
||||
model_cfg: dict | None,
|
||||
respond_with_final: bool,
|
||||
user_role: str,
|
||||
confirm_allow: frozenset,
|
||||
confirm_deny: frozenset,
|
||||
starting_round: int = 0,
|
||||
tool_list: list[str] | None = None,
|
||||
) -> tuple[str, OrchestrateCheckpoint | None]:
|
||||
"""
|
||||
Run the OpenAI ReAct loop from the current messages state.
|
||||
Returns (final_response, checkpoint) — checkpoint is set if confirmation is needed.
|
||||
"""
|
||||
final_response = ""
|
||||
budget = _context_budget(model_cfg)
|
||||
|
||||
per_model_limit = (model_cfg or {}).get("max_rounds") or settings.orchestrator_max_rounds
|
||||
effective_limit = min(per_model_limit, settings.orchestrator_max_rounds)
|
||||
|
||||
for round_num in range(starting_round, effective_limit):
|
||||
messages = _compact_messages(messages, budget)
|
||||
est = _estimate_tokens(messages)
|
||||
logger.info("OpenAI orchestrator round %d / %d model=%s ~%d tokens",
|
||||
round_num + 1, effective_limit, model_name, est)
|
||||
|
||||
call_kwargs: dict = {"model": model_name, "messages": messages}
|
||||
if active_tools:
|
||||
call_kwargs["tools"] = active_tools
|
||||
call_kwargs["tool_choice"] = "auto"
|
||||
reasoning_budget = (model_cfg or {}).get("reasoning_budget_tokens")
|
||||
if reasoning_budget:
|
||||
call_kwargs["extra_body"] = {"reasoning": {"budget_tokens": reasoning_budget}}
|
||||
response = await _chat_with_retry(client, **call_kwargs)
|
||||
|
||||
choice = response.choices[0]
|
||||
msg = choice.message
|
||||
|
||||
assistant_msg: dict = {"role": "assistant"}
|
||||
if msg.content:
|
||||
assistant_msg["content"] = msg.content
|
||||
if msg.tool_calls:
|
||||
assistant_msg["tool_calls"] = [
|
||||
{
|
||||
"id": tc.id,
|
||||
"type": "function",
|
||||
"function": {"name": tc.function.name, "arguments": tc.function.arguments},
|
||||
}
|
||||
for tc in msg.tool_calls
|
||||
]
|
||||
messages.append(assistant_msg)
|
||||
|
||||
# Some models set finish_reason="stop" even when tool_calls are present
|
||||
if msg.tool_calls and (choice.finish_reason in ("tool_calls", "stop", None)):
|
||||
# Snapshot state before tool responses for potential checkpoint
|
||||
pre_fn_state = list(messages)
|
||||
|
||||
pending_tools: list[dict] = []
|
||||
executed_results: list[dict] = []
|
||||
|
||||
for tc in msg.tool_calls:
|
||||
name = tc.function.name
|
||||
raw_args = tc.function.arguments or "{}"
|
||||
try:
|
||||
args_parsed = json.loads(raw_args)
|
||||
if not isinstance(args_parsed, dict):
|
||||
raise ValueError("args must be a JSON object")
|
||||
except (json.JSONDecodeError, ValueError) as e:
|
||||
logger.warning("Malformed tool args for %s: %s — args: %.200s", name, e, raw_args)
|
||||
args_parsed = {}
|
||||
|
||||
if name in effective_confirm:
|
||||
pending_tools.append({"name": name, "args": args_parsed, "tool_call_id": tc.id})
|
||||
logger.info("Tool %s blocked — confirmation required", name)
|
||||
else:
|
||||
result_str = await _execute_tool(name, tc.function.arguments, user_role, tool_list)
|
||||
logger.info("Tool %s → %d chars", name, len(result_str))
|
||||
executed_results.append({"name": name, "args": args_parsed, "result": result_str, "tool_call_id": tc.id})
|
||||
tool_call_log.append({"tool": name, "args": args_parsed, "result": result_str})
|
||||
messages.append({"role": "tool", "tool_call_id": tc.id, "content": result_str})
|
||||
|
||||
if pending_tools:
|
||||
# Add placeholder responses
|
||||
for pt in pending_tools:
|
||||
placeholder = f"[AWAITING USER CONFIRMATION for {pt['name']}]"
|
||||
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": "[awaiting confirmation]"})
|
||||
messages.append({"role": "tool", "tool_call_id": pt["tool_call_id"], "content": placeholder})
|
||||
|
||||
messages = _compact_messages(messages, budget)
|
||||
conf_call: dict = {"model": model_name, "messages": messages, "tool_choice": "none"}
|
||||
if active_tools:
|
||||
conf_call["tools"] = active_tools
|
||||
if reasoning_budget:
|
||||
conf_call["extra_body"] = {"reasoning": {"budget_tokens": reasoning_budget}}
|
||||
conf_resp = await _chat_with_retry(client, **conf_call)
|
||||
final_response = conf_resp.choices[0].message.content or (
|
||||
"This action requires your explicit confirmation before it can proceed."
|
||||
)
|
||||
|
||||
checkpoint = OrchestrateCheckpoint(
|
||||
engine="openai",
|
||||
pre_fn_state=pre_fn_state,
|
||||
executed_results=executed_results,
|
||||
pending_tools=pending_tools,
|
||||
tool_call_log=list(tool_call_log),
|
||||
task=task,
|
||||
model_cfg=model_cfg,
|
||||
respond_with_final=respond_with_final,
|
||||
user_role=user_role,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=confirm_allow,
|
||||
confirm_deny=confirm_deny,
|
||||
rounds_used=round_num + 2,
|
||||
)
|
||||
return final_response, checkpoint
|
||||
|
||||
else:
|
||||
final_response = msg.content or ""
|
||||
logger.info(
|
||||
"OpenAI orchestrator done after %d round(s). Tools used: %d",
|
||||
round_num + 1, len(tool_call_log),
|
||||
)
|
||||
break
|
||||
|
||||
else:
|
||||
logger.warning("OpenAI orchestrator hit max rounds (%d)", effective_limit)
|
||||
final_response = (
|
||||
f"Reached the tool iteration limit ({effective_limit} rounds). "
|
||||
"Here is what was gathered:\n\n"
|
||||
+ "\n\n".join(f"**{t['tool']}**: {t['result'][:500]}" for t in tool_call_log)
|
||||
)
|
||||
|
||||
return final_response, None
|
||||
|
||||
|
||||
_RETRY_STATUSES = {429, 500, 502, 503, 504}
|
||||
_MAX_API_RETRIES = 3
|
||||
|
||||
|
||||
async def _chat_with_retry(client, **kwargs):
|
||||
"""Wrap chat.completions.create with exponential backoff on transient errors."""
|
||||
last_exc: Exception = RuntimeError("No attempts made")
|
||||
for attempt in range(_MAX_API_RETRIES):
|
||||
try:
|
||||
return await client.chat.completions.create(**kwargs)
|
||||
except APIConnectionError as e:
|
||||
last_exc = e
|
||||
logger.warning("OpenAI connection error (attempt %d/%d): %s", attempt + 1, _MAX_API_RETRIES, e)
|
||||
except APIStatusError as e:
|
||||
if e.status_code in _RETRY_STATUSES:
|
||||
last_exc = e
|
||||
logger.warning("OpenAI status %d (attempt %d/%d): %s", e.status_code, attempt + 1, _MAX_API_RETRIES, e)
|
||||
else:
|
||||
raise
|
||||
if attempt < _MAX_API_RETRIES - 1:
|
||||
await asyncio.sleep(2 ** attempt) # 1s, 2s
|
||||
raise last_exc
|
||||
|
||||
|
||||
def _build_client(
|
||||
model_cfg: dict | None,
|
||||
user_role: str = "user",
|
||||
tool_list: list[str] | None = None,
|
||||
max_risk: str | None = None,
|
||||
risk_whitelist: list[str] | None = None,
|
||||
risk_blacklist: list[str] | None = None,
|
||||
) -> tuple:
|
||||
"""Build AsyncOpenAI client and return (client, model_name, active_tools)."""
|
||||
if not model_cfg:
|
||||
raise RuntimeError("model_cfg is required for the OpenAI orchestrator")
|
||||
api_url = model_cfg.get("api_url", "")
|
||||
api_key = model_cfg.get("api_key", "") or "none"
|
||||
model_name = model_cfg.get("model_name", "")
|
||||
host_type = model_cfg.get("host_type", "openwebui")
|
||||
if not api_url or not model_name:
|
||||
raise RuntimeError(
|
||||
f"model_cfg missing api_url or model_name: {model_cfg.get('label', model_cfg)}"
|
||||
)
|
||||
base_url = api_url.rstrip("/")
|
||||
if host_type == "openwebui":
|
||||
base_url = base_url + "/api"
|
||||
client = AsyncOpenAI(base_url=base_url, api_key=api_key, timeout=settings.timeout_local)
|
||||
if model_cfg.get("tools") is False:
|
||||
active_tools = []
|
||||
else:
|
||||
active_tools = _get_cached_tools(
|
||||
user_role, tool_list,
|
||||
max_risk=max_risk, whitelist=risk_whitelist, blacklist=risk_blacklist,
|
||||
)
|
||||
return client, model_name, active_tools
|
||||
|
||||
|
||||
async def _execute_tool(
|
||||
name: str,
|
||||
arguments_json: str,
|
||||
user_role: str = "user",
|
||||
tool_list: list[str] | None = None,
|
||||
max_risk: str | None = None,
|
||||
risk_whitelist: list[str] | None = None,
|
||||
risk_blacklist: list[str] | None = None,
|
||||
) -> str:
|
||||
"""Parse tool arguments and execute with role-filtered callables."""
|
||||
_, callables = get_tools_for_role(
|
||||
user_role, tool_list,
|
||||
max_risk=max_risk, whitelist=risk_whitelist, blacklist=risk_blacklist,
|
||||
)
|
||||
try:
|
||||
args = json.loads(arguments_json)
|
||||
except json.JSONDecodeError:
|
||||
args = {}
|
||||
try:
|
||||
return await call_tool(name, args, callables)
|
||||
except Exception as e:
|
||||
logger.warning("Tool %s failed: %s", name, e)
|
||||
return f"Tool error: {e}"
|
||||
|
||||
|
||||
async def _execute_tool_dict(
|
||||
name: str,
|
||||
args: dict,
|
||||
user_role: str = "user",
|
||||
tool_list: list[str] | None = None,
|
||||
) -> str:
|
||||
"""Execute a tool from a pre-parsed args dict."""
|
||||
_, callables = get_tools_for_role(user_role, tool_list)
|
||||
try:
|
||||
return await call_tool(name, args, callables)
|
||||
except Exception as e:
|
||||
logger.warning("Tool %s failed: %s", name, e)
|
||||
return f"Tool error: {e}"
|
||||
@@ -16,6 +16,7 @@ calls llm_client.complete() directly, which is faster and has no orchestration o
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
@@ -24,7 +25,10 @@ from google.genai import types
|
||||
|
||||
from config import settings
|
||||
from llm_client import complete
|
||||
from tools import TOOL_DECLARATIONS, call_tool
|
||||
from tools import TOOL_DECLARATIONS, call_tool, get_tools_for_role, CONFIRM_REQUIRED
|
||||
import usage_tracker
|
||||
import tool_audit
|
||||
from persona import _user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -43,12 +47,61 @@ Keep your summary factual and complete. Include relevant URLs, data, and specifi
|
||||
If no tools are needed, return an empty summary."""
|
||||
|
||||
|
||||
def _track_gemini_usage(response, model_name: str | None) -> None:
|
||||
meta = getattr(response, "usage_metadata", None)
|
||||
if not meta:
|
||||
return
|
||||
prompt_tokens = getattr(meta, "prompt_token_count", 0) or 0
|
||||
completion_tokens = getattr(meta, "candidates_token_count", 0) or 0
|
||||
if prompt_tokens or completion_tokens:
|
||||
try:
|
||||
asyncio.create_task(usage_tracker.record(
|
||||
username=_user.get(),
|
||||
backend="gemini_api",
|
||||
model_name=model_name or settings.orchestrator_model,
|
||||
prompt_tokens=prompt_tokens,
|
||||
completion_tokens=completion_tokens,
|
||||
))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
@dataclass
|
||||
class OrchestrateCheckpoint:
|
||||
"""Saved execution state for a job paused at a confirmation gate."""
|
||||
engine: str # "gemini" | "openai"
|
||||
pre_fn_state: list # conversation state before function responses
|
||||
executed_results: list[dict] # tools that already ran this round
|
||||
pending_tools: list[dict] # [{name, args}] awaiting confirmation
|
||||
tool_call_log: list[dict] # all tool calls so far
|
||||
task: str
|
||||
# Gemini-specific config (unused by openai engine)
|
||||
system_prompt: str = ""
|
||||
session_messages: list | None = None
|
||||
model_name: str | None = None
|
||||
gemini_api_key: str | None = None
|
||||
respond_with_claude: bool = True
|
||||
response_role: str = "chat"
|
||||
# OpenAI-specific config (unused by gemini engine)
|
||||
model_cfg: dict | None = None
|
||||
respond_with_final: bool = True
|
||||
# Common
|
||||
user_role: str = "user"
|
||||
tool_list: list[str] | None = None
|
||||
confirm_allow: frozenset = field(default_factory=frozenset)
|
||||
confirm_deny: frozenset = field(default_factory=frozenset)
|
||||
rounds_used: int = 0
|
||||
max_rounds: int | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class OrchestratorResult:
|
||||
response: str # final user-facing response (from Claude)
|
||||
tool_calls: list[dict] = field(default_factory=list) # [{tool, args, result}]
|
||||
backend: str = "claude" # model that produced the final response
|
||||
backend_label: str = "" # human-readable model label for display
|
||||
gemini_summary: str = "" # what Gemini handed to Claude (debug/display)
|
||||
checkpoint: OrchestrateCheckpoint | None = None # set when awaiting confirmation
|
||||
|
||||
|
||||
async def run(
|
||||
@@ -56,6 +109,17 @@ async def run(
|
||||
system_prompt: str = "",
|
||||
session_messages: list[dict] | None = None,
|
||||
respond_with_claude: bool = True,
|
||||
gemini_api_key: str | None = None,
|
||||
model_name: str | None = None,
|
||||
response_role: str = "chat",
|
||||
user_role: str = "user",
|
||||
tool_list: list[str] | None = None,
|
||||
confirm_allow: set[str] | None = None,
|
||||
confirm_deny: set[str] | None = None,
|
||||
max_rounds: int | None = None,
|
||||
max_risk: str | None = None,
|
||||
risk_whitelist: list[str] | None = None,
|
||||
risk_blacklist: list[str] | None = None,
|
||||
) -> OrchestratorResult:
|
||||
"""
|
||||
Run the full orchestration loop for a task.
|
||||
@@ -66,116 +130,331 @@ async def run(
|
||||
session_messages: Prior conversation history for session continuity
|
||||
respond_with_claude: If False, return Gemini's summary as the response (useful for
|
||||
background/cron tasks where a polished reply isn't needed)
|
||||
gemini_api_key: Per-user Gemini API key (falls back to GEMINI_API_KEY in .env)
|
||||
tool_list: Optional explicit tool allow-list from role config; intersected
|
||||
with user_role access-level filter (cannot elevate privileges)
|
||||
confirm_allow: Tools to bypass the confirmation gate for this user
|
||||
confirm_deny: Tools to always block for this user
|
||||
|
||||
Returns:
|
||||
OrchestratorResult with response, tool call log, backend used, and Gemini summary
|
||||
OrchestratorResult — if checkpoint is set, the job is awaiting confirmation
|
||||
"""
|
||||
if not settings.gemini_api_key:
|
||||
api_key = gemini_api_key or settings.gemini_api_key
|
||||
if not api_key:
|
||||
raise RuntimeError(
|
||||
"GEMINI_API_KEY not set — orchestrator requires Gemini API. "
|
||||
"Get a free key at https://aistudio.google.com/apikey and add it to .env"
|
||||
"No Gemini API key available — set GEMINI_API_KEY in .env or add a personal key "
|
||||
"via: manage_passwords.py gemini-key <username> <key>"
|
||||
)
|
||||
|
||||
client = genai.Client(api_key=settings.gemini_api_key)
|
||||
client = genai.Client(api_key=api_key)
|
||||
tool_audit.set_context("gemini", model_name or settings.orchestrator_model)
|
||||
|
||||
_confirm_allow = frozenset(confirm_allow or ())
|
||||
_confirm_deny = frozenset(confirm_deny or ())
|
||||
effective_confirm = (CONFIRM_REQUIRED - set(_confirm_allow)) | set(_confirm_deny)
|
||||
|
||||
# Seed Gemini with the task — include recent session context if available
|
||||
task_with_context = _build_task_prompt(task, session_messages)
|
||||
contents: list[types.Content] = [
|
||||
types.Content(role="user", parts=[types.Part(text=task_with_context)])
|
||||
]
|
||||
|
||||
tool_declarations, tool_callables = get_tools_for_role(
|
||||
user_role, tool_list, max_risk=max_risk,
|
||||
whitelist=risk_whitelist, blacklist=risk_blacklist,
|
||||
)
|
||||
tool_call_log: list[dict] = []
|
||||
gemini_summary = ""
|
||||
|
||||
# --- ReAct tool loop ---
|
||||
for round_num in range(settings.orchestrator_max_rounds):
|
||||
gemini_summary, checkpoint = await _run_from_contents(
|
||||
client=client,
|
||||
contents=contents,
|
||||
tool_declarations=tool_declarations,
|
||||
tool_callables=tool_callables,
|
||||
tool_call_log=tool_call_log,
|
||||
effective_confirm=effective_confirm,
|
||||
model_name=model_name,
|
||||
task=task,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_messages,
|
||||
respond_with_claude=respond_with_claude,
|
||||
response_role=response_role,
|
||||
user_role=user_role,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=_confirm_allow,
|
||||
confirm_deny=_confirm_deny,
|
||||
starting_round=0,
|
||||
gemini_api_key=api_key,
|
||||
max_rounds=max_rounds,
|
||||
)
|
||||
|
||||
if checkpoint:
|
||||
return OrchestratorResult(
|
||||
response=gemini_summary,
|
||||
tool_calls=list(tool_call_log),
|
||||
backend="gemini",
|
||||
gemini_summary=gemini_summary,
|
||||
checkpoint=checkpoint,
|
||||
)
|
||||
|
||||
return await _claude_handoff(
|
||||
task=task,
|
||||
tool_call_log=tool_call_log,
|
||||
gemini_summary=gemini_summary,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_messages,
|
||||
respond_with_claude=respond_with_claude,
|
||||
response_role=response_role,
|
||||
)
|
||||
|
||||
|
||||
async def resume(checkpoint: OrchestrateCheckpoint, confirmed: bool) -> OrchestratorResult:
|
||||
"""Continue a job that was paused at a confirmation gate."""
|
||||
api_key = checkpoint.gemini_api_key or settings.gemini_api_key
|
||||
client = genai.Client(api_key=api_key)
|
||||
tool_declarations, tool_callables = get_tools_for_role(
|
||||
checkpoint.user_role, checkpoint.tool_list,
|
||||
max_risk=getattr(checkpoint, "max_risk", None),
|
||||
whitelist=getattr(checkpoint, "risk_whitelist", None),
|
||||
blacklist=getattr(checkpoint, "risk_blacklist", None),
|
||||
)
|
||||
|
||||
effective_confirm = (CONFIRM_REQUIRED - set(checkpoint.confirm_allow)) | set(checkpoint.confirm_deny)
|
||||
|
||||
# Rebuild from saved state — strip "[awaiting confirmation]" placeholders
|
||||
contents = list(checkpoint.pre_fn_state)
|
||||
tool_call_log = [t for t in checkpoint.tool_call_log if t["result"] != "[awaiting confirmation]"]
|
||||
|
||||
# Build function responses for this round
|
||||
response_parts: list[types.Part] = []
|
||||
|
||||
for er in checkpoint.executed_results:
|
||||
response_parts.append(types.Part(function_response=types.FunctionResponse(
|
||||
name=er["name"], response={"result": er["result"]}
|
||||
)))
|
||||
|
||||
for pt in checkpoint.pending_tools:
|
||||
if confirmed:
|
||||
result_str = await _execute_tool(pt["name"], pt["args"], tool_callables)
|
||||
logger.info("Confirmed tool %s → %d chars", pt["name"], len(result_str))
|
||||
else:
|
||||
result_str = "Action denied by user."
|
||||
logger.info("Tool %s denied by user", pt["name"])
|
||||
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": result_str})
|
||||
response_parts.append(types.Part(function_response=types.FunctionResponse(
|
||||
name=pt["name"], response={"result": result_str}
|
||||
)))
|
||||
|
||||
contents.append(types.Content(role="user", parts=response_parts))
|
||||
|
||||
gemini_summary, new_checkpoint = await _run_from_contents(
|
||||
client=client,
|
||||
contents=contents,
|
||||
tool_declarations=tool_declarations,
|
||||
tool_callables=tool_callables,
|
||||
tool_call_log=tool_call_log,
|
||||
effective_confirm=effective_confirm,
|
||||
model_name=checkpoint.model_name,
|
||||
task=checkpoint.task,
|
||||
system_prompt=checkpoint.system_prompt,
|
||||
session_messages=checkpoint.session_messages,
|
||||
respond_with_claude=checkpoint.respond_with_claude,
|
||||
response_role=checkpoint.response_role,
|
||||
user_role=checkpoint.user_role,
|
||||
tool_list=checkpoint.tool_list,
|
||||
confirm_allow=checkpoint.confirm_allow,
|
||||
confirm_deny=checkpoint.confirm_deny,
|
||||
starting_round=checkpoint.rounds_used,
|
||||
gemini_api_key=api_key,
|
||||
max_rounds=checkpoint.max_rounds,
|
||||
)
|
||||
|
||||
if new_checkpoint:
|
||||
return OrchestratorResult(
|
||||
response=gemini_summary,
|
||||
tool_calls=list(tool_call_log),
|
||||
backend="gemini",
|
||||
gemini_summary=gemini_summary,
|
||||
checkpoint=new_checkpoint,
|
||||
)
|
||||
|
||||
return await _claude_handoff(
|
||||
task=checkpoint.task,
|
||||
tool_call_log=tool_call_log,
|
||||
gemini_summary=gemini_summary,
|
||||
system_prompt=checkpoint.system_prompt,
|
||||
session_messages=checkpoint.session_messages,
|
||||
respond_with_claude=checkpoint.respond_with_claude,
|
||||
response_role=checkpoint.response_role,
|
||||
)
|
||||
|
||||
|
||||
async def _run_from_contents(
|
||||
client,
|
||||
contents: list,
|
||||
tool_declarations: list,
|
||||
tool_callables: dict,
|
||||
tool_call_log: list[dict],
|
||||
effective_confirm: set[str],
|
||||
model_name: str | None,
|
||||
task: str,
|
||||
system_prompt: str,
|
||||
session_messages: list[dict] | None,
|
||||
respond_with_claude: bool,
|
||||
response_role: str,
|
||||
user_role: str,
|
||||
confirm_allow: frozenset,
|
||||
confirm_deny: frozenset,
|
||||
starting_round: int = 0,
|
||||
gemini_api_key: str | None = None,
|
||||
tool_list: list[str] | None = None,
|
||||
max_rounds: int | None = None,
|
||||
) -> tuple[str, OrchestrateCheckpoint | None]:
|
||||
"""
|
||||
Run the ReAct loop from the current contents state.
|
||||
Returns (gemini_summary, checkpoint) — checkpoint is set if confirmation is needed.
|
||||
"""
|
||||
gemini_summary = ""
|
||||
per_model_limit = max_rounds or settings.orchestrator_max_rounds
|
||||
effective_limit = min(per_model_limit, settings.orchestrator_max_rounds)
|
||||
|
||||
for round_num in range(starting_round, effective_limit):
|
||||
logger.info("Orchestrator round %d for task: %.80s", round_num + 1, task)
|
||||
|
||||
response = await asyncio.to_thread(
|
||||
client.models.generate_content,
|
||||
model=settings.orchestrator_model,
|
||||
model=model_name or settings.orchestrator_model,
|
||||
contents=contents,
|
||||
config=types.GenerateContentConfig(
|
||||
tools=TOOL_DECLARATIONS,
|
||||
tools=tool_declarations,
|
||||
system_instruction=_ORCHESTRATOR_SYSTEM,
|
||||
),
|
||||
)
|
||||
_track_gemini_usage(response, model_name)
|
||||
|
||||
candidate = response.candidates[0]
|
||||
parts = candidate.content.parts if candidate.content else []
|
||||
|
||||
# Check if Gemini wants to call any tools
|
||||
tool_call_parts = [
|
||||
p for p in parts
|
||||
if hasattr(p, "function_call") and p.function_call and p.function_call.name
|
||||
]
|
||||
|
||||
if not tool_call_parts:
|
||||
# No more tool calls — extract Gemini's text summary
|
||||
gemini_summary = "".join(
|
||||
p.text for p in parts if hasattr(p, "text") and p.text
|
||||
).strip()
|
||||
logger.info("Orchestrator done after %d round(s). Tools used: %d",
|
||||
round_num + 1, len(tool_call_log))
|
||||
break
|
||||
return gemini_summary, None
|
||||
|
||||
# Add Gemini's response (with function calls) to the conversation
|
||||
contents.append(candidate.content)
|
||||
|
||||
# Execute all tool calls in parallel
|
||||
tool_tasks = [
|
||||
_execute_tool(fc.function_call.name, dict(fc.function_call.args))
|
||||
for fc in tool_call_parts
|
||||
]
|
||||
tool_results = await asyncio.gather(*tool_tasks, return_exceptions=True)
|
||||
# Snapshot state before function responses — used if a checkpoint is needed
|
||||
pre_fn_state = list(contents)
|
||||
|
||||
# Build function response parts and update log
|
||||
response_parts: list[types.Part] = []
|
||||
for fc_part, result in zip(tool_call_parts, tool_results):
|
||||
fc = fc_part.function_call
|
||||
result_str = str(result) if not isinstance(result, Exception) else f"Error: {result}"
|
||||
logger.info("Tool %s → %d chars", fc.name, len(result_str))
|
||||
pending_tools: list[dict] = []
|
||||
executed_results: list[dict] = []
|
||||
|
||||
tool_call_log.append({
|
||||
"tool": fc.name,
|
||||
"args": dict(fc.args),
|
||||
"result": result_str,
|
||||
})
|
||||
response_parts.append(
|
||||
types.Part(
|
||||
function_response=types.FunctionResponse(
|
||||
name=fc.name,
|
||||
response={"result": result_str},
|
||||
for fc_part in tool_call_parts:
|
||||
fc = fc_part.function_call
|
||||
name = fc.name
|
||||
args = dict(fc.args)
|
||||
|
||||
if name in effective_confirm:
|
||||
pending_tools.append({"name": name, "args": args})
|
||||
logger.info("Tool %s blocked — confirmation required", name)
|
||||
else:
|
||||
result_str = await _execute_tool(name, args, tool_callables)
|
||||
logger.info("Tool %s → %d chars", name, len(result_str))
|
||||
executed_results.append({"name": name, "args": args, "result": result_str})
|
||||
tool_call_log.append({"tool": name, "args": args, "result": result_str})
|
||||
response_parts.append(types.Part(function_response=types.FunctionResponse(
|
||||
name=name, response={"result": result_str}
|
||||
)))
|
||||
|
||||
if pending_tools:
|
||||
# Add placeholder responses and get Gemini to produce the confirmation message
|
||||
for pt in pending_tools:
|
||||
placeholder = f"[AWAITING USER CONFIRMATION for {pt['name']}]"
|
||||
response_parts.append(types.Part(function_response=types.FunctionResponse(
|
||||
name=pt["name"], response={"result": placeholder}
|
||||
)))
|
||||
tool_call_log.append({"tool": pt["name"], "args": pt["args"], "result": "[awaiting confirmation]"})
|
||||
|
||||
contents.append(types.Content(role="user", parts=response_parts))
|
||||
|
||||
conf_response = await asyncio.to_thread(
|
||||
client.models.generate_content,
|
||||
model=model_name or settings.orchestrator_model,
|
||||
contents=contents,
|
||||
config=types.GenerateContentConfig(
|
||||
tools=tool_declarations,
|
||||
system_instruction=_ORCHESTRATOR_SYSTEM,
|
||||
),
|
||||
)
|
||||
_track_gemini_usage(conf_response, model_name)
|
||||
conf_parts = (
|
||||
conf_response.candidates[0].content.parts
|
||||
if conf_response.candidates and conf_response.candidates[0].content
|
||||
else []
|
||||
)
|
||||
gemini_summary = "".join(
|
||||
p.text for p in conf_parts if hasattr(p, "text") and p.text
|
||||
).strip() or "This action requires your explicit confirmation before it can proceed."
|
||||
|
||||
checkpoint = OrchestrateCheckpoint(
|
||||
engine="gemini",
|
||||
pre_fn_state=pre_fn_state,
|
||||
executed_results=executed_results,
|
||||
pending_tools=pending_tools,
|
||||
tool_call_log=list(tool_call_log),
|
||||
task=task,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_messages,
|
||||
model_name=model_name,
|
||||
gemini_api_key=gemini_api_key,
|
||||
respond_with_claude=respond_with_claude,
|
||||
response_role=response_role,
|
||||
user_role=user_role,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=confirm_allow,
|
||||
confirm_deny=confirm_deny,
|
||||
rounds_used=round_num + 2,
|
||||
max_rounds=max_rounds,
|
||||
)
|
||||
return gemini_summary, checkpoint
|
||||
|
||||
contents.append(types.Content(role="user", parts=response_parts))
|
||||
|
||||
else:
|
||||
# Hit the round limit — use whatever Gemini produced last
|
||||
logger.warning("Orchestrator hit max rounds (%d)", settings.orchestrator_max_rounds)
|
||||
logger.warning("Orchestrator hit max rounds (%d)", effective_limit)
|
||||
gemini_summary = (
|
||||
f"Reached the tool iteration limit ({settings.orchestrator_max_rounds} rounds). "
|
||||
f"Reached the tool iteration limit ({effective_limit} rounds). "
|
||||
"Here is what was gathered so far:\n\n"
|
||||
+ "\n\n".join(f"**{t['tool']}**: {t['result'][:500]}" for t in tool_call_log)
|
||||
)
|
||||
|
||||
# --- Claude handoff ---
|
||||
return gemini_summary, None
|
||||
|
||||
|
||||
async def _claude_handoff(
|
||||
task: str,
|
||||
tool_call_log: list[dict],
|
||||
gemini_summary: str,
|
||||
system_prompt: str,
|
||||
session_messages: list[dict] | None,
|
||||
respond_with_claude: bool,
|
||||
response_role: str,
|
||||
) -> OrchestratorResult:
|
||||
if respond_with_claude:
|
||||
claude_prompt = _build_claude_prompt(task, tool_call_log, gemini_summary)
|
||||
|
||||
# Merge with session history so Claude has conversation context
|
||||
messages = list(session_messages or [])
|
||||
messages.append({"role": "user", "content": claude_prompt})
|
||||
|
||||
response_text, backend = await complete(
|
||||
system_prompt=system_prompt,
|
||||
messages=messages,
|
||||
model="claude",
|
||||
role=response_role,
|
||||
)
|
||||
else:
|
||||
# Cron/background tasks: return Gemini's summary directly, no Claude call
|
||||
response_text = gemini_summary or "No information gathered."
|
||||
backend = "gemini"
|
||||
|
||||
@@ -187,10 +466,10 @@ async def run(
|
||||
)
|
||||
|
||||
|
||||
async def _execute_tool(name: str, args: dict) -> str:
|
||||
async def _execute_tool(name: str, args: dict, callables: dict | None = None) -> str:
|
||||
"""Execute a single tool call, catching all exceptions."""
|
||||
try:
|
||||
return await call_tool(name, args)
|
||||
return await call_tool(name, args, callables)
|
||||
except Exception as e:
|
||||
logger.warning("Tool %s failed: %s", name, e)
|
||||
return f"Tool error: {e}"
|
||||
@@ -201,12 +480,11 @@ def _build_task_prompt(task: str, session_messages: list[dict] | None) -> str:
|
||||
if not session_messages:
|
||||
return task
|
||||
|
||||
# Include last few turns for context (don't send the full history to keep tokens low)
|
||||
recent = session_messages[-6:] # last 3 turns
|
||||
recent = session_messages[-6:]
|
||||
history_lines = []
|
||||
for msg in recent:
|
||||
label = "User" if msg["role"] == "user" else "Assistant"
|
||||
history_lines.append(f"{label}: {msg['content'][:300]}") # truncate long messages
|
||||
history_lines.append(f"{label}: {msg['content'][:300]}")
|
||||
|
||||
context = "\n".join(history_lines)
|
||||
return f"<recent_conversation>\n{context}\n</recent_conversation>\n\nCurrent request: {task}"
|
||||
@@ -224,7 +502,6 @@ def _build_claude_prompt(
|
||||
parts.append("## Research gathered\n")
|
||||
for tc in tool_calls:
|
||||
parts.append(f"### {tc['tool']}({_format_args(tc['args'])})")
|
||||
# Truncate very long results — Claude gets the gist
|
||||
result = tc["result"]
|
||||
if len(result) > 2000:
|
||||
result = result[:2000] + "\n… [truncated]"
|
||||
|
||||
@@ -135,6 +135,27 @@ def _protocols(display_name: str) -> str:
|
||||
|
||||
---
|
||||
|
||||
## Tools & Modes
|
||||
|
||||
Cortex has two chat modes. Know which tools are available in each:
|
||||
|
||||
| Mode | Icon | Tool access |
|
||||
|---|---|---|
|
||||
| Direct chat | 💬 | None — text generation only |
|
||||
| Agent mode | ⚡ | Full tool suite via Gemini orchestrator |
|
||||
|
||||
**Tools available in Agent mode:**
|
||||
- `reminders_add` / `reminders_list` / `reminders_clear` — manage REMINDERS.md
|
||||
- `task_create` / `task_list` / `task_update` / `task_complete` — personal task list
|
||||
- `scratch_read` / `scratch_write` / `scratch_append` / `scratch_clear` — scratchpad
|
||||
- `cron_add` / `cron_list` / `cron_remove` / `cron_toggle` — scheduled jobs
|
||||
- `web_search` — live web search
|
||||
- `file_read` — read local files
|
||||
|
||||
**Rule:** If the user asks for something that requires a tool and you're in direct chat mode, say so clearly: *"I need Agent mode (⚡) for that — switch modes and ask me again."* Do not attempt workarounds or pretend the action was taken.
|
||||
|
||||
---
|
||||
|
||||
## Memory
|
||||
|
||||
- Long-term memory lives in MEMORY_LONG.md (auto-distilled monthly).
|
||||
|
||||
117
cortex/push_utils.py
Normal file
117
cortex/push_utils.py
Normal file
@@ -0,0 +1,117 @@
|
||||
"""
|
||||
Web Push (VAPID) helpers.
|
||||
|
||||
Subscriptions are stored per-user at:
|
||||
home/{user}/push_subscriptions.json → list of {endpoint, keys:{p256dh, auth}}
|
||||
|
||||
send_push(username, title, body, url) iterates all stored subscriptions for that
|
||||
user and fires a push. Stale endpoints (410 Gone) are pruned automatically.
|
||||
"""
|
||||
import asyncio
|
||||
import base64
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _subs_path(username: str) -> Path:
|
||||
return settings.home_root() / username / "push_subscriptions.json"
|
||||
|
||||
|
||||
def load_subscriptions(username: str) -> list[dict]:
|
||||
path = _subs_path(username)
|
||||
if not path.exists():
|
||||
return []
|
||||
try:
|
||||
return json.loads(path.read_text())
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
def _save_subscriptions(username: str, subs: list[dict]) -> None:
|
||||
path = _subs_path(username)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(subs, indent=2))
|
||||
|
||||
|
||||
def add_subscription(username: str, sub: dict) -> None:
|
||||
"""Upsert a subscription by endpoint URL."""
|
||||
subs = load_subscriptions(username)
|
||||
endpoint = sub.get("endpoint", "")
|
||||
subs = [s for s in subs if s.get("endpoint") != endpoint]
|
||||
subs.append(sub)
|
||||
_save_subscriptions(username, subs)
|
||||
|
||||
|
||||
def remove_subscription(username: str, endpoint: str) -> bool:
|
||||
subs = load_subscriptions(username)
|
||||
new_subs = [s for s in subs if s.get("endpoint") != endpoint]
|
||||
if len(new_subs) == len(subs):
|
||||
return False
|
||||
_save_subscriptions(username, new_subs)
|
||||
return True
|
||||
|
||||
|
||||
def _get_private_key_pem() -> str:
|
||||
"""Decode the base64-encoded PEM private key from settings."""
|
||||
raw = settings.vapid_private_key_b64.strip()
|
||||
if not raw:
|
||||
raise RuntimeError("VAPID_PRIVATE_KEY_B64 is not set in .env")
|
||||
return base64.b64decode(raw).decode()
|
||||
|
||||
|
||||
def _send_one(sub: dict, payload: dict) -> bool:
|
||||
"""Send a push to a single subscription. Returns False if the endpoint is stale (410)."""
|
||||
from pywebpush import webpush, WebPushException
|
||||
from py_vapid import Vapid
|
||||
|
||||
try:
|
||||
vapid = Vapid.from_pem(_get_private_key_pem().encode())
|
||||
webpush(
|
||||
subscription_info=sub,
|
||||
data=json.dumps(payload),
|
||||
vapid_private_key=vapid,
|
||||
vapid_claims={"sub": settings.vapid_contact},
|
||||
)
|
||||
return True
|
||||
except WebPushException as e:
|
||||
if e.response is not None and e.response.status_code == 410:
|
||||
logger.info("push endpoint gone (410), pruning: %s", sub.get("endpoint", "")[:60])
|
||||
return False
|
||||
logger.warning("push failed: %s", e)
|
||||
return True # keep the sub; might be transient
|
||||
|
||||
|
||||
async def send_push(username: str, title: str, body: str, url: str = "") -> dict:
|
||||
"""
|
||||
Send a push notification to all subscriptions for username.
|
||||
Returns {"sent": n, "pruned": m}.
|
||||
"""
|
||||
if not settings.vapid_public_key or not settings.vapid_private_key_b64:
|
||||
return {"error": "VAPID keys not configured"}
|
||||
|
||||
subs = load_subscriptions(username)
|
||||
if not subs:
|
||||
return {"error": f"No push subscriptions for {username}"}
|
||||
|
||||
payload = {"title": title, "body": body, "url": url}
|
||||
keep = []
|
||||
sent = 0
|
||||
pruned = 0
|
||||
|
||||
for sub in subs:
|
||||
alive = await asyncio.to_thread(_send_one, sub, payload)
|
||||
if alive:
|
||||
keep.append(sub)
|
||||
sent += 1
|
||||
else:
|
||||
pruned += 1
|
||||
|
||||
if pruned:
|
||||
_save_subscriptions(username, keep)
|
||||
|
||||
return {"sent": sent, "pruned": pruned}
|
||||
@@ -16,5 +16,20 @@ bcrypt>=4.0.0
|
||||
PyJWT>=2.8.0
|
||||
python-multipart>=0.0.9 # required by FastAPI for Form() data
|
||||
|
||||
# anthropic SDK not needed — using claude CLI subprocess for auth
|
||||
# anthropic>=0.40.0
|
||||
# Async HTTP client — used for local OpenAI-compatible backend (Open WebUI / Ollama)
|
||||
httpx>=0.27.0
|
||||
|
||||
# Web content extraction — strips ads/nav/boilerplate, returns clean article text
|
||||
trafilatura>=1.6.0
|
||||
|
||||
# OpenAI-compatible client — tool calling for OpenRouter / LiteLLM / any OAI-compat host
|
||||
openai>=1.0.0
|
||||
|
||||
# Web Push / VAPID — browser push notifications
|
||||
pywebpush>=2.0.0
|
||||
|
||||
# MariaDB / MySQL connector — used by ae_db_query orchestrator tool
|
||||
pymysql>=1.1.0
|
||||
|
||||
# Anthropic SDK — direct API key backend (alternative to CLI OAuth)
|
||||
anthropic>=0.40.0
|
||||
|
||||
122
cortex/routers/audit.py
Normal file
122
cortex/routers/audit.py
Normal file
@@ -0,0 +1,122 @@
|
||||
"""
|
||||
Tool audit log endpoints.
|
||||
|
||||
Self-service (any authenticated user, own data):
|
||||
GET /api/audit/files → list of available date strings (newest first)
|
||||
GET /api/audit/day?date=YYYY-MM-DD → entries for one day
|
||||
|
||||
Admin-only (cross-user aggregation):
|
||||
GET /api/audit/recent?user=scott&days=7&limit=200
|
||||
GET /api/audit/stats?user=scott&days=7
|
||||
"""
|
||||
import jwt
|
||||
from collections import Counter
|
||||
from datetime import date, timedelta
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token, get_user_role
|
||||
from config import settings
|
||||
import tool_audit
|
||||
from persona import list_users
|
||||
|
||||
router = APIRouter(prefix="/api/audit")
|
||||
|
||||
|
||||
def _session_user(request: Request) -> str:
|
||||
"""Return the authenticated username or raise 401."""
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
raise HTTPException(status_code=401, detail="Invalid session")
|
||||
|
||||
|
||||
def _require_admin(request: Request) -> str:
|
||||
username = _session_user(request)
|
||||
if get_user_role(username) != "admin":
|
||||
raise HTTPException(status_code=403, detail="Admin access required")
|
||||
return username
|
||||
|
||||
|
||||
@router.get("/files")
|
||||
async def audit_files(request: Request) -> dict:
|
||||
"""List available audit log dates for the current user (newest first)."""
|
||||
username = _session_user(request)
|
||||
audit_dir = settings.home_root() / username / "tool_audit"
|
||||
if not audit_dir.exists():
|
||||
return {"dates": []}
|
||||
dates = sorted(
|
||||
[p.stem for p in audit_dir.glob("*.jsonl") if p.stem],
|
||||
reverse=True,
|
||||
)
|
||||
return {"dates": dates}
|
||||
|
||||
|
||||
@router.get("/day")
|
||||
async def audit_day(
|
||||
request: Request,
|
||||
date: str = Query(..., description="YYYY-MM-DD"),
|
||||
) -> dict:
|
||||
"""Return all entries for a specific day (current user only)."""
|
||||
username = _session_user(request)
|
||||
try:
|
||||
from datetime import date as _date
|
||||
d = _date.fromisoformat(date)
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=400, detail="date must be YYYY-MM-DD")
|
||||
entries = tool_audit.read_day(username, date)
|
||||
return {"date": date, "entries": entries}
|
||||
|
||||
|
||||
@router.get("/recent")
|
||||
async def audit_recent(
|
||||
request: Request,
|
||||
user: str = Query(None, description="Username to filter (omit for all users)"),
|
||||
days: int = Query(7, ge=1, le=90),
|
||||
limit: int = Query(200, ge=1, le=1000),
|
||||
) -> dict:
|
||||
_require_admin(request)
|
||||
|
||||
if user:
|
||||
if user not in list_users():
|
||||
raise HTTPException(status_code=404, detail=f"User not found: {user}")
|
||||
entries = tool_audit.read_recent(user, days=days, limit=limit)
|
||||
else:
|
||||
entries = tool_audit.read_recent_all_users(days=days, limit=limit)
|
||||
|
||||
return {"entries": entries, "count": len(entries), "days": days}
|
||||
|
||||
|
||||
@router.get("/stats")
|
||||
async def audit_stats(
|
||||
request: Request,
|
||||
user: str = Query(None),
|
||||
days: int = Query(7, ge=1, le=90),
|
||||
) -> dict:
|
||||
_require_admin(request)
|
||||
|
||||
if user:
|
||||
if user not in list_users():
|
||||
raise HTTPException(status_code=404, detail=f"User not found: {user}")
|
||||
entries = tool_audit.read_recent(user, days=days, limit=10000)
|
||||
else:
|
||||
entries = tool_audit.read_recent_all_users(days=days, limit=10000)
|
||||
|
||||
tool_counts: Counter = Counter()
|
||||
status_counts: Counter = Counter()
|
||||
user_counts: Counter = Counter()
|
||||
|
||||
for e in entries:
|
||||
tool_counts[e.get("tool", "?")] += 1
|
||||
status_counts[e.get("status", "?")] += 1
|
||||
user_counts[e.get("user", "?")] += 1
|
||||
|
||||
return {
|
||||
"total": len(entries),
|
||||
"days": days,
|
||||
"by_tool": dict(tool_counts.most_common()),
|
||||
"by_status": dict(status_counts),
|
||||
"by_user": dict(user_counts.most_common()),
|
||||
}
|
||||
@@ -13,6 +13,7 @@ import logging
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from fastapi import APIRouter
|
||||
from config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/auth")
|
||||
@@ -20,7 +21,8 @@ router = APIRouter(prefix="/auth")
|
||||
CLAUDE_CREDS = Path.home() / ".claude" / ".credentials.json"
|
||||
GEMINI_CREDS = Path.home() / ".gemini" / "oauth_creds.json"
|
||||
GEMINI_ACCTS = Path.home() / ".gemini" / "google_accounts.json"
|
||||
WARN_HOURS = 24
|
||||
WARN_HOURS = 24 # no refresh token — warn a day ahead
|
||||
WARN_HOURS_REFRESH = 1 # refresh token present — only warn if CLI hasn't rotated in time
|
||||
|
||||
|
||||
def _claude_status() -> dict:
|
||||
@@ -31,11 +33,13 @@ def _claude_status() -> dict:
|
||||
expires_dt = datetime.fromtimestamp(oauth["expiresAt"] / 1000, tz=timezone.utc)
|
||||
now = datetime.now(tz=timezone.utc)
|
||||
hours_remaining = (expires_dt - now).total_seconds() / 3600
|
||||
# If a refresh token is present the session is long-lived (~1 year).
|
||||
# expiresAt only reflects the current access token window (~8 h) and
|
||||
# rotates automatically — do not warn based on it when a refresh token exists.
|
||||
warning = not has_refresh and hours_remaining < WARN_HOURS
|
||||
expired = hours_remaining <= 0 and not has_refresh
|
||||
# When a refresh token is present the CLI *should* auto-rotate the access
|
||||
# token, but sometimes it doesn't. Use a tight 1-hour window so a fresh
|
||||
# 8-hour token doesn't immediately trigger a warning, but a stale token
|
||||
# that the CLI missed will still surface before it expires.
|
||||
expired = hours_remaining <= 0
|
||||
threshold = WARN_HOURS_REFRESH if has_refresh else WARN_HOURS
|
||||
warning = expired or hours_remaining < threshold
|
||||
return {
|
||||
"ok": True,
|
||||
"has_refresh_token": has_refresh,
|
||||
@@ -68,9 +72,39 @@ def _gemini_status() -> dict:
|
||||
return {"ok": False, "error": str(e), "warning": True, "authenticated": False}
|
||||
|
||||
|
||||
async def _local_status(username: str = "scott") -> dict:
|
||||
"""Check reachability of the user's configured local model host."""
|
||||
import model_registry
|
||||
cfg = model_registry.get_best_local_model(username)
|
||||
if not cfg:
|
||||
return {"configured": False}
|
||||
api_url = cfg.get("api_url", "")
|
||||
if not api_url:
|
||||
return {"configured": False}
|
||||
try:
|
||||
import httpx
|
||||
url = api_url.rstrip("/") + "/api/models"
|
||||
headers = {}
|
||||
api_key = cfg.get("api_key", "")
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
async with httpx.AsyncClient(timeout=5) as client:
|
||||
resp = await client.get(url, headers=headers)
|
||||
reachable = resp.status_code < 400
|
||||
return {
|
||||
"configured": True,
|
||||
"reachable": reachable,
|
||||
"model": cfg.get("model_name", ""),
|
||||
"label": cfg.get("label", ""),
|
||||
}
|
||||
except Exception as e:
|
||||
return {"configured": True, "reachable": False, "error": str(e), "model": cfg.get("model_name", "")}
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
async def auth_status() -> dict:
|
||||
return {
|
||||
"claude": _claude_status(),
|
||||
"gemini": _gemini_status(),
|
||||
"local": await _local_status(),
|
||||
}
|
||||
|
||||
205
cortex/routers/auth_google.py
Normal file
205
cortex/routers/auth_google.py
Normal file
@@ -0,0 +1,205 @@
|
||||
"""
|
||||
Google OAuth 2.0 sign-in.
|
||||
|
||||
Flow:
|
||||
1. GET /auth/google → redirect to Google's consent page
|
||||
2. GET /auth/google/callback → exchange code, look up user, set JWT cookie
|
||||
|
||||
Users must be pre-registered by Scott before they can sign in:
|
||||
cd cortex && .venv/bin/python manage_passwords.py google-add <username> <email>
|
||||
|
||||
Routes are public (added to _PUBLIC_PREFIXES in auth_middleware.py).
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse, Response
|
||||
|
||||
from auth_utils import COOKIE_NAME, create_token, find_user_by_google, link_google
|
||||
from config import settings
|
||||
from persona import list_user_personas
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
_GOOGLE_AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth"
|
||||
_GOOGLE_TOKEN_URL = "https://oauth2.googleapis.com/token"
|
||||
_GOOGLE_USERINFO = "https://openidconnect.googleapis.com/v1/userinfo"
|
||||
_STATE_COOKIE = "oauth_state"
|
||||
_STATE_MAX_AGE = 600 # 10 minutes — plenty of time to complete the flow
|
||||
|
||||
|
||||
@router.get("/auth/google", include_in_schema=False)
|
||||
async def google_login():
|
||||
if not settings.google_client_id:
|
||||
return HTMLResponse("Google sign-in is not configured on this server.", status_code=503)
|
||||
|
||||
state = secrets.token_urlsafe(16)
|
||||
params = urllib.parse.urlencode({
|
||||
"client_id": settings.google_client_id,
|
||||
"redirect_uri": f"{settings.cortex_base_url}/auth/google/callback",
|
||||
"response_type": "code",
|
||||
"scope": "openid email profile",
|
||||
"state": state,
|
||||
"access_type": "online",
|
||||
"prompt": "select_account",
|
||||
})
|
||||
|
||||
resp = RedirectResponse(f"{_GOOGLE_AUTH_URL}?{params}", status_code=302)
|
||||
resp.set_cookie(_STATE_COOKIE, state, max_age=_STATE_MAX_AGE, httponly=True, samesite="lax")
|
||||
return resp
|
||||
|
||||
|
||||
@router.get("/auth/google/callback", include_in_schema=False)
|
||||
async def google_callback(
|
||||
request: Request,
|
||||
code: str = "",
|
||||
state: str = "",
|
||||
error: str = "",
|
||||
):
|
||||
if error:
|
||||
return _error_page(f"Google sign-in was cancelled or denied: {error}")
|
||||
|
||||
if not code:
|
||||
return _error_page("No authorisation code returned by Google.")
|
||||
|
||||
# CSRF check — state must match what we stored in the cookie
|
||||
stored_state = request.cookies.get(_STATE_COOKIE)
|
||||
if not stored_state or stored_state != state:
|
||||
return _error_page("State mismatch — please try signing in again.")
|
||||
|
||||
# Exchange authorisation code for tokens
|
||||
try:
|
||||
token_data = _exchange_code(code)
|
||||
except Exception as e:
|
||||
logger.error("Google token exchange failed: %s", e)
|
||||
return _error_page("Could not complete sign-in with Google. Please try again.")
|
||||
|
||||
access_token = token_data.get("access_token")
|
||||
if not access_token:
|
||||
return _error_page("No access token returned by Google.")
|
||||
|
||||
# Fetch the user's profile
|
||||
try:
|
||||
userinfo = _get_userinfo(access_token)
|
||||
except Exception as e:
|
||||
logger.error("Google userinfo fetch failed: %s", e)
|
||||
return _error_page("Could not retrieve your Google profile. Please try again.")
|
||||
|
||||
google_sub = userinfo.get("sub", "")
|
||||
google_email = userinfo.get("email", "")
|
||||
|
||||
if not google_sub or not google_email:
|
||||
return _error_page("Your Google account didn't return a usable email address.")
|
||||
|
||||
# Match to a Cortex user
|
||||
username = find_user_by_google(google_sub, google_email)
|
||||
if not username:
|
||||
logger.warning("Google sign-in rejected: no account for %s (%s)", google_sub, google_email)
|
||||
return _error_page(
|
||||
f"Your Google account (<strong>{google_email}</strong>) isn't registered with Cortex.<br><br>"
|
||||
"Contact Scott to get access."
|
||||
)
|
||||
|
||||
# Persist the stable sub so future lookups use it (not just email)
|
||||
link_google(username, google_sub, google_email)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
if not personas:
|
||||
return _error_page("No personas are configured for your account yet. Contact Scott.")
|
||||
|
||||
logger.info("Google sign-in: %s (%s)", username, google_email)
|
||||
resp = RedirectResponse(f"/{username}/{personas[0]}", status_code=302)
|
||||
_set_session_cookie(resp, username)
|
||||
resp.delete_cookie(_STATE_COOKIE)
|
||||
return resp
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Private helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _exchange_code(code: str) -> dict:
|
||||
body = urllib.parse.urlencode({
|
||||
"code": code,
|
||||
"client_id": settings.google_client_id,
|
||||
"client_secret": settings.google_client_secret,
|
||||
"redirect_uri": f"{settings.cortex_base_url}/auth/google/callback",
|
||||
"grant_type": "authorization_code",
|
||||
}).encode()
|
||||
req = urllib.request.Request(
|
||||
_GOOGLE_TOKEN_URL,
|
||||
data=body,
|
||||
headers={"Content-Type": "application/x-www-form-urlencoded"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
return json.loads(resp.read())
|
||||
|
||||
|
||||
def _get_userinfo(access_token: str) -> dict:
|
||||
req = urllib.request.Request(
|
||||
_GOOGLE_USERINFO,
|
||||
headers={"Authorization": f"Bearer {access_token}"},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=10) as resp:
|
||||
return json.loads(resp.read())
|
||||
|
||||
|
||||
def _set_session_cookie(response: Response, username: str) -> None:
|
||||
token = create_token(username)
|
||||
response.set_cookie(
|
||||
COOKIE_NAME,
|
||||
token,
|
||||
max_age=settings.jwt_expire_days * 86400,
|
||||
httponly=True,
|
||||
samesite="lax",
|
||||
secure=False, # set True if terminating TLS at the app layer (not behind a proxy)
|
||||
)
|
||||
|
||||
|
||||
def _error_page(message: str) -> HTMLResponse:
|
||||
html = f"""<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Sign In Failed</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
*, *::before, *::after {{ box-sizing: border-box; margin: 0; padding: 0; }}
|
||||
body {{
|
||||
min-height: 100vh; display: flex; align-items: center; justify-content: center;
|
||||
background: #0f1117; font-family: 'Inter', system-ui; font-weight: 450;
|
||||
-webkit-font-smoothing: antialiased; color: #e2e8f0;
|
||||
}}
|
||||
.card {{
|
||||
background: #1a1d27; border: 1px solid #2d3148; border-radius: 12px;
|
||||
padding: 2.5rem 2rem; width: 100%; max-width: 420px; text-align: center;
|
||||
}}
|
||||
h1 {{ font-size: 1.25rem; font-weight: 700; color: #f87171; margin-bottom: 1rem; }}
|
||||
p {{ font-size: 0.9rem; color: #94a3b8; margin-bottom: 1.75rem; line-height: 1.65; }}
|
||||
a {{
|
||||
display: inline-block; padding: 0.6rem 1.5rem;
|
||||
background: #7c3aed; border-radius: 6px; color: #fff;
|
||||
text-decoration: none; font-size: 0.9rem; font-weight: 600;
|
||||
transition: background 0.15s;
|
||||
}}
|
||||
a:hover {{ background: #6d28d9; }}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="card">
|
||||
<h1>Sign In Failed</h1>
|
||||
<p>{message}</p>
|
||||
<a href="/login">← Back to Sign In</a>
|
||||
</div>
|
||||
</body>
|
||||
</html>"""
|
||||
return HTMLResponse(html, status_code=403)
|
||||
@@ -1,34 +1,71 @@
|
||||
import asyncio
|
||||
import json
|
||||
from fastapi import APIRouter, HTTPException, Query
|
||||
import platform
|
||||
import jwt
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel
|
||||
from context_loader import load_context
|
||||
from llm_client import complete
|
||||
from session_logger import log_turn
|
||||
from session_store import load as load_session, save as save_session, list_all, generate_session_id, delete as delete_session
|
||||
from session_store import load as load_session, save as save_session, list_all, generate_session_id, delete as delete_session, rename as rename_session, get_name as get_session_name
|
||||
from config import settings
|
||||
from persona import set_context, validate as validate_persona
|
||||
from auth_utils import COOKIE_NAME, decode_token
|
||||
import model_registry
|
||||
import event_bus
|
||||
from model_registry import get_role_config
|
||||
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _backend_label(backend: str, username: str, role: str = "chat") -> str:
|
||||
"""Human-readable label for the model that handled a request (legacy path)."""
|
||||
if backend == "claude":
|
||||
return "Claude"
|
||||
if backend == "gemini":
|
||||
return "Gemini"
|
||||
if backend == "local":
|
||||
cfg = model_registry.get_best_local_model(username, role)
|
||||
if cfg:
|
||||
return cfg.get("label") or cfg.get("model_name") or "Local"
|
||||
return "Local"
|
||||
return backend.title()
|
||||
|
||||
|
||||
def _role_model_label(username: str, role: str, actual_backend: str) -> str:
|
||||
"""Return the model label for a role, falling back to the generic backend label."""
|
||||
cfg = model_registry.get_model_for_role(username, role)
|
||||
if cfg:
|
||||
return cfg.get("label") or cfg.get("model_name") or _backend_label(actual_backend, username, role)
|
||||
return _backend_label(actual_backend, username, role)
|
||||
|
||||
|
||||
class Attachment(BaseModel):
|
||||
filename: str
|
||||
mime_type: str
|
||||
data: str # base64 data URL for images (e.g. "data:image/png;base64,...")
|
||||
|
||||
|
||||
class ChatRequest(BaseModel):
|
||||
message: str
|
||||
session_id: str | None = None
|
||||
tier: int | None = None
|
||||
model: str | None = None # "claude" or "gemini" to override; None = use primary_backend
|
||||
model: str | None = None # legacy backend override ("claude"|"gemini"|"local")
|
||||
slot: str | None = None # Phase 3: explicit slot ("primary"|"backup_1"|"backup_2")
|
||||
chat_role: str = "chat" # active role: "chat"|"coder"|"research"|"distill" etc.
|
||||
include_long: bool = True
|
||||
include_mid: bool = True
|
||||
include_short: bool = True
|
||||
off_record: bool = False # skip session log (in-memory context preserved)
|
||||
user: str = "scott"
|
||||
persona: str = "inara"
|
||||
attachment: Attachment | None = None # image attachment (text files injected client-side)
|
||||
|
||||
|
||||
class BackendRequest(BaseModel):
|
||||
primary: str # "claude" or "gemini"
|
||||
primary: str # "claude", "gemini", or "local"
|
||||
|
||||
|
||||
class NoteRequest(BaseModel):
|
||||
@@ -62,19 +99,39 @@ async def _stream_chat(req: ChatRequest):
|
||||
session_id = req.session_id or generate_session_id()
|
||||
tier = req.tier or settings.default_tier
|
||||
|
||||
role_cfg = get_role_config(user, req.chat_role)
|
||||
system_prompt = load_context(
|
||||
tier,
|
||||
include_long=req.include_long,
|
||||
include_mid=req.include_mid,
|
||||
include_short=req.include_short,
|
||||
inject_datetime=role_cfg.get("inject_datetime", True),
|
||||
inject_mode=role_cfg.get("inject_mode", True),
|
||||
mode="otr" if req.off_record else "chat",
|
||||
)
|
||||
history = load_session(session_id)
|
||||
history.append({"role": "user", "content": req.message})
|
||||
|
||||
# req.message already contains the full user text:
|
||||
# - text files: client embedded content as a fenced code block
|
||||
# - images: client added "📎 filename.png" note; image data is in req.attachment
|
||||
# History always stores text only — base64 image data is never written to disk.
|
||||
llm_attachment: dict | None = None
|
||||
if req.attachment and req.attachment.mime_type.startswith("image/"):
|
||||
llm_attachment = {
|
||||
"filename": req.attachment.filename,
|
||||
"mime_type": req.attachment.mime_type,
|
||||
"data": req.attachment.data,
|
||||
}
|
||||
|
||||
history.append({"role": "user", "content": req.message, "off_record": req.off_record})
|
||||
|
||||
task = asyncio.create_task(complete(
|
||||
system_prompt=system_prompt,
|
||||
messages=history,
|
||||
model=req.model,
|
||||
role=req.chat_role,
|
||||
slot=req.slot,
|
||||
attachment=llm_attachment,
|
||||
))
|
||||
|
||||
try:
|
||||
@@ -90,17 +147,35 @@ async def _stream_chat(req: ChatRequest):
|
||||
|
||||
try:
|
||||
response_text, actual_backend = task.result()
|
||||
history.append({"role": "assistant", "content": response_text})
|
||||
if req.slot:
|
||||
slot_cfg = model_registry.get_model_for_slot(user, req.chat_role, req.slot)
|
||||
backend_label = (slot_cfg or {}).get("label") or _role_model_label(user, req.chat_role, actual_backend)
|
||||
else:
|
||||
backend_label = _role_model_label(user, req.chat_role, actual_backend)
|
||||
host = platform.node()
|
||||
history.append({
|
||||
"role": "assistant",
|
||||
"content": response_text,
|
||||
"backend": actual_backend,
|
||||
"backend_label": backend_label,
|
||||
"host": host,
|
||||
"off_record": req.off_record,
|
||||
})
|
||||
save_session(session_id, history)
|
||||
log_turn(session_id, req.message, response_text)
|
||||
if not req.off_record:
|
||||
log_turn(session_id, req.message, response_text, backend_label, host)
|
||||
|
||||
requested = req.model or settings.primary_backend
|
||||
# fallback_used only makes sense for explicit backend selections.
|
||||
# In auto mode (req.model is None), just report what responded.
|
||||
fallback_used = bool(req.model and actual_backend != req.model)
|
||||
payload = {
|
||||
"type": "response",
|
||||
"response": response_text,
|
||||
"session_id": session_id,
|
||||
"backend": actual_backend,
|
||||
"fallback_used": actual_backend != requested,
|
||||
"backend_label": backend_label,
|
||||
"host": host,
|
||||
"fallback_used": fallback_used,
|
||||
}
|
||||
yield f"data: {json.dumps(payload)}\n\n"
|
||||
|
||||
@@ -128,19 +203,111 @@ async def chat(req: ChatRequest) -> StreamingResponse:
|
||||
)
|
||||
|
||||
|
||||
_BACKEND_CYCLE = ("claude", "gemini", "local")
|
||||
_BACKEND_FALLBACK = {"claude": "gemini", "gemini": "claude", "local": "claude"}
|
||||
|
||||
|
||||
def _request_user(request: Request) -> str | None:
|
||||
"""Extract username from JWT cookie, or None."""
|
||||
try:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
return decode_token(token) if token else None
|
||||
except (jwt.InvalidTokenError, Exception):
|
||||
return None
|
||||
|
||||
|
||||
def _local_model_info(request: Request) -> dict | None:
|
||||
"""Return the best local model {label, model_name} for the session user, or None."""
|
||||
username = _request_user(request)
|
||||
if not username:
|
||||
return None
|
||||
try:
|
||||
cfg = model_registry.get_best_local_model(username, "chat")
|
||||
if cfg:
|
||||
return {"label": cfg.get("label", ""), "model_name": cfg.get("model_name", "")}
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def _chat_slot_models(username: str) -> list[dict]:
|
||||
"""Return [{slot, label, type}] for each configured slot in the chat role, primary first."""
|
||||
registry = model_registry.get_registry(username)
|
||||
role_slots = registry.get("roles", {}).get("chat", {})
|
||||
result = []
|
||||
for slot_key in model_registry.PRIORITY_KEYS:
|
||||
model_id = role_slots.get(slot_key)
|
||||
if not model_id:
|
||||
continue
|
||||
resolved = model_registry._resolve_model(registry, model_id)
|
||||
if resolved:
|
||||
result.append({
|
||||
"slot": slot_key,
|
||||
"label": resolved.get("label") or resolved.get("model_name") or "",
|
||||
"type": resolved.get("type", ""),
|
||||
})
|
||||
return result
|
||||
|
||||
|
||||
def _available_roles_for_toggle(username: str) -> list[dict]:
|
||||
"""Return roles with a primary model assigned (excluding orchestrator) for the UI toggle.
|
||||
|
||||
Returns [{role, label, model_label, type}] ordered by settings.defined_roles.
|
||||
"""
|
||||
registry = model_registry.get_registry(username)
|
||||
roles_cfg = registry.get("roles", {})
|
||||
result = []
|
||||
for role_name in settings.get_defined_roles():
|
||||
if role_name == "orchestrator":
|
||||
continue
|
||||
primary_id = roles_cfg.get(role_name, {}).get("primary")
|
||||
if not primary_id:
|
||||
continue
|
||||
resolved = model_registry._resolve_model(registry, primary_id)
|
||||
if resolved:
|
||||
result.append({
|
||||
"role": role_name,
|
||||
"label": role_name.title(),
|
||||
"model_label": resolved.get("label") or resolved.get("model_name") or "",
|
||||
"type": resolved.get("type", ""),
|
||||
})
|
||||
return result
|
||||
|
||||
|
||||
@router.get("/backend")
|
||||
async def get_backend() -> dict:
|
||||
other = "gemini" if settings.primary_backend == "claude" else "claude"
|
||||
return {"primary": settings.primary_backend, "fallback": other}
|
||||
async def get_backend(request: Request) -> dict:
|
||||
username = _request_user(request)
|
||||
chat_models = _chat_slot_models(username) if username else []
|
||||
available_roles = _available_roles_for_toggle(username) if username else []
|
||||
p = settings.primary_backend
|
||||
|
||||
orch_label = None
|
||||
if username:
|
||||
orch_cfg = model_registry.get_model_for_role(username, "orchestrator")
|
||||
if orch_cfg:
|
||||
orch_label = orch_cfg.get("label") or orch_cfg.get("model_name") or None
|
||||
|
||||
return {
|
||||
"chat_models": chat_models, # Phase 3: [{slot, label, type}] for chat-role slots
|
||||
"available_roles": available_roles, # kept for banner + backward compat
|
||||
"orchestrator_model": orch_label,
|
||||
# Legacy fields kept for backward compat
|
||||
"primary": p,
|
||||
"fallback": _BACKEND_FALLBACK.get(p, "claude"),
|
||||
"local_model": _local_model_info(request),
|
||||
}
|
||||
|
||||
|
||||
@router.post("/backend")
|
||||
async def set_backend(req: BackendRequest) -> dict:
|
||||
if req.primary not in ("claude", "gemini"):
|
||||
raise HTTPException(status_code=400, detail="primary must be 'claude' or 'gemini'")
|
||||
async def set_backend(req: BackendRequest, request: Request) -> dict:
|
||||
if req.primary not in _BACKEND_CYCLE:
|
||||
raise HTTPException(status_code=400, detail="primary must be 'claude', 'gemini', or 'local'")
|
||||
settings.primary_backend = req.primary
|
||||
other = "gemini" if req.primary == "claude" else "claude"
|
||||
return {"primary": settings.primary_backend, "fallback": other}
|
||||
return {
|
||||
"primary": req.primary,
|
||||
"fallback": _BACKEND_FALLBACK[req.primary],
|
||||
"local_model": _local_model_info(request),
|
||||
}
|
||||
|
||||
|
||||
def _set_ctx(user: str, persona: str) -> None:
|
||||
@@ -159,7 +326,8 @@ async def get_history(
|
||||
persona: str = Query("inara"),
|
||||
) -> dict:
|
||||
_set_ctx(user, persona)
|
||||
return {"session_id": session_id, "messages": load_session(session_id)}
|
||||
name = get_session_name(session_id)
|
||||
return {"session_id": session_id, "name": name, "messages": load_session(session_id)}
|
||||
|
||||
|
||||
@router.get("/sessions")
|
||||
@@ -171,6 +339,71 @@ async def list_sessions(
|
||||
return {"sessions": list_all()}
|
||||
|
||||
|
||||
class SessionRename(BaseModel):
|
||||
name: str
|
||||
|
||||
|
||||
@router.patch("/sessions/{session_id}")
|
||||
async def rename_session_endpoint(
|
||||
session_id: str,
|
||||
req: SessionRename,
|
||||
user: str = Query("scott"),
|
||||
persona: str = Query("inara"),
|
||||
) -> dict:
|
||||
_set_ctx(user, persona)
|
||||
found = rename_session(session_id, req.name.strip())
|
||||
if not found:
|
||||
raise HTTPException(status_code=404, detail=f"Session {session_id} not found")
|
||||
return {"ok": True, "session_id": session_id, "name": req.name.strip()}
|
||||
|
||||
|
||||
@router.post("/api/sessions/backfill-names")
|
||||
async def backfill_session_names(
|
||||
request: Request,
|
||||
user: str = Query(""),
|
||||
persona: str = Query(""),
|
||||
) -> dict:
|
||||
"""Name every unnamed session using its first user message (truncated to 60 chars).
|
||||
Idempotent — only touches sessions that have no name set.
|
||||
user/persona default to the JWT session user + last-used persona cookie."""
|
||||
# Resolve user from JWT if not provided
|
||||
if not user:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
try:
|
||||
user = decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
raise HTTPException(status_code=401, detail="Invalid session")
|
||||
|
||||
# Resolve persona from cookie if not provided
|
||||
if not persona:
|
||||
from persona import list_user_personas
|
||||
persona_cookie = request.cookies.get("cx_last_persona", "")
|
||||
available = list_user_personas(user)
|
||||
persona = persona_cookie if persona_cookie in available else (available[0] if available else "")
|
||||
if not persona:
|
||||
raise HTTPException(status_code=400, detail="No persona found for user")
|
||||
|
||||
_set_ctx(user, persona)
|
||||
sessions = list_all()
|
||||
named = 0
|
||||
for s in sessions:
|
||||
if s.get("name"):
|
||||
continue
|
||||
messages = load_session(s["session_id"])
|
||||
first_user = next((m for m in messages if m.get("role") == "user"), None)
|
||||
if not first_user:
|
||||
continue
|
||||
text = (first_user.get("content") or "").strip()
|
||||
if not text:
|
||||
continue
|
||||
auto_name = text[:60].rstrip() + ("…" if len(text) > 60 else "")
|
||||
rename_session(s["session_id"], auto_name)
|
||||
named += 1
|
||||
return {"ok": True, "named": named, "total": len(sessions)}
|
||||
|
||||
|
||||
@router.delete("/sessions/{session_id}")
|
||||
async def delete_session_endpoint(
|
||||
session_id: str,
|
||||
|
||||
479
cortex/routers/crons.py
Normal file
479
cortex/routers/crons.py
Normal file
@@ -0,0 +1,479 @@
|
||||
"""
|
||||
Schedules web UI — GET/POST /settings/crons/*
|
||||
|
||||
Lets users view, add, toggle, and remove cron jobs without going through the AI.
|
||||
Cron data lives in home/{user}/persona/{persona}/CRONS.json.
|
||||
Scheduler registration mirrors what tools/cron.py does so changes take effect immediately.
|
||||
"""
|
||||
|
||||
import html as _html
|
||||
import logging
|
||||
import secrets
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import jwt
|
||||
from fastapi import APIRouter, Form, Request
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token
|
||||
from cron_runner import load_crons, save_crons, parse_schedule
|
||||
from persona import list_user_personas
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
_STATIC = Path(__file__).parent.parent / "static"
|
||||
_LAST_PERSONA_COOKIE = "cx_last_persona"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_session_user(request: Request) -> str | None:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
return None
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
return None
|
||||
|
||||
|
||||
def _preferred_persona(request: Request, username: str) -> str:
|
||||
names = list_user_personas(username)
|
||||
if not names:
|
||||
return ""
|
||||
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
|
||||
if cookie_val in names:
|
||||
return cookie_val
|
||||
return names[0]
|
||||
|
||||
|
||||
def _integrations_nav(username: str) -> str:
|
||||
from auth_utils import _read_auth
|
||||
role = _read_auth(username).get("role", "user")
|
||||
if role == "admin":
|
||||
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
|
||||
return ""
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def _short_id() -> str:
|
||||
return "c_" + secrets.token_urlsafe(6)
|
||||
|
||||
|
||||
def _scheduler_add(job: dict, sched_kwargs: dict) -> None:
|
||||
import asyncio
|
||||
try:
|
||||
import scheduler as sched_module
|
||||
from cron_runner import run_job
|
||||
s = sched_module.get_scheduler()
|
||||
if s and s.running:
|
||||
sched_id = f"{job['user']}:{job['persona']}:{job['id']}"
|
||||
s.add_job(
|
||||
lambda j=job: asyncio.ensure_future(run_job(j)),
|
||||
"cron",
|
||||
id=sched_id,
|
||||
replace_existing=True,
|
||||
**sched_kwargs,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("scheduler_add failed: %s", e)
|
||||
|
||||
|
||||
def _scheduler_remove(job_id: str) -> None:
|
||||
try:
|
||||
import scheduler as sched_module
|
||||
s = sched_module.get_scheduler()
|
||||
if s and s.running:
|
||||
s.remove_job(job_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _scheduler_pause(job_id: str) -> None:
|
||||
try:
|
||||
import scheduler as sched_module
|
||||
s = sched_module.get_scheduler()
|
||||
if s and s.running:
|
||||
s.pause_job(job_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _scheduler_resume(job_id: str) -> None:
|
||||
try:
|
||||
import scheduler as sched_module
|
||||
s = sched_module.get_scheduler()
|
||||
if s and s.running:
|
||||
s.resume_job(job_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
_TYPE_CLASS = {
|
||||
"remind": "badge-remind", "note": "badge-note", "message": "badge-message",
|
||||
"brief": "badge-brief", "task": "badge-task",
|
||||
}
|
||||
|
||||
|
||||
def _render_cron_list(username: str) -> str:
|
||||
personas = list_user_personas(username)
|
||||
if not personas:
|
||||
return '<div class="empty-state">No personas found.</div>'
|
||||
|
||||
all_empty = True
|
||||
html_parts: list[str] = []
|
||||
|
||||
for persona in personas:
|
||||
crons = load_crons(username, persona)
|
||||
if not crons:
|
||||
continue
|
||||
all_empty = False
|
||||
|
||||
rows = []
|
||||
for c in crons:
|
||||
cid = _html.escape(c["id"])
|
||||
label = _html.escape(c.get("label", ""))
|
||||
schedule = _html.escape(c.get("schedule", ""))
|
||||
job_type = _html.escape(c.get("type", ""))
|
||||
payload = _html.escape(c.get("payload", ""))
|
||||
enabled = c.get("enabled", True)
|
||||
last_run = (c.get("last_run") or "")[:10] or "never"
|
||||
pers_esc = _html.escape(persona)
|
||||
|
||||
type_class = _TYPE_CLASS.get(c.get("type", ""), "badge-note")
|
||||
status_cls = "badge-enabled" if enabled else "badge-paused"
|
||||
status_txt = "enabled" if enabled else "paused"
|
||||
toggle_txt = "Pause" if enabled else "Resume"
|
||||
|
||||
rows.append(f"""
|
||||
<tr>
|
||||
<td>{label}</td>
|
||||
<td><code>{schedule}</code></td>
|
||||
<td><span class="badge {type_class}">{job_type}</span></td>
|
||||
<td class="payload-cell" title="{payload}">{payload}</td>
|
||||
<td>{last_run}</td>
|
||||
<td><span class="badge {status_cls}">{status_txt}</span></td>
|
||||
<td>
|
||||
<div class="cron-actions">
|
||||
<a href="/settings/crons/edit?cron_id={cid}&persona={pers_esc}"
|
||||
class="btn-cron">Edit</a>
|
||||
<form method="POST" action="/settings/crons/toggle" style="display:inline">
|
||||
<input type="hidden" name="cron_id" value="{cid}">
|
||||
<input type="hidden" name="persona" value="{pers_esc}">
|
||||
<button type="submit" class="btn-cron">{toggle_txt}</button>
|
||||
</form>
|
||||
<form method="POST" action="/settings/crons/remove" style="display:inline"
|
||||
onsubmit="return confirm('Delete this schedule?')">
|
||||
<input type="hidden" name="cron_id" value="{cid}">
|
||||
<input type="hidden" name="persona" value="{pers_esc}">
|
||||
<button type="submit" class="btn-cron btn-cron-del">Delete</button>
|
||||
</form>
|
||||
</div>
|
||||
</td>
|
||||
</tr>""")
|
||||
|
||||
html_parts.append(f"""
|
||||
<div class="persona-group">
|
||||
<p class="persona-group-label">{_html.escape(persona)}</p>
|
||||
<table class="cron-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Label</th><th>Schedule</th><th>Type</th>
|
||||
<th>Payload</th><th>Last run</th><th>Status</th><th></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>{"".join(rows)}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>""")
|
||||
|
||||
if all_empty:
|
||||
return '<div class="empty-state">No schedules yet. Add one below.</div>'
|
||||
|
||||
return "\n".join(html_parts)
|
||||
|
||||
|
||||
def _persona_options(username: str, selected: str = "") -> str:
|
||||
personas = list_user_personas(username)
|
||||
return "\n".join(
|
||||
f'<option value="{_html.escape(p)}"{"selected" if p == selected else ""}>{_html.escape(p)}</option>'
|
||||
for p in personas
|
||||
)
|
||||
|
||||
|
||||
_TYPE_OPTIONS = ("remind", "note", "message", "brief", "task")
|
||||
_TYPE_LABELS = {
|
||||
"remind": "remind — append to REMINDERS.md",
|
||||
"note": "note — append to SCRATCH.md",
|
||||
"message": "message — send payload as-is",
|
||||
"brief": "brief — LLM response, no tools",
|
||||
"task": "task — full orchestrator tool loop",
|
||||
}
|
||||
|
||||
|
||||
def _render_edit_form(job: dict, persona: str) -> str:
|
||||
cid = _html.escape(job["id"])
|
||||
pers_esc = _html.escape(persona)
|
||||
label = _html.escape(job.get("label", ""))
|
||||
schedule = _html.escape(job.get("schedule", ""))
|
||||
payload = _html.escape(job.get("payload", ""))
|
||||
cur_type = job.get("type", "remind")
|
||||
|
||||
type_opts = "\n".join(
|
||||
f'<option value="{t}" {"selected" if t == cur_type else ""}>{_html.escape(_TYPE_LABELS.get(t, t))}</option>'
|
||||
for t in _TYPE_OPTIONS
|
||||
)
|
||||
|
||||
return f"""
|
||||
<div class="section" style="border: 2px solid var(--pg-accent); border-radius: 8px; padding: 1rem;">
|
||||
<h2 style="margin-top:0">Edit schedule</h2>
|
||||
<form method="POST" action="/settings/crons/save">
|
||||
<input type="hidden" name="cron_id" value="{cid}">
|
||||
<input type="hidden" name="persona" value="{pers_esc}">
|
||||
<div class="add-form-grid">
|
||||
<div class="field">
|
||||
<label>Persona</label>
|
||||
<input type="text" value="{pers_esc}" disabled style="opacity:0.5">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="edit_job_type">Type</label>
|
||||
<select id="edit_job_type" name="job_type">{type_opts}</select>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="edit_label">Label</label>
|
||||
<input type="text" id="edit_label" name="label" value="{label}" required autocomplete="off">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="edit_schedule">Schedule</label>
|
||||
<input type="text" id="edit_schedule" name="schedule" value="{schedule}"
|
||||
required autocomplete="off" spellcheck="false">
|
||||
<p class="hint">
|
||||
hourly · daily · daily:HH:MM · weekly:DOW · weekly:DOW:HH:MM ·
|
||||
monthly · monthly:DD · monthly:DD:HH:MM · yearly:MM:DD · yearly:MM:DD:HH:MM
|
||||
</p>
|
||||
</div>
|
||||
<div class="field field-full">
|
||||
<label for="edit_payload">Payload / prompt</label>
|
||||
<textarea id="edit_payload" name="payload" rows="3" required>{payload}</textarea>
|
||||
</div>
|
||||
</div>
|
||||
<div style="display:flex; gap:0.5rem; align-items:center; margin-top:0.5rem">
|
||||
<button type="submit" class="btn-submit" style="margin-top:0">Save changes</button>
|
||||
<a href="/settings/crons" style="font-size:0.85rem; color:var(--pg-muted)">Cancel</a>
|
||||
</div>
|
||||
</form>
|
||||
</div>"""
|
||||
|
||||
|
||||
def _render_page(username: str, back_persona: str = "", success: str = "", error: str = "",
|
||||
edit_html: str = "") -> str:
|
||||
html = (_STATIC / "crons.html").read_text()
|
||||
html = html.replace("{{ edit_html }}", edit_html)
|
||||
html = html.replace("{{ cron_list_html }}", _render_cron_list(username))
|
||||
html = html.replace("{{ persona_options }}", _persona_options(username, back_persona))
|
||||
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
|
||||
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
|
||||
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
|
||||
if success:
|
||||
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{_html.escape(success)}</p>')
|
||||
if error:
|
||||
html = html.replace("<!-- ERROR -->", f'<p class="error">{_html.escape(error)}</p>')
|
||||
return html
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Routes
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@router.get("/settings/crons", include_in_schema=False)
|
||||
async def crons_page(request: Request):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
return HTMLResponse(_render_page(username, back_persona))
|
||||
|
||||
|
||||
@router.post("/settings/crons/add", include_in_schema=False)
|
||||
async def cron_add(
|
||||
request: Request,
|
||||
persona: str = Form(""),
|
||||
label: str = Form(""),
|
||||
schedule: str = Form(""),
|
||||
job_type: str = Form(""),
|
||||
payload: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
label = label.strip()
|
||||
schedule = schedule.strip()
|
||||
payload = payload.strip()
|
||||
persona = persona.strip()
|
||||
|
||||
_VALID_TYPES = ("remind", "note", "message", "brief", "task")
|
||||
if job_type not in _VALID_TYPES:
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Invalid type: {job_type}"))
|
||||
|
||||
try:
|
||||
sched_kwargs = parse_schedule(schedule)
|
||||
except ValueError as e:
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Bad schedule: {e}"))
|
||||
|
||||
if not label:
|
||||
return HTMLResponse(_render_page(username, back_persona, error="Label is required."))
|
||||
if not payload:
|
||||
return HTMLResponse(_render_page(username, back_persona, error="Payload is required."))
|
||||
|
||||
crons = load_crons(username, persona)
|
||||
job = {
|
||||
"id": _short_id(),
|
||||
"user": username,
|
||||
"persona": persona,
|
||||
"label": label,
|
||||
"schedule": schedule,
|
||||
"type": job_type,
|
||||
"payload": payload,
|
||||
"enabled": True,
|
||||
"created_at": _now(),
|
||||
"last_run": None,
|
||||
}
|
||||
crons.append(job)
|
||||
save_crons(crons, username, persona)
|
||||
_scheduler_add(job, sched_kwargs)
|
||||
|
||||
logger.info("cron added via UI: %s %s [%s]", job["id"], schedule, job_type)
|
||||
return HTMLResponse(_render_page(username, back_persona, success=f"Schedule '{label}' added."))
|
||||
|
||||
|
||||
@router.post("/settings/crons/toggle", include_in_schema=False)
|
||||
async def cron_toggle(
|
||||
request: Request,
|
||||
cron_id: str = Form(""),
|
||||
persona: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
crons = load_crons(username, persona)
|
||||
for c in crons:
|
||||
if c["id"] == cron_id:
|
||||
c["enabled"] = not c.get("enabled", True)
|
||||
save_crons(crons, username, persona)
|
||||
sched_id = f"{username}:{persona}:{cron_id}"
|
||||
if c["enabled"]:
|
||||
_scheduler_resume(sched_id)
|
||||
action = "resumed"
|
||||
else:
|
||||
_scheduler_pause(sched_id)
|
||||
action = "paused"
|
||||
logger.info("cron %s %s via UI", cron_id, action)
|
||||
return HTMLResponse(_render_page(username, back_persona, success=f"Schedule {action}."))
|
||||
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
|
||||
|
||||
|
||||
@router.post("/settings/crons/remove", include_in_schema=False)
|
||||
async def cron_remove(
|
||||
request: Request,
|
||||
cron_id: str = Form(""),
|
||||
persona: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
crons = load_crons(username, persona)
|
||||
before = len(crons)
|
||||
crons = [c for c in crons if c["id"] != cron_id]
|
||||
if len(crons) == before:
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
|
||||
|
||||
save_crons(crons, username, persona)
|
||||
_scheduler_remove(f"{username}:{persona}:{cron_id}")
|
||||
logger.info("cron %s removed via UI", cron_id)
|
||||
return HTMLResponse(_render_page(username, back_persona, success="Schedule deleted."))
|
||||
|
||||
|
||||
@router.get("/settings/crons/edit", include_in_schema=False)
|
||||
async def cron_edit_page(request: Request, cron_id: str = "", persona: str = ""):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
crons = load_crons(username, persona)
|
||||
job = next((c for c in crons if c["id"] == cron_id), None)
|
||||
if not job:
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
|
||||
|
||||
edit_html = _render_edit_form(job, persona)
|
||||
return HTMLResponse(_render_page(username, back_persona, edit_html=edit_html))
|
||||
|
||||
|
||||
@router.post("/settings/crons/save", include_in_schema=False)
|
||||
async def cron_save(
|
||||
request: Request,
|
||||
cron_id: str = Form(""),
|
||||
persona: str = Form(""),
|
||||
label: str = Form(""),
|
||||
schedule: str = Form(""),
|
||||
job_type: str = Form(""),
|
||||
payload: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
label = label.strip()
|
||||
schedule = schedule.strip()
|
||||
payload = payload.strip()
|
||||
|
||||
if job_type not in _TYPE_OPTIONS:
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Invalid type: {job_type}"))
|
||||
if not label:
|
||||
return HTMLResponse(_render_page(username, back_persona, error="Label is required."))
|
||||
if not payload:
|
||||
return HTMLResponse(_render_page(username, back_persona, error="Payload is required."))
|
||||
|
||||
try:
|
||||
sched_kwargs = parse_schedule(schedule)
|
||||
except ValueError as e:
|
||||
# Re-render with the edit form still open so the user can fix the schedule
|
||||
crons = load_crons(username, persona)
|
||||
job = next((c for c in crons if c["id"] == cron_id), None)
|
||||
edit_html = _render_edit_form(job, persona) if job else ""
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Bad schedule: {e}",
|
||||
edit_html=edit_html))
|
||||
|
||||
crons = load_crons(username, persona)
|
||||
for c in crons:
|
||||
if c["id"] == cron_id:
|
||||
c["label"] = label
|
||||
c["schedule"] = schedule
|
||||
c["type"] = job_type
|
||||
c["payload"] = payload
|
||||
save_crons(crons, username, persona)
|
||||
# Replace the live scheduler job with the updated schedule
|
||||
sched_id = f"{username}:{persona}:{cron_id}"
|
||||
_scheduler_remove(sched_id)
|
||||
if c.get("enabled", True):
|
||||
_scheduler_add(c, sched_kwargs)
|
||||
logger.info("cron %s updated via UI [%s]", cron_id, schedule)
|
||||
return HTMLResponse(_render_page(username, back_persona,
|
||||
success=f"Schedule '{label}' updated."))
|
||||
|
||||
return HTMLResponse(_render_page(username, back_persona, error=f"Schedule not found: {cron_id}"))
|
||||
@@ -5,21 +5,99 @@ Manual memory distillation endpoints.
|
||||
POST /distill/mid — summarize short → MEMORY_MID.md (LLM)
|
||||
POST /distill/long — integrate mid → MEMORY_LONG.md (LLM)
|
||||
POST /distill/all — run all three in sequence
|
||||
POST /distill/rebuild — wipe mid + long, then run all three from scratch
|
||||
|
||||
All endpoints require ?user=<username>&persona=<name> query params.
|
||||
|
||||
Concurrency: one distillation at a time per persona. A second request while one
|
||||
is running returns 409 immediately — no silent queuing.
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
import asyncio
|
||||
from datetime import datetime, timedelta
|
||||
from fastapi import APIRouter, HTTPException, Query
|
||||
from memory_distiller import distill_short, distill_mid, distill_long
|
||||
from persona import validate as validate_persona, set_context, persona_path as _persona_path
|
||||
import scheduler
|
||||
|
||||
router = APIRouter(prefix="/distill")
|
||||
|
||||
# Per-persona asyncio lock. Key: (user, persona)
|
||||
_LOCKS: dict[tuple, asyncio.Lock] = {}
|
||||
_LOCKS_META: dict[tuple, str] = {} # key → which step is currently running
|
||||
|
||||
# Minimum time between successive runs of each endpoint, per persona.
|
||||
# Prevents accidental rapid-fire runs and token waste.
|
||||
_COOLDOWNS: dict[tuple, timedelta] = {
|
||||
"short": timedelta(minutes=1),
|
||||
"mid": timedelta(minutes=30),
|
||||
"long": timedelta(hours=6),
|
||||
"all": timedelta(hours=1),
|
||||
"rebuild": timedelta(hours=6),
|
||||
}
|
||||
_LAST_RUN: dict[tuple, datetime] = {} # key: (user, persona, endpoint)
|
||||
|
||||
|
||||
def _get_lock(user: str, persona: str) -> asyncio.Lock:
|
||||
key = (user, persona)
|
||||
if key not in _LOCKS:
|
||||
_LOCKS[key] = asyncio.Lock()
|
||||
return _LOCKS[key]
|
||||
|
||||
|
||||
def _resolve(user: str, persona: str) -> tuple[str, str]:
|
||||
try:
|
||||
u, p = validate_persona(user, persona)
|
||||
except Exception:
|
||||
raise HTTPException(status_code=404, detail=f"Persona not found: {user}/{persona}")
|
||||
set_context(u, p)
|
||||
return u, p
|
||||
|
||||
|
||||
def _check_lock(user: str, persona: str) -> asyncio.Lock:
|
||||
"""Return the lock if free, raise 409 if already held."""
|
||||
lock = _get_lock(user, persona)
|
||||
if lock.locked():
|
||||
step = _LOCKS_META.get((user, persona), "distillation")
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"A {step} is already running for {persona} — please wait for it to finish.",
|
||||
)
|
||||
return lock
|
||||
|
||||
|
||||
def _check_cooldown(user: str, persona: str, endpoint: str) -> None:
|
||||
"""Raise 429 if the endpoint was run too recently for this persona."""
|
||||
cooldown = _COOLDOWNS.get(endpoint)
|
||||
if not cooldown:
|
||||
return
|
||||
key = (user, persona, endpoint)
|
||||
last = _LAST_RUN.get(key)
|
||||
if last:
|
||||
elapsed = datetime.now() - last
|
||||
if elapsed < cooldown:
|
||||
remaining = cooldown - elapsed
|
||||
mins = int(remaining.total_seconds() // 60)
|
||||
secs = int(remaining.total_seconds() % 60)
|
||||
wait = f"{mins}m {secs}s" if mins else f"{secs}s"
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail=f"{endpoint} was just run — please wait {wait} before running again.",
|
||||
)
|
||||
|
||||
|
||||
def _record_run(user: str, persona: str, endpoint: str) -> None:
|
||||
_LAST_RUN[(user, persona, endpoint)] = datetime.now()
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
async def distill_status() -> dict:
|
||||
"""Show auto-distillation schedule and next run times."""
|
||||
from config import settings
|
||||
# Include which personas are currently distilling
|
||||
active = [f"{u}/{p}" for (u, p), lock in _LOCKS.items() if lock.locked()]
|
||||
return {
|
||||
"enabled": settings.auto_distill,
|
||||
"jobs": scheduler.status(),
|
||||
"active": active,
|
||||
"config": {
|
||||
"short": settings.auto_distill_short,
|
||||
"mid": settings.auto_distill_mid,
|
||||
@@ -29,32 +107,132 @@ async def distill_status() -> dict:
|
||||
|
||||
|
||||
@router.post("/short")
|
||||
async def do_distill_short() -> dict:
|
||||
return {"ok": True, **distill_short()}
|
||||
async def do_distill_short(
|
||||
user: str = Query(...),
|
||||
persona: str = Query(...),
|
||||
) -> dict:
|
||||
u, p = _resolve(user, persona)
|
||||
_check_cooldown(u, p, "short")
|
||||
lock = _check_lock(u, p)
|
||||
async with lock:
|
||||
_LOCKS_META[(u, p)] = "short distill"
|
||||
try:
|
||||
result = distill_short(u, p)
|
||||
_record_run(u, p, "short")
|
||||
return {"ok": True, **result}
|
||||
finally:
|
||||
_LOCKS_META.pop((u, p), None)
|
||||
|
||||
|
||||
@router.post("/mid")
|
||||
async def do_distill_mid() -> dict:
|
||||
result = await distill_mid()
|
||||
async def do_distill_mid(
|
||||
user: str = Query(...),
|
||||
persona: str = Query(...),
|
||||
) -> dict:
|
||||
u, p = _resolve(user, persona)
|
||||
_check_cooldown(u, p, "mid")
|
||||
lock = _check_lock(u, p)
|
||||
async with lock:
|
||||
_LOCKS_META[(u, p)] = "mid distill"
|
||||
try:
|
||||
result = await distill_mid(u, p)
|
||||
if "error" not in result:
|
||||
_record_run(u, p, "mid")
|
||||
return {"ok": "error" not in result, **result}
|
||||
finally:
|
||||
_LOCKS_META.pop((u, p), None)
|
||||
|
||||
|
||||
@router.post("/long")
|
||||
async def do_distill_long() -> dict:
|
||||
result = await distill_long()
|
||||
async def do_distill_long(
|
||||
user: str = Query(...),
|
||||
persona: str = Query(...),
|
||||
) -> dict:
|
||||
u, p = _resolve(user, persona)
|
||||
_check_cooldown(u, p, "long")
|
||||
lock = _check_lock(u, p)
|
||||
async with lock:
|
||||
_LOCKS_META[(u, p)] = "long distill"
|
||||
try:
|
||||
result = await distill_long(u, p)
|
||||
if "error" not in result:
|
||||
_record_run(u, p, "long")
|
||||
return {"ok": "error" not in result, **result}
|
||||
finally:
|
||||
_LOCKS_META.pop((u, p), None)
|
||||
|
||||
|
||||
@router.post("/all")
|
||||
async def do_distill_all() -> dict:
|
||||
short_result = distill_short()
|
||||
mid_result = await distill_mid()
|
||||
async def do_distill_all(
|
||||
user: str = Query(...),
|
||||
persona: str = Query(...),
|
||||
) -> dict:
|
||||
u, p = _resolve(user, persona)
|
||||
_check_cooldown(u, p, "all")
|
||||
lock = _check_lock(u, p)
|
||||
async with lock:
|
||||
_LOCKS_META[(u, p)] = "full distill"
|
||||
try:
|
||||
short_result = distill_short(u, p)
|
||||
mid_result = await distill_mid(u, p)
|
||||
if "error" in mid_result:
|
||||
return {"ok": False, "short": short_result, "mid": mid_result}
|
||||
long_result = await distill_long()
|
||||
long_result = await distill_long(u, p)
|
||||
ok = "error" not in long_result
|
||||
if ok:
|
||||
_record_run(u, p, "all")
|
||||
return {
|
||||
"ok": "error" not in long_result,
|
||||
"ok": ok,
|
||||
"short": short_result,
|
||||
"mid": mid_result,
|
||||
"long": long_result,
|
||||
}
|
||||
finally:
|
||||
_LOCKS_META.pop((u, p), None)
|
||||
|
||||
|
||||
@router.post("/rebuild")
|
||||
async def do_distill_rebuild(
|
||||
user: str = Query(...),
|
||||
persona: str = Query(...),
|
||||
) -> dict: # noqa: E501
|
||||
"""Wipe MEMORY_MID and MEMORY_LONG (with backups), then run short → mid → long.
|
||||
|
||||
Use when memories have drifted, been corrupted, or you want a clean slate
|
||||
rebuilt purely from session logs. Hand-edited content will be replaced.
|
||||
"""
|
||||
u, p = _resolve(user, persona)
|
||||
_check_cooldown(u, p, "rebuild")
|
||||
lock = _check_lock(u, p)
|
||||
async with lock:
|
||||
_LOCKS_META[(u, p)] = "memory rebuild"
|
||||
try:
|
||||
from memory_distiller import _rotate_backup, _read
|
||||
inara_dir = _persona_path(u, p)
|
||||
|
||||
# Back up then wipe mid and long before rebuilding
|
||||
for name in ("MEMORY_MID.md", "MEMORY_LONG.md"):
|
||||
path = inara_dir / name
|
||||
if path.exists():
|
||||
_rotate_backup(path)
|
||||
path.write_text(
|
||||
f"# {name}\n\n*Cleared for rebuild — {__import__('datetime').datetime.now().strftime('%Y-%m-%d %H:%M')}.*\n"
|
||||
)
|
||||
|
||||
short_result = distill_short(u, p)
|
||||
mid_result = await distill_mid(u, p)
|
||||
if "error" in mid_result:
|
||||
return {"ok": False, "short": short_result, "mid": mid_result, "rebuilt": True}
|
||||
long_result = await distill_long(u, p)
|
||||
ok = "error" not in long_result
|
||||
if ok:
|
||||
_record_run(u, p, "rebuild")
|
||||
return {
|
||||
"ok": ok,
|
||||
"short": short_result,
|
||||
"mid": mid_result,
|
||||
"long": long_result,
|
||||
"rebuilt": True,
|
||||
}
|
||||
finally:
|
||||
_LOCKS_META.pop((u, p), None)
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
"""
|
||||
Read/write the Inara identity markdown files.
|
||||
Read/write Inara identity markdown files, and search past session logs.
|
||||
Only whitelisted filenames are accessible — no path traversal possible.
|
||||
"""
|
||||
import re
|
||||
from fastapi import APIRouter, HTTPException, Query
|
||||
from pydantic import BaseModel
|
||||
from persona import persona_path, set_context, validate as validate_persona
|
||||
from config import settings as _settings
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -18,9 +20,29 @@ ALLOWED = {
|
||||
"MEMORY_LONG.md",
|
||||
"MEMORY_MID.md",
|
||||
"MEMORY_SHORT.md",
|
||||
"MEMORY_LONG.bak1.md",
|
||||
"MEMORY_LONG.bak2.md",
|
||||
"MEMORY_MID.bak1.md",
|
||||
"MEMORY_MID.bak2.md",
|
||||
"MEMORY_SHORT.bak1.md",
|
||||
"MEMORY_SHORT.bak2.md",
|
||||
"HELP.md",
|
||||
# Agent private notes — backups only; AGENT_NOTES.md itself is agent-only
|
||||
"AGENT_NOTES.bak1.md",
|
||||
"AGENT_NOTES.bak2.md",
|
||||
"AGENT_NOTES.bak3.md",
|
||||
}
|
||||
|
||||
# Files that can be read via the panel but not written by users
|
||||
READ_ONLY = {
|
||||
"AGENT_NOTES.bak1.md",
|
||||
"AGENT_NOTES.bak2.md",
|
||||
"AGENT_NOTES.bak3.md",
|
||||
}
|
||||
|
||||
# Files served from home/{user}/ instead of persona path
|
||||
USER_FILES = {"email_allowlist.json", "usage.json"}
|
||||
|
||||
|
||||
def _resolve(user: str, persona: str) -> None:
|
||||
"""Validate and set context from query params. Raises HTTPException on bad input."""
|
||||
@@ -31,7 +53,11 @@ def _resolve(user: str, persona: str) -> None:
|
||||
raise HTTPException(status_code=404, detail=str(e))
|
||||
|
||||
|
||||
def _path(filename: str):
|
||||
def _path(filename: str, user: str = ""):
|
||||
if filename in USER_FILES:
|
||||
if not user:
|
||||
raise HTTPException(status_code=400, detail="user param required for this file")
|
||||
return _settings.home_root() / user / filename
|
||||
if filename not in ALLOWED:
|
||||
raise HTTPException(status_code=404, detail=f"File not found: {filename}")
|
||||
return persona_path() / filename
|
||||
@@ -47,10 +73,22 @@ async def list_files(
|
||||
files = []
|
||||
for name in sorted(ALLOWED):
|
||||
p = persona_dir / name
|
||||
st = p.stat() if p.exists() else None
|
||||
files.append({
|
||||
"name": name,
|
||||
"exists": p.exists(),
|
||||
"size": p.stat().st_size if p.exists() else 0,
|
||||
"size": st.st_size if st else 0,
|
||||
"modified": st.st_mtime if st else None,
|
||||
})
|
||||
for name in sorted(USER_FILES):
|
||||
p = _settings.home_root() / user / name
|
||||
st = p.stat() if p.exists() else None
|
||||
files.append({
|
||||
"name": name,
|
||||
"exists": p.exists(),
|
||||
"size": st.st_size if st else 0,
|
||||
"modified": st.st_mtime if st else None,
|
||||
"scope": "user",
|
||||
})
|
||||
return {"files": files}
|
||||
|
||||
@@ -62,10 +100,14 @@ async def get_file(
|
||||
persona: str = Query("inara"),
|
||||
) -> dict:
|
||||
_resolve(user, persona)
|
||||
p = _path(filename)
|
||||
p = _path(filename, user=user)
|
||||
if not p.exists():
|
||||
raise HTTPException(status_code=404, detail=f"{filename} does not exist")
|
||||
return {"name": filename, "content": p.read_text()}
|
||||
return {
|
||||
"name": filename,
|
||||
"content": p.read_text(),
|
||||
"readonly": filename in READ_ONLY,
|
||||
}
|
||||
|
||||
|
||||
class FileWrite(BaseModel):
|
||||
@@ -79,7 +121,65 @@ async def save_file(
|
||||
user: str = Query("scott"),
|
||||
persona: str = Query("inara"),
|
||||
) -> dict:
|
||||
if filename in READ_ONLY:
|
||||
raise HTTPException(status_code=403, detail=f"{filename} is read-only.")
|
||||
_resolve(user, persona)
|
||||
p = _path(filename)
|
||||
p = _path(filename, user=user)
|
||||
p.write_text(req.content)
|
||||
return {"ok": True, "name": filename, "size": len(req.content)}
|
||||
|
||||
|
||||
# ── Session search ────────────────────────────────────────────────────────────
|
||||
|
||||
_CONTEXT_CHARS = 120 # chars of context to include around each match
|
||||
|
||||
|
||||
@router.get("/sessions/search")
|
||||
async def search_sessions(
|
||||
q: str = Query(..., min_length=2),
|
||||
user: str = Query("scott"),
|
||||
persona: str = Query("inara"),
|
||||
limit: int = Query(20, ge=1, le=100),
|
||||
) -> dict:
|
||||
"""Full-text search across past session logs.
|
||||
|
||||
Returns up to `limit` matches, newest sessions first.
|
||||
Each match includes a short excerpt (120 chars before/after) for context.
|
||||
"""
|
||||
_resolve(user, persona)
|
||||
sessions_dir = persona_path() / "sessions"
|
||||
if not sessions_dir.exists():
|
||||
return {"query": q, "matches": [], "total_files_searched": 0}
|
||||
|
||||
pattern = re.compile(re.escape(q), re.IGNORECASE)
|
||||
session_files = sorted(sessions_dir.glob("*.md"), reverse=True) # newest first
|
||||
|
||||
matches = []
|
||||
for sf in session_files:
|
||||
if len(matches) >= limit:
|
||||
break
|
||||
try:
|
||||
text = sf.read_text()
|
||||
except OSError:
|
||||
continue
|
||||
for m in pattern.finditer(text):
|
||||
if len(matches) >= limit:
|
||||
break
|
||||
start = max(0, m.start() - _CONTEXT_CHARS)
|
||||
end = min(len(text), m.end() + _CONTEXT_CHARS)
|
||||
excerpt = text[start:end].strip()
|
||||
# Prefix with ellipsis if we truncated the left side
|
||||
if start > 0:
|
||||
excerpt = "…" + excerpt
|
||||
if end < len(text):
|
||||
excerpt = excerpt + "…"
|
||||
matches.append({
|
||||
"date": sf.stem, # YYYY-MM-DD
|
||||
"excerpt": excerpt,
|
||||
})
|
||||
|
||||
return {
|
||||
"query": q,
|
||||
"matches": matches,
|
||||
"total_files_searched": len(session_files),
|
||||
}
|
||||
|
||||
@@ -3,14 +3,16 @@ import logging
|
||||
from fastapi import APIRouter, HTTPException, Request, Response
|
||||
from google.auth.transport import requests as google_requests
|
||||
from google.oauth2 import id_token
|
||||
from auth_utils import get_user_channels
|
||||
from context_loader import load_context
|
||||
from llm_client import complete
|
||||
from persona import set_context
|
||||
from session_logger import log_turn
|
||||
from session_store import load as load_session, save as save_session
|
||||
from config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/channels/google-chat")
|
||||
router = APIRouter()
|
||||
|
||||
# Workspace Add-on Chat apps: JWT is issued by accounts.google.com.
|
||||
# (Legacy standalone Chat bots used chat@system.gserviceaccount.com — different format.)
|
||||
@@ -35,7 +37,7 @@ def _msg(text: str) -> dict:
|
||||
}
|
||||
|
||||
|
||||
def _verify_system_id_token(token: str) -> None:
|
||||
def _verify_system_id_token(token: str, audience: str) -> None:
|
||||
"""Verify the systemIdToken from authorizationEventObject.
|
||||
|
||||
For Workspace Add-on Chat apps Google sends the token in the request body
|
||||
@@ -44,13 +46,13 @@ def _verify_system_id_token(token: str) -> None:
|
||||
|
||||
Claims verified:
|
||||
iss = "https://accounts.google.com"
|
||||
aud = settings.google_chat_audience (the endpoint URL)
|
||||
aud = the per-user audience from channels.json (the endpoint URL)
|
||||
"""
|
||||
try:
|
||||
claims = id_token.verify_oauth2_token(
|
||||
token,
|
||||
google_requests.Request(),
|
||||
audience=settings.google_chat_audience,
|
||||
audience=audience,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("Google Chat JWT verification failed: %s", exc)
|
||||
@@ -60,17 +62,30 @@ def _verify_system_id_token(token: str) -> None:
|
||||
raise HTTPException(status_code=401, detail="Wrong issuer")
|
||||
|
||||
|
||||
@router.post("")
|
||||
async def receive(request: Request):
|
||||
@router.post("/channels/google-chat/{username}")
|
||||
async def receive(username: str, request: Request):
|
||||
channels = get_user_channels(username)
|
||||
cfg = channels.get("google_chat")
|
||||
if not cfg:
|
||||
logger.warning("Google Chat: no channel config for user %r", username)
|
||||
raise HTTPException(status_code=404, detail="Channel not configured for this user")
|
||||
|
||||
persona_name = cfg.get("persona", "inara")
|
||||
audience = cfg.get("audience", "")
|
||||
backend = cfg.get("backend", settings.primary_backend)
|
||||
timeout = cfg.get("timeout", 25)
|
||||
|
||||
set_context(username, persona_name)
|
||||
|
||||
body = await request.json()
|
||||
|
||||
# Verify the systemIdToken embedded in the request body
|
||||
if settings.google_chat_audience:
|
||||
if audience:
|
||||
token = body.get("authorizationEventObject", {}).get("systemIdToken", "")
|
||||
if not token:
|
||||
logger.warning("Google Chat: missing systemIdToken")
|
||||
logger.warning("Google Chat: missing systemIdToken for %s", username)
|
||||
raise HTTPException(status_code=401, detail="Missing token")
|
||||
_verify_system_id_token(token)
|
||||
_verify_system_id_token(token, audience)
|
||||
|
||||
chat = body.get("chat", {})
|
||||
|
||||
@@ -79,8 +94,8 @@ async def receive(request: Request):
|
||||
if "addedToSpacePayload" in chat:
|
||||
space_type = chat["addedToSpacePayload"].get("space", {}).get("type", "")
|
||||
if space_type == "DM":
|
||||
return _msg(f"✨ Hello! I'm {settings.agent_name}. What can I help you with?")
|
||||
return _msg(f"✨ Hello! I'm {settings.agent_name}. Send me a message and I'll do my best to help.")
|
||||
return _msg(f"✨ Hello! I'm {persona_name.capitalize()}. What can I help you with?")
|
||||
return _msg(f"✨ Hello! I'm {persona_name.capitalize()}. Send me a message and I'll do my best to help.")
|
||||
|
||||
if "removedFromSpacePayload" in chat:
|
||||
return Response(status_code=200)
|
||||
@@ -107,7 +122,7 @@ async def receive(request: Request):
|
||||
logger.warning("Google Chat: empty user_text, ignoring")
|
||||
return Response(status_code=200)
|
||||
|
||||
session_id = "gc_" + space_name.replace("/", "_")
|
||||
session_id = f"gc_{username}_{space_name.replace('/', '_')}"
|
||||
system_prompt = load_context(settings.default_tier)
|
||||
history = load_session(session_id)
|
||||
history.append({"role": "user", "content": user_text})
|
||||
@@ -117,9 +132,9 @@ async def receive(request: Request):
|
||||
complete(
|
||||
system_prompt=system_prompt,
|
||||
messages=history,
|
||||
model=settings.google_chat_backend,
|
||||
model=backend,
|
||||
),
|
||||
timeout=settings.google_chat_timeout,
|
||||
timeout=timeout,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning("Google Chat request timed out for session %s", session_id)
|
||||
|
||||
70
cortex/routers/help.py
Normal file
70
cortex/routers/help.py
Normal file
@@ -0,0 +1,70 @@
|
||||
"""
|
||||
Help page router.
|
||||
|
||||
Routes:
|
||||
GET /help → full-page help viewer (requires auth)
|
||||
"""
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import jwt
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token, _read_auth
|
||||
from persona import list_user_personas
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
_STATIC = Path(__file__).parent.parent / "static"
|
||||
|
||||
|
||||
_LAST_PERSONA_COOKIE = "cx_last_persona"
|
||||
|
||||
|
||||
def _get_session_user(request: Request) -> str | None:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
return None
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
return None
|
||||
|
||||
|
||||
def _preferred_persona(request: Request, username: str) -> str:
|
||||
names = list_user_personas(username)
|
||||
if not names:
|
||||
return ""
|
||||
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
|
||||
if cookie_val in names:
|
||||
return cookie_val
|
||||
return names[0]
|
||||
|
||||
|
||||
@router.get("/help", include_in_schema=False)
|
||||
async def help_page(request: Request, persona: str = ""):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
# Use persona from query param if valid, else prefer last-visited from cookie
|
||||
if persona and persona in personas:
|
||||
back_persona = persona
|
||||
else:
|
||||
back_persona = _preferred_persona(request, username)
|
||||
back_href = f"/{username}/{back_persona}" if back_persona else "/"
|
||||
|
||||
html = (_STATIC / "help.html").read_text()
|
||||
config_tag = (
|
||||
f'<script>window.HELP_CONFIG = '
|
||||
f'{{user: "{username}", persona: "{back_persona}", backHref: "{back_href}"}};</script>'
|
||||
)
|
||||
html = html.replace("</head>", f"{config_tag}\n</head>", 1)
|
||||
nav = '<a href="/settings/integrations" class="nav-link">Integrations</a>' \
|
||||
if _read_auth(username).get("role", "user") == "admin" else ""
|
||||
html = html.replace("{{ integrations_nav }}", nav)
|
||||
return HTMLResponse(html)
|
||||
199
cortex/routers/homeassistant.py
Normal file
199
cortex/routers/homeassistant.py
Normal file
@@ -0,0 +1,199 @@
|
||||
"""
|
||||
Home Assistant webhook router — POST /webhook/ha/{username}/{webhook_id}
|
||||
|
||||
Receives event payloads from HA automations and routes them to Inara.
|
||||
Auth: the webhook_id in the URL acts as the shared secret (same model HA uses).
|
||||
Response is delivered async via notify() — NC Talk, web push, etc.
|
||||
|
||||
channels.json schema:
|
||||
"homeassistant": {
|
||||
"webhook_id": "your-secret-id",
|
||||
"persona": "inara",
|
||||
"tier": 2,
|
||||
"role": "chat",
|
||||
"tools": false
|
||||
}
|
||||
|
||||
HA automation example (rest_command):
|
||||
rest_command:
|
||||
cortex_notify:
|
||||
url: "https://cortex.dgrzone.com/webhook/ha/scott/your-secret-id"
|
||||
method: POST
|
||||
content_type: "application/json"
|
||||
payload: '{"message": "{{message}}", "entity_id": "{{entity_id}}", "state": "{{state}}"}'
|
||||
|
||||
automation:
|
||||
trigger:
|
||||
- trigger: state
|
||||
entity_id: binary_sensor.front_door
|
||||
to: "on"
|
||||
action:
|
||||
- action: rest_command.cortex_notify
|
||||
data:
|
||||
message: "Front door opened"
|
||||
entity_id: "binary_sensor.front_door"
|
||||
state: "on"
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response
|
||||
|
||||
from auth_utils import get_user_channels, get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
|
||||
from context_loader import load_context
|
||||
from llm_client import complete
|
||||
from notification import notify
|
||||
from persona import set_context
|
||||
from session_logger import log_turn
|
||||
from session_store import load as load_session, save as save_session
|
||||
from config import settings
|
||||
import event_bus
|
||||
import model_registry
|
||||
import orchestrator_engine
|
||||
import openai_orchestrator
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _build_task(body: dict) -> str:
|
||||
"""Turn an HA event payload into a natural-language prompt for Inara."""
|
||||
if "message" in body:
|
||||
msg = str(body["message"])
|
||||
extras = {k: body[k] for k in ("entity_id", "state", "trigger", "event", "area") if k in body}
|
||||
if extras:
|
||||
msg += "\n\nContext: " + json.dumps(extras)
|
||||
return msg
|
||||
return "Home Assistant event:\n" + json.dumps(body, indent=2)
|
||||
|
||||
|
||||
async def _process_event(username: str, body: dict, cfg: dict) -> None:
|
||||
persona_name = cfg.get("persona", "inara")
|
||||
tier = cfg.get("tier") or settings.default_tier
|
||||
role = cfg.get("role", "chat")
|
||||
use_tools = cfg.get("tools", False)
|
||||
|
||||
set_context(username, persona_name)
|
||||
|
||||
task = _build_task(body)
|
||||
session_id = f"ha_{username}"
|
||||
history = load_session(session_id)
|
||||
session_msgs = list(history)
|
||||
|
||||
logger.info("HA event for %s: %r", username, task[:80])
|
||||
|
||||
backend = "unknown"
|
||||
try:
|
||||
if use_tools:
|
||||
role_cfg = model_registry.get_role_config(username, role)
|
||||
system_prompt = load_context(
|
||||
tier,
|
||||
role_append=role_cfg.get("system_append", ""),
|
||||
inject_datetime=role_cfg.get("inject_datetime", True),
|
||||
inject_mode=role_cfg.get("inject_mode", True),
|
||||
)
|
||||
orch_model = model_registry.get_model_for_role(username, "orchestrator")
|
||||
user_role_val = get_user_role(username)
|
||||
tool_list = role_cfg.get("tools")
|
||||
policy = get_tool_policy(username)
|
||||
c_allow = set(policy.get("allow", []))
|
||||
c_deny = set(policy.get("deny", []))
|
||||
max_risk, risk_wl, risk_bl = get_risk_policy(username)
|
||||
|
||||
if orch_model and orch_model.get("type") == "local_openai":
|
||||
result = await openai_orchestrator.run(
|
||||
task=task,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_msgs or None,
|
||||
model_cfg=orch_model,
|
||||
user_role=user_role_val,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=c_allow,
|
||||
confirm_deny=c_deny,
|
||||
max_risk=max_risk,
|
||||
risk_whitelist=risk_wl,
|
||||
risk_blacklist=risk_bl,
|
||||
)
|
||||
else:
|
||||
gemini_key = (
|
||||
(orch_model.get("api_key") if orch_model else None)
|
||||
or get_user_gemini_key(username)
|
||||
)
|
||||
result = await orchestrator_engine.run(
|
||||
task=task,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_msgs or None,
|
||||
respond_with_claude=True,
|
||||
gemini_api_key=gemini_key,
|
||||
model_name=orch_model.get("model_name") if orch_model else None,
|
||||
response_role=role,
|
||||
user_role=user_role_val,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=c_allow,
|
||||
confirm_deny=c_deny,
|
||||
max_risk=max_risk,
|
||||
risk_whitelist=risk_wl,
|
||||
risk_blacklist=risk_bl,
|
||||
)
|
||||
response_text = result.response
|
||||
backend = result.backend
|
||||
|
||||
else:
|
||||
system_prompt = load_context(tier)
|
||||
msgs = list(session_msgs) + [{"role": "user", "content": task}]
|
||||
response_text, backend = await complete(system_prompt=system_prompt, messages=msgs)
|
||||
|
||||
except Exception as e:
|
||||
logger.error("HA event error for %s: %s", username, e)
|
||||
return
|
||||
|
||||
logger.info("HA response via %s (%d chars)", backend, len(response_text))
|
||||
|
||||
history.append({"role": "user", "content": task})
|
||||
history.append({"role": "assistant", "content": response_text})
|
||||
save_session(session_id, history)
|
||||
log_turn(session_id, task, response_text)
|
||||
|
||||
await event_bus.publish({
|
||||
"type": "ha_event",
|
||||
"session_id": session_id,
|
||||
"response": response_text,
|
||||
"backend": backend,
|
||||
})
|
||||
|
||||
await notify(username, response_text)
|
||||
|
||||
|
||||
@router.post("/webhook/ha/{username}/{webhook_id}")
|
||||
async def ha_webhook(
|
||||
username: str,
|
||||
webhook_id: str,
|
||||
request: Request,
|
||||
background_tasks: BackgroundTasks,
|
||||
) -> Response:
|
||||
"""Receive an event from a Home Assistant automation and route it to Inara."""
|
||||
channels = get_user_channels(username)
|
||||
cfg = channels.get("homeassistant")
|
||||
if not cfg:
|
||||
raise HTTPException(status_code=404, detail="Channel not configured")
|
||||
|
||||
if webhook_id != cfg.get("webhook_id", ""):
|
||||
logger.warning("HA webhook: bad webhook_id for user %r", username)
|
||||
raise HTTPException(status_code=401, detail="Invalid webhook ID")
|
||||
|
||||
content_type = request.headers.get("content-type", "")
|
||||
if "application/json" in content_type:
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
raise HTTPException(status_code=400, detail="Invalid JSON")
|
||||
else:
|
||||
form = await request.form()
|
||||
body = dict(form)
|
||||
|
||||
if not body:
|
||||
return Response(status_code=200)
|
||||
|
||||
background_tasks.add_task(_process_event, username, body, cfg)
|
||||
return Response(status_code=200)
|
||||
849
cortex/routers/local_llm.py
Normal file
849
cortex/routers/local_llm.py
Normal file
@@ -0,0 +1,849 @@
|
||||
"""
|
||||
Model Registry settings — providers, hosts, models, and role assignments.
|
||||
|
||||
Routes:
|
||||
GET /settings/models → settings page (canonical)
|
||||
GET /settings/local → redirect to /settings/models
|
||||
POST /settings/local/host → save/create a local host
|
||||
POST /settings/local/host/{id}/remove → remove a host (and its models)
|
||||
POST /settings/local/google-account → save/create a Google account
|
||||
POST /settings/local/google-account/{id}/remove → remove a Google account
|
||||
POST /settings/local/anthropic-key → save/create an Anthropic API key
|
||||
POST /settings/local/anthropic-key/{id}/remove → remove an Anthropic API key
|
||||
POST /settings/local/models/add → add a model (any provider)
|
||||
POST /settings/local/models/{id}/edit → edit an existing model entry
|
||||
POST /settings/local/models/{id}/remove → remove a model
|
||||
POST /settings/local/roles/add → add a custom role (redirects to #roles)
|
||||
POST /settings/local/roles/remove → remove a custom role (redirects to #roles)
|
||||
POST /api/models/role → AJAX: set a role assignment
|
||||
GET /api/local-llm/fetch-models → proxy to host /api/models (JSON)
|
||||
"""
|
||||
import json as _json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import jwt
|
||||
from fastapi import APIRouter, Form, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token, _read_auth
|
||||
from config import settings as app_settings
|
||||
from persona import list_user_personas
|
||||
import model_registry as reg
|
||||
from tools import TOOL_CATEGORIES
|
||||
|
||||
_LAST_PERSONA_COOKIE = "cx_last_persona"
|
||||
|
||||
|
||||
def _preferred_persona(request: Request, username: str) -> str:
|
||||
names = list_user_personas(username)
|
||||
if not names:
|
||||
return ""
|
||||
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
|
||||
if cookie_val in names:
|
||||
return cookie_val
|
||||
return names[0]
|
||||
|
||||
|
||||
def _integrations_nav(username: str) -> str:
|
||||
role = _read_auth(username).get("role", "user")
|
||||
if role == "admin":
|
||||
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
|
||||
return ""
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
_STATIC = Path(__file__).parent.parent / "static"
|
||||
|
||||
|
||||
def _host_row_html(h: dict) -> str:
|
||||
"""Return the HTML for one host config row (edit form + remove link)."""
|
||||
api_key = h.get("api_key", "")
|
||||
key_hint = f"…{api_key[-4:]}" if api_key else "not set"
|
||||
ht = h.get("host_type", "openwebui")
|
||||
ow = ' selected' if ht == "openwebui" else ''
|
||||
ai = ' selected' if ht == "openai" else ''
|
||||
hid = h["id"]
|
||||
hlbl = h.get("label", "")
|
||||
hurl = h.get("api_url", "")
|
||||
maxc = h.get("max_concurrent", 3)
|
||||
return f'''
|
||||
<div class="host-row">
|
||||
<form method="POST" action="/settings/local/host" class="host-form">
|
||||
<input type="hidden" name="host_id" value="{hid}">
|
||||
<div class="field-row">
|
||||
<div class="field">
|
||||
<label>Label</label>
|
||||
<input type="text" name="label" value="{hlbl}"
|
||||
placeholder="Gaming Laptop" autocomplete="off" data-form-type="other">
|
||||
</div>
|
||||
<div class="field" style="flex:2">
|
||||
<label>API URL</label>
|
||||
<input type="text" name="api_url" value="{hurl}"
|
||||
placeholder="http://192.168.x.x:3000"
|
||||
autocomplete="off" spellcheck="false" data-form-type="other">
|
||||
</div>
|
||||
</div>
|
||||
<div class="field-row">
|
||||
<div class="field">
|
||||
<label>API Key</label>
|
||||
<input type="password" name="api_key" placeholder="Leave blank to keep existing"
|
||||
autocomplete="new-password" data-1p-ignore data-lpignore="true"
|
||||
data-form-type="other">
|
||||
<p class="key-status">Current: {key_hint}</p>
|
||||
</div>
|
||||
<div class="field" style="flex:0 0 auto">
|
||||
<label>Type</label>
|
||||
<select name="host_type">
|
||||
<option value="openwebui"{ow}>Open WebUI / Ollama</option>
|
||||
<option value="openai"{ai}>OpenAI-compatible API</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="field" style="flex:0 0 auto; width:6rem">
|
||||
<label>Max parallel</label>
|
||||
<input type="number" name="max_concurrent" min="1" max="20"
|
||||
value="{maxc}" style="width:100%">
|
||||
</div>
|
||||
</div>
|
||||
<div class="btn-row">
|
||||
<button type="submit" class="btn btn-secondary btn-sm">Save</button>
|
||||
<button type="button" class="btn btn-secondary btn-sm fetch-btn"
|
||||
data-host-id="{hid}">Fetch models</button>
|
||||
<span class="fetch-status" id="fetch-{hid}"></span>
|
||||
</div>
|
||||
</form>
|
||||
<form method="POST" action="/settings/local/host/{hid}/remove"
|
||||
onsubmit="return confirm('Remove host and all its models?')"
|
||||
style="margin-top:0.5rem">
|
||||
<button type="submit" class="btn-link danger">Remove host</button>
|
||||
</form>
|
||||
</div>'''
|
||||
|
||||
|
||||
# ── Auth helper ───────────────────────────────────────────────────────────────
|
||||
|
||||
def _get_user(request: Request) -> str | None:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
return None
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
return None
|
||||
|
||||
|
||||
# ── Page renderer ─────────────────────────────────────────────────────────────
|
||||
|
||||
def _render(username: str, request: Request | None = None, success: str = "", error: str = "") -> str:
|
||||
registry = reg.get_registry(username)
|
||||
hosts = registry.get("hosts", [])
|
||||
models = registry.get("models", [])
|
||||
roles = registry.get("roles", {})
|
||||
builtins = reg._builtins()
|
||||
host_by_id = {h["id"]: h for h in hosts}
|
||||
goog_accts = registry.get("providers", {}).get("google", {}).get("accounts", [])
|
||||
|
||||
# ── Google account rows ───────────────────────────────────────────────────
|
||||
google_account_rows = ""
|
||||
for a in goog_accts:
|
||||
hint = (a.get("api_key") or "")[:10] + "…" if a.get("api_key") else "no key"
|
||||
google_account_rows += f'''
|
||||
<div class="account-row">
|
||||
<div>
|
||||
<span class="account-label">{a.get("label") or "Unnamed"}</span>
|
||||
<span class="account-hint">{hint}</span>
|
||||
</div>
|
||||
<form method="POST" action="/settings/local/google-account/{a["id"]}/remove"
|
||||
onsubmit="return confirm('Remove this Google account?')">
|
||||
<button type="submit" class="btn-link danger">Remove</button>
|
||||
</form>
|
||||
</div>'''
|
||||
if not google_account_rows:
|
||||
google_account_rows = '<p class="empty-note">No accounts configured yet.</p>'
|
||||
|
||||
# ── Host rows — split cloud (openai) vs local (openwebui) ─────────────────
|
||||
cloud_hosts = [h for h in hosts if h.get("host_type") == "openai"]
|
||||
local_hosts = [h for h in hosts if h.get("host_type", "openwebui") != "openai"]
|
||||
|
||||
cloud_host_rows = "".join(_host_row_html(h) for h in cloud_hosts)
|
||||
local_host_rows = "".join(_host_row_html(h) for h in local_hosts)
|
||||
if not cloud_host_rows:
|
||||
cloud_host_rows = '<p class="empty-note">No cloud API services configured yet. Add one below.</p>'
|
||||
if not local_host_rows:
|
||||
local_host_rows = '<p class="empty-note">No local hosts configured yet. Add one below.</p>'
|
||||
|
||||
host_options = "".join(
|
||||
f'<option value="{h["id"]}">{h.get("label") or h["api_url"]}</option>'
|
||||
for h in hosts
|
||||
)
|
||||
|
||||
# ── Anthropic API key rows ────────────────────────────────────────────────
|
||||
anthropic_api_keys = reg.get_anthropic_api_keys(username)
|
||||
anthropic_keys_js = _json.dumps(anthropic_api_keys)
|
||||
anthropic_key_rows = ""
|
||||
for c in anthropic_api_keys:
|
||||
hint = c.get("hint", "no key")
|
||||
anthropic_key_rows += f'''
|
||||
<div class="account-row">
|
||||
<div>
|
||||
<span class="account-label">{c.get("label") or "API Key"}</span>
|
||||
<span class="account-hint">{hint}</span>
|
||||
</div>
|
||||
<form method="POST" action="/settings/local/anthropic-key/{c["id"]}/remove"
|
||||
onsubmit="return confirm('Remove this Anthropic API key?')">
|
||||
<button type="submit" class="btn-link danger">Remove</button>
|
||||
</form>
|
||||
</div>'''
|
||||
if not anthropic_key_rows:
|
||||
anthropic_key_rows = '<p class="empty-note">No API keys configured. Add one below or use Claude CLI (OAuth).</p>'
|
||||
|
||||
# ── Model rows (all providers) ────────────────────────────────────────────
|
||||
_PROVIDER_BADGE = {
|
||||
"claude_cli": ('<span class="pbadge pb-anthropic">Anthropic</span>', "Claude CLI"),
|
||||
"anthropic_api": ('<span class="pbadge pb-anthropic">Anthropic</span>', "API Key"),
|
||||
"gemini_api": ('<span class="pbadge pb-google">Google</span>', ""),
|
||||
"local_openai": ('<span class="pbadge pb-local">Local</span>', ""),
|
||||
}
|
||||
model_rows = ""
|
||||
for m in models:
|
||||
resolved = reg._resolve_model(registry, m["id"])
|
||||
if not resolved:
|
||||
continue
|
||||
mtype = m.get("type", "local_openai")
|
||||
badge, default_secondary = _PROVIDER_BADGE.get(mtype, ("", ""))
|
||||
|
||||
if mtype == "local_openai":
|
||||
h = host_by_id.get(m.get("host_id", ""), {})
|
||||
secondary = h.get("label") or h.get("api_url", "")
|
||||
elif mtype == "gemini_api":
|
||||
acct = next((a for a in goog_accts if a["id"] == m.get("account_id")), None)
|
||||
secondary = acct["label"] if acct else ""
|
||||
else:
|
||||
secondary = default_secondary
|
||||
|
||||
ctx = f'<span class="ctx-badge">{m.get("context_k",0)}k</span>' if m.get("context_k") else ""
|
||||
no_tools = '' if m.get("tools", True) else '<span class="pbadge pb-notools">no tools</span>'
|
||||
tags_html = " ".join(f'<span class="tag">{t}</span>' for t in (m.get("tags") or []))
|
||||
sec = f'<span class="model-host">{secondary}</span>' if secondary else ""
|
||||
|
||||
# ── Inline edit form fields (type-specific) ───────────────────────────
|
||||
if mtype == "local_openai":
|
||||
host_opts = "".join(
|
||||
f'<option value="{h["id"]}"'
|
||||
f'{" selected" if h["id"] == m.get("host_id") else ""}>'
|
||||
f'{h.get("label") or h.get("api_url","")}</option>'
|
||||
for h in hosts
|
||||
)
|
||||
mid = m["id"]
|
||||
extra_fields = (
|
||||
f'<div class="field"><label>Host</label>'
|
||||
f'<select name="host_id" id="edit-host-{mid}">{host_opts}</select></div>'
|
||||
f'<div class="btn-row" style="margin-bottom:0.75rem">'
|
||||
f'<button type="button" class="btn btn-secondary btn-sm edit-fetch-btn" data-id="{mid}">Fetch models</button>'
|
||||
f'<span class="fetch-status" id="edit-fetch-status-{mid}"></span>'
|
||||
f'</div>'
|
||||
f'<div id="edit-model-select-wrap-{mid}" style="display:none; margin-bottom:0.75rem">'
|
||||
f'<label>Pick from host</label>'
|
||||
f'<select id="edit-model-picker-{mid}"><option value="">— fetch first —</option></select>'
|
||||
f'</div>'
|
||||
)
|
||||
elif mtype == "gemini_api":
|
||||
acct_opts = "".join(
|
||||
f'<option value="{a["id"]}"'
|
||||
f'{" selected" if a["id"] == m.get("account_id") else ""}>'
|
||||
f'{a.get("label","Unnamed")}</option>'
|
||||
for a in goog_accts
|
||||
)
|
||||
extra_fields = (
|
||||
f'<div class="field"><label>Google Account</label>'
|
||||
f'<select name="account_id">{acct_opts}</select></div>'
|
||||
)
|
||||
elif mtype == "anthropic_api":
|
||||
key_opts = "".join(
|
||||
f'<option value="{c["id"]}"'
|
||||
f'{" selected" if c["id"] == m.get("credential_id") else ""}>'
|
||||
f'{c.get("label","API Key")} ({c.get("hint","")})</option>'
|
||||
for c in anthropic_api_keys
|
||||
)
|
||||
extra_fields = (
|
||||
f'<div class="field"><label>API Key</label>'
|
||||
f'<select name="credential_id">{key_opts or "<option value=\"\">No API keys configured</option>"}</select></div>'
|
||||
)
|
||||
else:
|
||||
extra_fields = '<input type="hidden" name="credential_id" value="cli">'
|
||||
|
||||
cur_label = m.get("label", "")
|
||||
cur_model_name = m.get("model_name", "")
|
||||
cur_ctx = m.get("context_k", 0) or 0
|
||||
cur_max_rounds = m.get("max_rounds") or 0
|
||||
cur_tools = m.get("tools", True)
|
||||
cur_tags = ", ".join(m.get("tags") or [])
|
||||
cur_reasoning_budget = m.get("reasoning_budget_tokens") or 0
|
||||
_rb_levels = [(0, "Off — Non-think"), (1024, "Light"), (4096, "Moderate"), (8192, "High"), (32768, "Max")]
|
||||
reasoning_opts = "".join(
|
||||
f'<option value="{v}" {"selected" if cur_reasoning_budget == v else ""}>{lbl}</option>'
|
||||
for v, lbl in _rb_levels
|
||||
)
|
||||
|
||||
model_rows += f'''
|
||||
<div class="model-row" id="model-{m["id"]}">
|
||||
<div class="model-row-header">
|
||||
<div class="model-info">
|
||||
<div>{badge}<span class="model-label">{m.get("label") or m.get("model_name","")}</span>{ctx}{no_tools}</div>
|
||||
<span class="model-name">{m.get("model_name","")}</span>
|
||||
{sec}
|
||||
<div class="tag-row">{tags_html}</div>
|
||||
</div>
|
||||
<div class="model-btns">
|
||||
<button type="button" class="row-btn model-edit-btn" data-id="{m["id"]}">Edit</button>
|
||||
<form method="POST" action="/settings/local/models/{m["id"]}/remove"
|
||||
onsubmit="return confirm('Remove this model?')" style="margin:0">
|
||||
<button type="submit" class="row-btn danger">Remove</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
<form class="model-edit-form" id="edit-form-{m["id"]}" style="display:none"
|
||||
method="POST" action="/settings/local/models/{m["id"]}/edit">
|
||||
<input type="hidden" name="mtype" value="{mtype}">
|
||||
<div class="field-row">
|
||||
<div class="field">
|
||||
<label>Display label</label>
|
||||
<input type="text" name="label" value="{cur_label}"
|
||||
placeholder="My Model" autocomplete="off" data-form-type="other">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>Model name / ID</label>
|
||||
<input type="text" name="model_name" value="{cur_model_name}"
|
||||
placeholder="provider/model-name" autocomplete="off"
|
||||
spellcheck="false" data-form-type="other" required>
|
||||
</div>
|
||||
</div>
|
||||
{extra_fields}
|
||||
<div class="field-row">
|
||||
<div class="field" style="flex:0 0 auto">
|
||||
<label title="Context window size in thousands of tokens. 0 = assume 32k.">Context (k)</label>
|
||||
<input type="number" name="context_k" value="{cur_ctx}" min="0"
|
||||
title="Context window size in thousands of tokens. 0 = assume 32k (compaction budget ~24k tokens).">
|
||||
</div>
|
||||
<div class="field" style="flex:0 0 auto">
|
||||
<label title="Per-model tool loop cap. 0 = use the global default (orchestrator_max_rounds).">Max rounds</label>
|
||||
<input type="number" name="max_rounds" value="{cur_max_rounds}" min="0"
|
||||
title="Per-model tool loop cap. 0 = use the global default (orchestrator_max_rounds).">
|
||||
</div>
|
||||
<div class="field" style="flex:0 0 auto">
|
||||
<label title="Reasoning depth via OpenRouter's reasoning.budget_tokens. Off = Non-think. Light ~1k, Moderate ~4k, High ~8k, Max ~32k tokens.">Reasoning</label>
|
||||
<select name="reasoning_budget_tokens"
|
||||
title="Reasoning depth via OpenRouter's reasoning.budget_tokens. Off = Non-think. Light ~1k, Moderate ~4k, High ~8k, Max ~32k tokens.">
|
||||
{reasoning_opts}
|
||||
</select>
|
||||
</div>
|
||||
<div class="field" style="flex:0 0 auto">
|
||||
<label title="Whether this model supports tool calling. If not supported, requests skip the tool loop entirely.">Tool calling</label>
|
||||
<select name="tools"
|
||||
title="Whether this model supports tool calling. If not supported, requests skip the tool loop entirely.">
|
||||
<option value="1" {'selected' if cur_tools else ''}>Supported</option>
|
||||
<option value="0" {'' if cur_tools else 'selected'}>Not supported</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>Tags</label>
|
||||
<input type="text" name="tags" value="{cur_tags}"
|
||||
placeholder="fast, code, vision" autocomplete="off" data-form-type="other">
|
||||
</div>
|
||||
</div>
|
||||
<div class="btn-row" style="margin-top:0.5rem">
|
||||
<button type="submit" class="btn btn-primary btn-sm">Save</button>
|
||||
<button type="button" class="model-edit-cancel btn btn-secondary btn-sm"
|
||||
data-id="{m["id"]}">Cancel</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>'''
|
||||
if not model_rows:
|
||||
model_rows = '<p class="empty-note">No models added yet.</p>'
|
||||
|
||||
# ── Role assignment rows ──────────────────────────────────────────────────
|
||||
model_opts = '<option value="">— .env default —</option>\n'
|
||||
model_opts += '<optgroup label="Built-in">\n'
|
||||
for bid, bm in builtins.items():
|
||||
model_opts += f' <option value="{bid}">{bm["label"]}</option>\n'
|
||||
model_opts += '</optgroup>\n'
|
||||
if models:
|
||||
model_opts += '<optgroup label="Configured models">\n'
|
||||
for m in models:
|
||||
lbl = m.get("label") or m.get("model_name", m["id"])
|
||||
model_opts += f' <option value="{m["id"]}">{lbl}</option>\n'
|
||||
model_opts += '</optgroup>\n'
|
||||
|
||||
all_roles = reg.get_all_roles(username)
|
||||
|
||||
role_rows = ""
|
||||
for role in all_roles:
|
||||
is_required = role in reg.REQUIRED_ROLES
|
||||
role_cfg = roles.get(role, {})
|
||||
role_title = role.replace("_", " ").title()
|
||||
required_badge = (
|
||||
'<span class="required-badge">required</span>'
|
||||
if is_required else ''
|
||||
)
|
||||
rcp_danger = (
|
||||
'' if is_required else
|
||||
f'<div class="rcp-danger">'
|
||||
f'<form method="POST" action="/settings/local/roles/remove" class="remove-role-form">'
|
||||
f'<input type="hidden" name="role_name" value="{role}">'
|
||||
f'<button type="submit" class="btn-link danger" data-role="{role}">Remove this role…</button>'
|
||||
f'</form>'
|
||||
f'</div>'
|
||||
)
|
||||
role_rows += (
|
||||
f'<div class="role-row" data-role="{role}">'
|
||||
f'<div class="role-name-col">'
|
||||
f'<span class="role-name">{role_title}</span>'
|
||||
f'{required_badge}'
|
||||
f'</div>'
|
||||
f'<div class="role-slots">'
|
||||
)
|
||||
for slot in reg.PRIORITY_KEYS[:2]:
|
||||
slot_label = slot.replace("_", " ").title()
|
||||
sel = (
|
||||
f'<select class="role-select" data-role="{role}" '
|
||||
f'data-slot="{slot}" title="{slot_label}">\n{model_opts}\n</select>'
|
||||
)
|
||||
role_rows += f'<div class="role-slot"><span class="slot-label">{slot_label}</span>{sel}</div>'
|
||||
role_rows += (
|
||||
f'</div>'
|
||||
f'<button class="role-cfg-btn" data-role="{role}" title="Configure">⚙</button>'
|
||||
f'</div>'
|
||||
f'<div class="role-config-panel" id="rcp-{role}">'
|
||||
f'<div class="rcp-field">'
|
||||
f'<label class="rcp-label">System prompt addition</label>'
|
||||
f'<textarea class="rcp-textarea" data-role="{role}" rows="3" '
|
||||
f'placeholder="Extra instructions injected into the system prompt when this role is active…"></textarea>'
|
||||
f'</div>'
|
||||
f'<div class="rcp-field">'
|
||||
f'<div style="display:flex;flex-direction:column;gap:0.3rem">'
|
||||
f'<label class="rcp-check">'
|
||||
f'<input type="checkbox" class="rcp-datetime-cb" data-role="{role}" checked>'
|
||||
f'<span>Inject current date & time into system prompt</span>'
|
||||
f'</label>'
|
||||
f'<label class="rcp-check">'
|
||||
f'<input type="checkbox" class="rcp-mode-cb" data-role="{role}" checked>'
|
||||
f'<span>Inject session mode (Chat / Off The Record) into system prompt</span>'
|
||||
f'</label>'
|
||||
f'</div>'
|
||||
f'<p class="rcp-hint" style="margin-top:0.4rem">Disable both for pure processing roles (summarizer, classifier, translator).</p>'
|
||||
f'</div>'
|
||||
f'<div class="rcp-field">'
|
||||
f'<label class="rcp-label">Tool allow-list '
|
||||
f'<span class="rcp-hint">— all checked means no restriction (use all accessible tools)</span></label>'
|
||||
f'<div class="rcp-tools" id="rcp-tools-{role}"></div>'
|
||||
f'</div>'
|
||||
f'<div class="rcp-actions">'
|
||||
f'<button class="btn btn-primary btn-sm rcp-save" data-role="{role}">Save</button>'
|
||||
f'<button class="btn btn-secondary btn-sm rcp-cancel" data-role="{role}">Cancel</button>'
|
||||
f'</div>'
|
||||
f'{rcp_danger}'
|
||||
f'</div>'
|
||||
)
|
||||
|
||||
role_data_js = _json.dumps({
|
||||
role: {slot: (roles.get(role, {}).get(slot) or "") for slot in reg.PRIORITY_KEYS[:2]}
|
||||
for role in all_roles
|
||||
})
|
||||
|
||||
role_config_data_js = _json.dumps({
|
||||
role: {
|
||||
"system_append": roles.get(role, {}).get("system_append", ""),
|
||||
"tools": roles.get(role, {}).get("tools") or None,
|
||||
"inject_datetime": roles.get(role, {}).get("inject_datetime", True),
|
||||
"inject_mode": roles.get(role, {}).get("inject_mode", True),
|
||||
}
|
||||
for role in all_roles
|
||||
})
|
||||
tool_categories_js = _json.dumps(TOOL_CATEGORIES)
|
||||
|
||||
# ── Catalog data + Google accounts for JS ─────────────────────────────────
|
||||
google_accounts_js = _json.dumps(reg.get_google_accounts(username))
|
||||
google_catalog_js = _json.dumps(reg.get_catalog("google"))
|
||||
anthropic_catalog_js = _json.dumps(reg.get_catalog("anthropic"))
|
||||
cloud_catalog_js = _json.dumps(reg.get_catalog("cloud"))
|
||||
has_hosts = "true" if hosts else "false"
|
||||
|
||||
html = (_STATIC / "local_llm.html").read_text()
|
||||
replacements = {
|
||||
"{{ username }}": username,
|
||||
"{{ google_account_rows }}": google_account_rows,
|
||||
"{{ anthropic_key_rows }}": anthropic_key_rows,
|
||||
"{{ cloud_host_rows }}": cloud_host_rows,
|
||||
"{{ local_host_rows }}": local_host_rows,
|
||||
"{{ model_rows }}": model_rows,
|
||||
"{{ host_options }}": host_options,
|
||||
"{{ role_rows }}": role_rows,
|
||||
"{{ role_data_js }}": role_data_js,
|
||||
"{{ role_config_data_js }}": role_config_data_js,
|
||||
"{{ tool_categories_js }}": tool_categories_js,
|
||||
"{{ google_accounts_js }}": google_accounts_js,
|
||||
"{{ anthropic_keys_js }}": anthropic_keys_js,
|
||||
"{{ google_catalog_js }}": google_catalog_js,
|
||||
"{{ anthropic_catalog_js }}": anthropic_catalog_js,
|
||||
"{{ cloud_catalog_js }}": cloud_catalog_js,
|
||||
"{{ has_hosts }}": has_hosts,
|
||||
}
|
||||
for key, val in replacements.items():
|
||||
html = html.replace(key, val)
|
||||
|
||||
back_persona = _preferred_persona(request, username) if request else ""
|
||||
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
|
||||
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
|
||||
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
|
||||
|
||||
if success:
|
||||
html = html.replace("<!-- SUCCESS -->", f'<p class="msg success">{success}</p>')
|
||||
if error:
|
||||
html = html.replace("<!-- ERROR -->", f'<p class="msg error">{error}</p>')
|
||||
return html
|
||||
|
||||
|
||||
# ── Routes ────────────────────────────────────────────────────────────────────
|
||||
|
||||
@router.get("/settings/models", include_in_schema=False)
|
||||
async def models_page_canonical(request: Request):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
return HTMLResponse(_render(username, request))
|
||||
|
||||
|
||||
@router.get("/settings/local", include_in_schema=False)
|
||||
async def models_page_legacy(request: Request):
|
||||
return RedirectResponse("/settings/models", status_code=301)
|
||||
|
||||
|
||||
@router.post("/settings/local/google-account", include_in_schema=False)
|
||||
async def save_google_account(
|
||||
request: Request,
|
||||
account_id: str = Form(""),
|
||||
label: str = Form(""),
|
||||
api_key: str = Form(""),
|
||||
):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
if not api_key.strip() and not account_id.strip():
|
||||
return HTMLResponse(_render(username, request, error="API key is required."))
|
||||
reg.save_google_account(username, account_id or None, label, api_key)
|
||||
return HTMLResponse(_render(username, request, success="Google account saved."))
|
||||
|
||||
|
||||
@router.post("/settings/local/google-account/{account_id}/remove", include_in_schema=False)
|
||||
async def remove_google_account(request: Request, account_id: str):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
reg.remove_google_account(username, account_id)
|
||||
return HTMLResponse(_render(username, request, success="Google account removed."))
|
||||
|
||||
|
||||
@router.post("/settings/local/anthropic-key", include_in_schema=False)
|
||||
async def save_anthropic_api_key(
|
||||
request: Request,
|
||||
key_id: str = Form(""),
|
||||
label: str = Form(""),
|
||||
api_key: str = Form(""),
|
||||
):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
if not api_key.strip() and not key_id.strip():
|
||||
return HTMLResponse(_render(username, request, error="API key is required."))
|
||||
reg.save_anthropic_api_key(username, key_id or None, label, api_key)
|
||||
return HTMLResponse(_render(username, request, success="Anthropic API key saved."))
|
||||
|
||||
|
||||
@router.post("/settings/local/anthropic-key/{key_id}/remove", include_in_schema=False)
|
||||
async def remove_anthropic_api_key(request: Request, key_id: str):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
reg.remove_anthropic_api_key(username, key_id)
|
||||
return HTMLResponse(_render(username, request, success="Anthropic API key removed."))
|
||||
|
||||
|
||||
@router.post("/settings/local/host", include_in_schema=False)
|
||||
async def save_host(
|
||||
request: Request,
|
||||
host_id: str = Form(""),
|
||||
label: str = Form(""),
|
||||
api_url: str = Form(""),
|
||||
api_key: str = Form(""),
|
||||
host_type: str = Form("openwebui"),
|
||||
max_concurrent: int = Form(3),
|
||||
):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
if not api_url.strip():
|
||||
return HTMLResponse(_render(username, request, error="API URL is required."))
|
||||
reg.save_host(username, host_id or None, label, api_url, api_key, host_type, max_concurrent)
|
||||
return HTMLResponse(_render(username, request, success="Host saved."))
|
||||
|
||||
|
||||
@router.post("/settings/local/host/{host_id}/remove", include_in_schema=False)
|
||||
async def remove_host(request: Request, host_id: str):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
reg.remove_host(username, host_id)
|
||||
return HTMLResponse(_render(username, request, success="Host removed."))
|
||||
|
||||
|
||||
@router.post("/settings/local/models/add", include_in_schema=False)
|
||||
async def add_model(
|
||||
request: Request,
|
||||
provider: str = Form("local"),
|
||||
label: str = Form(""),
|
||||
context_k: int = Form(0),
|
||||
max_rounds: int = Form(0),
|
||||
tools: int = Form(1),
|
||||
tags: str = Form(""),
|
||||
reasoning_budget_tokens: int = Form(0),
|
||||
# local-only fields
|
||||
host_id: str = Form(""),
|
||||
model_name: str = Form(""),
|
||||
# cloud-only fields
|
||||
cloud_model_name: str = Form(""),
|
||||
account_id: str = Form(""),
|
||||
credential_id: str = Form("cli"),
|
||||
):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
tag_list = [t.strip() for t in tags.split(",") if t.strip()]
|
||||
max_rounds_ = max_rounds or None
|
||||
tools_bool = tools != 0
|
||||
reasoning_budget_ = reasoning_budget_tokens or None
|
||||
|
||||
if provider == "local":
|
||||
if not model_name.strip():
|
||||
return HTMLResponse(_render(username, request, error="Model name is required."))
|
||||
if not host_id.strip():
|
||||
return HTMLResponse(_render(username, request, error="Select a host."))
|
||||
reg.save_model(username, None, host_id, label, model_name, context_k, tag_list,
|
||||
max_rounds=max_rounds_, tools=tools_bool,
|
||||
reasoning_budget_tokens=reasoning_budget_)
|
||||
display = label or model_name
|
||||
|
||||
elif provider in ("google", "anthropic"):
|
||||
if not cloud_model_name.strip():
|
||||
return HTMLResponse(_render(username, request, error="Select a model from the catalog."))
|
||||
if provider == "google" and not account_id.strip():
|
||||
return HTMLResponse(_render(username, request, error="Select a Google account."))
|
||||
reg.save_cloud_model(
|
||||
username, None, provider, cloud_model_name, label,
|
||||
account_id=account_id or None,
|
||||
credential_id=credential_id or None,
|
||||
context_k=context_k, tags=tag_list,
|
||||
max_rounds=max_rounds_, tools=tools_bool,
|
||||
)
|
||||
display = label or cloud_model_name
|
||||
else:
|
||||
return HTMLResponse(_render(username, request, error=f"Unknown provider: {provider}"))
|
||||
|
||||
logger.info("model added: %s / %s (%s)", username, display, provider)
|
||||
return HTMLResponse(_render(username, request, success=f'Model "{display}" added.'))
|
||||
|
||||
|
||||
@router.post("/settings/local/models/{model_id}/edit", include_in_schema=False)
|
||||
async def edit_model(
|
||||
request: Request,
|
||||
model_id: str,
|
||||
mtype: str = Form(""),
|
||||
label: str = Form(""),
|
||||
model_name: str = Form(""),
|
||||
context_k: int = Form(0),
|
||||
max_rounds: int = Form(0),
|
||||
tools: int = Form(1),
|
||||
tags: str = Form(""),
|
||||
reasoning_budget_tokens: int = Form(0),
|
||||
host_id: str = Form(""),
|
||||
account_id: str = Form(""),
|
||||
credential_id: str = Form("cli"),
|
||||
):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
if not model_name.strip():
|
||||
return HTMLResponse(_render(username, request, error="Model name is required."))
|
||||
tag_list = [t.strip() for t in tags.split(",") if t.strip()]
|
||||
max_rounds_ = max_rounds or None
|
||||
tools_bool = tools != 0
|
||||
reasoning_budget_ = reasoning_budget_tokens or None
|
||||
if mtype == "local_openai":
|
||||
if not host_id.strip():
|
||||
return HTMLResponse(_render(username, request, error="Select a host for this model."))
|
||||
reg.save_model(username, model_id, host_id, label, model_name, context_k, tag_list,
|
||||
max_rounds=max_rounds_, tools=tools_bool,
|
||||
reasoning_budget_tokens=reasoning_budget_)
|
||||
elif mtype == "gemini_api":
|
||||
reg.save_cloud_model(username, model_id, "google", model_name, label,
|
||||
account_id=account_id or None, context_k=context_k, tags=tag_list,
|
||||
max_rounds=max_rounds_, tools=tools_bool)
|
||||
elif mtype in ("claude_cli", "anthropic_api"):
|
||||
reg.save_cloud_model(username, model_id, "anthropic", model_name, label,
|
||||
credential_id=credential_id or "cli", context_k=context_k, tags=tag_list,
|
||||
max_rounds=max_rounds_, tools=tools_bool)
|
||||
else:
|
||||
return HTMLResponse(_render(username, request, error=f"Unknown model type: {mtype}"))
|
||||
display = label.strip() or model_name.strip()
|
||||
logger.info("model edited: %s / %s (%s)", username, display, mtype)
|
||||
return HTMLResponse(_render(username, request, success=f'Model "{display}" updated.'))
|
||||
|
||||
|
||||
@router.post("/settings/local/models/{model_id}/remove", include_in_schema=False)
|
||||
async def remove_model(request: Request, model_id: str):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
reg.remove_model(username, model_id)
|
||||
return HTMLResponse(_render(username, request, success="Model removed."))
|
||||
|
||||
|
||||
@router.post("/settings/local/roles/add", include_in_schema=False)
|
||||
async def add_custom_role_route(
|
||||
request: Request,
|
||||
role_name: str = Form(""),
|
||||
):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
name = role_name.strip().lower()
|
||||
if not name or not name[0].isalpha():
|
||||
return HTMLResponse(_render(username, request, error="Role name must start with a letter."))
|
||||
ok = reg.add_custom_role(username, name)
|
||||
if not ok:
|
||||
return HTMLResponse(_render(username, request, error=f'"{name}" is a required role and cannot be re-added.'))
|
||||
logger.info("custom role added: %s / %s", username, name)
|
||||
return RedirectResponse("/settings/models#roles", status_code=303)
|
||||
|
||||
|
||||
@router.post("/settings/local/roles/remove", include_in_schema=False)
|
||||
async def remove_custom_role_route(
|
||||
request: Request,
|
||||
role_name: str = Form(""),
|
||||
):
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
name = role_name.strip()
|
||||
ok = reg.remove_custom_role(username, name)
|
||||
if not ok:
|
||||
return HTMLResponse(_render(username, request, error=f'"{name}" is a required role and cannot be removed.'))
|
||||
logger.info("custom role removed: %s / %s", username, name)
|
||||
return RedirectResponse("/settings/models#roles", status_code=303)
|
||||
|
||||
|
||||
@router.post("/api/models/role")
|
||||
async def set_role(request: Request) -> JSONResponse:
|
||||
"""AJAX: assign a model to a role priority slot.
|
||||
|
||||
Body: {"role": "chat", "slot": "primary", "model_id": "abc123" | ""}
|
||||
"""
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return JSONResponse({"error": "Not authenticated"}, status_code=401)
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
|
||||
|
||||
role = body.get("role", "").strip()
|
||||
slot = body.get("slot", "").strip()
|
||||
model_id = body.get("model_id", "").strip() or None
|
||||
|
||||
if not role or not slot:
|
||||
return JSONResponse({"error": "role and slot are required"}, status_code=400)
|
||||
|
||||
ok = reg.set_role(username, role, slot, model_id)
|
||||
if not ok:
|
||||
return JSONResponse({"error": "Invalid slot or model_id not found"}, status_code=400)
|
||||
|
||||
logger.info("role set: %s %s.%s = %s", username, role, slot, model_id)
|
||||
return JSONResponse({"ok": True})
|
||||
|
||||
|
||||
@router.post("/api/models/role-config")
|
||||
async def set_role_config(request: Request) -> JSONResponse:
|
||||
"""AJAX: save system_append, tool allow-list, and inject_datetime flag for a role.
|
||||
|
||||
Body: {"role": "coder", "system_append": "...", "tools": [...] | null, "inject_datetime": true}
|
||||
tools=null clears the allow-list (role uses all accessible tools).
|
||||
inject_datetime=false suppresses the date/time header for pure processing roles.
|
||||
"""
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return JSONResponse({"error": "Not authenticated"}, status_code=401)
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
|
||||
|
||||
role = body.get("role", "").strip()
|
||||
system_append = body.get("system_append", "")
|
||||
tools = body.get("tools") # list[str] or None
|
||||
inject_datetime = body.get("inject_datetime", True)
|
||||
inject_mode = body.get("inject_mode", True)
|
||||
|
||||
if not role:
|
||||
return JSONResponse({"error": "role is required"}, status_code=400)
|
||||
if tools is not None and not isinstance(tools, list):
|
||||
return JSONResponse({"error": "tools must be a list or null"}, status_code=400)
|
||||
|
||||
reg.set_role_config(username, role, system_append, tools,
|
||||
inject_datetime=bool(inject_datetime),
|
||||
inject_mode=bool(inject_mode))
|
||||
logger.info("role config saved: %s %s (tools=%s inject_datetime=%s inject_mode=%s)",
|
||||
username, role, len(tools) if tools is not None else "all",
|
||||
inject_datetime, inject_mode)
|
||||
return JSONResponse({"ok": True})
|
||||
|
||||
|
||||
@router.get("/api/local-llm/fetch-models")
|
||||
async def fetch_models(request: Request, host_id: str = "") -> JSONResponse:
|
||||
"""Proxy to the host's models endpoint. host_id selects which host."""
|
||||
username = _get_user(request)
|
||||
if not username:
|
||||
return JSONResponse({"error": "Not authenticated"}, status_code=401)
|
||||
|
||||
registry = reg.get_registry(username)
|
||||
hosts = registry.get("hosts", [])
|
||||
|
||||
host = next((h for h in hosts if h["id"] == host_id), None) if host_id else (hosts[0] if hosts else None)
|
||||
|
||||
if host:
|
||||
api_url, api_key, host_type = host.get("api_url",""), host.get("api_key",""), host.get("host_type","openwebui")
|
||||
else:
|
||||
api_url, api_key, host_type = app_settings.local_api_url, app_settings.local_api_key, "openwebui"
|
||||
|
||||
if not api_url:
|
||||
return JSONResponse({"error": "No host configured."}, status_code=400)
|
||||
|
||||
models_path = "/models" if host_type == "openai" else "/api/models"
|
||||
url = api_url.rstrip("/") + models_path
|
||||
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=8) as client:
|
||||
resp = await client.get(url, headers=headers)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
models = sorted(
|
||||
[{"id": m["id"], "name": m.get("name") or m["id"]} for m in data.get("data", [])],
|
||||
key=lambda m: m["name"].lower(),
|
||||
)
|
||||
return JSONResponse({"models": models})
|
||||
except httpx.HTTPStatusError as e:
|
||||
return JSONResponse({"error": f"Host returned {e.response.status_code}"}, status_code=502)
|
||||
except Exception as e:
|
||||
return JSONResponse({"error": str(e)}, status_code=502)
|
||||
@@ -3,17 +3,21 @@ import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, Response
|
||||
|
||||
from config import settings
|
||||
from auth_utils import get_user_channels, get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
|
||||
from context_loader import load_context
|
||||
from llm_client import complete
|
||||
from notification import _send_nct_message
|
||||
from persona import set_context
|
||||
from session_logger import log_turn
|
||||
from session_store import load as load_session, save as save_session
|
||||
from config import settings
|
||||
import event_bus
|
||||
import model_registry
|
||||
import orchestrator_engine
|
||||
import openai_orchestrator
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.setLevel(logging.DEBUG)
|
||||
@@ -26,58 +30,44 @@ if not logger.handlers:
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _verify_signature(body: bytes, random_header: str, sig_header: str) -> bool:
|
||||
def _verify_signature(body: bytes, random_header: str, sig_header: str, secret: str) -> bool:
|
||||
"""Nextcloud signs requests with HMAC-SHA256(key=secret, msg=random+body)."""
|
||||
expected = hmac.new(
|
||||
settings.nextcloud_talk_bot_secret.encode(),
|
||||
secret.encode(),
|
||||
(random_header + body.decode("utf-8", errors="replace")).encode(),
|
||||
hashlib.sha256,
|
||||
).hexdigest()
|
||||
return hmac.compare_digest(expected, sig_header.lower())
|
||||
|
||||
|
||||
async def _send_reply(conversation_token: str, message: str) -> None:
|
||||
async def _send_reply(conversation_token: str, message: str, nextcloud_url: str, secret: str) -> None:
|
||||
"""Post a message to Nextcloud Talk as the bot."""
|
||||
url = (
|
||||
f"{settings.nextcloud_url}/ocs/v2.php/apps/spreed/api/v1"
|
||||
f"/bot/{conversation_token}/message"
|
||||
)
|
||||
# NC Talk verifies HMAC over (random + message_text), NOT the raw body.
|
||||
# See BotController::getBotFromHeaders → checksumVerificationService::validateRequest($random, $sig, $secret, $message)
|
||||
body_dict = {"message": message}
|
||||
body_bytes = json.dumps(body_dict, ensure_ascii=False).encode("utf-8")
|
||||
random_str = secrets.token_hex(32)
|
||||
sig = hmac.new(
|
||||
settings.nextcloud_talk_bot_secret.encode(),
|
||||
(random_str + message).encode("utf-8"),
|
||||
hashlib.sha256,
|
||||
).hexdigest()
|
||||
|
||||
logger.info("NCT _send_reply → %s (body: %s)", url, body_bytes.decode())
|
||||
try:
|
||||
async with httpx.AsyncClient() as client:
|
||||
resp = await client.post(
|
||||
url,
|
||||
content=body_bytes,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"OCS-APIRequest": "true",
|
||||
"X-Nextcloud-Talk-Bot-Random": random_str,
|
||||
"X-Nextcloud-Talk-Bot-Signature": sig,
|
||||
},
|
||||
timeout=15,
|
||||
)
|
||||
logger.info("NCT reply: %s — %s", resp.status_code, resp.text[:400])
|
||||
except Exception as e:
|
||||
logger.error("NCT reply error: %s", e)
|
||||
logger.info("NCT _send_reply → room %s (%d chars)", conversation_token, len(message))
|
||||
await _send_nct_message(nextcloud_url, secret, conversation_token, message)
|
||||
|
||||
|
||||
async def _process_message(conversation_token: str, user_text: str, actor_name: str) -> None:
|
||||
async def _process_message(
|
||||
conversation_token: str,
|
||||
user_text: str,
|
||||
actor_name: str,
|
||||
username: str,
|
||||
persona_name: str,
|
||||
nextcloud_url: str,
|
||||
secret: str,
|
||||
timeout: int,
|
||||
cfg: dict,
|
||||
) -> None:
|
||||
logger.info("NCT process: token=%s user=%s text=%r", conversation_token, actor_name, user_text)
|
||||
session_id = f"nct_{conversation_token}"
|
||||
system_prompt = load_context(settings.default_tier)
|
||||
|
||||
set_context(username, persona_name)
|
||||
|
||||
tier = cfg.get("tier") or settings.default_tier
|
||||
role = cfg.get("role", "chat")
|
||||
use_tools = cfg.get("tools", False)
|
||||
|
||||
session_id = f"nct_{username}_{conversation_token}"
|
||||
history = load_session(session_id)
|
||||
history.append({"role": "user", "content": user_text})
|
||||
session_msgs = list(history) # snapshot before we append
|
||||
|
||||
await event_bus.publish({
|
||||
"type": "nct_message",
|
||||
@@ -87,21 +77,88 @@ async def _process_message(conversation_token: str, user_text: str, actor_name:
|
||||
"actor": actor_name,
|
||||
})
|
||||
|
||||
backend = "unknown"
|
||||
try:
|
||||
response_text, backend = await asyncio.wait_for(
|
||||
complete(system_prompt=system_prompt, messages=history),
|
||||
timeout=settings.nextcloud_talk_timeout,
|
||||
if use_tools:
|
||||
await _send_reply(conversation_token, "⏳ Working on it…", nextcloud_url, secret)
|
||||
|
||||
role_cfg = model_registry.get_role_config(username, role)
|
||||
system_prompt = load_context(
|
||||
tier,
|
||||
role_append=role_cfg.get("system_append", ""),
|
||||
inject_datetime=role_cfg.get("inject_datetime", True),
|
||||
inject_mode=role_cfg.get("inject_mode", True),
|
||||
)
|
||||
orch_model = model_registry.get_model_for_role(username, "orchestrator")
|
||||
user_role_val = get_user_role(username)
|
||||
tool_list = role_cfg.get("tools")
|
||||
policy = get_tool_policy(username)
|
||||
c_allow = set(policy.get("allow", []))
|
||||
c_deny = set(policy.get("deny", []))
|
||||
max_risk, risk_wl, risk_bl = get_risk_policy(username)
|
||||
|
||||
if orch_model and orch_model.get("type") == "local_openai":
|
||||
result = await openai_orchestrator.run(
|
||||
task=user_text,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_msgs or None,
|
||||
model_cfg=orch_model,
|
||||
user_role=user_role_val,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=c_allow,
|
||||
confirm_deny=c_deny,
|
||||
max_risk=max_risk,
|
||||
risk_whitelist=risk_wl,
|
||||
risk_blacklist=risk_bl,
|
||||
)
|
||||
else:
|
||||
gemini_key = (
|
||||
(orch_model.get("api_key") if orch_model else None)
|
||||
or get_user_gemini_key(username)
|
||||
)
|
||||
result = await orchestrator_engine.run(
|
||||
task=user_text,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_msgs or None,
|
||||
respond_with_claude=True,
|
||||
gemini_api_key=gemini_key,
|
||||
model_name=orch_model.get("model_name") if orch_model else None,
|
||||
response_role=role,
|
||||
user_role=user_role_val,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=c_allow,
|
||||
confirm_deny=c_deny,
|
||||
max_risk=max_risk,
|
||||
risk_whitelist=risk_wl,
|
||||
risk_blacklist=risk_bl,
|
||||
)
|
||||
|
||||
response_text = result.response
|
||||
backend = result.backend
|
||||
|
||||
if result.checkpoint:
|
||||
response_text += "\n\n_(This action requires confirmation — use the web UI to approve or deny.)_"
|
||||
|
||||
else:
|
||||
system_prompt = load_context(tier)
|
||||
history_for_llm = list(session_msgs) + [{"role": "user", "content": user_text}]
|
||||
response_text, backend = await asyncio.wait_for(
|
||||
complete(system_prompt=system_prompt, messages=history_for_llm),
|
||||
timeout=timeout,
|
||||
)
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning("NCT timeout for %s", conversation_token)
|
||||
await _send_reply(conversation_token, "⏳ Still thinking — this is taking longer than usual.")
|
||||
await _send_reply(conversation_token, "⏳ Still thinking — this is taking longer than usual.", nextcloud_url, secret)
|
||||
return
|
||||
except Exception as e:
|
||||
logger.error("NCT LLM error for %s: %s", conversation_token, e)
|
||||
await _send_reply(conversation_token, "⚠️ Something went wrong on my end.")
|
||||
await _send_reply(conversation_token, "⚠️ Something went wrong on my end.", nextcloud_url, secret)
|
||||
return
|
||||
|
||||
logger.info("NCT LLM responded via %s (%d chars)", backend, len(response_text))
|
||||
|
||||
history.append({"role": "user", "content": user_text})
|
||||
history.append({"role": "assistant", "content": response_text})
|
||||
save_session(session_id, history)
|
||||
log_turn(session_id, user_text, response_text)
|
||||
@@ -114,22 +171,33 @@ async def _process_message(conversation_token: str, user_text: str, actor_name:
|
||||
"backend": backend,
|
||||
})
|
||||
|
||||
await _send_reply(conversation_token, response_text)
|
||||
await _send_reply(conversation_token, response_text, nextcloud_url, secret)
|
||||
|
||||
|
||||
@router.post("/inara-nextcloud-talk-webhook")
|
||||
async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundTasks):
|
||||
body = await request.body()
|
||||
@router.post("/webhook/nextcloud/{username}")
|
||||
async def nextcloud_talk_webhook(username: str, request: Request, background_tasks: BackgroundTasks):
|
||||
channels = get_user_channels(username)
|
||||
cfg = channels.get("nextcloud")
|
||||
if not cfg:
|
||||
logger.warning("NCT webhook: no channel config for user %r", username)
|
||||
raise HTTPException(status_code=404, detail="Channel not configured for this user")
|
||||
|
||||
if not settings.nextcloud_talk_bot_secret:
|
||||
logger.error("nextcloud_talk_bot_secret not configured")
|
||||
persona_name = cfg.get("persona", "inara")
|
||||
nextcloud_url = cfg.get("url", "")
|
||||
secret = cfg.get("bot_secret", "")
|
||||
timeout = cfg.get("timeout", 55)
|
||||
|
||||
if not secret:
|
||||
logger.error("NCT webhook: bot_secret missing for user %r", username)
|
||||
return Response(status_code=500)
|
||||
|
||||
body = await request.body()
|
||||
|
||||
random_header = request.headers.get("X-Nextcloud-Talk-Random", "")
|
||||
sig_header = request.headers.get("X-Nextcloud-Talk-Signature", "")
|
||||
|
||||
if not _verify_signature(body, random_header, sig_header):
|
||||
logger.warning("NCT webhook: signature mismatch")
|
||||
if not _verify_signature(body, random_header, sig_header, secret):
|
||||
logger.warning("NCT webhook: signature mismatch for %s", username)
|
||||
raise HTTPException(status_code=401, detail="Invalid signature")
|
||||
|
||||
try:
|
||||
@@ -158,7 +226,7 @@ async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundT
|
||||
except (json.JSONDecodeError, AttributeError):
|
||||
user_text = (obj.get("name") or obj.get("content", "")).strip()
|
||||
|
||||
mention_prefix = f"@{settings.agent_name.lower()}"
|
||||
mention_prefix = f"@{persona_name.lower()}"
|
||||
if user_text.lower().startswith(mention_prefix):
|
||||
user_text = user_text[len(mention_prefix):].strip()
|
||||
|
||||
@@ -168,5 +236,9 @@ async def nextcloud_talk_webhook(request: Request, background_tasks: BackgroundT
|
||||
actor_name = actor.get("name", "User")
|
||||
logger.info("NCT message from %s in %s: %r", actor_name, conversation_token, user_text[:60])
|
||||
|
||||
background_tasks.add_task(_process_message, conversation_token, user_text, actor_name)
|
||||
background_tasks.add_task(
|
||||
_process_message,
|
||||
conversation_token, user_text, actor_name,
|
||||
username, persona_name, nextcloud_url, secret, timeout, cfg,
|
||||
)
|
||||
return Response(status_code=200)
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
"""
|
||||
Onboarding router — invite-based setup + persona creation.
|
||||
Onboarding router — invite-based setup + persona creation + model connect.
|
||||
|
||||
Routes:
|
||||
GET /setup/{token} → show password setup form (step 1)
|
||||
POST /setup/{token} → set password, redirect to persona step
|
||||
GET /setup/persona → show persona creation form (step 2, requires auth)
|
||||
POST /setup/persona → create persona, redirect to /{user}/{persona}
|
||||
POST /setup/persona → create persona, redirect to /setup/model
|
||||
GET /setup/model → OpenRouter quick-connect (step 3, also standalone)
|
||||
POST /setup/model → save host + model + assign to chat role, redirect to chat
|
||||
"""
|
||||
|
||||
import logging
|
||||
@@ -21,6 +23,7 @@ from auth_utils import (
|
||||
)
|
||||
from persona_template import create_persona
|
||||
from persona import list_user_personas, validate as validate_persona
|
||||
import model_registry
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/setup")
|
||||
@@ -114,7 +117,11 @@ async def persona_submit(
|
||||
description=description.strip(),
|
||||
)
|
||||
logger.info("persona created: %s/%s", username, persona_name)
|
||||
return RedirectResponse(f"/{username}/{persona_name}", status_code=302)
|
||||
# Step 3: guided model setup before entering the chat
|
||||
resp = RedirectResponse("/setup/model", status_code=302)
|
||||
# Remember which persona to land on after model setup
|
||||
resp.set_cookie("cx_setup_persona", f"{username}/{persona_name}", max_age=3600, httponly=True, samesite="lax")
|
||||
return resp
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -178,3 +185,126 @@ async def setup_submit(
|
||||
return resp
|
||||
|
||||
return HTMLResponse(_setup_page("Unknown step."), status_code=400)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Step 3 — model connect (OpenRouter quick-connect, also standalone)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Curated model list shown in the Step 3 dropdown.
|
||||
_OPENROUTER_MODELS = [
|
||||
("anthropic/claude-3-5-haiku-20241022", "Claude 3.5 Haiku — Fast & affordable"),
|
||||
("anthropic/claude-3-7-sonnet-20250219", "Claude 3.7 Sonnet — Smarter Claude"),
|
||||
("google/gemini-2.0-flash-001", "Gemini 2.0 Flash — Fast Google model"),
|
||||
("meta-llama/llama-3.3-70b-instruct", "Llama 3.3 70B — Open source"),
|
||||
]
|
||||
|
||||
|
||||
def _model_page(error: str = "", from_setup: bool = False) -> str:
|
||||
html = (_STATIC / "setup.html").read_text()
|
||||
# Hide steps 1 and 2 inline; show step 3
|
||||
html = html.replace('<div id="step-password">', '<div id="step-password" style="display:none">')
|
||||
html = html.replace('<div id="step-persona" style="display:none">', '<div id="step-persona" style="display:none">')
|
||||
html = html.replace('<div id="step-model" style="display:none">', '<div id="step-model">')
|
||||
if from_setup:
|
||||
html = html.replace("<!-- SETUP_STEP3_LABEL -->", "Step 3 of 3")
|
||||
if error:
|
||||
html = html.replace("<!-- ERROR_MODEL -->", f'<p class="error">{error}</p>')
|
||||
return html
|
||||
|
||||
|
||||
@router.post("/model/skip", include_in_schema=False)
|
||||
async def model_skip(request: Request):
|
||||
"""Skip model setup — redirect to the remembered persona or user root."""
|
||||
from auth_utils import decode_token
|
||||
import jwt
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
username = None
|
||||
if token:
|
||||
try:
|
||||
username = decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
pass
|
||||
|
||||
dest_cookie = request.cookies.get("cx_setup_persona", "")
|
||||
dest = f"/{dest_cookie}" if dest_cookie else (f"/{username}" if username else "/")
|
||||
resp = RedirectResponse(dest, status_code=302)
|
||||
resp.delete_cookie("cx_setup_persona")
|
||||
return resp
|
||||
|
||||
|
||||
@router.get("/model", include_in_schema=False)
|
||||
async def model_page(request: Request):
|
||||
from auth_utils import decode_token
|
||||
import jwt
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
try:
|
||||
decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
from_setup = bool(request.cookies.get("cx_setup_persona"))
|
||||
return HTMLResponse(_model_page(from_setup=from_setup))
|
||||
|
||||
|
||||
@router.post("/model", include_in_schema=False)
|
||||
async def model_submit(
|
||||
request: Request,
|
||||
api_key: str = Form(...),
|
||||
model_name: str = Form(...),
|
||||
):
|
||||
from auth_utils import decode_token
|
||||
import jwt
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
try:
|
||||
username = decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
api_key = api_key.strip()
|
||||
model_name = model_name.strip()
|
||||
|
||||
if not api_key:
|
||||
from_setup = bool(request.cookies.get("cx_setup_persona"))
|
||||
return HTMLResponse(_model_page("API key is required.", from_setup=from_setup), status_code=422)
|
||||
|
||||
# Save OpenRouter as a host
|
||||
host_id = model_registry.save_host(
|
||||
username=username,
|
||||
host_id=None,
|
||||
label="OpenRouter",
|
||||
api_url="https://openrouter.ai/api/v1",
|
||||
api_key=api_key,
|
||||
host_type="openai",
|
||||
)
|
||||
|
||||
# Find label for selected model
|
||||
label = next((lbl for mn, lbl in _OPENROUTER_MODELS if mn == model_name), model_name)
|
||||
label = label.split(" — ")[0] # keep just the model name part
|
||||
|
||||
# Save model entry
|
||||
mid = model_registry.save_model(
|
||||
username=username,
|
||||
model_id=None,
|
||||
host_id=host_id,
|
||||
label=label,
|
||||
model_name=model_name,
|
||||
context_k=128,
|
||||
tools=True,
|
||||
)
|
||||
|
||||
# Assign as chat role primary
|
||||
model_registry.set_role(username, "chat", "primary", mid)
|
||||
logger.info("openrouter setup complete: %s → %s", username, model_name)
|
||||
|
||||
# Redirect to chat (use remembered persona, or user root)
|
||||
dest_cookie = request.cookies.get("cx_setup_persona", "")
|
||||
dest = f"/{dest_cookie}" if dest_cookie else f"/{username}"
|
||||
|
||||
resp = RedirectResponse(dest, status_code=302)
|
||||
resp.delete_cookie("cx_setup_persona")
|
||||
return resp
|
||||
|
||||
@@ -12,28 +12,36 @@ Designed to be triggered from:
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import platform
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from fastapi import APIRouter
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from auth_utils import get_user_gemini_key, get_user_role, get_tool_policy, get_risk_policy
|
||||
from config import settings
|
||||
from context_loader import load_context
|
||||
from persona import set_context, validate as validate_persona
|
||||
import model_registry
|
||||
import orchestrator_engine
|
||||
import openai_orchestrator
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/orchestrate", tags=["orchestrator"])
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# In-memory job store
|
||||
# Jobs are keyed by UUID. For this phase, memory is fine — jobs are short-lived.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_jobs: dict[str, dict] = {}
|
||||
_jobs_lock = asyncio.Lock()
|
||||
|
||||
# Checkpoints are stored separately — they hold Python objects (types.Content, etc.)
|
||||
# that can't be included in the JSON-serializable job dict.
|
||||
_checkpoints: dict[str, orchestrator_engine.OrchestrateCheckpoint] = {}
|
||||
_checkpoints_lock = asyncio.Lock()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request / response models
|
||||
@@ -49,11 +57,13 @@ class OrchestrateRequest(BaseModel):
|
||||
include_short: bool = True
|
||||
user: str = "scott"
|
||||
persona: str = "inara"
|
||||
chat_role: str = "chat" # role used for the final response (decoupled from tool-loop model)
|
||||
off_record: bool = False # skip session log; inject OTR mode line into system prompt
|
||||
|
||||
|
||||
class OrchestrateResponse(BaseModel):
|
||||
job_id: str
|
||||
status: str # "queued" | "running" | "complete" | "error"
|
||||
status: str # "queued" | "running" | "complete" | "error" | "awaiting_confirmation"
|
||||
|
||||
|
||||
class JobStatusResponse(BaseModel):
|
||||
@@ -66,8 +76,11 @@ class JobStatusResponse(BaseModel):
|
||||
response: str | None = None
|
||||
tool_calls: list[dict] | None = None
|
||||
backend: str | None = None
|
||||
backend_label: str | None = None
|
||||
host: str | None = None
|
||||
gemini_summary: str | None = None
|
||||
error: str | None = None
|
||||
pending_confirmation: dict | None = None # {tools: [{name, args}], message: str}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -81,7 +94,6 @@ async def orchestrate(req: OrchestrateRequest) -> OrchestrateResponse:
|
||||
user, persona = validate_persona(req.user, req.persona)
|
||||
set_context(user, persona)
|
||||
except ValueError as e:
|
||||
from fastapi import HTTPException
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
|
||||
job_id = str(uuid.uuid4())
|
||||
@@ -93,18 +105,21 @@ async def orchestrate(req: OrchestrateRequest) -> OrchestrateResponse:
|
||||
"task": req.task,
|
||||
"created_at": now,
|
||||
"completed_at": None,
|
||||
"session_id": None,
|
||||
"response": None,
|
||||
"tool_calls": None,
|
||||
"backend": None,
|
||||
"gemini_summary": None,
|
||||
"error": None,
|
||||
"pending_confirmation": None,
|
||||
"_user": user,
|
||||
"_off_record": req.off_record,
|
||||
}
|
||||
|
||||
async with _jobs_lock:
|
||||
_jobs[job_id] = job
|
||||
|
||||
# Run in background — caller polls GET /orchestrate/{job_id}
|
||||
asyncio.create_task(_run_job(job_id, req))
|
||||
asyncio.create_task(_run_job(job_id, req, user))
|
||||
logger.info("Orchestrator job queued: %s — %.80s", job_id, req.task)
|
||||
return OrchestrateResponse(job_id=job_id, status="queued")
|
||||
|
||||
@@ -116,10 +131,9 @@ async def job_status(job_id: str) -> JobStatusResponse:
|
||||
job = _jobs.get(job_id)
|
||||
|
||||
if job is None:
|
||||
from fastapi import HTTPException
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
|
||||
return JobStatusResponse(**job)
|
||||
return JobStatusResponse(**{k: v for k, v in job.items() if not k.startswith("_")})
|
||||
|
||||
|
||||
@router.get("", response_model=list[JobStatusResponse])
|
||||
@@ -127,14 +141,58 @@ async def list_jobs() -> list[JobStatusResponse]:
|
||||
"""List all jobs (most recent first). Useful for debugging."""
|
||||
async with _jobs_lock:
|
||||
jobs = sorted(_jobs.values(), key=lambda j: j["created_at"], reverse=True)
|
||||
return [JobStatusResponse(**j) for j in jobs]
|
||||
return [JobStatusResponse(**{k: v for k, v in j.items() if not k.startswith("_")}) for j in jobs]
|
||||
|
||||
|
||||
@router.post("/{job_id}/confirm", response_model=OrchestrateResponse)
|
||||
async def confirm_job(job_id: str) -> OrchestrateResponse:
|
||||
"""Confirm a pending tool call — the blocked tool will execute and the job continues."""
|
||||
async with _checkpoints_lock:
|
||||
checkpoint = _checkpoints.pop(job_id, None)
|
||||
|
||||
if checkpoint is None:
|
||||
raise HTTPException(status_code=404, detail="No pending confirmation for this job")
|
||||
|
||||
async with _jobs_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if not job or job["status"] != "awaiting_confirmation":
|
||||
raise HTTPException(status_code=409, detail="Job is not awaiting confirmation")
|
||||
_jobs[job_id]["status"] = "running"
|
||||
_jobs[job_id]["pending_confirmation"] = None
|
||||
user = job.get("_user", "scott")
|
||||
|
||||
asyncio.create_task(_resume_job(job_id, checkpoint, confirmed=True, user=user))
|
||||
logger.info("Orchestrator job %s confirmed — resuming", job_id)
|
||||
return OrchestrateResponse(job_id=job_id, status="running")
|
||||
|
||||
|
||||
@router.post("/{job_id}/deny", response_model=OrchestrateResponse)
|
||||
async def deny_job(job_id: str) -> OrchestrateResponse:
|
||||
"""Deny a pending tool call — the tool is skipped and the job produces a final response."""
|
||||
async with _checkpoints_lock:
|
||||
checkpoint = _checkpoints.pop(job_id, None)
|
||||
|
||||
if checkpoint is None:
|
||||
raise HTTPException(status_code=404, detail="No pending confirmation for this job")
|
||||
|
||||
async with _jobs_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if not job or job["status"] != "awaiting_confirmation":
|
||||
raise HTTPException(status_code=409, detail="Job is not awaiting confirmation")
|
||||
_jobs[job_id]["status"] = "running"
|
||||
_jobs[job_id]["pending_confirmation"] = None
|
||||
user = job.get("_user", "scott")
|
||||
|
||||
asyncio.create_task(_resume_job(job_id, checkpoint, confirmed=False, user=user))
|
||||
logger.info("Orchestrator job %s denied — resuming with skip", job_id)
|
||||
return OrchestrateResponse(job_id=job_id, status="running")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Background runner
|
||||
# Background runners
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def _run_job(job_id: str, req: OrchestrateRequest) -> None:
|
||||
async def _run_job(job_id: str, req: OrchestrateRequest, user: str) -> None:
|
||||
"""Execute the orchestration job and update the job store."""
|
||||
async with _jobs_lock:
|
||||
_jobs[job_id]["status"] = "running"
|
||||
@@ -142,47 +200,91 @@ async def _run_job(job_id: str, req: OrchestrateRequest) -> None:
|
||||
try:
|
||||
from session_store import load as load_session, save as save_session, generate_session_id
|
||||
|
||||
# Load Inara's system prompt (same as the chat router does)
|
||||
tier = req.tier or settings.default_tier
|
||||
role_cfg = model_registry.get_role_config(user, req.chat_role)
|
||||
system_prompt = load_context(
|
||||
tier,
|
||||
include_long=req.include_long,
|
||||
include_mid=req.include_mid,
|
||||
include_short=req.include_short,
|
||||
role_append=role_cfg.get("system_append", ""),
|
||||
inject_datetime=role_cfg.get("inject_datetime", True),
|
||||
inject_mode=role_cfg.get("inject_mode", True),
|
||||
mode="otr" if req.off_record else "chat",
|
||||
)
|
||||
|
||||
# Load session history if a session_id was provided
|
||||
session_id = req.session_id or generate_session_id()
|
||||
history = load_session(session_id)
|
||||
session_messages = history or None
|
||||
|
||||
orch_model = model_registry.get_model_for_role(user, "orchestrator")
|
||||
user_role = get_user_role(user)
|
||||
tool_list = role_cfg.get("tools")
|
||||
|
||||
policy = get_tool_policy(user)
|
||||
confirm_allow = set(policy.get("allow", []))
|
||||
confirm_deny = set(policy.get("deny", []))
|
||||
max_risk, risk_wl, risk_bl = get_risk_policy(user)
|
||||
|
||||
if orch_model and orch_model.get("type") == "local_openai":
|
||||
result = await openai_orchestrator.run(
|
||||
task=req.task,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_messages,
|
||||
model_cfg=orch_model,
|
||||
respond_with_final=req.respond_with_claude,
|
||||
user_role=user_role,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=confirm_allow,
|
||||
confirm_deny=confirm_deny,
|
||||
max_risk=max_risk,
|
||||
risk_whitelist=risk_wl,
|
||||
risk_blacklist=risk_bl,
|
||||
)
|
||||
else:
|
||||
gemini_key = (
|
||||
(orch_model.get("api_key") if orch_model else None)
|
||||
or get_user_gemini_key(user)
|
||||
)
|
||||
result = await orchestrator_engine.run(
|
||||
task=req.task,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=session_messages,
|
||||
respond_with_claude=req.respond_with_claude,
|
||||
gemini_api_key=gemini_key,
|
||||
model_name=orch_model.get("model_name") if orch_model else None,
|
||||
response_role=req.chat_role,
|
||||
user_role=user_role,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=confirm_allow,
|
||||
confirm_deny=confirm_deny,
|
||||
max_rounds=orch_model.get("max_rounds") if orch_model else None,
|
||||
max_risk=max_risk,
|
||||
risk_whitelist=risk_wl,
|
||||
risk_blacklist=risk_bl,
|
||||
)
|
||||
|
||||
# Save the turn to the session store so it survives a page refresh
|
||||
history.append({"role": "user", "content": req.task})
|
||||
history.append({"role": "assistant", "content": result.response})
|
||||
save_session(session_id, history)
|
||||
|
||||
from session_logger import log_turn
|
||||
log_turn(session_id, req.task, result.response)
|
||||
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
if result.checkpoint:
|
||||
async with _checkpoints_lock:
|
||||
_checkpoints[job_id] = result.checkpoint
|
||||
async with _jobs_lock:
|
||||
_jobs[job_id].update({
|
||||
"status": "complete",
|
||||
"completed_at": now,
|
||||
"session_id": session_id,
|
||||
"status": "awaiting_confirmation",
|
||||
"response": result.response,
|
||||
"tool_calls": result.tool_calls,
|
||||
"backend": result.backend,
|
||||
"gemini_summary": result.gemini_summary,
|
||||
"session_id": session_id,
|
||||
"pending_confirmation": {
|
||||
"tools": result.checkpoint.pending_tools,
|
||||
"message": result.response,
|
||||
},
|
||||
})
|
||||
logger.info("Orchestrator job complete: %s (%d tool calls)", job_id, len(result.tool_calls))
|
||||
logger.info("Orchestrator job %s awaiting confirmation — %d tool(s) blocked",
|
||||
job_id, len(result.checkpoint.pending_tools))
|
||||
return
|
||||
|
||||
await _finalize_job(job_id, result, session_id, req.task, history, off_record=req.off_record)
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Orchestrator job failed: %s", job_id)
|
||||
@@ -193,3 +295,100 @@ async def _run_job(job_id: str, req: OrchestrateRequest) -> None:
|
||||
"completed_at": now,
|
||||
"error": str(e),
|
||||
})
|
||||
|
||||
|
||||
async def _resume_job(
|
||||
job_id: str,
|
||||
checkpoint: orchestrator_engine.OrchestrateCheckpoint,
|
||||
confirmed: bool,
|
||||
user: str,
|
||||
) -> None:
|
||||
"""Resume a job after the user confirms or denies a pending tool call."""
|
||||
try:
|
||||
if checkpoint.engine == "gemini":
|
||||
result = await orchestrator_engine.resume(checkpoint, confirmed)
|
||||
else:
|
||||
result = await openai_orchestrator.resume(checkpoint, confirmed)
|
||||
|
||||
if result.checkpoint:
|
||||
# Another confirmation needed (chained gates)
|
||||
async with _checkpoints_lock:
|
||||
_checkpoints[job_id] = result.checkpoint
|
||||
async with _jobs_lock:
|
||||
_jobs[job_id].update({
|
||||
"status": "awaiting_confirmation",
|
||||
"response": result.response,
|
||||
"tool_calls": result.tool_calls,
|
||||
"backend": result.backend,
|
||||
"gemini_summary": result.gemini_summary,
|
||||
"pending_confirmation": {
|
||||
"tools": result.checkpoint.pending_tools,
|
||||
"message": result.response,
|
||||
},
|
||||
})
|
||||
logger.info("Orchestrator job %s awaiting another confirmation", job_id)
|
||||
return
|
||||
|
||||
async with _jobs_lock:
|
||||
session_id = _jobs[job_id].get("session_id") or ""
|
||||
task = _jobs[job_id].get("task", "")
|
||||
off_record = _jobs[job_id].get("_off_record", False)
|
||||
|
||||
from session_store import load as load_session
|
||||
history = load_session(session_id) if session_id else []
|
||||
await _finalize_job(job_id, result, session_id, task, history, off_record=off_record)
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Orchestrator resume failed: %s", job_id)
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
async with _jobs_lock:
|
||||
_jobs[job_id].update({
|
||||
"status": "error",
|
||||
"completed_at": now,
|
||||
"error": str(e),
|
||||
})
|
||||
|
||||
|
||||
async def _finalize_job(
|
||||
job_id: str,
|
||||
result: orchestrator_engine.OrchestratorResult,
|
||||
session_id: str,
|
||||
task: str,
|
||||
history: list,
|
||||
off_record: bool = False,
|
||||
) -> None:
|
||||
"""Save session, log the turn, and mark the job complete."""
|
||||
from session_store import save as save_session, generate_session_id
|
||||
from session_logger import log_turn
|
||||
|
||||
if not session_id:
|
||||
session_id = generate_session_id()
|
||||
|
||||
host = platform.node()
|
||||
history.append({"role": "user", "content": task, "off_record": off_record})
|
||||
history.append({
|
||||
"role": "assistant",
|
||||
"content": result.response,
|
||||
"backend": result.backend,
|
||||
"backend_label": result.backend_label,
|
||||
"host": host,
|
||||
"off_record": off_record,
|
||||
})
|
||||
save_session(session_id, history)
|
||||
if not off_record:
|
||||
log_turn(session_id, task, result.response)
|
||||
|
||||
now = datetime.now(timezone.utc).isoformat()
|
||||
async with _jobs_lock:
|
||||
_jobs[job_id].update({
|
||||
"status": "complete",
|
||||
"completed_at": now,
|
||||
"session_id": session_id,
|
||||
"response": result.response,
|
||||
"tool_calls": result.tool_calls,
|
||||
"backend": result.backend,
|
||||
"backend_label": result.backend_label,
|
||||
"host": host,
|
||||
"gemini_summary": result.gemini_summary,
|
||||
})
|
||||
logger.info("Orchestrator job complete: %s (%d tool calls)", job_id, len(result.tool_calls))
|
||||
|
||||
120
cortex/routers/push.py
Normal file
120
cortex/routers/push.py
Normal file
@@ -0,0 +1,120 @@
|
||||
"""
|
||||
Web Push endpoints.
|
||||
|
||||
GET /api/push/vapid-key → public VAPID key for browser PushManager.subscribe()
|
||||
POST /api/push/subscribe → save a push subscription for the logged-in user
|
||||
DELETE /api/push/subscribe → remove a subscription by endpoint
|
||||
"""
|
||||
import jwt
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from pydantic import BaseModel
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token
|
||||
from config import settings
|
||||
import push_utils
|
||||
|
||||
router = APIRouter(prefix="/api/push")
|
||||
|
||||
|
||||
def _require_user(request: Request) -> str:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
raise HTTPException(status_code=401, detail="Invalid session")
|
||||
|
||||
|
||||
@router.get("/vapid-key")
|
||||
async def get_vapid_key() -> dict:
|
||||
"""Return the VAPID public key. Public endpoint — needed before login to subscribe."""
|
||||
key = settings.vapid_public_key
|
||||
if not key:
|
||||
raise HTTPException(status_code=503, detail="Push notifications not configured")
|
||||
return {"public_key": key}
|
||||
|
||||
|
||||
class SubscribeRequest(BaseModel):
|
||||
subscription: dict # full PushSubscription JSON from browser
|
||||
|
||||
|
||||
class UnsubscribeRequest(BaseModel):
|
||||
endpoint: str
|
||||
|
||||
|
||||
@router.post("/subscribe")
|
||||
async def subscribe(req: SubscribeRequest, request: Request) -> dict:
|
||||
username = _require_user(request)
|
||||
sub = req.subscription
|
||||
if not sub.get("endpoint"):
|
||||
raise HTTPException(status_code=400, detail="subscription.endpoint is required")
|
||||
push_utils.add_subscription(username, sub)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@router.delete("/subscribe")
|
||||
async def unsubscribe(req: UnsubscribeRequest, request: Request) -> dict:
|
||||
username = _require_user(request)
|
||||
found = push_utils.remove_subscription(username, req.endpoint)
|
||||
return {"ok": True, "found": found}
|
||||
|
||||
|
||||
@router.post("/test")
|
||||
async def notify_test(request: Request) -> dict:
|
||||
"""Send a test notification via the user's configured notification channel.
|
||||
|
||||
Useful for verifying channel setup (web push, NCT, email, etc.) without
|
||||
waiting for a cron job or reminder to fire naturally.
|
||||
"""
|
||||
username = _require_user(request)
|
||||
from notification import notify
|
||||
await notify(username, "Test notification from Cortex — your notification channel is working.")
|
||||
return {"ok": True, "user": username}
|
||||
|
||||
|
||||
@router.post("/reminders/check")
|
||||
async def reminder_check_now(request: Request) -> dict:
|
||||
"""Run the reminder check for the current user immediately.
|
||||
|
||||
Same logic as the daily 09:00 scheduler job, but scoped to one user
|
||||
and fired on demand. Returns how many reminders were found and whether
|
||||
a notification was sent.
|
||||
"""
|
||||
import re
|
||||
username = _require_user(request)
|
||||
|
||||
from persona import list_user_personas, set_context
|
||||
from notification import notify
|
||||
|
||||
total_sent = 0
|
||||
for persona_name in list_user_personas(username):
|
||||
set_context(username, persona_name)
|
||||
from tools.reminders import load_due_reminders
|
||||
content = load_due_reminders()
|
||||
if not content:
|
||||
continue
|
||||
|
||||
entries = []
|
||||
for line in content.splitlines():
|
||||
m = re.match(r"^\d+\.\s+(.+)", line.strip())
|
||||
if m:
|
||||
text = re.sub(r"\[(OVERDUE|due TODAY|due: \S+)\]", "", m.group(1)).strip()
|
||||
if text:
|
||||
entries.append(text)
|
||||
|
||||
if not entries:
|
||||
continue
|
||||
|
||||
count = len(entries)
|
||||
if count == 1:
|
||||
msg = f"Reminder: {entries[0]}"
|
||||
else:
|
||||
bullet_list = "\n".join(f"• {e}" for e in entries[:3])
|
||||
tail = f"\n…and {count - 3} more" if count > 3 else ""
|
||||
msg = f"{count} reminders due:\n{bullet_list}{tail}"
|
||||
|
||||
await notify(username, msg)
|
||||
total_sent += count
|
||||
|
||||
return {"ok": True, "user": username, "reminders_found": total_sent}
|
||||
499
cortex/routers/settings.py
Normal file
499
cortex/routers/settings.py
Normal file
@@ -0,0 +1,499 @@
|
||||
"""
|
||||
Account settings router.
|
||||
|
||||
Routes:
|
||||
GET /settings → show account settings page (requires auth)
|
||||
POST /settings/password → change password
|
||||
POST /settings/username → rename the user account (forces re-login)
|
||||
POST /settings/persona/rename → rename a persona directory
|
||||
"""
|
||||
|
||||
import html as _html
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import jwt
|
||||
from fastapi import APIRouter, Form, Request
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token, check_credentials, set_password, _read_auth, _write_auth, get_user_channels
|
||||
from persona import list_user_personas
|
||||
from config import settings as app_settings
|
||||
|
||||
_SLUG_RE = re.compile(r"^[a-z_][a-z0-9_-]{0,31}$")
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
_STATIC = Path(__file__).parent.parent / "static"
|
||||
|
||||
|
||||
_LAST_PERSONA_COOKIE = "cx_last_persona"
|
||||
|
||||
|
||||
def _get_session_user(request: Request) -> str | None:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
return None
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
return None
|
||||
|
||||
|
||||
def _preferred_persona(request: Request, username: str) -> str:
|
||||
names = list_user_personas(username)
|
||||
if not names:
|
||||
return ""
|
||||
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
|
||||
if cookie_val in names:
|
||||
return cookie_val
|
||||
return names[0]
|
||||
|
||||
|
||||
def _integrations_nav(username: str) -> str:
|
||||
"""Return the Integrations nav link for admin users, empty string otherwise."""
|
||||
role = _read_auth(username).get("role", "user")
|
||||
if role == "admin":
|
||||
return '<a href="/settings/integrations" class="nav-link">Integrations</a>'
|
||||
return ""
|
||||
|
||||
|
||||
def _notifications_page(username: str, back_persona: str = "", success: str = "", error: str = "") -> str:
|
||||
html = (_STATIC / "notifications.html").read_text()
|
||||
channels = get_user_channels(username)
|
||||
nct = channels.get("nextcloud") or {}
|
||||
|
||||
notify_ch = _html.escape(channels.get("notification_channel", "") or "")
|
||||
notify_email = _html.escape(channels.get("notification_email", "") or "")
|
||||
nc_url = _html.escape(nct.get("url", "") or "")
|
||||
nc_bot_secret = _html.escape(nct.get("bot_secret", "") or "")
|
||||
nc_room = _html.escape(nct.get("notification_room", "") or "")
|
||||
nc_username = _html.escape(nct.get("nc_username", "") or "")
|
||||
nc_app_password = _html.escape(nct.get("nc_app_password", "") or "")
|
||||
gc_webhook = _html.escape((channels.get("google_chat") or {}).get("outbound_webhook", "") or "")
|
||||
ha = channels.get("homeassistant") or {}
|
||||
ha_url = _html.escape(ha.get("url", "") or "")
|
||||
ha_webhook_id = _html.escape(ha.get("webhook_id", "") or "")
|
||||
ha_tools_checked = "checked" if ha.get("tools", False) else ""
|
||||
|
||||
html = html.replace("{{ notify_channel }}", notify_ch)
|
||||
html = html.replace("{{ notify_email_override }}", notify_email)
|
||||
html = html.replace("{{ nc_url }}", nc_url)
|
||||
html = html.replace("{{ nc_bot_secret }}", nc_bot_secret)
|
||||
html = html.replace("{{ nc_notify_room }}", nc_room)
|
||||
html = html.replace("{{ nc_username }}", nc_username)
|
||||
html = html.replace("{{ nc_app_password }}", nc_app_password)
|
||||
html = html.replace("{{ gc_webhook }}", gc_webhook)
|
||||
html = html.replace("{{ ha_url }}", ha_url)
|
||||
html = html.replace("{{ ha_webhook_id }}", ha_webhook_id)
|
||||
html = html.replace("{{ ha_tools_checked }}", ha_tools_checked)
|
||||
html = html.replace("{{ ha_username }}", username)
|
||||
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
|
||||
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
|
||||
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
|
||||
if success:
|
||||
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
|
||||
if error:
|
||||
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
|
||||
return html
|
||||
|
||||
|
||||
def _settings_page(username: str, personas: list[str], back_persona: str = "", success: str = "", error: str = "") -> str:
|
||||
html = (_STATIC / "settings.html").read_text()
|
||||
html = html.replace("{{ username }}", username)
|
||||
|
||||
# Connected Google account (OAuth sign-in)
|
||||
auth_data = _read_auth(username)
|
||||
google_email = auth_data.get("google_email") or ""
|
||||
html = html.replace("{{ google_email }}", google_email)
|
||||
|
||||
role = auth_data.get("role", "user")
|
||||
html = html.replace("{{ user_role }}", role)
|
||||
|
||||
al_path = app_settings.home_root() / username / "email_allowlist.json"
|
||||
try:
|
||||
patterns = json.loads(al_path.read_text())
|
||||
allowlist_text = _html.escape("\n".join(str(p) for p in patterns if str(p).strip()))
|
||||
except Exception:
|
||||
allowlist_text = ""
|
||||
html = html.replace("{{ email_allowlist }}", allowlist_text)
|
||||
|
||||
http_al_path = app_settings.home_root() / username / "http_allowlist.json"
|
||||
try:
|
||||
http_prefixes = json.loads(http_al_path.read_text())
|
||||
http_allowlist_text = _html.escape("\n".join(str(p) for p in http_prefixes if str(p).strip()))
|
||||
except Exception:
|
||||
http_allowlist_text = ""
|
||||
html = html.replace("{{ http_allowlist }}", http_allowlist_text)
|
||||
|
||||
persona_items = "\n".join(
|
||||
f'''<li>
|
||||
<a href="/{username}/{p}" class="persona-link">{p}</a>
|
||||
<button class="persona-rename-toggle" data-persona="{p}" title="Rename">✏</button>
|
||||
<form class="persona-rename-form" data-persona="{p}"
|
||||
method="POST" action="/settings/persona/rename" style="display:none">
|
||||
<input type="hidden" name="old_name" value="{p}">
|
||||
<input type="text" name="new_name" value="{p}"
|
||||
pattern="[a-z_][a-z0-9_\\-]{{0,31}}" required>
|
||||
<button type="submit" class="btn-save">Save</button>
|
||||
<button type="button" class="btn-cancel persona-rename-cancel">Cancel</button>
|
||||
</form>
|
||||
</li>''' for p in personas
|
||||
)
|
||||
html = html.replace("{{ persona_items }}", persona_items or "<li><em>No personas yet.</em></li>")
|
||||
if not back_persona:
|
||||
back_persona = personas[0] if personas else ""
|
||||
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
|
||||
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
|
||||
html = html.replace("{{ integrations_nav }}", _integrations_nav(username))
|
||||
if success:
|
||||
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
|
||||
if error:
|
||||
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
|
||||
return html
|
||||
|
||||
|
||||
def _integrations_page(username: str, back_persona: str = "", success: str = "", error: str = "") -> str:
|
||||
html = (_STATIC / "integrations.html").read_text()
|
||||
channels = get_user_channels(username)
|
||||
ae_db = channels.get("aether_db") or {}
|
||||
|
||||
html = html.replace("{{ ae_db_host }}", _html.escape(ae_db.get("host", "") or ""))
|
||||
html = html.replace("{{ ae_db_port }}", _html.escape(str(ae_db.get("port", 3306))))
|
||||
html = html.replace("{{ ae_db_name }}", _html.escape(ae_db.get("name", "") or ""))
|
||||
html = html.replace("{{ ae_db_user }}", _html.escape(ae_db.get("user", "") or ""))
|
||||
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
|
||||
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
|
||||
if success:
|
||||
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
|
||||
if error:
|
||||
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
|
||||
return html
|
||||
|
||||
|
||||
@router.get("/settings", include_in_schema=False)
|
||||
async def settings_page(request: Request):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
personas = list_user_personas(username)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona=back_persona))
|
||||
|
||||
|
||||
@router.post("/settings/password", include_in_schema=False)
|
||||
async def change_password(
|
||||
request: Request,
|
||||
current_password: str = Form(...),
|
||||
new_password: str = Form(...),
|
||||
confirm_password: str = Form(...),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
if not check_credentials(username, current_password):
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, error="Current password is incorrect."))
|
||||
|
||||
if len(new_password) < 8:
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, error="New password must be at least 8 characters."))
|
||||
|
||||
if new_password != confirm_password:
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, error="New passwords do not match."))
|
||||
|
||||
set_password(username, new_password)
|
||||
logger.info("password changed: %s", username)
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, success="Password updated successfully."))
|
||||
|
||||
|
||||
@router.post("/settings/username", include_in_schema=False)
|
||||
async def rename_username(
|
||||
request: Request,
|
||||
new_username: str = Form(...),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
new_username = new_username.strip().lower()
|
||||
|
||||
if not _SLUG_RE.match(new_username):
|
||||
return HTMLResponse(_settings_page(
|
||||
username, personas, back_persona,
|
||||
error="Invalid username. Use lowercase letters, digits, _ or - only."))
|
||||
|
||||
if new_username == username:
|
||||
return RedirectResponse("/settings", status_code=302)
|
||||
|
||||
home_root = app_settings.home_root()
|
||||
old_dir = home_root / username
|
||||
new_dir = home_root / new_username
|
||||
|
||||
if new_dir.exists():
|
||||
return HTMLResponse(_settings_page(
|
||||
username, personas, back_persona,
|
||||
error=f"Username '{new_username}' is already taken."))
|
||||
|
||||
old_dir.rename(new_dir)
|
||||
logger.info("user renamed: %s → %s", username, new_username)
|
||||
|
||||
# Clear the auth cookie — old JWT now refers to a non-existent user
|
||||
resp = RedirectResponse("/login?msg=username_changed", status_code=302)
|
||||
resp.delete_cookie(COOKIE_NAME)
|
||||
return resp
|
||||
|
||||
|
||||
@router.post("/settings/gemini-key", include_in_schema=False)
|
||||
async def save_gemini_key(
|
||||
request: Request,
|
||||
gemini_api_key: str = Form(...),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
gemini_api_key = gemini_api_key.strip()
|
||||
|
||||
data = _read_auth(username)
|
||||
if gemini_api_key:
|
||||
data["gemini_api_key"] = gemini_api_key
|
||||
msg = "Gemini API key saved."
|
||||
else:
|
||||
data.pop("gemini_api_key", None)
|
||||
msg = "Gemini API key removed — using server key."
|
||||
_write_auth(username, data)
|
||||
logger.info("gemini key updated: %s", username)
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, success=msg))
|
||||
|
||||
|
||||
@router.post("/settings/persona/rename", include_in_schema=False)
|
||||
async def rename_persona(
|
||||
request: Request,
|
||||
old_name: str = Form(...),
|
||||
new_name: str = Form(...),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
new_name = new_name.strip().lower()
|
||||
|
||||
if not _SLUG_RE.match(new_name):
|
||||
return HTMLResponse(_settings_page(
|
||||
username, personas, back_persona,
|
||||
error="Invalid name. Use lowercase letters, digits, _ or - only."))
|
||||
|
||||
if new_name == old_name:
|
||||
return RedirectResponse("/settings", status_code=302)
|
||||
|
||||
persona_root = app_settings.home_root() / username / "persona"
|
||||
old_dir = persona_root / old_name
|
||||
new_dir = persona_root / new_name
|
||||
|
||||
if not old_dir.exists():
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, error=f"Persona '{old_name}' not found."))
|
||||
|
||||
if new_dir.exists():
|
||||
return HTMLResponse(_settings_page(
|
||||
username, personas, back_persona,
|
||||
error=f"A persona named '{new_name}' already exists."))
|
||||
|
||||
old_dir.rename(new_dir)
|
||||
logger.info("persona renamed: %s/%s → %s", username, old_name, new_name)
|
||||
return RedirectResponse("/settings", status_code=302)
|
||||
|
||||
|
||||
@router.get("/settings/notifications", include_in_schema=False)
|
||||
async def notifications_page(request: Request):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
return HTMLResponse(_notifications_page(username, back_persona))
|
||||
|
||||
|
||||
@router.post("/settings/notifications", include_in_schema=False)
|
||||
async def save_notifications(
|
||||
request: Request,
|
||||
notification_channel: str = Form(""),
|
||||
notification_email: str = Form(""),
|
||||
nc_url: str = Form(""),
|
||||
nc_bot_secret: str = Form(""),
|
||||
nc_notification_room: str = Form(""),
|
||||
nc_username: str = Form(""),
|
||||
nc_app_password: str = Form(""),
|
||||
gc_outbound_webhook: str = Form(""),
|
||||
ha_url: str = Form(""),
|
||||
ha_token: str = Form(""),
|
||||
ha_webhook_id: str = Form(""),
|
||||
ha_tools: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
channels_path = app_settings.home_root() / username / "channels.json"
|
||||
try:
|
||||
channels = json.loads(channels_path.read_text())
|
||||
except Exception:
|
||||
channels = {}
|
||||
|
||||
# Top-level notification preference
|
||||
notification_channel = notification_channel.strip()
|
||||
if notification_channel in ("web_push", "email", "nextcloud", "google_chat"):
|
||||
channels["notification_channel"] = notification_channel
|
||||
else:
|
||||
channels.pop("notification_channel", None)
|
||||
|
||||
# Optional email address override (blank = use login email)
|
||||
notification_email = notification_email.strip()
|
||||
if notification_email:
|
||||
channels["notification_email"] = notification_email
|
||||
else:
|
||||
channels.pop("notification_email", None)
|
||||
|
||||
# Nextcloud Talk — full config nested under "nextcloud"
|
||||
if "nextcloud" not in channels:
|
||||
channels["nextcloud"] = {}
|
||||
nct = channels["nextcloud"]
|
||||
if nc_url.strip():
|
||||
nct["url"] = nc_url.strip().rstrip("/")
|
||||
# Only overwrite secrets if a new value was provided (blank = keep existing)
|
||||
if nc_bot_secret.strip():
|
||||
nct["bot_secret"] = nc_bot_secret.strip()
|
||||
nct["notification_room"] = nc_notification_room.strip()
|
||||
if nc_username.strip():
|
||||
nct["nc_username"] = nc_username.strip()
|
||||
if nc_app_password.strip():
|
||||
nct["nc_app_password"] = nc_app_password.strip()
|
||||
|
||||
# Google Chat outbound webhook — nested under "google_chat"
|
||||
if "google_chat" not in channels:
|
||||
channels["google_chat"] = {}
|
||||
channels["google_chat"]["outbound_webhook"] = gc_outbound_webhook.strip()
|
||||
|
||||
# Home Assistant — nested under "homeassistant"
|
||||
if "homeassistant" not in channels:
|
||||
channels["homeassistant"] = {}
|
||||
ha = channels["homeassistant"]
|
||||
if ha_url.strip():
|
||||
ha["url"] = ha_url.strip().rstrip("/")
|
||||
if ha_token.strip():
|
||||
ha["token"] = ha_token.strip()
|
||||
if ha_webhook_id.strip():
|
||||
ha["webhook_id"] = ha_webhook_id.strip()
|
||||
ha["tools"] = ha_tools == "1"
|
||||
|
||||
channels_path.write_text(json.dumps(channels, indent=2) + "\n")
|
||||
logger.info("notifications updated for %s (channel=%s)", username, notification_channel or "none")
|
||||
return HTMLResponse(_notifications_page(username, back_persona, success="Notification settings saved."))
|
||||
|
||||
|
||||
@router.post("/settings/email-allowlist", include_in_schema=False)
|
||||
async def save_email_allowlist(
|
||||
request: Request,
|
||||
patterns: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
lines = [ln.strip() for ln in patterns.splitlines() if ln.strip()]
|
||||
path = app_settings.home_root() / username / "email_allowlist.json"
|
||||
path.write_text(json.dumps(lines, indent=2))
|
||||
logger.info("email allowlist updated for %s (%d patterns)", username, len(lines))
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, success=f"Email allowlist saved ({len(lines)} pattern{'s' if len(lines) != 1 else ''})."))
|
||||
|
||||
|
||||
@router.post("/settings/http-allowlist", include_in_schema=False)
|
||||
async def save_http_allowlist(
|
||||
request: Request,
|
||||
prefixes: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
lines = [ln.strip() for ln in prefixes.splitlines() if ln.strip()]
|
||||
path = app_settings.home_root() / username / "http_allowlist.json"
|
||||
path.write_text(json.dumps(lines, indent=2))
|
||||
logger.info("http allowlist updated for %s (%d prefixes)", username, len(lines))
|
||||
return HTMLResponse(_settings_page(username, personas, back_persona, success=f"HTTP allowlist saved ({len(lines)} prefix{'es' if len(lines) != 1 else ''})."))
|
||||
|
||||
|
||||
def _require_admin(username: str) -> bool:
|
||||
return _read_auth(username).get("role", "user") == "admin"
|
||||
|
||||
|
||||
@router.get("/settings/integrations", include_in_schema=False)
|
||||
async def integrations_page(request: Request):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
if not _require_admin(username):
|
||||
return RedirectResponse("/settings", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
return HTMLResponse(_integrations_page(username, back_persona))
|
||||
|
||||
|
||||
@router.post("/settings/integrations", include_in_schema=False)
|
||||
async def save_integrations(
|
||||
request: Request,
|
||||
ae_db_host: str = Form(""),
|
||||
ae_db_port: str = Form("3306"),
|
||||
ae_db_name: str = Form(""),
|
||||
ae_db_user: str = Form(""),
|
||||
ae_db_password: str = Form(""),
|
||||
):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
if not _require_admin(username):
|
||||
return RedirectResponse("/settings", status_code=302)
|
||||
|
||||
back_persona = _preferred_persona(request, username)
|
||||
|
||||
channels_path = app_settings.home_root() / username / "channels.json"
|
||||
try:
|
||||
channels = json.loads(channels_path.read_text())
|
||||
except Exception:
|
||||
channels = {}
|
||||
|
||||
if "aether_db" not in channels:
|
||||
channels["aether_db"] = {}
|
||||
db = channels["aether_db"]
|
||||
|
||||
if ae_db_host.strip():
|
||||
db["host"] = ae_db_host.strip()
|
||||
try:
|
||||
db["port"] = int(ae_db_port.strip()) if ae_db_port.strip() else 3306
|
||||
except ValueError:
|
||||
db["port"] = 3306
|
||||
if ae_db_name.strip():
|
||||
db["name"] = ae_db_name.strip()
|
||||
if ae_db_user.strip():
|
||||
db["user"] = ae_db_user.strip()
|
||||
if ae_db_password.strip():
|
||||
db["password"] = ae_db_password.strip()
|
||||
|
||||
channels_path.write_text(json.dumps(channels, indent=2) + "\n")
|
||||
logger.info("integrations updated for %s", username)
|
||||
return HTMLResponse(_integrations_page(username, back_persona, success="Integration settings saved."))
|
||||
193
cortex/routers/tools_settings.py
Normal file
193
cortex/routers/tools_settings.py
Normal file
@@ -0,0 +1,193 @@
|
||||
"""
|
||||
Tool settings router.
|
||||
|
||||
Routes:
|
||||
GET /settings/tools → tool risk policy page
|
||||
POST /settings/tools → save max_risk + per-tool overrides
|
||||
"""
|
||||
|
||||
import html as _html
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
import jwt
|
||||
from fastapi import APIRouter, Form, Request
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token, get_tool_policy, save_tool_policy, _read_auth
|
||||
from persona import list_user_personas
|
||||
from tools import TOOL_CATEGORIES, TOOL_RISK, CONFIRM_REQUIRED
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
_STATIC = Path(__file__).parent.parent / "static"
|
||||
_LAST_PERSONA_COOKIE = "cx_last_persona"
|
||||
|
||||
|
||||
def _get_session_user(request: Request) -> str | None:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
return None
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
return None
|
||||
|
||||
|
||||
def _preferred_persona(request: Request, username: str) -> str:
|
||||
names = list_user_personas(username)
|
||||
if not names:
|
||||
return ""
|
||||
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
|
||||
return cookie_val if cookie_val in names else (names[0] if names else "")
|
||||
|
||||
|
||||
def _build_tool_table(policy: dict) -> str:
|
||||
"""Generate the per-tool override table rows grouped by category."""
|
||||
whitelist = set(policy.get("whitelist") or [])
|
||||
blacklist = set(policy.get("blacklist") or [])
|
||||
|
||||
rows: list[str] = []
|
||||
for category, tools in TOOL_CATEGORIES.items():
|
||||
# Category header spanning all columns
|
||||
escaped_cat = _html.escape(category)
|
||||
rows.append(f'<tr class="tool-cat-row"><td colspan="4">{escaped_cat}</td></tr>')
|
||||
for tool in tools:
|
||||
risk = TOOL_RISK.get(tool, "medium")
|
||||
risk_cls = f"risk-{risk}"
|
||||
risk_html = f'<span class="risk {risk_cls}">{_html.escape(risk)}</span>'
|
||||
|
||||
# Override select value
|
||||
if tool in whitelist:
|
||||
override_val = "whitelist"
|
||||
elif tool in blacklist:
|
||||
override_val = "blacklist"
|
||||
else:
|
||||
override_val = "default"
|
||||
|
||||
def _opt(val: str, label: str) -> str:
|
||||
sel = 'selected' if override_val == val else ''
|
||||
return f'<option value="{val}" {sel}>{label}</option>'
|
||||
|
||||
override_sel = (
|
||||
f'<select name="override_{_html.escape(tool)}" '
|
||||
f'class="override-sel" data-tool="{_html.escape(tool)}">'
|
||||
+ _opt("default", "Default (auto)")
|
||||
+ _opt("whitelist", "Force include")
|
||||
+ _opt("blacklist", "Force exclude")
|
||||
+ '</select>'
|
||||
)
|
||||
|
||||
rows.append(
|
||||
f'<tr data-tool-risk="{_html.escape(risk)}">'
|
||||
f'<td class="tool-name">{_html.escape(tool)}</td>'
|
||||
f'<td>{risk_html}</td>'
|
||||
f'<td><span class="auto-pill"></span></td>'
|
||||
f'<td>{override_sel}</td>'
|
||||
f'</tr>'
|
||||
)
|
||||
|
||||
table_body = "\n".join(rows)
|
||||
return (
|
||||
'<table class="tool-table">'
|
||||
'<thead><tr>'
|
||||
'<th>Tool</th><th>Risk</th><th>Auto status</th><th>Override</th>'
|
||||
'</tr></thead>'
|
||||
f'<tbody>{table_body}</tbody>'
|
||||
'</table>'
|
||||
)
|
||||
|
||||
|
||||
def _tools_page(
|
||||
username: str,
|
||||
back_persona: str = "",
|
||||
success: str = "",
|
||||
error: str = "",
|
||||
) -> str:
|
||||
html = (_STATIC / "tools_settings.html").read_text()
|
||||
policy = get_tool_policy(username)
|
||||
max_risk = policy.get("max_risk") or ""
|
||||
|
||||
# Max risk select options
|
||||
html = html.replace("{{ sel_none }}", "selected" if max_risk == "" else "")
|
||||
html = html.replace("{{ sel_low }}", "selected" if max_risk == "low" else "")
|
||||
html = html.replace("{{ sel_medium }}", "selected" if max_risk == "medium" else "")
|
||||
html = html.replace("{{ sel_high }}", "selected" if max_risk == "high" else "")
|
||||
|
||||
html = html.replace("{{ tool_table_html }}", _build_tool_table(policy))
|
||||
html = html.replace("{{ tool_risk_json }}", json.dumps(TOOL_RISK))
|
||||
html = html.replace("{{ confirm_required_tools }}", _html.escape(", ".join(sorted(CONFIRM_REQUIRED))))
|
||||
html = html.replace("{{ tool_allow }}", _html.escape("\n".join(policy.get("allow") or [])))
|
||||
html = html.replace("{{ tool_deny }}", _html.escape("\n".join(policy.get("deny") or [])))
|
||||
html = html.replace("{{ back_href }}", f"/{username}/{back_persona}" if back_persona else "/")
|
||||
html = html.replace("{{ help_href }}", f"/help?persona={back_persona}" if back_persona else "/help")
|
||||
nav = '<a href="/settings/integrations" class="nav-link">Integrations</a>' \
|
||||
if _read_auth(username).get("role", "user") == "admin" else ""
|
||||
html = html.replace("{{ integrations_nav }}", nav)
|
||||
|
||||
if success:
|
||||
html = html.replace("<!-- SUCCESS -->", f'<p class="success">{success}</p>')
|
||||
if error:
|
||||
html = html.replace("<!-- ERROR -->", f'<p class="error">{error}</p>')
|
||||
return html
|
||||
|
||||
|
||||
@router.get("/settings/tools", include_in_schema=False)
|
||||
async def tools_page(request: Request):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
back_persona = _preferred_persona(request, username)
|
||||
return HTMLResponse(_tools_page(username, back_persona))
|
||||
|
||||
|
||||
@router.post("/settings/tools", include_in_schema=False)
|
||||
async def save_tools(request: Request):
|
||||
username = _get_session_user(request)
|
||||
if not username:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
|
||||
back_persona = _preferred_persona(request, username)
|
||||
form = await request.form()
|
||||
|
||||
max_risk = (form.get("max_risk") or "").strip()
|
||||
if max_risk not in ("", "low", "medium", "high"):
|
||||
max_risk = ""
|
||||
|
||||
whitelist: list[str] = []
|
||||
blacklist: list[str] = []
|
||||
|
||||
all_tools = [t for tools in TOOL_CATEGORIES.values() for t in tools]
|
||||
for tool in all_tools:
|
||||
val = (form.get(f"override_{tool}") or "").strip()
|
||||
if val == "whitelist":
|
||||
whitelist.append(tool)
|
||||
elif val == "blacklist":
|
||||
blacklist.append(tool)
|
||||
|
||||
allow_tools = [ln.strip() for ln in (form.get("allow_list") or "").splitlines() if ln.strip()]
|
||||
deny_tools = [ln.strip() for ln in (form.get("deny_list") or "").splitlines() if ln.strip()]
|
||||
|
||||
policy = get_tool_policy(username)
|
||||
if max_risk:
|
||||
policy["max_risk"] = max_risk
|
||||
else:
|
||||
policy.pop("max_risk", None)
|
||||
|
||||
policy["whitelist"] = whitelist
|
||||
policy["blacklist"] = blacklist
|
||||
policy["allow"] = allow_tools
|
||||
policy["deny"] = deny_tools
|
||||
|
||||
save_tool_policy(username, policy)
|
||||
logger.info(
|
||||
"tool policy saved for %s: max_risk=%s whitelist=%d blacklist=%d allow=%d deny=%d",
|
||||
username, max_risk or "none", len(whitelist), len(blacklist), len(allow_tools), len(deny_tools),
|
||||
)
|
||||
return HTMLResponse(_tools_page(
|
||||
username, back_persona,
|
||||
success=f"Tool policy saved — max risk: {max_risk or 'none'}, "
|
||||
f"{len(whitelist)} whitelisted, {len(blacklist)} blacklisted.",
|
||||
))
|
||||
@@ -56,12 +56,52 @@ def _set_cookie(response: Response, username: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
_LAST_PERSONA_COOKIE = "cx_last_persona"
|
||||
|
||||
|
||||
def _first_persona(username: str) -> str | None:
|
||||
"""Return the first available persona for a user, or None."""
|
||||
names = list_user_personas(username)
|
||||
return names[0] if names else None
|
||||
|
||||
|
||||
def _preferred_persona(request: Request, username: str) -> str | None:
|
||||
"""Return the last-visited persona from cookie if valid, else the first available."""
|
||||
names = list_user_personas(username)
|
||||
if not names:
|
||||
return None
|
||||
cookie_val = request.cookies.get(_LAST_PERSONA_COOKIE, "")
|
||||
if cookie_val in names:
|
||||
return cookie_val
|
||||
return names[0]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Favicon — default sparkle; persona pages override via JS
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_FAVICON_SVG = (
|
||||
"<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'>"
|
||||
"<text y='.9em' font-size='90'>✨</text></svg>"
|
||||
)
|
||||
|
||||
@router.get("/favicon.ico", include_in_schema=False)
|
||||
async def favicon():
|
||||
return Response(content=_FAVICON_SVG, media_type="image/svg+xml")
|
||||
|
||||
|
||||
@router.get("/sw.js", include_in_schema=False)
|
||||
async def service_worker():
|
||||
from fastapi.responses import FileResponse
|
||||
return FileResponse(str(_STATIC / "sw.js"), media_type="application/javascript")
|
||||
|
||||
|
||||
@router.get("/manifest.json", include_in_schema=False)
|
||||
async def web_manifest():
|
||||
from fastapi.responses import FileResponse
|
||||
return FileResponse(str(_STATIC / "manifest.json"), media_type="application/manifest+json")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Root redirect
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -71,7 +111,7 @@ async def root(request: Request):
|
||||
user = _get_session_user(request)
|
||||
if not user:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
persona = _first_persona(user)
|
||||
persona = _preferred_persona(request, user)
|
||||
if not persona:
|
||||
return HTMLResponse("<h1>No personas configured for your account.</h1>", status_code=500)
|
||||
return RedirectResponse(f"/{user}/{persona}", status_code=302)
|
||||
@@ -86,7 +126,7 @@ async def login_page(request: Request):
|
||||
user = _get_session_user(request)
|
||||
if user:
|
||||
# Already logged in — redirect home
|
||||
persona = _first_persona(user)
|
||||
persona = _preferred_persona(request, user)
|
||||
if persona:
|
||||
return RedirectResponse(f"/{user}/{persona}", status_code=302)
|
||||
return HTMLResponse((_STATIC / "login.html").read_text())
|
||||
@@ -123,6 +163,112 @@ async def logout():
|
||||
return resp
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# User landing — /{username} → persona picker
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@router.get("/{username}", include_in_schema=False)
|
||||
async def user_landing(username: str, request: Request):
|
||||
session_user = _get_session_user(request)
|
||||
if not session_user:
|
||||
return RedirectResponse("/login", status_code=302)
|
||||
if session_user != username:
|
||||
return RedirectResponse(f"/{session_user}", status_code=302)
|
||||
|
||||
personas = list_user_personas(username)
|
||||
if not personas:
|
||||
return HTMLResponse("<h1>No personas configured.</h1>", status_code=404)
|
||||
|
||||
cards_html = ""
|
||||
for p in personas:
|
||||
emoji = "✨"
|
||||
identity_path = persona_path(username, p) / "IDENTITY.md"
|
||||
if identity_path.exists():
|
||||
m = re.search(r"\|\s*Emoji\s*\|\s*(.+?)\s*\|", identity_path.read_text())
|
||||
if m:
|
||||
emoji = m.group(1).strip()
|
||||
cards_html += (
|
||||
f'<a href="/{username}/{p}" class="persona-card">'
|
||||
f'<span class="p-emoji">{emoji}</span>'
|
||||
f'<span class="p-name">{p.capitalize()}</span>'
|
||||
f'</a>\n'
|
||||
)
|
||||
|
||||
html = f"""<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — {username}</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
*, *::before, *::after {{ box-sizing: border-box; margin: 0; padding: 0; }}
|
||||
body {{
|
||||
min-height: 100vh;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: #1a1228;
|
||||
font-family: 'Inter', system-ui, -apple-system, sans-serif;
|
||||
font-weight: 450;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
color: #e8e0f0;
|
||||
padding: 2rem 1.5rem;
|
||||
}}
|
||||
.card {{
|
||||
background: #221840;
|
||||
border: 1px solid #3a2852;
|
||||
border-radius: 14px;
|
||||
padding: 2.5rem 2rem;
|
||||
width: 100%;
|
||||
max-width: 400px;
|
||||
text-align: center;
|
||||
}}
|
||||
h1 {{ font-size: 1.3rem; font-weight: 700; color: #c4935a; margin-bottom: 0.4rem; }}
|
||||
.sub {{ font-size: 0.82rem; color: #b0a2c8; margin-bottom: 2rem; }}
|
||||
.personas {{ display: flex; flex-direction: column; gap: 0.75rem; }}
|
||||
.persona-card {{
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1rem;
|
||||
padding: 0.85rem 1.2rem;
|
||||
background: #1a1228;
|
||||
border: 1px solid #3a2852;
|
||||
border-radius: 10px;
|
||||
color: #e8e0f0;
|
||||
text-decoration: none;
|
||||
font-size: 1rem;
|
||||
font-weight: 500;
|
||||
transition: border-color 0.15s, background 0.15s;
|
||||
}}
|
||||
.persona-card:hover {{ border-color: #c4935a; background: #261d42; }}
|
||||
.p-emoji {{ font-size: 1.6rem; line-height: 1; }}
|
||||
.p-name {{ color: #c4935a; font-weight: 600; }}
|
||||
.settings-link {{
|
||||
display: inline-block;
|
||||
margin-top: 1.5rem;
|
||||
font-size: 0.78rem;
|
||||
color: #b0a2c8;
|
||||
text-decoration: none;
|
||||
}}
|
||||
.settings-link:hover {{ color: #e8e0f0; }}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="card">
|
||||
<h1>Cortex</h1>
|
||||
<p class="sub">Signed in as <strong>{username}</strong> — choose a persona</p>
|
||||
<div class="personas">
|
||||
{cards_html} </div>
|
||||
<a href="/settings" class="settings-link">Account settings</a>
|
||||
</div>
|
||||
</body>
|
||||
</html>"""
|
||||
return HTMLResponse(html)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Main UI — /{username}/{persona}
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -134,7 +280,16 @@ async def api_personas(request: Request) -> dict:
|
||||
if not user:
|
||||
from fastapi import HTTPException
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
return {"user": user, "personas": list_user_personas(user)}
|
||||
personas_with_emoji = []
|
||||
for p in list_user_personas(user):
|
||||
emoji = "✨"
|
||||
identity_path = persona_path(user, p) / "IDENTITY.md"
|
||||
if identity_path.exists():
|
||||
m = re.search(r"\|\s*Emoji\s*\|\s*(.+?)\s*\|", identity_path.read_text())
|
||||
if m:
|
||||
emoji = m.group(1).strip()
|
||||
personas_with_emoji.append({"name": p, "emoji": emoji})
|
||||
return {"user": user, "personas": personas_with_emoji}
|
||||
|
||||
|
||||
@router.get("/{username}/{persona}", include_in_schema=False)
|
||||
@@ -168,4 +323,6 @@ async def serve_ui(username: str, persona: str, request: Request):
|
||||
f'{{user: "{username}", persona: "{persona}", emoji: "{emoji}"}};</script>'
|
||||
)
|
||||
html = html.replace("</head>", f"{config_tag}\n</head>", 1)
|
||||
return HTMLResponse(html)
|
||||
resp = HTMLResponse(html)
|
||||
resp.set_cookie(_LAST_PERSONA_COOKIE, persona, max_age=365 * 86400, httponly=False, samesite="lax")
|
||||
return resp
|
||||
|
||||
104
cortex/routers/usage.py
Normal file
104
cortex/routers/usage.py
Normal file
@@ -0,0 +1,104 @@
|
||||
"""
|
||||
Usage / token-tracking endpoints.
|
||||
|
||||
Self-service (any authenticated user, own data):
|
||||
GET /api/usage → full usage dict {date: {model_key: {calls, prompt_tokens, completion_tokens}}}
|
||||
GET /api/usage/summary → aggregate totals per model key, with friendly labels resolved from registry
|
||||
|
||||
Admin-only (cross-user aggregation):
|
||||
GET /api/usage/all → summary for every user {username: summary_dict}
|
||||
"""
|
||||
import jwt
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
|
||||
from auth_utils import COOKIE_NAME, decode_token, get_user_role
|
||||
from persona import list_users
|
||||
import model_registry
|
||||
import usage_tracker
|
||||
|
||||
router = APIRouter(prefix="/api/usage")
|
||||
|
||||
|
||||
def _session_user(request: Request) -> str:
|
||||
token = request.cookies.get(COOKIE_NAME)
|
||||
if not token:
|
||||
raise HTTPException(status_code=401, detail="Not authenticated")
|
||||
try:
|
||||
return decode_token(token)
|
||||
except jwt.InvalidTokenError:
|
||||
raise HTTPException(status_code=401, detail="Invalid session")
|
||||
|
||||
|
||||
def _build_label_map(username: str) -> dict[str, str]:
|
||||
"""Build a map from usage key (backend/model_name) → registered label."""
|
||||
label_map: dict[str, str] = {}
|
||||
try:
|
||||
for m in model_registry.get_all_models(username):
|
||||
model_name = m.get("model_name", "")
|
||||
label = m.get("label", "")
|
||||
host_type = m.get("host_type", "")
|
||||
if not model_name or not label:
|
||||
continue
|
||||
# local models: key is "local/{model_name}"
|
||||
if host_type in ("openwebui", "ollama", "openai_compatible"):
|
||||
label_map[f"local/{model_name}"] = label
|
||||
# cloud Gemini: key is "gemini_api/{model_name}"
|
||||
elif host_type == "google":
|
||||
label_map[f"gemini_api/{model_name}"] = label
|
||||
except Exception:
|
||||
pass
|
||||
return label_map
|
||||
|
||||
|
||||
def _summarize(data: dict, label_map: dict[str, str] | None = None) -> list[dict]:
|
||||
"""Collapse date-keyed usage dict into per-model totals, sorted by total tokens desc."""
|
||||
totals: dict[str, dict] = {}
|
||||
for _date, models in data.items():
|
||||
for key, counts in models.items():
|
||||
t = totals.setdefault(key, {"calls": 0, "prompt_tokens": 0, "completion_tokens": 0})
|
||||
t["calls"] += counts.get("calls", 0)
|
||||
t["prompt_tokens"] += counts.get("prompt_tokens", 0)
|
||||
t["completion_tokens"] += counts.get("completion_tokens", 0)
|
||||
|
||||
result = []
|
||||
for key, counts in totals.items():
|
||||
entry = {
|
||||
"key": key,
|
||||
"label": (label_map or {}).get(key) or key,
|
||||
"calls": counts["calls"],
|
||||
"prompt_tokens": counts["prompt_tokens"],
|
||||
"completion_tokens": counts["completion_tokens"],
|
||||
"total_tokens": counts["prompt_tokens"] + counts["completion_tokens"],
|
||||
}
|
||||
result.append(entry)
|
||||
|
||||
result.sort(key=lambda x: x["total_tokens"], reverse=True)
|
||||
return result
|
||||
|
||||
|
||||
@router.get("")
|
||||
async def get_usage(request: Request) -> dict:
|
||||
"""Return the raw daily usage log for the authenticated user."""
|
||||
username = _session_user(request)
|
||||
return usage_tracker.read_usage(username)
|
||||
|
||||
|
||||
@router.get("/summary")
|
||||
async def get_usage_summary(request: Request) -> list:
|
||||
"""Return per-model totals (all time) for the authenticated user, with friendly labels."""
|
||||
username = _session_user(request)
|
||||
label_map = _build_label_map(username)
|
||||
return _summarize(usage_tracker.read_usage(username), label_map)
|
||||
|
||||
|
||||
@router.get("/all")
|
||||
async def get_all_usage(request: Request) -> dict:
|
||||
"""Admin: return per-model summary for every user."""
|
||||
username = _session_user(request)
|
||||
if get_user_role(username) != "admin":
|
||||
raise HTTPException(status_code=403, detail="Admin access required")
|
||||
result = {}
|
||||
for user in list_users():
|
||||
label_map = _build_label_map(user)
|
||||
result[user] = _summarize(usage_tracker.read_usage(user), label_map)
|
||||
return result
|
||||
@@ -19,37 +19,95 @@ logger = logging.getLogger(__name__)
|
||||
_scheduler: AsyncIOScheduler | None = None
|
||||
|
||||
|
||||
def _all_personas() -> list[tuple[str, str]]:
|
||||
"""Return [(username, persona_name)] for every persona on this instance."""
|
||||
from persona import list_users, list_user_personas
|
||||
pairs = []
|
||||
for u in list_users():
|
||||
for p in list_user_personas(u):
|
||||
pairs.append((u, p))
|
||||
return pairs
|
||||
|
||||
|
||||
async def _run_short() -> None:
|
||||
from memory_distiller import distill_short
|
||||
for u, p in _all_personas():
|
||||
try:
|
||||
result = distill_short()
|
||||
logger.info("auto distill short: %d files, %d chars", result["files_included"], result["chars_written"])
|
||||
result = distill_short(u, p)
|
||||
logger.info("auto distill short [%s/%s]: %d files, %d chars", u, p, result["files_included"], result["chars_written"])
|
||||
except Exception as e:
|
||||
logger.error("auto distill short failed: %s", e)
|
||||
logger.error("auto distill short [%s/%s] failed: %s", u, p, e)
|
||||
|
||||
|
||||
async def _run_mid() -> None:
|
||||
from memory_distiller import distill_mid
|
||||
from notification import notify
|
||||
for u, p in _all_personas():
|
||||
try:
|
||||
result = await distill_mid()
|
||||
result = await distill_mid(u, p)
|
||||
if "error" in result:
|
||||
logger.warning("auto distill mid skipped: %s", result["error"])
|
||||
logger.warning("auto distill mid [%s/%s] skipped: %s", u, p, result["error"])
|
||||
else:
|
||||
logger.info("auto distill mid: %d chars via %s", result["chars_written"], result["backend"])
|
||||
logger.info("auto distill mid [%s/%s]: %d chars via %s", u, p, result["chars_written"], result["backend"])
|
||||
await notify(u, f"📝 Weekly memory digest complete ({result['chars_written']} chars via {result['backend']}).")
|
||||
except Exception as e:
|
||||
logger.error("auto distill mid failed: %s", e)
|
||||
logger.error("auto distill mid [%s/%s] failed: %s", u, p, e)
|
||||
|
||||
|
||||
async def _run_long() -> None:
|
||||
from memory_distiller import distill_long
|
||||
from notification import notify
|
||||
for u, p in _all_personas():
|
||||
try:
|
||||
result = await distill_long()
|
||||
result = await distill_long(u, p)
|
||||
if "error" in result:
|
||||
logger.warning("auto distill long skipped: %s", result["error"])
|
||||
logger.warning("auto distill long [%s/%s] skipped: %s", u, p, result["error"])
|
||||
else:
|
||||
logger.info("auto distill long: %d chars via %s", result["chars_written"], result["backend"])
|
||||
logger.info("auto distill long [%s/%s]: %d chars via %s", u, p, result["chars_written"], result["backend"])
|
||||
await notify(u, f"🧠 Monthly long-term memory integration complete ({result['chars_written']} chars via {result['backend']}). Worth a quick review.")
|
||||
except Exception as e:
|
||||
logger.error("auto distill long failed: %s", e)
|
||||
logger.error("auto distill long [%s/%s] failed: %s", u, p, e)
|
||||
|
||||
|
||||
async def _run_reminder_check() -> None:
|
||||
"""Notify users of any due or overdue reminders (fires once daily at 09:00)."""
|
||||
import re
|
||||
from notification import notify
|
||||
from persona import set_context
|
||||
|
||||
for u, p in _all_personas():
|
||||
try:
|
||||
set_context(u, p)
|
||||
from tools.reminders import load_due_reminders
|
||||
content = load_due_reminders()
|
||||
if not content:
|
||||
continue
|
||||
|
||||
# Extract numbered entries (lines like "1. [label] text" or "1. text")
|
||||
entries = []
|
||||
for line in content.splitlines():
|
||||
m = re.match(r"^\d+\.\s+(.+)", line.strip())
|
||||
if m:
|
||||
# Strip status tags ([OVERDUE], [due TODAY], etc.) for display
|
||||
text = re.sub(r"\[(OVERDUE|due TODAY|due: \S+)\]", "", m.group(1)).strip()
|
||||
if text:
|
||||
entries.append(text)
|
||||
|
||||
if not entries:
|
||||
continue
|
||||
|
||||
count = len(entries)
|
||||
if count == 1:
|
||||
msg = f"Reminder: {entries[0]}"
|
||||
else:
|
||||
bullet_list = "\n".join(f"• {e}" for e in entries[:3])
|
||||
tail = f"\n…and {count - 3} more" if count > 3 else ""
|
||||
msg = f"{count} reminders due:\n{bullet_list}{tail}"
|
||||
|
||||
await notify(u, msg)
|
||||
logger.info("reminder check [%s/%s]: notified %d reminder(s)", u, p, count)
|
||||
except Exception as e:
|
||||
logger.error("reminder check [%s/%s] failed: %s", u, p, e)
|
||||
|
||||
|
||||
def get_scheduler() -> AsyncIOScheduler | None:
|
||||
@@ -76,6 +134,10 @@ def start() -> None:
|
||||
_scheduler.add_job(_run_long, "cron", day=1, hour=4, minute=0, id="distill_long")
|
||||
logger.info("scheduled: distill_long monthly on 1st at 04:00")
|
||||
|
||||
# Daily reminder notification check — 09:00
|
||||
_scheduler.add_job(_run_reminder_check, "cron", hour=9, minute=0, id="reminder_check")
|
||||
logger.info("scheduled: reminder_check daily at 09:00")
|
||||
|
||||
# Load user-defined cron jobs from CRONS.json
|
||||
_load_user_crons()
|
||||
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
from datetime import datetime
|
||||
from config import settings
|
||||
from persona import persona_path
|
||||
from persona import persona_path, get_user, get_persona
|
||||
|
||||
|
||||
def log_turn(session_id: str, user_msg: str, assistant_msg: str) -> None:
|
||||
def log_turn(
|
||||
session_id: str,
|
||||
user_msg: str,
|
||||
assistant_msg: str,
|
||||
backend_label: str = "",
|
||||
host: str = "",
|
||||
) -> None:
|
||||
today = datetime.now().strftime("%Y-%m-%d")
|
||||
sessions_dir = persona_path() / "sessions"
|
||||
sessions_dir.mkdir(exist_ok=True)
|
||||
@@ -12,11 +17,18 @@ def log_turn(session_id: str, user_msg: str, assistant_msg: str) -> None:
|
||||
timestamp = datetime.now().strftime("%H:%M")
|
||||
is_new = not log_file.exists()
|
||||
|
||||
meta_parts = [p for p in [backend_label, host] if p]
|
||||
meta = f" · {' / '.join(meta_parts)}" if meta_parts else ""
|
||||
|
||||
# Use the actual user/persona names from the current request context
|
||||
user_label = get_user().title()
|
||||
persona_label = get_persona().title()
|
||||
|
||||
with open(log_file, "a") as f:
|
||||
if is_new:
|
||||
f.write(f"# Session Log — {today}\n")
|
||||
f.write(
|
||||
f"\n### [{timestamp}] `{session_id}`\n"
|
||||
f"**{settings.user_name}:** {user_msg}\n\n"
|
||||
f"**{settings.agent_name}:** {assistant_msg}\n"
|
||||
f"\n### [{timestamp}] `{session_id}`{meta}\n"
|
||||
f"**{user_label}:** {user_msg}\n\n"
|
||||
f"**{persona_label}:** {assistant_msg}\n"
|
||||
)
|
||||
|
||||
@@ -62,12 +62,40 @@ def save(session_id: str, messages: list[dict]) -> None:
|
||||
# Enforce rolling window
|
||||
windowed = messages[-settings.max_history_messages:]
|
||||
|
||||
path.write_text(json.dumps({
|
||||
data = {
|
||||
"session_id": session_id,
|
||||
"created": existing.get("created", datetime.now().isoformat()),
|
||||
"updated": datetime.now().isoformat(),
|
||||
"messages": windowed,
|
||||
}, indent=2))
|
||||
}
|
||||
if "name" in existing:
|
||||
data["name"] = existing["name"]
|
||||
path.write_text(json.dumps(data, indent=2))
|
||||
|
||||
|
||||
def get_name(session_id: str) -> str:
|
||||
"""Return the friendly name for a session, or '' if none set."""
|
||||
path = _path(session_id)
|
||||
if not path.exists():
|
||||
return ""
|
||||
try:
|
||||
return json.loads(path.read_text()).get("name", "")
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def rename(session_id: str, name: str) -> bool:
|
||||
"""Set (or clear) the friendly name on a session. Returns False if not found."""
|
||||
path = _path(session_id)
|
||||
if not path.exists():
|
||||
return False
|
||||
data = json.loads(path.read_text())
|
||||
if name:
|
||||
data["name"] = name
|
||||
else:
|
||||
data.pop("name", None)
|
||||
path.write_text(json.dumps(data, indent=2))
|
||||
return True
|
||||
|
||||
|
||||
def delete(session_id: str) -> bool:
|
||||
@@ -84,14 +112,17 @@ def list_all() -> list[dict]:
|
||||
if not d.exists():
|
||||
return []
|
||||
results = []
|
||||
for f in sorted(d.glob("*.json"), key=lambda p: p.stat().st_mtime, reverse=True):
|
||||
for f in d.glob("*.json"):
|
||||
try:
|
||||
data = json.loads(f.read_text())
|
||||
results.append({
|
||||
"session_id": data["session_id"],
|
||||
"name": data.get("name", ""),
|
||||
"updated": data.get("updated"),
|
||||
"message_count": len(data.get("messages", [])),
|
||||
"_sort_key": data.get("updated") or f.stat().st_mtime,
|
||||
})
|
||||
except Exception:
|
||||
pass
|
||||
results.sort(key=lambda s: s.pop("_sort_key"), reverse=True)
|
||||
return results
|
||||
|
||||
518
cortex/static/HELP.md
Normal file
518
cortex/static/HELP.md
Normal file
@@ -0,0 +1,518 @@
|
||||
# Cortex UI — Help & Reference
|
||||
|
||||
<!-- SHARED BASE: cortex/static/HELP.md
|
||||
This file is served to all users regardless of persona.
|
||||
Persona-specific additions live in home/{username}/persona/{name}/HELP.md
|
||||
and are appended automatically by help.html when present.
|
||||
-->
|
||||
|
||||
*Last updated: 2026-05-13*
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
If this is your first time using Cortex, you need one thing before the chat will work: an AI model connected to your account.
|
||||
|
||||
**Fastest path — OpenRouter:**
|
||||
OpenRouter gives you access to Claude, Gemini, and dozens of other models with a single API key.
|
||||
|
||||
1. Get a free API key at [openrouter.ai/keys](https://openrouter.ai/keys)
|
||||
2. Go to **☰ → Account → [Set up OpenRouter →]** (shown automatically if no model is configured)
|
||||
3. Paste your key, pick a starting model, click **Connect**
|
||||
|
||||
That's it — you're ready to chat.
|
||||
|
||||
**Already past setup but seeing errors?** Go to **☰ → Account → Model Registry → Manage models** and confirm a model is assigned to the **Chat** role (Primary slot). If all slots are empty, add a model first.
|
||||
|
||||
---
|
||||
|
||||
## Header Controls
|
||||
|
||||
| Button | What it does |
|
||||
|---|---|
|
||||
| **Sessions** | Open the sessions panel — list, resume, or start sessions |
|
||||
| **N** (sliders icon) | Open the Context & Memory panel (N = current context tier) |
|
||||
| **☰** | Settings menu — Files, push notification toggle, Account, Sign Out |
|
||||
| **?** | Open this help panel |
|
||||
|
||||
The **Context & Memory** panel (sliders icon with tier number) contains all configuration options:
|
||||
|
||||
| Section | Controls |
|
||||
|---|---|
|
||||
| **Context Tier** | T1 – T4 context depth |
|
||||
| **Memory Layers** | Toggle Long / Mid / Short memory on/off |
|
||||
| **Distill Memory** | Manually trigger Short / Mid / Long / All distillation |
|
||||
| **Model** | Active chat model — click to cycle through your configured slot models (Primary → Backup 1 → …) |
|
||||
| **Display** | **Aa** cycles font size · **☾** toggles theme · **S/M/L** cycles input area height · **⌃↵** toggles send shortcut |
|
||||
|
||||
All settings persist in `localStorage` across page refreshes.
|
||||
|
||||
---
|
||||
|
||||
## Chat
|
||||
|
||||
- **Send:** `Ctrl+Enter` by default. Click `⌃↵` in the input controls to toggle to plain `Enter` mode.
|
||||
- **Stop:** Click **Stop** to cancel an in-progress response at any time.
|
||||
- **Edit a message:** Hover over any message → click **edit**. `Ctrl+Enter` saves, `Esc` cancels.
|
||||
- **Delete a message:** Hover over any message → click **del**, then **confirm delete**.
|
||||
- **Copy:** Hover over any message → click **copy**.
|
||||
- **New line while typing:** `Shift+Enter` (in `Ctrl+Enter` mode) or `Shift+Enter` / Enter (in Enter mode).
|
||||
|
||||
Each assistant response shows a small **model tag** below the message identifying which model and host responded.
|
||||
|
||||
---
|
||||
|
||||
## Tools (⚡)
|
||||
|
||||
Click the **⚡** button in the input row to enable the Tools toggle. When lit (amber), **Send** changes to **Run** and messages are routed through the **orchestrator** instead of directly to the chat model.
|
||||
|
||||
The orchestrator runs a multi-step tool loop:
|
||||
|
||||
1. The **orchestrator model** reasons about the request and calls tools as needed
|
||||
2. Tool results are fed back into the conversation; the loop continues until the model has what it needs
|
||||
3. The model produces the final user-facing reply — when the orchestrator role uses Gemini, Claude writes the final response; when it uses a local model, that same model writes it
|
||||
4. Expandable tool-call cards appear above the response — click any card to see the arguments sent and the result returned
|
||||
|
||||
The ⚡ toggle is **independent of the Role selector** — you can use any role (chat, coder, research, etc.) with or without tools. The orchestrator model is configured in **Account → Model Registry → Role Assignments → Orchestrator**.
|
||||
|
||||
Tools mode is best for tasks requiring research, multi-step reasoning, or side effects (e.g. "search for X", "add a task", "what's on my list?", "append this to my journal"). Regular chat is faster for conversational turns.
|
||||
|
||||
Orchestrated sessions persist to history exactly like regular chat.
|
||||
|
||||
### Available Tools
|
||||
|
||||
69 tools across 17 categories. Tool schemas are narrowed per-message using keyword routing — only categories relevant to your request are sent, keeping token overhead low. Per-role tool sets provide additional filtering.
|
||||
|
||||
| Category | Tools |
|
||||
|---|---|
|
||||
| **Web** | `web_search`, `http_fetch`, `web_read`, `http_post` |
|
||||
| **Project Files** | `project_file_read`, `project_file_list`, `file_stat`, `file_grep`, `file_diff`, `file_syntax_check` |
|
||||
| **Files** (admin) | `file_read`, `file_list`, `file_write`, `session_read`, `session_search` |
|
||||
| **Git** | `git_status`, `git_log`, `git_diff` |
|
||||
| **Shell** | `shell_exec`, `claude_allow_dir` |
|
||||
| **System** | `cortex_restart`, `cortex_logs`, `cortex_status`, `cortex_update` |
|
||||
| **Tasks** | `task_list`, `task_create`, `task_update`, `task_complete` |
|
||||
| **Cron** | `cron_list`, `cron_add`, `cron_remove`, `cron_toggle` |
|
||||
| **Reminders** | `reminders_add`, `reminders_list`, `reminders_remove`, `reminders_clear` |
|
||||
| **Scratchpad** | `scratch_read`, `scratch_write`, `scratch_append`, `scratch_clear` |
|
||||
| **Notifications** | `web_push`, `email_send`, `nc_talk_send`, `nc_talk_history` |
|
||||
| **Aether Journals** | `ae_journal_list/search`, `ae_journal_entries_list`, `ae_journal_entry_read/create/update/disable/append/prepend` |
|
||||
| **Aether Tasks** | `ae_task_list` |
|
||||
| **Aether Database** (admin) | `ae_db_query`, `ae_db_describe`, `ae_db_show_view` |
|
||||
| **Agent Notes** | `agent_notes_read`, `agent_notes_write`, `agent_notes_append`, `agent_notes_clear` |
|
||||
| **Agents** | `spawn_agent`, `aider_run` |
|
||||
| **Home Assistant** | `ha_get_state`, `ha_get_states`, `ha_call_service` |
|
||||
|
||||
Files, Shell, System, Aether Database, Agents, and some Notification/Web tools are **admin-only** and not visible to regular users.
|
||||
`http_post` requires a URL prefix allowlist in `home/{user}/http_allowlist.json`.
|
||||
`nc_talk_history` requires `nc_username` and `nc_app_password` in `channels.json` under `nextcloud`.
|
||||
`ae_db_*` tools require Aether DB credentials configured in **Integrations** settings. All queries are SELECT-only — no writes possible.
|
||||
`aider_run` requires Aider installed (`pip install aider-chat`) and a model configured via `AIDER_MODEL` env var or the project's `.aider.conf.yml`. Supports any OpenAI-compatible backend — DeepSeek, OpenRouter, Ollama, etc.
|
||||
|
||||
### Per-Role Tool Sets
|
||||
|
||||
Each role can be configured with a specific subset of tool categories. When a role has a tool subset configured, only those tools are sent to the orchestrator — the rest are invisible to the model for that session.
|
||||
|
||||
**Example:** a Coder role might only need Web, Files, Shell, and Agent Notes. A Research role might only need Web. Configuring this avoids sending schemas for 30+ irrelevant tools on every call.
|
||||
|
||||
Configure per-role tool sets in **Account → Model Registry → Role Assignments** — expand a role card to see the category checkboxes. The default (no checkboxes selected) sends all tools the user has access to.
|
||||
|
||||
---
|
||||
|
||||
## Sessions
|
||||
|
||||
Sessions are named conversation threads that persist across page refreshes.
|
||||
|
||||
- Click **Sessions** → **+ New** to start a fresh session.
|
||||
- Click any listed session to resume it — full history loads instantly.
|
||||
- Sessions from Nextcloud Talk appear as `nct_*` prefixed IDs.
|
||||
- A blue **●** badge appears on the Sessions button when Talk activity arrives in a session you're not currently viewing.
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
Notes are injected into a session without triggering an LLM response.
|
||||
|
||||
- Click **Note** to toggle note mode. The input border changes colour.
|
||||
- **Private note** (amber border) — visible only in the UI, never sent to the LLM.
|
||||
- **Context note** (teal border) — persisted to session history so the LLM sees it on the next turn. Useful for nudging context without a full message.
|
||||
- Click the `private / public` label to switch between note types.
|
||||
|
||||
---
|
||||
|
||||
## Install as App (PWA)
|
||||
|
||||
Cortex supports installation as a Progressive Web App — it runs in its own window with no browser chrome.
|
||||
|
||||
- **Chrome / Edge (desktop):** Look for the install icon in the address bar, or open the browser menu → **Install Cortex…**
|
||||
- **Android (Chrome):** Tap ⋮ → **Add to Home Screen**
|
||||
- **iOS (Safari):** Tap the Share button → **Add to Home Screen**
|
||||
|
||||
Once installed, opening Cortex from the home screen or app launcher skips the browser UI entirely.
|
||||
|
||||
---
|
||||
|
||||
## Switching Models
|
||||
|
||||
The **Model** button in the Context & Memory panel cycles through the slot models configured for your active role (Primary → Backup 1). Click it to switch between models mid-session.
|
||||
|
||||
- The button label shows the active model (e.g. "GPT-4o", "Gemini 2.5 Flash")
|
||||
- The selected slot is sent with each chat request so the correct model is used
|
||||
- If only one model is configured, the toggle does nothing
|
||||
- A system message appears in the chat when you switch models
|
||||
|
||||
If the active model fails, the next configured backup slot is tried automatically.
|
||||
|
||||
Each response shows a **model tag** (bottom-right of message) with the model label and host, so you always know what responded.
|
||||
|
||||
---
|
||||
|
||||
## Account Settings
|
||||
|
||||
**Navigate to:** ☰ (top-right menu) → **Account**
|
||||
|
||||
| Section | What you can do |
|
||||
|---|---|
|
||||
| **Account** | View your username, role badge (Admin / User), rename your username |
|
||||
| **Connected Accounts** | See which Google account is linked for OAuth sign-in |
|
||||
| **Email Allowlist** | Regex patterns controlling which addresses the `email_send` tool can reach |
|
||||
| **Notifications** | Dedicated page — set channel (Browser Push, NC Talk, Google Chat, email) for proactive messages; configure Home Assistant inbound webhook; test buttons for instant verification |
|
||||
| **Schedules** | View, add, edit, pause, and delete scheduled jobs directly — without going through the AI |
|
||||
| **Tool Permissions** | Allow or block specific orchestrator tools for your account |
|
||||
| **Usage** | Token consumption by model — see below |
|
||||
| **Browser Cache** | Clear UI preferences stored locally (theme, font size, session ID, etc.) |
|
||||
| **Model Registry** | Configure AI providers, local hosts, and role assignments |
|
||||
| **Change Password** | Update your login password |
|
||||
| **Personas** | List and rename your personas |
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
Token consumption is tracked automatically for API-backed models. **Navigate to:** ☰ → **Account** → **Usage** section.
|
||||
|
||||
The table shows all-time totals per model key, with columns for:
|
||||
|
||||
| Column | Meaning |
|
||||
|---|---|
|
||||
| **Model** | `backend/model-name` key (e.g. `gemini_api/gemini-2.5-flash`, `local/deepseek-v4`) |
|
||||
| **Calls** | Number of API calls made |
|
||||
| **Prompt** | Input tokens sent |
|
||||
| **Output** | Completion tokens received |
|
||||
| **Total** | Prompt + Output |
|
||||
|
||||
Values ≥ 1,000 are displayed as `k` (e.g. `24.3k`).
|
||||
|
||||
**What is and isn't tracked:**
|
||||
|
||||
- ✅ Gemini API calls (orchestrator, distillation)
|
||||
- ✅ Local OpenAI-compatible calls (Open WebUI, Ollama, OpenRouter)
|
||||
- ✗ Claude CLI — no structured token data is returned by the subprocess
|
||||
- ✗ Gemini CLI — same reason
|
||||
|
||||
The raw data lives in `home/{username}/usage.json` and is also accessible via the Files panel or the API.
|
||||
|
||||
---
|
||||
|
||||
## Model Registry
|
||||
|
||||
Configure which AI models are available and which handles each task type.
|
||||
|
||||
**New user quick path:** ☰ → **Account** → **Set up OpenRouter →** (the guided wizard adds a host, model, and role assignment in one step).
|
||||
|
||||
**Full manual path:** ☰ → **Account** → scroll to **Model Registry** → **Manage models →**
|
||||
|
||||
---
|
||||
|
||||
### Step 1 — Set up providers and hosts
|
||||
|
||||
Do this before adding models — models need a provider account or local host to attach to.
|
||||
|
||||
**Anthropic (Claude):** Two options:
|
||||
- **CLI (OAuth):** Nothing to configure — uses your existing `claude auth login` session. If Claude isn't working, run `claude auth login` in a terminal.
|
||||
- **Direct API key:** Scroll to **Cloud Providers → Anthropic** → click **+ Add API key**. Enter a label and your `sk-ant-…` key from [console.anthropic.com/keys](https://console.anthropic.com/keys). When you add a model using an API key credential, it routes through the Anthropic SDK instead of the CLI.
|
||||
|
||||
**Google (Gemini):** Add one entry per API key you want to use:
|
||||
1. Scroll to **Cloud Providers → Google** → click **+ Add Google account**
|
||||
2. Enter a label (e.g. "Work", "Personal") and your API key
|
||||
3. Get a free key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey)
|
||||
|
||||
**OpenRouter** (recommended for new users — one key for many models):
|
||||
1. Get a key at [openrouter.ai/keys](https://openrouter.ai/keys)
|
||||
2. Scroll to **Local Hosts** → **+ Add host**
|
||||
3. Label: "OpenRouter", URL: `https://openrouter.ai/api/v1`, paste your key, Type: OpenAI-compatible
|
||||
4. Click **Fetch models** to verify, then add models from the fetched list
|
||||
|
||||
**Other local hosts** (Open WebUI, Ollama, LM Studio, etc.):
|
||||
1. Scroll to **Local Hosts** → click **+ Add host** to expand the form
|
||||
2. Enter a label, the API URL (e.g. `http://192.168.1.100:3000`), and optional API key
|
||||
3. Set **Type**: Open WebUI / Ollama, or OpenAI-compatible
|
||||
4. Click **Fetch models** on the saved host card to verify connectivity
|
||||
|
||||
---
|
||||
|
||||
### Step 2 — Add models
|
||||
|
||||
Scroll to **Add Model**. Select the provider tab, fill in the details, click **Add Model**:
|
||||
|
||||
| Tab | What you need |
|
||||
|---|---|
|
||||
| **Local** | Select a host (from Step 1) → enter model name, or use **Fetch from host** to pick from a live list |
|
||||
| **Google** | Select a Gemini model from the catalog → select a Google account (from Step 1) |
|
||||
| **Anthropic** | Select a credential (CLI OAuth or an API key added in Step 1) → select a Claude model from the catalog |
|
||||
|
||||
The label and context window size auto-fill from the catalog — edit them if you want. Tags are optional.
|
||||
|
||||
---
|
||||
|
||||
### Step 3 — Assign models to roles
|
||||
|
||||
Scroll to **Role Assignments** at the bottom of the page. Each role has **Primary** and **Backup 1** slots — Primary is tried first, then Backup 1. Changes save automatically.
|
||||
|
||||
**Required roles** (always present, cannot be removed):
|
||||
|
||||
| Role | Used for |
|
||||
|---|---|
|
||||
| **Chat** | Regular conversation |
|
||||
| **Orchestrator** | Agent mode tool loop |
|
||||
| **Distill** | Memory distillation (short / mid / long) |
|
||||
|
||||
**Custom roles** — Click **+ Add custom role** to create your own. Each custom role gets its own model selection, tool set, and system prompt addition. Good examples:
|
||||
|
||||
| Example | Purpose |
|
||||
|---|---|
|
||||
| **Coder** | Code-focused tasks — larger context window, code-aware model |
|
||||
| **Research** | Long-context research — high-token model, web tools prioritized |
|
||||
|
||||
Switch roles via the **Role** selector in the Context & Memory panel (⚙). Leave all slots empty to use the server default.
|
||||
|
||||
**Per-role tool sets:** Expand any role card to configure which tool categories the orchestrator can use when that role is active. Unchecked categories are hidden from the model entirely — reducing token overhead on every orchestrated call. Leaving all categories unchecked means all tools the user has access to are available (the default).
|
||||
|
||||
**Inject timestamp:** Each role card has an "Inject current date & time into system prompt" checkbox (default on). Disable it for pure processing roles (summarizer, classifier, translator) that don't need clock awareness.
|
||||
|
||||
---
|
||||
|
||||
## Nextcloud Talk Bot
|
||||
|
||||
The Cortex bot is registered in Nextcloud Talk.
|
||||
|
||||
- Messages sent in enabled Talk conversations are received by Cortex, processed, and replied to.
|
||||
- The webhook returns `200 OK` immediately; the reply happens asynchronously.
|
||||
- Real-time updates stream to the web UI via SSE — you see Talk messages and responses appear live.
|
||||
- To enable the bot in a conversation: open Talk conversation settings → Bots → enable the bot.
|
||||
|
||||
---
|
||||
|
||||
## Google Chat Bot
|
||||
|
||||
The Cortex bot is available in Google Chat (One Sky IT Workspace).
|
||||
|
||||
- Send the bot a direct message in Google Chat to start a conversation.
|
||||
- Each DM thread is its own session (`gc_spaces/*` prefix) — history persists across messages.
|
||||
- Responses are synchronous — Google Chat displays the reply directly in the thread.
|
||||
- To add the bot to a space: open the space, click **Add people & apps**, and search for the Cortex bot.
|
||||
- Sessions from Google Chat appear as `gc_*` prefixed IDs in the Sessions panel.
|
||||
|
||||
---
|
||||
|
||||
## Files (Identity Editor)
|
||||
|
||||
The **Files** button opens an editor for your persona's identity and memory files:
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `SOUL.md` | Core personality, values, and voice |
|
||||
| `IDENTITY.md` | Role, capabilities, and context |
|
||||
| `USER.md` | Your profile, preferences, and history |
|
||||
| `PROTOCOLS.md` | Behavioural rules and communication protocols |
|
||||
| `CONTEXT_TIERS.md` | Defines what gets loaded at each context tier |
|
||||
| `MEMORY_LONG.md` | Permanent curated long-term memory |
|
||||
| `MEMORY_MID.md` | Rolling mid-term digest (LLM-distilled) |
|
||||
| `MEMORY_SHORT.md` | Recent session rollup (auto-aggregated) |
|
||||
| `HELP.md` | This file — persona-specific additions appended below |
|
||||
| `email_allowlist.json` | Regex patterns for permitted `email_send` recipients (one per line) |
|
||||
|
||||
Toggle **preview** / **edit** to switch between rendered markdown and raw text. **Ctrl+S** saves, **Esc** closes.
|
||||
|
||||
The **Audit Log** group at the bottom of the sidebar (collapsed by default) lists tool call logs by date (`YYYY-MM-DD.jsonl`). Click any date to view a read-only table of every orchestrator tool call: time, tool name, status, model, args, and result snippet. Status is colour-coded: green = ok, red = error, amber = denied.
|
||||
|
||||
---
|
||||
|
||||
## Push Notifications
|
||||
|
||||
Cortex can send browser push notifications — even when the tab is closed.
|
||||
|
||||
- Open **☰ → Enable notifications** and accept the browser permission prompt.
|
||||
- Once enabled, the button shows **Notifications on** (in accent colour).
|
||||
- Click again to disable. Subscriptions are stored per-device.
|
||||
- The orchestrator's `web_push` tool lets your persona send you a push proactively (e.g. when a long task completes).
|
||||
|
||||
**Notification channel settings:** ☰ → **Account** → **Notification settings →** — choose Browser Push, Email, Nextcloud Talk, or Google Chat as the channel your persona uses for scheduled reminders, cron job completions, and memory digests. Use the **Send Test Notification** button to verify your setup, or **Check Reminders Now** to trigger the reminder check immediately.
|
||||
|
||||
---
|
||||
|
||||
## Context & Memory ( ⚙ panel )
|
||||
|
||||
### Context Tiers
|
||||
|
||||
Controls how much context is prepended to each LLM call:
|
||||
|
||||
| Tier | Loads | ~Tokens |
|
||||
|---|---|---|
|
||||
| **Min** | SOUL + IDENTITY + USER summary | ~1,500 |
|
||||
| **Std** | + USER full + PROTOCOLS + HELP + memory layers | ~5,000 |
|
||||
| **Ext** | + last 2 raw session logs | ~15,000 |
|
||||
| **Full** | + last 7 raw session logs | ~50,000 |
|
||||
|
||||
Default is **Std**. Use **Min** for small/local models. Use **Ext** or **Full** for complex multi-session tasks.
|
||||
|
||||
### Memory Layers
|
||||
|
||||
Three independently toggleable memory files, loaded **Long → Mid → Short**:
|
||||
|
||||
| Layer | File | Contents |
|
||||
|---|---|---|
|
||||
| **Long** | `MEMORY_LONG.md` | Permanent facts — origin, key decisions, profile highlights |
|
||||
| **Mid** | `MEMORY_MID.md` | Rolling digest of recent weeks — LLM-distilled from Short |
|
||||
| **Short** | `MEMORY_SHORT.md` | Recent session rollup — auto-aggregated from session logs |
|
||||
|
||||
Toggle any layer off to save tokens for a focused conversation.
|
||||
|
||||
### Memory Distillation
|
||||
|
||||
Distillation builds up the memory layers from raw session logs. Runs automatically on a schedule; trigger manually via the ⚙ panel:
|
||||
|
||||
| Button | What it does |
|
||||
|---|---|
|
||||
| **short** | Rolls recent session log files → `MEMORY_SHORT.md` (fast, no LLM) |
|
||||
| **mid** | LLM summarizes `MEMORY_SHORT.md` → `MEMORY_MID.md` |
|
||||
| **long** | LLM integrates `MEMORY_MID.md` → `MEMORY_LONG.md` |
|
||||
| **all** | Runs short → mid → long in sequence |
|
||||
|
||||
**Recommended workflow:** run **short** after any productive session; **mid** weekly; **long** monthly.
|
||||
|
||||
---
|
||||
|
||||
## Scheduled Jobs
|
||||
|
||||
Cortex can run recurring jobs on a schedule — reminders, daily briefings, automated research, and more. Manage them by asking your persona to set them up, or go directly to **☰ → Account → Schedules**.
|
||||
|
||||
### Job Types
|
||||
|
||||
| Type | What it does |
|
||||
|---|---|
|
||||
| `remind` | Appends to `REMINDERS.md` — automatically surfaced in chat context |
|
||||
| `note` | Appends to `SCRATCH.md` — read on demand via the scratchpad |
|
||||
| `message` | Sends the payload text directly to your notification channel |
|
||||
| `brief` | Calls the AI with your payload as the prompt, sends the response to your notification channel. Good for morning briefings, check-ins. |
|
||||
| `task` | Runs the full orchestrator tool loop with your payload as the request, sends Claude's response to your notification channel. Use this for agentic scheduled work: research, file updates, summaries that need tool access. |
|
||||
|
||||
For `task` jobs: tools that require confirmation are skipped in scheduled context. Pre-approve them in **Settings → Tools** to allow them in scheduled tasks.
|
||||
|
||||
### Schedule Formats
|
||||
|
||||
| Format | When it runs |
|
||||
|---|---|
|
||||
| `hourly` | Every hour at :00 |
|
||||
| `daily` | Every day at 09:00 |
|
||||
| `daily:HH:MM` | Every day at the specified time |
|
||||
| `weekly:DOW` | Every specified day at 09:00 (e.g. `weekly:mon`) |
|
||||
| `weekly:DOW:HH:MM` | Every specified day at the specified time (e.g. `weekly:fri:17:00`) |
|
||||
| `monthly` | 1st of every month at 09:00 |
|
||||
| `monthly:DD` | Specific day of month at 09:00 (e.g. `monthly:15`) |
|
||||
| `monthly:DD:HH:MM` | Specific day of month at the specified time |
|
||||
| `yearly:MM:DD` | Every year on that date at 09:00 — for birthdays, anniversaries (e.g. `yearly:03:15`) |
|
||||
| `yearly:MM:DD:HH:MM` | Every year on that date at the specified time |
|
||||
|
||||
DOW values: `mon tue wed thu fri sat sun`. All times are server-local.
|
||||
|
||||
Schedules take effect immediately when added or edited — no restart needed. Paused jobs stay in the list and can be resumed at any time.
|
||||
|
||||
### Home Assistant Integration
|
||||
|
||||
HA automations can trigger your persona via webhook. Configure in **Notifications → Home Assistant → Inbound webhook**:
|
||||
|
||||
- Set a **Webhook ID** (long random string — this is your secret URL component)
|
||||
- Your endpoint: `https://cortex.dgrzone.com/webhook/ha/{username}/{webhook_id}`
|
||||
- **Enable orchestrator tools** — when checked, HA events trigger the full tool loop; when unchecked, events get a direct LLM response (faster, no tools)
|
||||
|
||||
HA payload fields recognized: `message`, `entity_id`, `state`, `trigger`, `event`, `area`.
|
||||
|
||||
---
|
||||
|
||||
## Keyboard Shortcuts
|
||||
|
||||
| Keys | Action |
|
||||
|---|---|
|
||||
| `Ctrl+Enter` | Send message (default mode) |
|
||||
| `Enter` | Send (when in Enter mode) |
|
||||
| `Shift+Enter` | New line in message input |
|
||||
| `Ctrl+Enter` | Save inline message edit |
|
||||
| `Esc` | Cancel inline edit / close any open modal |
|
||||
| `Ctrl+S` | Save file (Files modal) |
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
For direct access or scripting:
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|---|---|---|
|
||||
| `POST` | `/chat` | Send a message — returns SSE stream |
|
||||
| `GET` | `/backend` | Get current primary/fallback backends |
|
||||
| `POST` | `/backend` | Set primary backend (`{"primary": "claude"}`) |
|
||||
| `GET` | `/sessions` | List all sessions |
|
||||
| `GET` | `/history/{id}` | Get session message history |
|
||||
| `PUT` | `/history/{id}` | Replace full session history |
|
||||
| `GET` | `/events` | SSE stream for real-time Talk activity |
|
||||
| `POST` | `/note` | Inject a context note into a session |
|
||||
| `GET` | `/files` | List identity files |
|
||||
| `GET` | `/files/{name}` | Read a file |
|
||||
| `PUT` | `/files/{name}` | Write a file |
|
||||
| `POST` | `/distill/short` | Aggregate session logs → MEMORY_SHORT |
|
||||
| `POST` | `/distill/mid` | Summarize short → MEMORY_MID (LLM) |
|
||||
| `POST` | `/distill/long` | Integrate mid → MEMORY_LONG (LLM) |
|
||||
| `POST` | `/distill/all` | Run all three distillation steps |
|
||||
| `GET` | `/distill/status` | Scheduler status and next run times |
|
||||
| `POST` | `/orchestrate` | Submit an agent task — returns `{"job_id": "..."}` |
|
||||
| `GET` | `/orchestrate/{job_id}` | Poll job status and result |
|
||||
| `GET` | `/settings/models` | Model registry UI |
|
||||
| `POST` | `/api/models/role` | Set a role assignment (JSON body) |
|
||||
| `POST` | `/api/models/role-config` | Set per-role tool list and system prompt append |
|
||||
| `GET` | `/api/push/vapid-key` | VAPID public key (for push subscription) |
|
||||
| `POST` | `/api/push/subscribe` | Register a push subscription |
|
||||
| `DELETE` | `/api/push/subscribe` | Remove a push subscription |
|
||||
| `POST` | `/api/push/test` | Send a test notification via configured channel |
|
||||
| `POST` | `/api/push/reminders/check` | Run reminder check immediately; returns `{"reminders_found": n}` |
|
||||
| `GET` | `/api/audit/files` | List available audit log dates (own data) |
|
||||
| `GET` | `/api/audit/day?date=` | Tool call entries for a specific date (own data) |
|
||||
| `GET` | `/api/audit/recent` | Recent tool calls across days (admin) |
|
||||
| `GET` | `/api/audit/stats` | Tool call counts by tool/status/user (admin) |
|
||||
| `GET` | `/api/usage` | Full daily token usage log (own data) |
|
||||
| `GET` | `/api/usage/summary` | Per-model token totals, all time (own data) |
|
||||
| `GET` | `/api/usage/all` | Per-model totals for all users (admin) |
|
||||
| `GET` | `/setup/model` | Guided OpenRouter setup form (Step 3 / standalone) |
|
||||
| `POST` | `/setup/model` | Save OpenRouter host + model + assign to chat role |
|
||||
| `GET` | `/health` | Health check — returns `{"status": "ok"}` |
|
||||
|
||||
Chat request body (`POST /chat`):
|
||||
```json
|
||||
{
|
||||
"message": "string",
|
||||
"session_id": "string | null",
|
||||
"tier": 2,
|
||||
"chat_role": "chat",
|
||||
"slot": "primary | backup_1 | backup_2 | null",
|
||||
"include_long": true,
|
||||
"include_mid": true,
|
||||
"include_short": true,
|
||||
"off_record": false
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Cortex is a self-hosted personal AI platform. Named after the 'verse-wide communications network in Firefly.*
|
||||
123
cortex/static/TOOLS.md
Normal file
123
cortex/static/TOOLS.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Tool Reference
|
||||
|
||||
> This reference covers all 45 orchestrator tools available when the ⚡ toggle is on.
|
||||
> Tools are invoked automatically by the orchestrator — you don't call them directly.
|
||||
|
||||
¹ **Admin only** — requires the `admin` role. Invisible to regular users.
|
||||
² **Confirmation required** — the orchestrator pauses and shows **Confirm / Deny** buttons before executing.
|
||||
|
||||
---
|
||||
|
||||
## Web
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `web_search` | DuckDuckGo search — returns titles, URLs, and snippets for the top results |
|
||||
| `http_fetch` | Fetch a specific URL and return the response body (8 192 char cap) |
|
||||
|
||||
## Files ¹
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `file_read` ¹ | Read any file under the persona home directory |
|
||||
| `file_list` ¹ | List files and directories with sizes (200 entry cap) |
|
||||
| `file_write` ¹ ² | Write or append to a file under the persona home directory |
|
||||
|
||||
## Shell ¹
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `shell_exec` ¹ ² | Run any shell command on the Cortex host; timeout 1–120 s |
|
||||
| `claude_allow_dir` ¹ | Add a directory to Claude Code's auto-allowed paths |
|
||||
|
||||
## System ¹
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `cortex_restart` ¹ ² | Restart the Cortex service (5 s delay); connection drops — refresh the page |
|
||||
| `cortex_logs` ¹ | Recent lines from the systemd journal (default 50, max 200) |
|
||||
| `cortex_status` ¹ | Current git branch, commit, ahead/behind remote, and service state |
|
||||
| `cortex_update` ¹ ² | `git pull` + syntax check all `.py` files; reports what changed. Does **not** restart automatically — call `cortex_restart` after reviewing |
|
||||
|
||||
## Tasks
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `task_list` | List personal tasks; pass `include_done=true` to include completed |
|
||||
| `task_create` | Create a task with title, optional notes and due date |
|
||||
| `task_update` | Update any fields on an existing task |
|
||||
| `task_complete` | Mark a task as complete |
|
||||
|
||||
## Cron
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `cron_list` | List all scheduled jobs for this persona |
|
||||
| `cron_add` | Add a scheduled job — accepts cron syntax or plain-English interval |
|
||||
| `cron_remove` ² | Remove a scheduled job by ID |
|
||||
| `cron_toggle` | Enable or disable a job without removing it |
|
||||
|
||||
## Reminders
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `reminders_add` | Add a reminder with optional label; surfaced in context at Tier 2+ |
|
||||
| `reminders_list` | List all pending reminders, numbered for easy removal |
|
||||
| `reminders_remove` | Remove a single reminder by number (call `reminders_list` first) |
|
||||
| `reminders_clear` ² | Clear all reminders at once |
|
||||
|
||||
## Scratchpad
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `scratch_read` | Read the current scratchpad |
|
||||
| `scratch_write` | Overwrite the scratchpad with new content |
|
||||
| `scratch_append` | Append a timestamped section to the scratchpad |
|
||||
| `scratch_clear` | Erase the scratchpad |
|
||||
|
||||
## Notifications ¹
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `web_push` | Send a browser push notification to the active user's registered devices |
|
||||
| `email_send` ¹ | Send an email via SMTP; recipient must match your `email_allowlist.json` |
|
||||
| `nc_talk_send` ¹ | Send a message to a Nextcloud Talk conversation |
|
||||
|
||||
## Aether Journals
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `ae_journal_list` | List all journals for the configured AE account (returns names + IDs) |
|
||||
| `ae_journal_search` | Search entries by keyword, tag, date range, type, status, or priority |
|
||||
| `ae_journal_entries_list` | Browse all entries in a specific journal, newest first; paginated |
|
||||
| `ae_journal_entry_read` | Read the full content of a single entry by ID |
|
||||
| `ae_journal_entry_create` | Create a new entry with title, content, tags, and summary |
|
||||
| `ae_journal_entry_update` | Patch any fields on an existing entry (title, content, tags, summary, enable) |
|
||||
| `ae_journal_entry_disable` | Soft-delete an entry (`enable=false`) without permanently removing it |
|
||||
| `ae_journal_entry_append` | Append a timestamped section to the bottom of an entry's content |
|
||||
| `ae_journal_entry_prepend` | Prepend a timestamped section to the top of an entry's content |
|
||||
|
||||
## Aether Tasks ¹
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `ae_task_list` ¹ | List tasks from the agents_sync Kanban board |
|
||||
|
||||
## Agent Notes
|
||||
|
||||
Private, durable notes visible only to the orchestrator — not surfaced to users. Persist across sessions. Only available in orchestrated (tool-enabled) sessions.
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `agent_notes_read` | Read the current private notes file |
|
||||
| `agent_notes_write` | Overwrite the notes file completely |
|
||||
| `agent_notes_append` | Append a timestamped entry (keeps last 3 backups automatically) |
|
||||
| `agent_notes_clear` | Erase all notes (backs up first) |
|
||||
|
||||
## Agents ¹
|
||||
|
||||
Spawn sub-agents that run their own tool loop using a specific role's model and tools.
|
||||
|
||||
| Tool | What it does |
|
||||
|---|---|
|
||||
| `spawn_agent` ¹ | Spawn a sub-agent synchronously — blocks until the task completes or times out. Params: `task`, `role` (default `chat`), `tier` (1–4, default 1), `timeout` seconds, `max_rounds` override. Only works with `local_openai` and `gemini_api` models. |
|
||||
1737
cortex/static/app.js
1737
cortex/static/app.js
File diff suppressed because it is too large
Load Diff
172
cortex/static/crons.html
Normal file
172
cortex/static/crons.html
Normal file
@@ -0,0 +1,172 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Schedules</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
corePlugins: { preflight: false },
|
||||
darkMode: ['selector', '[data-theme="dark"]'],
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
pg: {
|
||||
bg: 'var(--pg-bg)',
|
||||
surface: 'var(--pg-surface)',
|
||||
border: 'var(--pg-border)',
|
||||
text: 'var(--pg-text)',
|
||||
muted: 'var(--pg-muted)',
|
||||
dim: 'var(--pg-dim)',
|
||||
dimmer: 'var(--pg-dimmer)',
|
||||
bright: 'var(--pg-bright)',
|
||||
accent: 'var(--pg-accent)',
|
||||
action: 'var(--pg-action)',
|
||||
}
|
||||
},
|
||||
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<link rel="stylesheet" href="/static/pg.css">
|
||||
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
|
||||
<style>
|
||||
/* ── Server-generated table + badges ── */
|
||||
.cron-table {
|
||||
width: 100%; border-collapse: collapse; font-size: 0.82rem;
|
||||
margin-bottom: 1.5rem;
|
||||
}
|
||||
.cron-table th {
|
||||
text-align: left; padding: 0.4rem 0.6rem;
|
||||
border-bottom: 2px solid var(--pg-border);
|
||||
color: var(--pg-muted); font-weight: 600; font-size: 0.75rem;
|
||||
text-transform: uppercase; letter-spacing: 0.04em;
|
||||
}
|
||||
.cron-table td {
|
||||
padding: 0.5rem 0.6rem; border-bottom: 1px solid var(--pg-border);
|
||||
vertical-align: middle;
|
||||
}
|
||||
.cron-table tr:last-child td { border-bottom: none; }
|
||||
.cron-table tr:hover td { background: var(--pg-hover); }
|
||||
|
||||
.badge {
|
||||
display: inline-block; padding: 0.15rem 0.45rem;
|
||||
border-radius: 4px; font-size: 0.72rem; font-weight: 600;
|
||||
text-transform: uppercase; letter-spacing: 0.03em;
|
||||
}
|
||||
.badge-enabled { background: color-mix(in srgb, var(--pg-accent) 18%, transparent); color: var(--pg-accent); }
|
||||
.badge-paused { background: color-mix(in srgb, var(--pg-muted) 18%, transparent); color: var(--pg-muted); }
|
||||
.badge-remind { background: color-mix(in srgb, #a78bfa 15%, transparent); color: #a78bfa; }
|
||||
.badge-note { background: color-mix(in srgb, #60a5fa 15%, transparent); color: #60a5fa; }
|
||||
.badge-message { background: color-mix(in srgb, #34d399 15%, transparent); color: #34d399; }
|
||||
.badge-brief { background: color-mix(in srgb, #fb923c 15%, transparent); color: #fb923c; }
|
||||
.badge-task { background: color-mix(in srgb, #f472b6 15%, transparent); color: #f472b6; }
|
||||
|
||||
.cron-actions { display: flex; gap: 0.35rem; }
|
||||
.btn-cron {
|
||||
padding: 0.2rem 0.55rem; border-radius: 4px; border: 1px solid var(--pg-border);
|
||||
background: transparent; color: var(--pg-muted); font-size: 0.75rem; cursor: pointer;
|
||||
font-family: inherit;
|
||||
}
|
||||
.btn-cron:hover { border-color: var(--pg-accent); color: var(--pg-accent); }
|
||||
.btn-cron-del { color: var(--pg-dimmer); }
|
||||
.btn-cron-del:hover { border-color: #ef4444; color: #ef4444; }
|
||||
|
||||
.payload-cell {
|
||||
max-width: 240px; overflow: hidden; text-overflow: ellipsis;
|
||||
white-space: nowrap; color: var(--pg-dimmer);
|
||||
}
|
||||
|
||||
.persona-group-label {
|
||||
font-size: 0.72rem; font-weight: 700; text-transform: uppercase;
|
||||
letter-spacing: 0.06em; color: var(--pg-dimmer); margin: 1.25rem 0 0.5rem;
|
||||
}
|
||||
|
||||
.empty-state {
|
||||
text-align: center; padding: 2rem 1rem;
|
||||
color: var(--pg-dimmer); font-size: 0.85rem;
|
||||
border: 1px dashed var(--pg-border); border-radius: 8px;
|
||||
margin-bottom: 1.5rem;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<nav class="page-nav">
|
||||
<a href="{{ back_href }}" class="nav-link">← Chat</a>
|
||||
<a href="{{ help_href }}" class="nav-link">Help</a>
|
||||
<a href="/settings" class="nav-link">Settings</a>
|
||||
<a href="/settings/models" class="nav-link">Models</a>
|
||||
<a href="/settings/notifications" class="nav-link">Notifications</a>
|
||||
<a href="/settings/tools" class="nav-link">Tools</a>
|
||||
<a href="/settings/crons" class="nav-link active">Schedules</a>
|
||||
{{ integrations_nav }}
|
||||
<span class="nav-spacer"></span>
|
||||
<a href="/logout" class="nav-link nav-logout">Sign out</a>
|
||||
</nav>
|
||||
<div class="page-wrap">
|
||||
<h1 class="page-title">Schedules</h1>
|
||||
<p class="page-subtitle">Recurring jobs — reminders, notes, briefings, and agentic tasks.</p>
|
||||
|
||||
<!-- SUCCESS -->
|
||||
<!-- ERROR -->
|
||||
|
||||
<!-- Edit form (shown only when editing) -->
|
||||
{{ edit_html }}
|
||||
|
||||
<!-- Cron list -->
|
||||
{{ cron_list_html }}
|
||||
|
||||
<!-- Add new schedule -->
|
||||
<div class="section">
|
||||
<h2>Add schedule</h2>
|
||||
<form method="POST" action="/settings/crons/add">
|
||||
<div class="grid grid-cols-2 gap-x-3">
|
||||
<div class="field">
|
||||
<label for="add_persona">Persona</label>
|
||||
<select id="add_persona" name="persona">
|
||||
{{ persona_options }}
|
||||
</select>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="add_job_type">Type</label>
|
||||
<select id="add_job_type" name="job_type">
|
||||
<option value="remind">remind — append to REMINDERS.md</option>
|
||||
<option value="note">note — append to SCRATCH.md</option>
|
||||
<option value="message">message — send payload as-is</option>
|
||||
<option value="brief">brief — LLM response, no tools</option>
|
||||
<option value="task">task — full orchestrator tool loop</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="add_label">Label</label>
|
||||
<input type="text" id="add_label" name="label"
|
||||
placeholder="Monday morning summary"
|
||||
required autocomplete="off">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="add_schedule">Schedule</label>
|
||||
<input type="text" id="add_schedule" name="schedule"
|
||||
placeholder="weekly:mon:08:00"
|
||||
required autocomplete="off" spellcheck="false">
|
||||
<p class="hint">
|
||||
hourly · daily · daily:HH:MM · weekly:DOW · weekly:DOW:HH:MM ·
|
||||
monthly · monthly:DD · monthly:DD:HH:MM · yearly:MM:DD · yearly:MM:DD:HH:MM
|
||||
</p>
|
||||
</div>
|
||||
<div class="field col-span-2">
|
||||
<label for="add_payload">Payload / prompt</label>
|
||||
<textarea id="add_payload" name="payload" rows="3"
|
||||
placeholder="Check my open tasks and send a summary." required></textarea>
|
||||
</div>
|
||||
</div>
|
||||
<button type="submit" class="btn-submit w-full md:w-96">Add schedule</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
239
cortex/static/help.html
Normal file
239
cortex/static/help.html
Normal file
@@ -0,0 +1,239 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Help & Reference</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
corePlugins: { preflight: false },
|
||||
darkMode: ['selector', '[data-theme="dark"]'],
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
pg: {
|
||||
bg: 'var(--pg-bg)',
|
||||
surface: 'var(--pg-surface)',
|
||||
border: 'var(--pg-border)',
|
||||
text: 'var(--pg-text)',
|
||||
muted: 'var(--pg-muted)',
|
||||
dim: 'var(--pg-dim)',
|
||||
dimmer: 'var(--pg-dimmer)',
|
||||
bright: 'var(--pg-bright)',
|
||||
accent: 'var(--pg-accent)',
|
||||
action: 'var(--pg-action)',
|
||||
}
|
||||
},
|
||||
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<link rel="stylesheet" href="/static/pg.css">
|
||||
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
|
||||
<style>
|
||||
/* ── Tab panels (JS-toggled display) ── */
|
||||
.tab-panel { display: none; }
|
||||
.tab-panel.active { display: block; }
|
||||
|
||||
/* ── Dynamically-rendered markdown content ── */
|
||||
.help-body { line-height: 1.7; }
|
||||
|
||||
details {
|
||||
margin-bottom: 0.75rem;
|
||||
background: var(--pg-surface);
|
||||
border: 1px solid var(--pg-border);
|
||||
border-radius: 8px; overflow: hidden;
|
||||
}
|
||||
summary {
|
||||
padding: 0.85rem 1rem; font-weight: 600; font-size: 0.95rem;
|
||||
color: var(--pg-bright); cursor: pointer; list-style: none;
|
||||
display: flex; align-items: center; gap: 0.5rem;
|
||||
}
|
||||
summary::before {
|
||||
content: '▶'; font-size: 0.65rem; color: var(--pg-muted);
|
||||
transition: transform 0.15s;
|
||||
}
|
||||
details[open] summary::before { transform: rotate(90deg); }
|
||||
summary::-webkit-details-marker { display: none; }
|
||||
details > *:not(summary) { padding: 0 1rem 1rem; }
|
||||
|
||||
.help-body p { margin: 0.5rem 0; font-size: 0.9rem; color: var(--pg-bright); }
|
||||
.help-body ul { margin: 0.5rem 0 0.5rem 1.25rem; }
|
||||
.help-body li { font-size: 0.9rem; color: var(--pg-bright); margin-bottom: 0.25rem; }
|
||||
.help-body strong { color: var(--pg-text); }
|
||||
.help-body code {
|
||||
background: var(--pg-bg); border: 1px solid var(--pg-border);
|
||||
border-radius: 4px; padding: 0.1em 0.4em;
|
||||
font-size: 0.85em; color: var(--pg-accent);
|
||||
}
|
||||
.help-body a { color: var(--pg-accent); }
|
||||
.help-body h1 { font-size: 1.1rem; font-weight: 700; color: var(--pg-text); margin: 0.75rem 0 0.5rem; }
|
||||
.help-body h3 {
|
||||
font-size: 0.8rem; font-weight: 600; color: var(--pg-muted);
|
||||
text-transform: uppercase; letter-spacing: 0.05em; margin: 0.75rem 0 0.25rem;
|
||||
}
|
||||
.help-body table { width: 100%; border-collapse: collapse; font-size: 0.88rem; margin: 0.5rem 0 0.75rem; }
|
||||
.help-body th, .help-body td { padding: 0.45rem 0.7rem; text-align: left; border-bottom: 1px solid var(--pg-border); }
|
||||
.help-body th { color: var(--pg-muted); font-weight: 600; font-size: 0.8rem; text-transform: uppercase; letter-spacing: 0.04em; }
|
||||
.help-body td { color: var(--pg-bright); }
|
||||
.help-body pre { background: var(--pg-bg); border: 1px solid var(--pg-border); border-radius: 6px; padding: 0.75rem 1rem; overflow-x: auto; margin: 0.5rem 0; }
|
||||
.help-body pre code { background: none; border: none; padding: 0; font-size: 0.85em; color: var(--pg-muted); }
|
||||
.help-body hr { border: none; border-top: 1px solid var(--pg-border); margin: 0.5rem 0; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<nav class="page-nav" id="page-nav">
|
||||
<a id="nav-chat" href="/" class="nav-link">← Chat</a>
|
||||
<a href="/help" class="nav-link active">Help</a>
|
||||
<a href="/settings" class="nav-link" id="nav-settings">Settings</a>
|
||||
<a href="/settings/models" class="nav-link">Models</a>
|
||||
<a href="/settings/notifications" class="nav-link">Notifications</a>
|
||||
<a href="/settings/tools" class="nav-link">Tools</a>
|
||||
<a href="/settings/crons" class="nav-link">Schedules</a>
|
||||
{{ integrations_nav }}
|
||||
<span class="nav-spacer"></span>
|
||||
<a href="/logout" class="nav-link nav-logout">Sign out</a>
|
||||
</nav>
|
||||
|
||||
<div class="max-w-3xl mx-auto px-6 py-8 pb-16">
|
||||
<div class="mb-6 pb-4 border-b border-pg-border">
|
||||
<h1 class="text-xl font-bold text-pg-accent">Help & Reference</h1>
|
||||
<p id="persona-label" class="text-xs text-pg-muted mt-1"></p>
|
||||
</div>
|
||||
|
||||
<!-- Tab bar -->
|
||||
<div class="tab-bar">
|
||||
<button class="tab-btn active" data-tab="ui">UI Guide</button>
|
||||
<button class="tab-btn" data-tab="tools">Tools</button>
|
||||
<button class="tab-btn" data-tab="persona" id="tab-btn-persona">Persona</button>
|
||||
</div>
|
||||
|
||||
<div id="tab-ui" class="tab-panel active"><div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
|
||||
<div id="tab-tools" class="tab-panel"> <div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
|
||||
<div id="tab-persona" class="tab-panel"> <div class="help-body"><p class="text-pg-dimmer text-sm text-center py-8">Loading…</p></div></div>
|
||||
</div>
|
||||
|
||||
<style>
|
||||
.tab-bar {
|
||||
display: flex; gap: 0.25rem; margin-bottom: 1.5rem;
|
||||
border-bottom: 1px solid var(--pg-border);
|
||||
}
|
||||
.tab-btn {
|
||||
padding: 0.45rem 1rem; font-size: 0.85rem; font-weight: 500;
|
||||
color: var(--pg-dim); background: none; border: none;
|
||||
border-bottom: 2px solid transparent; margin-bottom: -1px;
|
||||
cursor: pointer; transition: color 0.15s, border-color 0.15s;
|
||||
font-family: inherit;
|
||||
}
|
||||
.tab-btn:hover { color: var(--pg-bright); }
|
||||
.tab-btn.active { color: var(--pg-accent); border-bottom-color: var(--pg-accent); }
|
||||
</style>
|
||||
|
||||
<script>
|
||||
const cfg = window.HELP_CONFIG || {};
|
||||
const user = cfg.user || 'scott';
|
||||
const persona = cfg.persona || 'inara';
|
||||
const params = `user=${encodeURIComponent(user)}&persona=${encodeURIComponent(persona)}`;
|
||||
|
||||
document.getElementById('nav-chat').href = cfg.backHref || '/';
|
||||
if (persona) {
|
||||
document.getElementById('persona-label').textContent =
|
||||
`${persona.charAt(0).toUpperCase() + persona.slice(1)} · ${user}`;
|
||||
}
|
||||
|
||||
// Rename Persona tab to the actual persona name
|
||||
const personaTabBtn = document.getElementById('tab-btn-persona');
|
||||
personaTabBtn.textContent = persona.charAt(0).toUpperCase() + persona.slice(1);
|
||||
|
||||
// ── Tab switching ────────────────────────────────────────────────
|
||||
const tabBtns = document.querySelectorAll('.tab-btn');
|
||||
const tabPanels = document.querySelectorAll('.tab-panel');
|
||||
const TAB_KEY = `cx_help_tab_${user}_${persona}`;
|
||||
|
||||
function activateTab(name) {
|
||||
tabBtns.forEach(b => b.classList.toggle('active', b.dataset.tab === name));
|
||||
tabPanels.forEach(p => p.classList.toggle('active', p.id === `tab-${name}`));
|
||||
try { localStorage.setItem(TAB_KEY, name); } catch (_) {}
|
||||
}
|
||||
|
||||
tabBtns.forEach(btn => btn.addEventListener('click', () => activateTab(btn.dataset.tab)));
|
||||
|
||||
// Restore last active tab
|
||||
try {
|
||||
const saved = localStorage.getItem(TAB_KEY);
|
||||
if (saved) activateTab(saved);
|
||||
} catch (_) {}
|
||||
|
||||
// ── Collapsible h2 sections ──────────────────────────────────────
|
||||
function makeCollapsible(container, openAll = false, openSet = null) {
|
||||
container.querySelectorAll('h2').forEach(h2 => {
|
||||
const title = h2.textContent.trim();
|
||||
const details = document.createElement('details');
|
||||
if (openAll || (openSet && openSet.has(title))) details.open = true;
|
||||
|
||||
const summary = document.createElement('summary');
|
||||
summary.textContent = title;
|
||||
details.appendChild(summary);
|
||||
|
||||
const siblings = [];
|
||||
let node = h2.nextSibling;
|
||||
while (node && node.nodeName !== 'H2') { siblings.push(node); node = node.nextSibling; }
|
||||
siblings.forEach(s => details.appendChild(s));
|
||||
h2.parentNode.replaceChild(details, h2);
|
||||
});
|
||||
}
|
||||
|
||||
// ── Render markdown into a panel ────────────────────────────────
|
||||
function render(panelId, markdown, openAll, openSet) {
|
||||
const panel = document.querySelector(`#${panelId} .help-body`);
|
||||
panel.innerHTML = marked.parse(markdown);
|
||||
panel.querySelectorAll('a').forEach(a => { a.target = '_blank'; a.rel = 'noopener noreferrer'; });
|
||||
makeCollapsible(panel, openAll, openSet);
|
||||
}
|
||||
|
||||
// ── Load all three tabs in parallel ─────────────────────────────
|
||||
const UI_OPEN = new Set(['Getting Started', 'Chat', 'Sessions', 'Model Registry']);
|
||||
|
||||
async function loadAll() {
|
||||
// UI Guide
|
||||
fetch('/static/HELP.md')
|
||||
.then(r => r.ok ? r.text() : Promise.reject(r.status))
|
||||
.then(md => render('tab-ui', md, false, UI_OPEN))
|
||||
.catch(e => { document.querySelector('#tab-ui .help-body').innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">Failed to load: ${e}</p>`; });
|
||||
|
||||
// Tools
|
||||
fetch('/static/TOOLS.md')
|
||||
.then(r => r.ok ? r.text() : Promise.reject(r.status))
|
||||
.then(md => render('tab-tools', md, true, null))
|
||||
.catch(e => { document.querySelector('#tab-tools .help-body').innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">Failed to load: ${e}</p>`; });
|
||||
|
||||
// Persona-specific HELP.md
|
||||
const personaPanel = document.querySelector('#tab-persona .help-body');
|
||||
try {
|
||||
const res = await fetch(`/files/HELP.md?${params}`);
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
const content = (data.content || '').trim();
|
||||
if (content) {
|
||||
render('tab-persona', content, true, null);
|
||||
} else {
|
||||
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet. Edit <code>HELP.md</code> in the Files panel to add them.</p>`;
|
||||
}
|
||||
} else {
|
||||
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet.</p>`;
|
||||
}
|
||||
} catch (_) {
|
||||
personaPanel.innerHTML = `<p class="text-pg-dimmer text-sm text-center py-8">No ${persona}-specific notes yet.</p>`;
|
||||
}
|
||||
}
|
||||
|
||||
loadAll();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
BIN
cortex/static/icon-192.png
Normal file
BIN
cortex/static/icon-192.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.2 KiB |
BIN
cortex/static/icon-512.png
Normal file
BIN
cortex/static/icon-512.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 6.7 KiB |
4
cortex/static/icon.svg
Normal file
4
cortex/static/icon.svg
Normal file
@@ -0,0 +1,4 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
|
||||
<rect width="512" height="512" rx="96" fill="#1a1228"/>
|
||||
<text x="256" y="390" font-size="340" text-anchor="middle" font-family="system-ui, -apple-system, sans-serif">✨</text>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 251 B |
@@ -5,19 +5,35 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Inara</title>
|
||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>✨</text></svg>">
|
||||
<link rel="manifest" href="/manifest.json">
|
||||
<meta name="theme-color" content="#1a1228" id="meta-theme-color">
|
||||
<meta name="mobile-web-app-capable" content="yes">
|
||||
<meta name="apple-mobile-web-app-capable" content="yes">
|
||||
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
|
||||
<meta name="apple-mobile-web-app-title" content="Cortex">
|
||||
<link rel="apple-touch-icon" href="/static/icon-192.png">
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<!-- Apply saved theme + font size before first paint to avoid flash -->
|
||||
<script>
|
||||
(function(){
|
||||
var t = localStorage.getItem('theme');
|
||||
if (!t) t = window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
|
||||
document.documentElement.setAttribute('data-theme', t);
|
||||
var sizes = { normal: '16px', large: '18px', small: '14px' };
|
||||
var sizes = { normal: '21px', large: '25px', small: '17px' };
|
||||
var fs = localStorage.getItem('font-size') || 'normal';
|
||||
document.documentElement.style.fontSize = sizes[fs] || '16px';
|
||||
document.documentElement.style.fontSize = sizes[fs] || '21px';
|
||||
})();
|
||||
</script>
|
||||
<link rel="stylesheet" href="/static/style.css">
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/codemirror@5.65.17/lib/codemirror.min.css">
|
||||
<script src="https://cdn.jsdelivr.net/npm/codemirror@5.65.17/lib/codemirror.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/codemirror@5.65.17/mode/markdown/markdown.min.js"></script>
|
||||
<script src="/static/marked.min.js"></script>
|
||||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/styles/atom-one-dark.min.css">
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.11.1/highlight.min.js"></script>
|
||||
<script src="https://unpkg.com/lucide@latest/dist/umd/lucide.min.js"></script>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
@@ -25,15 +41,45 @@
|
||||
<div class="persona-switcher" id="persona-switcher">
|
||||
<div class="name" id="persona-name">Inara</div>
|
||||
<div class="subtitle">Cortex · Local</div>
|
||||
<div id="session-id"></div>
|
||||
<div class="persona-dropdown" id="persona-dropdown"></div>
|
||||
</div>
|
||||
<button id="sessions-btn" class="hdr-btn">Sessions</button>
|
||||
<button id="files-btn" class="hdr-btn">Files</button>
|
||||
<button id="ctx-open-btn" class="hdr-btn" title="Settings">⚙<span class="tier-badge">2</span></button>
|
||||
<button id="help-btn" class="hdr-btn" title="Help & reference">?</button>
|
||||
|
||||
<nav id="hdr-nav">
|
||||
<button id="sessions-btn" class="hdr-btn" title="Sessions">
|
||||
<svg data-lucide="history" class="btn-icon"></svg>
|
||||
<span class="btn-label">Sessions</span>
|
||||
</button>
|
||||
<button id="ctx-open-btn" class="hdr-btn" title="Context & memory">
|
||||
<svg data-lucide="sliders-horizontal" class="btn-icon"></svg><span class="tier-badge">2</span>
|
||||
</button>
|
||||
<div class="hdr-dropdown-wrap" id="settings-wrap">
|
||||
<button class="hdr-btn" id="settings-btn" title="Settings">
|
||||
<svg data-lucide="menu" class="btn-icon"></svg>
|
||||
</button>
|
||||
<div class="hdr-dropdown" id="settings-dropdown">
|
||||
<button id="files-btn" class="hdr-dd-item">
|
||||
<svg data-lucide="folder-open" class="btn-icon"></svg> Files
|
||||
</button>
|
||||
<a href="/settings" class="hdr-dd-item">
|
||||
<svg data-lucide="user" class="btn-icon"></svg> Account
|
||||
</a>
|
||||
<button id="push-btn" class="hdr-dd-item" style="display:none">
|
||||
<svg data-lucide="bell" class="btn-icon"></svg>
|
||||
<span id="push-btn-label">Enable notifications</span>
|
||||
</button>
|
||||
<div class="hdr-dd-divider"></div>
|
||||
<form method="POST" action="/logout" style="margin:0">
|
||||
<button type="submit" class="hdr-btn" title="Sign out" id="logout-btn">⏏</button>
|
||||
<button type="submit" class="hdr-dd-item">
|
||||
<svg data-lucide="log-out" class="btn-icon"></svg> Sign Out
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
<a id="help-link" href="/help" class="hdr-btn" title="Help & reference" style="text-decoration:none">
|
||||
<svg data-lucide="circle-help" class="btn-icon"></svg>
|
||||
</a>
|
||||
</nav>
|
||||
|
||||
<div id="sessions-panel"></div>
|
||||
|
||||
@@ -42,10 +88,10 @@
|
||||
<div class="ctx-section">
|
||||
<div class="ctx-section-title">Context Tier</div>
|
||||
<div class="ctx-row">
|
||||
<button class="ctx-btn" data-tier="1" id="tier-1" title="Minimal (~1.5k tokens)">T1</button>
|
||||
<button class="ctx-btn active" data-tier="2" id="tier-2" title="Standard (~5k tokens)">T2</button>
|
||||
<button class="ctx-btn" data-tier="3" id="tier-3" title="Extended (~15k tokens)">T3</button>
|
||||
<button class="ctx-btn" data-tier="4" id="tier-4" title="Full (~50k tokens)">T4</button>
|
||||
<button class="ctx-btn" data-tier="1" id="tier-1" title="Minimal — identity only (~1.5k tokens)">Min</button>
|
||||
<button class="ctx-btn active" data-tier="2" id="tier-2" title="Standard — memory + user profile (~5k tokens)">Std</button>
|
||||
<button class="ctx-btn" data-tier="3" id="tier-3" title="Extended — + last 2 sessions (~15k tokens)">Ext</button>
|
||||
<button class="ctx-btn" data-tier="4" id="tier-4" title="Full — + last 7 sessions (~50k tokens)">Full</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="ctx-section">
|
||||
@@ -59,25 +105,29 @@
|
||||
<div class="ctx-section">
|
||||
<div class="ctx-section-title">Distill Memory</div>
|
||||
<div class="ctx-row">
|
||||
<button class="ctx-btn" id="distill-short-btn" title="Roll session logs → MEMORY_SHORT (no LLM)">short</button>
|
||||
<button class="ctx-btn" id="distill-mid-btn" title="Summarize short → MEMORY_MID (LLM)">mid</button>
|
||||
<button class="ctx-btn" id="distill-long-btn" title="Integrate mid → MEMORY_LONG (LLM)">long</button>
|
||||
<button class="ctx-btn" id="distill-all-btn" title="Run all three steps in sequence">all</button>
|
||||
<button class="ctx-btn" id="distill-short-btn" title="Roll today's sessions → MEMORY_SHORT.md (fast, no LLM)">Short</button>
|
||||
<button class="ctx-btn" id="distill-mid-btn" title="Summarize SHORT → MID memory (uses LLM)">Mid</button>
|
||||
<button class="ctx-btn" id="distill-long-btn" title="Integrate MID → LONG memory (uses LLM)">Long</button>
|
||||
<button class="ctx-btn" id="distill-all-btn" title="Run Short → Mid → Long in sequence">All</button>
|
||||
<button class="ctx-btn ctx-btn-danger" id="distill-rebuild-btn" title="⚠ Wipe Mid + Long memories and rebuild from session logs. Hand-edited content will be replaced.">Rebuild</button>
|
||||
</div>
|
||||
<div id="ctx-distill-status"></div>
|
||||
<div id="ctx-schedule"></div>
|
||||
</div>
|
||||
<div class="ctx-section">
|
||||
<div class="ctx-section-title">Backend</div>
|
||||
<div class="ctx-section-title">Role</div>
|
||||
<div class="ctx-row">
|
||||
<button id="backend-toggle" class="ctx-btn" title="Click to switch primary backend">claude</button>
|
||||
<button id="backend-toggle" class="ctx-btn" title="Active role — click to cycle">chat</button>
|
||||
</div>
|
||||
<div id="backend-model-hint"></div>
|
||||
</div>
|
||||
<div class="ctx-section">
|
||||
<div class="ctx-section-title">Display</div>
|
||||
<div class="ctx-row">
|
||||
<button id="font-size-btn" class="ctx-btn" title="Cycle font size: normal → large → small">Aa</button>
|
||||
<button id="theme-btn" class="ctx-btn" title="Toggle light/dark mode">☾</button>
|
||||
<button id="font-size-btn" class="ctx-btn" title="Cycle font size: Normal → Large → Small">Aa</button>
|
||||
<button id="theme-btn" class="ctx-btn" title="Toggle light / dark theme">☾</button>
|
||||
<button id="height-cycle-btn" class="ctx-btn" title="Input size: Compact — click to cycle">S</button>
|
||||
<button id="enter-toggle" class="ctx-btn" title="Toggle send shortcut: Ctrl+Enter ↔ Enter">⌃↵</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -88,65 +138,71 @@
|
||||
<div id="file-modal-inner">
|
||||
<div id="file-modal-header">
|
||||
<span id="file-modal-title">Context Files</span>
|
||||
<select id="file-select"></select>
|
||||
<span class="fm-spacer"></span>
|
||||
<button class="fm-btn" id="file-raw-btn">edit</button>
|
||||
<button class="fm-btn active" id="file-preview-btn">preview</button>
|
||||
<button class="fm-btn save" id="file-save-btn">Save</button>
|
||||
<span id="file-saved-msg">saved ✓</span>
|
||||
<button class="fm-btn" id="file-close-btn">✕</button>
|
||||
</div>
|
||||
<div id="file-modal-content">
|
||||
<div id="file-sidebar-wrap">
|
||||
<div id="file-sidebar"></div>
|
||||
<div id="session-search-wrap">
|
||||
<div id="session-search-label">Session Search</div>
|
||||
<div id="session-search-row">
|
||||
<input id="session-search-input" type="search" placeholder="Search sessions…" autocomplete="off">
|
||||
<button id="session-search-btn">Go</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="file-modal-body">
|
||||
<textarea id="file-editor" spellcheck="false"></textarea>
|
||||
<div id="file-editor-wrap"></div>
|
||||
<div id="file-preview"></div>
|
||||
<div id="session-search-results" style="display:none"></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Help modal -->
|
||||
<div id="help-modal">
|
||||
<div id="help-modal-inner">
|
||||
<div id="help-modal-header">
|
||||
<h2>Cortex — Help & Reference</h2>
|
||||
<button class="fm-btn" id="help-close-btn">✕</button>
|
||||
</div>
|
||||
<div id="help-modal-body"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Auth warning banner — shown when Claude CLI token is near expiry -->
|
||||
<div id="auth-banner">
|
||||
<div id="auth-banner-text">
|
||||
<span id="auth-banner-msg"></span>
|
||||
<span id="auth-banner-hint"></span>
|
||||
</div>
|
||||
<button id="auth-banner-close" title="Dismiss">✕</button>
|
||||
</div>
|
||||
|
||||
<div id="messages"></div>
|
||||
<div id="session-id"></div>
|
||||
|
||||
<div id="input-area">
|
||||
<textarea id="input" rows="1" placeholder="Message Inara… (Ctrl+Enter to send)" autofocus></textarea>
|
||||
<div id="right-col">
|
||||
<!-- Semi-hidden: appear when content > ~3 lines -->
|
||||
<div id="height-row">
|
||||
<span>↕</span>
|
||||
<select id="height-sel">
|
||||
<option value="120">5 lines</option>
|
||||
<option value="240">10 lines</option>
|
||||
<option value="480">20 lines</option>
|
||||
</select>
|
||||
<!-- Mode select — compact dropdown, opens upward, MRU sorted -->
|
||||
<div id="mode-select">
|
||||
<button id="mode-select-btn" title="Input mode">
|
||||
<span id="mode-icon">💬</span>
|
||||
<span id="mode-label">Chat</span>
|
||||
<span class="mode-arrow">▲</span>
|
||||
</button>
|
||||
<!-- Populated dynamically in MRU order -->
|
||||
<div id="mode-dropdown"></div>
|
||||
<!-- Note visibility sub-toggle — only shown when note mode is active -->
|
||||
<button id="note-vis-btn" title="Toggle note visibility (private / public)">prv</button>
|
||||
<!-- Tools toggle — routes through the orchestrator tool loop when active -->
|
||||
<button id="tools-toggle" title="Tools disabled — click to enable">⚡</button>
|
||||
<!-- Attach file — images (vision) or text/code files -->
|
||||
<button id="attach-btn" title="Attach image or text file">📎</button>
|
||||
<input type="file" id="file-input" style="display:none"
|
||||
accept="image/png,image/jpeg,image/webp,image/gif,text/plain,text/markdown,.md,.txt,.py,.js,.ts,.jsx,.tsx,.json,.yaml,.yml,.toml,.html,.css,.sh,.csv,.xml,.rs,.go,.java,.c,.cpp,.h,.rb,.php,.swift,.kt,.sql">
|
||||
</div>
|
||||
<button id="enter-toggle" title="Toggle send shortcut">⌃↵</button>
|
||||
<!-- Note mode controls -->
|
||||
<button id="note-type-btn">private</button>
|
||||
<button id="note-btn">Note</button>
|
||||
<button id="agent-mode-btn" title="Agent mode — Gemini tool loop + Claude response">Agent</button>
|
||||
<!-- Attachment preview — shown when a file is pending -->
|
||||
<div id="attachment-row" style="display:none">
|
||||
<div id="attachment-preview">
|
||||
<img id="attachment-thumb" alt="" style="display:none">
|
||||
<span id="attachment-icon">📎</span>
|
||||
<span id="attachment-name"></span>
|
||||
<button id="attachment-clear" title="Remove attachment">✕</button>
|
||||
</div>
|
||||
</div>
|
||||
<textarea id="input" rows="1" placeholder="Message…" autofocus></textarea>
|
||||
<div id="send-col">
|
||||
<button id="send">Send</button>
|
||||
<button id="stop">Stop</button>
|
||||
<button id="stop"><svg data-lucide="square" width="14" height="14" class="btn-icon"></svg> Stop</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="sessions-backdrop"></div>
|
||||
<div id="toast-container"></div>
|
||||
<script src="/static/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
134
cortex/static/integrations.html
Normal file
134
cortex/static/integrations.html
Normal file
@@ -0,0 +1,134 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Integrations</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
corePlugins: { preflight: false },
|
||||
darkMode: ['selector', '[data-theme="dark"]'],
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
pg: {
|
||||
bg: 'var(--pg-bg)',
|
||||
surface: 'var(--pg-surface)',
|
||||
border: 'var(--pg-border)',
|
||||
text: 'var(--pg-text)',
|
||||
muted: 'var(--pg-muted)',
|
||||
dim: 'var(--pg-dim)',
|
||||
dimmer: 'var(--pg-dimmer)',
|
||||
bright: 'var(--pg-bright)',
|
||||
accent: 'var(--pg-accent)',
|
||||
action: 'var(--pg-action)',
|
||||
}
|
||||
},
|
||||
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<link rel="stylesheet" href="/static/pg.css">
|
||||
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
|
||||
<style>
|
||||
details.channel-block summary::-webkit-details-marker { display: none; }
|
||||
details.channel-block summary::before {
|
||||
content: '▶'; font-size: 0.65rem; color: var(--pg-dimmer);
|
||||
transition: transform 0.15s; flex-shrink: 0;
|
||||
}
|
||||
details.channel-block[open] summary::before { transform: rotate(90deg); }
|
||||
details.channel-block[open] summary { border-bottom: 1px solid var(--pg-border); }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<nav class="page-nav">
|
||||
<a href="{{ back_href }}" class="nav-link">← Chat</a>
|
||||
<a href="{{ help_href }}" class="nav-link">Help</a>
|
||||
<a href="/settings" class="nav-link">Settings</a>
|
||||
<a href="/settings/models" class="nav-link">Models</a>
|
||||
<a href="/settings/notifications" class="nav-link">Notifications</a>
|
||||
<a href="/settings/tools" class="nav-link">Tools</a>
|
||||
<a href="/settings/crons" class="nav-link">Schedules</a>
|
||||
<a href="/settings/integrations" class="nav-link active">Integrations</a>
|
||||
<span class="nav-spacer"></span>
|
||||
<a href="/logout" class="nav-link nav-logout">Sign out</a>
|
||||
</nav>
|
||||
<div class="page-wrap">
|
||||
<h1 class="page-title">Integrations</h1>
|
||||
<p class="page-subtitle">External service connections — admin only.</p>
|
||||
|
||||
<!-- SUCCESS -->
|
||||
<!-- ERROR -->
|
||||
|
||||
<form method="POST" action="/settings/integrations">
|
||||
|
||||
<div class="section">
|
||||
<h2>Aether Platform Database</h2>
|
||||
<p class="section-note">
|
||||
Gives the orchestrator direct read-only access to the Aether MariaDB via the
|
||||
<code>ae_db_query</code>, <code>ae_db_describe</code>, and <code>ae_db_show_view</code> tools.
|
||||
Only SELECT, SHOW, DESCRIBE, and EXPLAIN are permitted — no writes possible.
|
||||
</p>
|
||||
|
||||
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
|
||||
{{ ae_db_host and 'open' or '' }}>
|
||||
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
|
||||
Connection
|
||||
</summary>
|
||||
<div class="px-4 pt-4 pb-2">
|
||||
<p class="text-xs text-pg-dimmer mb-4 -mt-1 leading-relaxed">
|
||||
Use the same credentials as
|
||||
<code class="font-mono text-pg-accent bg-pg-bg border border-pg-border rounded px-1 text-xs">agents_sync/mcp/scripts/sql_inspector.py</code>.
|
||||
Leave the password blank to keep the stored value.
|
||||
</p>
|
||||
<div class="grid grid-cols-[1fr_7rem] gap-3 items-start">
|
||||
<div class="field">
|
||||
<label for="ae_db_host">Host</label>
|
||||
<input type="text" id="ae_db_host" name="ae_db_host"
|
||||
value="{{ ae_db_host }}"
|
||||
placeholder="192.168.64.5"
|
||||
autocomplete="off" spellcheck="false">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="ae_db_port">Port</label>
|
||||
<input type="number" id="ae_db_port" name="ae_db_port"
|
||||
value="{{ ae_db_port }}"
|
||||
placeholder="3306" min="1" max="65535"
|
||||
autocomplete="off">
|
||||
</div>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="ae_db_name">Database name</label>
|
||||
<input type="text" id="ae_db_name" name="ae_db_name"
|
||||
value="{{ ae_db_name }}"
|
||||
placeholder="aether_dev"
|
||||
autocomplete="off" spellcheck="false">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="ae_db_user">Username</label>
|
||||
<input type="text" id="ae_db_user" name="ae_db_user"
|
||||
value="{{ ae_db_user }}"
|
||||
placeholder="aether_dev"
|
||||
autocomplete="off" spellcheck="false">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="ae_db_password">Password</label>
|
||||
<input type="password" id="ae_db_password" name="ae_db_password"
|
||||
value=""
|
||||
placeholder="Leave blank to keep existing value"
|
||||
autocomplete="new-password" spellcheck="false">
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
|
||||
<button type="submit" class="btn-submit w-full md:w-96">Save integrations</button>
|
||||
</form>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
1055
cortex/static/local_llm.html
Normal file
1055
cortex/static/local_llm.html
Normal file
File diff suppressed because it is too large
Load Diff
@@ -4,6 +4,9 @@
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Sign In</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
|
||||
|
||||
@@ -13,7 +16,10 @@
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: #0f1117;
|
||||
font-family: system-ui, -apple-system, sans-serif;
|
||||
font-family: 'Inter', system-ui, -apple-system, sans-serif;
|
||||
font-weight: 450;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
color: #e2e8f0;
|
||||
}
|
||||
|
||||
@@ -40,7 +46,7 @@
|
||||
|
||||
.logo p {
|
||||
font-size: 0.8rem;
|
||||
color: #64748b;
|
||||
color: #94a3b8;
|
||||
margin-top: 0.25rem;
|
||||
}
|
||||
|
||||
@@ -84,6 +90,40 @@
|
||||
|
||||
button[type="submit"]:hover { background: #6d28d9; }
|
||||
|
||||
.divider {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.75rem;
|
||||
margin: 1.25rem 0;
|
||||
color: #475569;
|
||||
font-size: 0.78rem;
|
||||
}
|
||||
.divider::before, .divider::after {
|
||||
content: '';
|
||||
flex: 1;
|
||||
border-top: 1px solid #2d3148;
|
||||
}
|
||||
|
||||
.google-btn {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 0.6rem;
|
||||
width: 100%;
|
||||
padding: 0.65rem;
|
||||
background: #fff;
|
||||
border: 1px solid #dadce0;
|
||||
border-radius: 6px;
|
||||
color: #3c4043;
|
||||
font-size: 0.95rem;
|
||||
font-weight: 500;
|
||||
font-family: inherit;
|
||||
cursor: pointer;
|
||||
text-decoration: none;
|
||||
transition: background 0.15s, box-shadow 0.15s;
|
||||
}
|
||||
.google-btn:hover { background: #f8f9fa; box-shadow: 0 1px 4px rgba(0,0,0,0.2); }
|
||||
|
||||
.error {
|
||||
color: #f87171;
|
||||
font-size: 0.85rem;
|
||||
@@ -101,6 +141,18 @@
|
||||
|
||||
<!-- ERROR -->
|
||||
|
||||
<a href="/auth/google" class="google-btn">
|
||||
<svg width="18" height="18" viewBox="0 0 18 18" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M17.64 9.2c0-.637-.057-1.251-.164-1.84H9v3.481h4.844c-.209 1.125-.843 2.078-1.796 2.717v2.258h2.908c1.702-1.567 2.684-3.875 2.684-6.615z" fill="#4285F4"/>
|
||||
<path d="M9 18c2.43 0 4.467-.806 5.956-2.18l-2.908-2.259c-.806.54-1.837.86-3.048.86-2.344 0-4.328-1.584-5.036-3.711H.957v2.332A8.997 8.997 0 0 0 9 18z" fill="#34A853"/>
|
||||
<path d="M3.964 10.71A5.41 5.41 0 0 1 3.682 9c0-.593.102-1.17.282-1.71V4.958H.957A8.996 8.996 0 0 0 0 9c0 1.452.348 2.827.957 4.042l3.007-2.332z" fill="#FBBC05"/>
|
||||
<path d="M9 3.58c1.321 0 2.508.454 3.44 1.345l2.582-2.58C13.463.891 11.426 0 9 0A8.997 8.997 0 0 0 .957 4.958L3.964 7.29C4.672 5.163 6.656 3.58 9 3.58z" fill="#EA4335"/>
|
||||
</svg>
|
||||
Sign in with Google
|
||||
</a>
|
||||
|
||||
<div class="divider">or</div>
|
||||
|
||||
<form method="POST" action="/login">
|
||||
<div class="field">
|
||||
<label for="username">Username</label>
|
||||
|
||||
30
cortex/static/manifest.json
Normal file
30
cortex/static/manifest.json
Normal file
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"name": "Cortex · Inara",
|
||||
"short_name": "Cortex",
|
||||
"description": "Personal AI assistant",
|
||||
"start_url": "/",
|
||||
"scope": "/",
|
||||
"display": "standalone",
|
||||
"background_color": "#1a1228",
|
||||
"theme_color": "#1a1228",
|
||||
"icons": [
|
||||
{
|
||||
"src": "/static/icon-192.png",
|
||||
"sizes": "192x192",
|
||||
"type": "image/png",
|
||||
"purpose": "any"
|
||||
},
|
||||
{
|
||||
"src": "/static/icon-512.png",
|
||||
"sizes": "512x512",
|
||||
"type": "image/png",
|
||||
"purpose": "any maskable"
|
||||
},
|
||||
{
|
||||
"src": "/static/icon.svg",
|
||||
"sizes": "any",
|
||||
"type": "image/svg+xml",
|
||||
"purpose": "any"
|
||||
}
|
||||
]
|
||||
}
|
||||
348
cortex/static/notifications.html
Normal file
348
cortex/static/notifications.html
Normal file
@@ -0,0 +1,348 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Notifications</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
corePlugins: { preflight: false },
|
||||
darkMode: ['selector', '[data-theme="dark"]'],
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
pg: {
|
||||
bg: 'var(--pg-bg)',
|
||||
surface: 'var(--pg-surface)',
|
||||
border: 'var(--pg-border)',
|
||||
text: 'var(--pg-text)',
|
||||
muted: 'var(--pg-muted)',
|
||||
dim: 'var(--pg-dim)',
|
||||
dimmer: 'var(--pg-dimmer)',
|
||||
bright: 'var(--pg-bright)',
|
||||
accent: 'var(--pg-accent)',
|
||||
action: 'var(--pg-action)',
|
||||
}
|
||||
},
|
||||
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<link rel="stylesheet" href="/static/pg.css">
|
||||
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
|
||||
<style>
|
||||
/* ── Channel collapsible arrow ── */
|
||||
details.channel-block summary::-webkit-details-marker { display: none; }
|
||||
details.channel-block summary::before {
|
||||
content: '▶'; font-size: 0.65rem; color: var(--pg-dimmer);
|
||||
transition: transform 0.15s; flex-shrink: 0;
|
||||
}
|
||||
details.channel-block[open] summary::before { transform: rotate(90deg); }
|
||||
details.channel-block[open] summary { border-bottom: 1px solid var(--pg-border); }
|
||||
|
||||
/* ── Test result feedback (JS-toggled display) ── */
|
||||
#test-result { display: none; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<nav class="page-nav">
|
||||
<a href="{{ back_href }}" class="nav-link">← Chat</a>
|
||||
<a href="{{ help_href }}" class="nav-link">Help</a>
|
||||
<a href="/settings" class="nav-link">Settings</a>
|
||||
<a href="/settings/models" class="nav-link">Models</a>
|
||||
<a href="/settings/notifications" class="nav-link active">Notifications</a>
|
||||
<a href="/settings/tools" class="nav-link">Tools</a>
|
||||
<a href="/settings/crons" class="nav-link">Schedules</a>
|
||||
{{ integrations_nav }}
|
||||
<span class="nav-spacer"></span>
|
||||
<a href="/logout" class="nav-link nav-logout">Sign out</a>
|
||||
</nav>
|
||||
<div class="page-wrap">
|
||||
<h1 class="page-title">Notifications</h1>
|
||||
<p class="page-subtitle">How your persona reaches out proactively — reminders, cron jobs, and memory digests.</p>
|
||||
|
||||
<!-- SUCCESS -->
|
||||
<!-- ERROR -->
|
||||
|
||||
<form method="POST" action="/settings/notifications">
|
||||
|
||||
<!-- Channel selector -->
|
||||
<div class="section">
|
||||
<h2>Channel</h2>
|
||||
<div class="field">
|
||||
<label for="notification_channel">Default outbound channel</label>
|
||||
<select id="notification_channel" name="notification_channel"
|
||||
data-value="{{ notify_channel }}">
|
||||
<option value="">None (disabled)</option>
|
||||
<option value="web_push">Browser Push Notification</option>
|
||||
<option value="email">Email</option>
|
||||
<option value="nextcloud">Nextcloud Talk</option>
|
||||
<option value="google_chat">Google Chat</option>
|
||||
</select>
|
||||
<p class="hint">Used for reminder alerts, distillation summaries, and cron job notifications.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="notification_email">
|
||||
Email address override
|
||||
<span class="font-normal text-pg-dim">(optional)</span>
|
||||
</label>
|
||||
<input type="email" id="notification_email" name="notification_email"
|
||||
value="{{ notify_email_override }}"
|
||||
placeholder="Leave blank to use your login email"
|
||||
autocomplete="off">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Nextcloud Talk -->
|
||||
<div class="section">
|
||||
<h2>Nextcloud Talk</h2>
|
||||
<p class="section-note">
|
||||
Configure to send and receive messages via your Nextcloud Talk bot.
|
||||
<strong>Sending</strong> requires the bot URL, secret, and notification room.
|
||||
<strong>Reading history</strong> (<code>nc_talk_history</code> tool) additionally
|
||||
requires a Nextcloud username and app password.
|
||||
</p>
|
||||
|
||||
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
|
||||
{{ nc_url and 'open' or '' }}>
|
||||
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
|
||||
Bot credentials (sending)
|
||||
</summary>
|
||||
<div class="px-4 pt-4 pb-2">
|
||||
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
|
||||
Set these up in your Nextcloud Talk room → Bot settings.
|
||||
See the <a href="/help" class="text-pg-accent">setup guide</a> for step-by-step instructions.
|
||||
</p>
|
||||
<div class="field">
|
||||
<label for="nc_url">Nextcloud URL</label>
|
||||
<input type="url" id="nc_url" name="nc_url"
|
||||
value="{{ nc_url }}"
|
||||
placeholder="https://cloud.example.com"
|
||||
autocomplete="off" spellcheck="false">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="nc_bot_secret">Bot secret</label>
|
||||
<input type="password" id="nc_bot_secret" name="nc_bot_secret"
|
||||
value="{{ nc_bot_secret }}"
|
||||
placeholder="Leave blank to keep existing value"
|
||||
autocomplete="new-password" spellcheck="false">
|
||||
<p class="hint">Generated when you registered the bot in Nextcloud Talk.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="nc_notification_room">Notification room token</label>
|
||||
<input type="text" id="nc_notification_room" name="nc_notification_room"
|
||||
value="{{ nc_notify_room }}"
|
||||
placeholder="Token from the Talk room URL"
|
||||
autocomplete="off" spellcheck="false">
|
||||
<p class="hint">The token at the end of the Talk room URL — e.g. <code>abc123def</code>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
|
||||
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
|
||||
{{ nc_username and 'open' or '' }}>
|
||||
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
|
||||
API credentials (reading history)
|
||||
</summary>
|
||||
<div class="px-4 pt-4 pb-2">
|
||||
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
|
||||
Required for the <code>nc_talk_history</code> orchestrator tool.
|
||||
Generate an app password in Nextcloud → Settings → Security → App passwords.
|
||||
</p>
|
||||
<div class="field">
|
||||
<label for="nc_username">Nextcloud username</label>
|
||||
<input type="text" id="nc_username" name="nc_username"
|
||||
value="{{ nc_username }}"
|
||||
placeholder="Your Nextcloud login username"
|
||||
autocomplete="off" spellcheck="false">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="nc_app_password">App password</label>
|
||||
<input type="password" id="nc_app_password" name="nc_app_password"
|
||||
value="{{ nc_app_password }}"
|
||||
placeholder="Leave blank to keep existing value"
|
||||
autocomplete="new-password" spellcheck="false">
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
|
||||
<!-- Home Assistant -->
|
||||
<div class="section">
|
||||
<h2>Home Assistant</h2>
|
||||
<p class="section-note">
|
||||
Receive events from HA automations and let your persona call the HA REST API
|
||||
(read states, control devices). Webhook ID is the shared secret used in your
|
||||
HA <code>rest_command</code> URL.
|
||||
</p>
|
||||
|
||||
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
|
||||
{{ ha_url and 'open' or '' }}>
|
||||
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
|
||||
Connection
|
||||
</summary>
|
||||
<div class="px-4 pt-4 pb-2">
|
||||
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
|
||||
HA URL and a Long-Lived Access Token (Profile → scroll to bottom →
|
||||
Long-Lived Access Tokens → Create Token).
|
||||
</p>
|
||||
<div class="field">
|
||||
<label for="ha_url">Home Assistant URL</label>
|
||||
<input type="url" id="ha_url" name="ha_url"
|
||||
value="{{ ha_url }}"
|
||||
placeholder="https://ha.yourdomain.com"
|
||||
autocomplete="off" spellcheck="false">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="ha_token">Long-Lived Access Token</label>
|
||||
<input type="password" id="ha_token" name="ha_token"
|
||||
value=""
|
||||
placeholder="Leave blank to keep existing token"
|
||||
autocomplete="new-password" spellcheck="false">
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
|
||||
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
|
||||
{{ ha_webhook_id and 'open' or '' }}>
|
||||
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
|
||||
Inbound webhook (HA → Cortex)
|
||||
</summary>
|
||||
<div class="px-4 pt-4 pb-2">
|
||||
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
|
||||
The webhook ID is the shared secret in your HA <code>rest_command</code> URL.
|
||||
Your endpoint: <code>https://cortex.dgrzone.com/webhook/ha/{{ ha_username }}/<webhook_id></code>
|
||||
</p>
|
||||
<div class="field">
|
||||
<label for="ha_webhook_id">Webhook ID</label>
|
||||
<input type="text" id="ha_webhook_id" name="ha_webhook_id"
|
||||
value="{{ ha_webhook_id }}"
|
||||
placeholder="Paste or generate a random secret"
|
||||
autocomplete="off" spellcheck="false">
|
||||
<p class="hint">Treat this like a password — use a long, random string.</p>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label class="checkbox-label">
|
||||
<input type="checkbox" name="ha_tools" value="1" {{ ha_tools_checked }}>
|
||||
Enable orchestrator tools
|
||||
</label>
|
||||
<p class="hint">When checked, HA events trigger the full tool loop (research, home control, tasks). When unchecked, events get a direct LLM response — faster but no tools.</p>
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
|
||||
<!-- Google Chat -->
|
||||
<div class="section">
|
||||
<h2>Google Chat</h2>
|
||||
<p class="section-note">
|
||||
Outbound webhook for proactive messages to a Google Chat space.
|
||||
Incoming messages are handled separately via the Google Chat Add-on.
|
||||
</p>
|
||||
|
||||
<details class="channel-block border border-pg-border rounded-lg overflow-hidden mb-3"
|
||||
{{ gc_webhook and 'open' or '' }}>
|
||||
<summary class="flex items-center gap-2 px-4 py-3 text-sm font-semibold text-pg-muted cursor-pointer select-none bg-pg-bg">
|
||||
Outbound webhook
|
||||
</summary>
|
||||
<div class="px-4 pt-4 pb-2">
|
||||
<p class="text-xs text-pg-dimmer -mt-1 mb-4 leading-relaxed">
|
||||
Create a webhook in your Google Chat space → Manage webhooks. Paste the full URL here.
|
||||
</p>
|
||||
<div class="field">
|
||||
<label for="gc_outbound_webhook">Webhook URL</label>
|
||||
<input type="url" id="gc_outbound_webhook" name="gc_outbound_webhook"
|
||||
value="{{ gc_webhook }}"
|
||||
placeholder="https://chat.googleapis.com/v1/spaces/…"
|
||||
autocomplete="off" spellcheck="false">
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
|
||||
<button type="submit" class="btn-submit w-full md:w-96">Save notification settings</button>
|
||||
</form>
|
||||
|
||||
<!-- Test -->
|
||||
<div class="section" style="margin-top:2rem;">
|
||||
<h2>Test</h2>
|
||||
<p class="section-note">
|
||||
Fire a notification via your configured channel or run the reminder check
|
||||
immediately — no need to wait for the daily 09:00 scheduler job.
|
||||
</p>
|
||||
<div class="flex gap-3 mt-2">
|
||||
<button class="flex-1 px-3 py-2.5 text-sm font-medium border border-pg-border rounded-md bg-pg-bg text-pg-text hover:border-pg-action hover:text-pg-accent transition-colors cursor-pointer disabled:opacity-50"
|
||||
id="btn-test-notify">Send Test Notification</button>
|
||||
<button class="flex-1 px-3 py-2.5 text-sm font-medium border border-pg-border rounded-md bg-pg-bg text-pg-text hover:border-pg-action hover:text-pg-accent transition-colors cursor-pointer disabled:opacity-50"
|
||||
id="btn-check-reminders">Check Reminders Now</button>
|
||||
</div>
|
||||
<div id="test-result"
|
||||
class="mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Set channel select to saved value
|
||||
const sel = document.getElementById('notification_channel');
|
||||
if (sel) {
|
||||
const saved = sel.dataset.value;
|
||||
if (saved) {
|
||||
for (const opt of sel.options) {
|
||||
if (opt.value === saved) { opt.selected = true; break; }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Test buttons
|
||||
const resultEl = document.getElementById('test-result');
|
||||
|
||||
function showResult(ok, msg) {
|
||||
resultEl.textContent = msg;
|
||||
resultEl.className = ok
|
||||
? 'mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed bg-green-950 text-green-400 border border-green-800'
|
||||
: 'mt-3 px-3 py-2.5 rounded-md text-sm leading-relaxed bg-red-950 text-red-400 border border-red-800';
|
||||
resultEl.style.display = 'block';
|
||||
}
|
||||
|
||||
async function apiPost(url, btnEl, label) {
|
||||
btnEl.disabled = true;
|
||||
btnEl.textContent = label + '…';
|
||||
resultEl.style.display = 'none';
|
||||
try {
|
||||
const r = await fetch(url, { method: 'POST' });
|
||||
const data = await r.json();
|
||||
if (r.ok && data.ok) {
|
||||
if (url.includes('reminders')) {
|
||||
const n = data.reminders_found ?? 0;
|
||||
showResult(true, n > 0
|
||||
? `Found ${n} due reminder${n !== 1 ? 's' : ''} — notification sent.`
|
||||
: 'No due reminders found — nothing sent.');
|
||||
} else {
|
||||
showResult(true, 'Notification sent. Check your configured channel.');
|
||||
}
|
||||
} else {
|
||||
showResult(false, data.detail || 'Request failed.');
|
||||
}
|
||||
} catch (e) {
|
||||
showResult(false, 'Network error: ' + e.message);
|
||||
} finally {
|
||||
btnEl.disabled = false;
|
||||
btnEl.textContent = label;
|
||||
}
|
||||
}
|
||||
|
||||
document.getElementById('btn-test-notify').addEventListener('click', function() {
|
||||
apiPost('/api/push/test', this, 'Send Test Notification');
|
||||
});
|
||||
|
||||
document.getElementById('btn-check-reminders').addEventListener('click', function() {
|
||||
apiPost('/api/push/reminders/check', this, 'Check Reminders Now');
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
189
cortex/static/pg.css
Normal file
189
cortex/static/pg.css
Normal file
@@ -0,0 +1,189 @@
|
||||
/* ─── Cortex settings pages — shared stylesheet ───────────────────────────── */
|
||||
|
||||
/* ── Variables ── */
|
||||
:root {
|
||||
--pg-bg: #0f1117; --pg-surface: #1a1d27; --pg-border: #2d3148;
|
||||
--pg-text: #e2e8f0; --pg-muted: #94a3b8;
|
||||
--pg-dim: #64748b; --pg-dimmer: #475569;
|
||||
--pg-bright: #cbd5e1; --pg-nav-hover: rgba(255,255,255,0.05);
|
||||
--pg-accent: #a78bfa; /* heading/highlight purple */
|
||||
--pg-action: #7c3aed; /* button/focus purple */
|
||||
}
|
||||
[data-theme="light"] {
|
||||
--pg-bg: #f4f2fa; --pg-surface: #ffffff; --pg-border: #d0c8e8;
|
||||
--pg-text: #1a1228; --pg-muted: #5a5478;
|
||||
--pg-dim: #7a7290; --pg-dimmer: #9e98b0;
|
||||
--pg-bright: #1a1228; --pg-nav-hover: rgba(0,0,0,0.05);
|
||||
--pg-accent: #7c3aed;
|
||||
--pg-action: #6d28d9;
|
||||
}
|
||||
|
||||
/* ── Reset ── */
|
||||
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
|
||||
|
||||
/* ── Base ── */
|
||||
body {
|
||||
min-height: 100vh;
|
||||
background: var(--pg-bg);
|
||||
font-family: 'Inter', system-ui, -apple-system, sans-serif;
|
||||
font-weight: 450;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
color: var(--pg-text);
|
||||
}
|
||||
|
||||
/* ── Top nav ── */
|
||||
.page-nav {
|
||||
display: flex; align-items: center; gap: 0.25rem;
|
||||
padding: 0.5rem 1rem; background: var(--pg-surface);
|
||||
border-bottom: 1px solid var(--pg-border); flex-wrap: wrap;
|
||||
}
|
||||
.nav-link {
|
||||
display: inline-flex; align-items: center;
|
||||
padding: 0.3rem 0.6rem; border-radius: 6px;
|
||||
font-size: 0.8rem; font-weight: 500; color: var(--pg-dim);
|
||||
text-decoration: none; transition: color 0.15s, background 0.15s;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.nav-link:hover { color: var(--pg-bright); background: var(--pg-nav-hover); }
|
||||
.nav-link.active { color: var(--pg-accent); }
|
||||
.nav-spacer { flex: 1; min-width: 0.5rem; }
|
||||
.nav-link.nav-logout { color: var(--pg-dimmer); }
|
||||
.nav-link.nav-logout:hover { color: var(--pg-muted); background: none; }
|
||||
|
||||
/* ── Page container ── */
|
||||
.page-wrap {
|
||||
max-width: 860px; margin: 0 auto;
|
||||
padding: 2rem 1.5rem 4rem; width: 100%;
|
||||
}
|
||||
|
||||
/* ── Page heading ── */
|
||||
.page-title {
|
||||
font-size: 1.4rem; font-weight: 700; color: var(--pg-accent);
|
||||
}
|
||||
.page-subtitle {
|
||||
font-size: 0.8rem; color: var(--pg-muted);
|
||||
margin-top: 0.2rem; margin-bottom: 1.75rem; line-height: 1.5;
|
||||
}
|
||||
|
||||
/* ── Sections (settings-style, bottom-bordered h2) ── */
|
||||
.section { margin-bottom: 2rem; }
|
||||
.section > h2 {
|
||||
font-size: 0.9rem; font-weight: 600; color: var(--pg-muted);
|
||||
margin-bottom: 1rem; padding-bottom: 0.4rem;
|
||||
border-bottom: 1px solid var(--pg-border);
|
||||
}
|
||||
|
||||
/* ── Form elements ── */
|
||||
.field { margin-bottom: 1rem; }
|
||||
|
||||
label {
|
||||
display: block; font-size: 0.8rem; font-weight: 500;
|
||||
color: var(--pg-muted); margin-bottom: 0.4rem;
|
||||
}
|
||||
|
||||
input, select, textarea {
|
||||
width: 100%; padding: 0.65rem 0.85rem;
|
||||
background: var(--pg-bg); border: 1px solid var(--pg-border);
|
||||
border-radius: 6px; color: var(--pg-text); font-size: 0.95rem;
|
||||
font-family: inherit; outline: none; transition: border-color 0.15s;
|
||||
}
|
||||
input:focus, select:focus, textarea:focus { border-color: var(--pg-action); }
|
||||
input[readonly] { color: var(--pg-muted); cursor: default; }
|
||||
input[type="password"] { font-family: monospace; letter-spacing: 0.05em; }
|
||||
input[type="checkbox"], input[type="radio"] { width: auto; padding: 0; }
|
||||
|
||||
textarea {
|
||||
font-family: 'SF Mono', 'Fira Mono', 'Menlo', monospace;
|
||||
font-size: 0.88rem; line-height: 1.55; resize: vertical;
|
||||
}
|
||||
|
||||
/* ── Buttons ── */
|
||||
|
||||
/* Primary form submit */
|
||||
.btn-submit {
|
||||
padding: 0.6rem 1.5rem; margin-top: 0.25rem;
|
||||
background: var(--pg-action); border: none; border-radius: 6px;
|
||||
color: #fff; font-size: 0.9rem; font-weight: 600;
|
||||
cursor: pointer; transition: opacity 0.15s;
|
||||
}
|
||||
.btn-submit:hover { opacity: 0.88; }
|
||||
|
||||
/* Compact inline primary (e.g. rename save, inline forms) */
|
||||
.btn-save {
|
||||
padding: 0.4rem 0.9rem; background: var(--pg-action); border: none;
|
||||
border-radius: 6px; color: #fff; font-size: 0.9rem;
|
||||
font-weight: 600; cursor: pointer; transition: opacity 0.15s;
|
||||
}
|
||||
.btn-save:hover { opacity: 0.88; }
|
||||
|
||||
/* Outline secondary (e.g. clear cache, cancel, test actions) */
|
||||
.btn-secondary {
|
||||
padding: 0.5rem 1rem; background: none;
|
||||
border: 1px solid var(--pg-border); border-radius: 6px;
|
||||
color: var(--pg-muted); font-size: 0.88rem; font-weight: 500;
|
||||
cursor: pointer; transition: border-color 0.15s, color 0.15s;
|
||||
}
|
||||
.btn-secondary:hover { border-color: var(--pg-muted); color: var(--pg-text); }
|
||||
.btn-secondary:disabled { opacity: 0.5; cursor: default; }
|
||||
|
||||
/* Inline cancel */
|
||||
.btn-cancel {
|
||||
padding: 0.4rem 0.75rem; background: none;
|
||||
border: 1px solid var(--pg-border); border-radius: 6px;
|
||||
color: var(--pg-muted); font-size: 0.9rem;
|
||||
cursor: pointer; transition: border-color 0.15s, color 0.15s;
|
||||
}
|
||||
.btn-cancel:hover { border-color: var(--pg-muted); color: var(--pg-text); }
|
||||
|
||||
/* Button-styled link (purple, used for "Settings →" style CTAs) */
|
||||
.action-link {
|
||||
display: inline-block; padding: 0.5rem 1rem;
|
||||
background: var(--pg-action); border-radius: 6px;
|
||||
color: #fff; font-size: 0.88rem; font-weight: 600;
|
||||
text-decoration: none; transition: opacity 0.15s;
|
||||
}
|
||||
.action-link:hover { opacity: 0.88; }
|
||||
|
||||
/* Inline button row */
|
||||
.btn-row { display: flex; gap: 0.5rem; margin-top: 0.5rem; }
|
||||
|
||||
/* ── Text utilities ── */
|
||||
|
||||
/* Small muted helper text below inputs */
|
||||
.hint { font-size: 0.78rem; color: var(--pg-dim); margin-top: 0.35rem; line-height: 1.5; }
|
||||
|
||||
/* Section-level description paragraph */
|
||||
.section-note { font-size: 0.8rem; color: var(--pg-muted); margin-bottom: 0.85rem; line-height: 1.55; }
|
||||
|
||||
/* Inline code */
|
||||
code {
|
||||
font-size: 0.82rem; font-family: 'SF Mono', 'Fira Mono', 'Menlo', monospace;
|
||||
background: var(--pg-bg); border: 1px solid var(--pg-border);
|
||||
padding: 0.1rem 0.35rem; border-radius: 4px; color: var(--pg-accent);
|
||||
}
|
||||
|
||||
/* ── Feedback messages ── */
|
||||
.success { color: #4ade80; font-size: 0.85rem; text-align: center; margin-bottom: 1rem; }
|
||||
.error { color: #f87171; font-size: 0.85rem; text-align: center; margin-bottom: 1rem; }
|
||||
|
||||
/* ── Usage table (JS-rendered in settings) ── */
|
||||
.usage-table { border-collapse: collapse; width: 100%; min-width: 360px; }
|
||||
.usage-table th {
|
||||
padding: 0.35rem 0.5rem; font-size: 0.75rem; color: var(--pg-muted);
|
||||
font-weight: 600; text-align: right; border-bottom: 1px solid var(--pg-border);
|
||||
}
|
||||
.usage-table th:first-child { padding-left: 0; text-align: left; }
|
||||
.usage-table td {
|
||||
padding: 0.4rem 0.5rem; font-size: 0.82rem; color: var(--pg-muted); text-align: right;
|
||||
}
|
||||
.usage-table td:first-child { padding-left: 0; color: var(--pg-text); text-align: left; white-space: nowrap; }
|
||||
.usage-table td:last-child { color: var(--pg-text); font-weight: 600; }
|
||||
|
||||
/* ── Tool category header row (tools_settings.py generated) ── */
|
||||
.tool-cat-row td {
|
||||
padding: 0.75rem 0.9rem 0.3rem;
|
||||
font-size: 0.72rem; font-weight: 700; letter-spacing: 0.07em;
|
||||
text-transform: uppercase; color: var(--pg-dimmer);
|
||||
border-bottom: 1px solid var(--pg-border);
|
||||
}
|
||||
397
cortex/static/settings.html
Normal file
397
cortex/static/settings.html
Normal file
@@ -0,0 +1,397 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Account Settings</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
corePlugins: { preflight: false },
|
||||
darkMode: ['selector', '[data-theme="dark"]'],
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
pg: {
|
||||
bg: 'var(--pg-bg)',
|
||||
surface: 'var(--pg-surface)',
|
||||
border: 'var(--pg-border)',
|
||||
text: 'var(--pg-text)',
|
||||
muted: 'var(--pg-muted)',
|
||||
dim: 'var(--pg-dim)',
|
||||
dimmer: 'var(--pg-dimmer)',
|
||||
bright: 'var(--pg-bright)',
|
||||
accent: 'var(--pg-accent)',
|
||||
action: 'var(--pg-action)',
|
||||
}
|
||||
},
|
||||
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<link rel="stylesheet" href="/static/pg.css">
|
||||
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
|
||||
<style>
|
||||
/* ── Server-generated persona list ── */
|
||||
.persona-list {
|
||||
list-style: none; display: flex; flex-direction: column;
|
||||
gap: 0.5rem; margin-top: 0.5rem;
|
||||
}
|
||||
.persona-list li { display: flex; align-items: center; gap: 0.5rem; flex-wrap: wrap; }
|
||||
.persona-link {
|
||||
display: inline-block; padding: 0.3rem 0.75rem;
|
||||
background: var(--pg-bg); border: 1px solid var(--pg-border);
|
||||
border-radius: 20px; color: var(--pg-accent); font-size: 0.85rem;
|
||||
text-decoration: none; transition: border-color 0.15s;
|
||||
}
|
||||
.persona-link:hover { border-color: var(--pg-action); }
|
||||
.persona-list li em { color: var(--pg-muted); font-size: 0.85rem; }
|
||||
.persona-rename-toggle {
|
||||
background: none; border: 1px solid var(--pg-border);
|
||||
border-radius: 6px; color: var(--pg-muted); font-size: 0.8rem;
|
||||
padding: 0.3rem 0.6rem; margin-top: 0.25rem;
|
||||
cursor: pointer; opacity: 0.7; transition: opacity 0.15s, color 0.15s;
|
||||
}
|
||||
.persona-rename-toggle:hover { opacity: 1; color: var(--pg-accent); }
|
||||
.persona-rename-form { display: flex; align-items: center; gap: 0.4rem; }
|
||||
.persona-rename-form input[type="text"] {
|
||||
width: 12rem; padding: 0.3rem 0.6rem;
|
||||
border-color: var(--pg-action); font-size: 0.9rem;
|
||||
}
|
||||
.persona-rename-form .btn-save { padding: 0.3rem 0.75rem; font-size: 0.85rem; }
|
||||
|
||||
/* ── Server-generated role badge ── */
|
||||
.role-badge {
|
||||
display: inline-block; padding: 0.25rem 0.75rem;
|
||||
border-radius: 20px; font-size: 0.78rem; font-weight: 600;
|
||||
text-transform: uppercase; letter-spacing: 0.06em;
|
||||
}
|
||||
.role-badge.role-admin {
|
||||
background: rgba(124,58,237,0.15); color: var(--pg-accent);
|
||||
border: 1px solid rgba(124,58,237,0.4);
|
||||
}
|
||||
.role-badge.role-user {
|
||||
background: rgba(100,116,139,0.12); color: var(--pg-muted);
|
||||
border: 1px solid var(--pg-border);
|
||||
}
|
||||
|
||||
/* ── JS-toggled states ── */
|
||||
#clear-ls-ok { display: none; margin-left: 0.75rem; font-size: 0.8rem; color: #4ade80; }
|
||||
.usage-wrap { overflow-x: auto; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<nav class="page-nav">
|
||||
<a href="{{ back_href }}" class="nav-link">← Chat</a>
|
||||
<a href="{{ help_href }}" class="nav-link">Help</a>
|
||||
<a href="/settings" class="nav-link active">Settings</a>
|
||||
<a href="/settings/models" class="nav-link">Models</a>
|
||||
<a href="/settings/notifications" class="nav-link">Notifications</a>
|
||||
<a href="/settings/tools" class="nav-link">Tools</a>
|
||||
<a href="/settings/crons" class="nav-link">Schedules</a>
|
||||
{{ integrations_nav }}
|
||||
<span class="nav-spacer"></span>
|
||||
<a href="/logout" class="nav-link nav-logout">Sign out</a>
|
||||
</nav>
|
||||
<div class="page-wrap">
|
||||
<h1 class="page-title">Account Settings</h1>
|
||||
<p class="page-subtitle">Manage your account and personas.</p>
|
||||
|
||||
<!-- SUCCESS -->
|
||||
<!-- ERROR -->
|
||||
|
||||
<!-- OpenRouter quickstart (shown by JS when no model is configured) -->
|
||||
<div id="openrouter-quickstart"
|
||||
class="hidden rounded-xl border border-amber-800 bg-amber-950 p-4 mb-5">
|
||||
<p class="text-xs font-semibold text-amber-400 mb-1">⚡ You're on the server default model</p>
|
||||
<p class="text-xs text-amber-600 mb-3 leading-relaxed">
|
||||
You can chat now, but adding your own model gives you more choices, lets you pick
|
||||
role-specific models, and tracks your usage separately.
|
||||
OpenRouter is the easiest way to get started — one key, many models.
|
||||
</p>
|
||||
<a href="/setup/model"
|
||||
class="inline-block px-3 py-2 rounded-md bg-amber-900 text-amber-100 text-sm font-medium hover:bg-amber-800 transition-colors">
|
||||
Set up OpenRouter →
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<!-- Account info -->
|
||||
<div class="section">
|
||||
<h2>Account</h2>
|
||||
<div class="field">
|
||||
<label>Username</label>
|
||||
<input type="text" value="{{ username }}" readonly>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>Role</label>
|
||||
<span class="role-badge role-{{ user_role }}">{{ user_role }}</span>
|
||||
</div>
|
||||
<button type="button" id="show-rename-user" class="persona-rename-toggle">
|
||||
✏ Change username
|
||||
</button>
|
||||
<form id="rename-user-form" method="POST" action="/settings/username" style="display:none; margin-top:0.75rem;">
|
||||
<div class="field">
|
||||
<label for="new_username">New username</label>
|
||||
<input type="text" id="new_username" name="new_username"
|
||||
value="{{ username }}"
|
||||
pattern="[a-z_][a-z0-9_\-]{0,31}" required autofocus
|
||||
autocomplete="off" data-form-type="other">
|
||||
<p class="hint">Lowercase letters, digits, _ or - only. You will be logged out after renaming.</p>
|
||||
</div>
|
||||
<div class="btn-row">
|
||||
<button type="submit" class="btn-save">Save</button>
|
||||
<button type="button" id="cancel-rename-user" class="btn-cancel">Cancel</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<!-- Connected accounts -->
|
||||
<div class="section">
|
||||
<h2>Connected Accounts</h2>
|
||||
<div class="field">
|
||||
<label>Google Account</label>
|
||||
<input type="text" value="{{ google_email }}" readonly
|
||||
placeholder="No Google account linked"
|
||||
style="{{ google_email == '' and 'color:var(--pg-dimmer)' or '' }}">
|
||||
</div>
|
||||
<p class="hint" style="margin-top:-0.5rem;">To link or change your Google account, contact Scott.</p>
|
||||
</div>
|
||||
|
||||
<!-- Email Allowlist -->
|
||||
<div class="section">
|
||||
<h2>Email Allowlist</h2>
|
||||
<p class="section-note">
|
||||
One regex pattern per line. The <code>email_send</code> tool will only send to addresses
|
||||
that match at least one pattern. Leave blank to block all outbound email.
|
||||
</p>
|
||||
<form method="POST" action="/settings/email-allowlist">
|
||||
<div class="field">
|
||||
<label for="email_allowlist_ta">Allowed patterns</label>
|
||||
<textarea id="email_allowlist_ta" name="patterns" rows="6"
|
||||
placeholder=".*@example\.com alice@example\.com"
|
||||
spellcheck="false">{{ email_allowlist }}</textarea>
|
||||
</div>
|
||||
<button type="submit" class="btn-submit w-full md:w-96">Save allowlist</button>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<!-- HTTP POST Allowlist -->
|
||||
<div class="section">
|
||||
<h2>HTTP POST Allowlist</h2>
|
||||
<p class="section-note">
|
||||
One URL prefix per line. The <code>http_post</code> tool will only POST to URLs that
|
||||
start with a listed prefix. Leave blank to block all outbound POST requests.
|
||||
</p>
|
||||
<form method="POST" action="/settings/http-allowlist">
|
||||
<div class="field">
|
||||
<label for="http_allowlist_ta">Allowed URL prefixes</label>
|
||||
<textarea id="http_allowlist_ta" name="prefixes" rows="5"
|
||||
placeholder="https://ha.dgrzone.com/api/webhook/ https://n8n.dgrzone.com/webhook/"
|
||||
spellcheck="false">{{ http_allowlist }}</textarea>
|
||||
</div>
|
||||
<button type="submit" class="btn-submit w-full md:w-96">Save allowlist</button>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<!-- Usage summary -->
|
||||
<div class="section" id="usage-section">
|
||||
<h2>Usage</h2>
|
||||
<p class="section-note">
|
||||
Token consumption tracked for API-backed models (Gemini API, local OpenAI-compatible).
|
||||
Claude CLI calls are not metered.
|
||||
</p>
|
||||
<div id="usage-table-wrap" class="usage-wrap">
|
||||
<p class="section-note">Loading…</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Browser Cache -->
|
||||
<div class="section">
|
||||
<h2>Browser Cache</h2>
|
||||
<p class="section-note">
|
||||
Clears UI preferences stored in this browser: active mode, session ID, memory toggles,
|
||||
theme, font size, and context tier. Does not sign you out.
|
||||
</p>
|
||||
<button type="button" id="clear-ls-btn" class="btn-secondary">Clear browser cache</button>
|
||||
<span id="clear-ls-ok">Cleared.</span>
|
||||
</div>
|
||||
|
||||
<!-- Change Password -->
|
||||
<div class="section">
|
||||
<h2>Change Password</h2>
|
||||
<form method="POST" action="/settings/password" id="password-form">
|
||||
<div class="field">
|
||||
<label for="current_password">Current password</label>
|
||||
<input type="password" id="current_password" name="current_password"
|
||||
autocomplete="current-password" required>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="new_password">New password</label>
|
||||
<input type="password" id="new_password" name="new_password"
|
||||
autocomplete="new-password" required minlength="8">
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="confirm_password">Confirm new password</label>
|
||||
<input type="password" id="confirm_password" name="confirm_password"
|
||||
autocomplete="new-password" required>
|
||||
</div>
|
||||
<button type="submit" class="btn-submit w-full md:w-96">Update password</button>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<!-- Sessions -->
|
||||
<div class="section">
|
||||
<h2>Sessions</h2>
|
||||
<p class="section-note">
|
||||
Auto-name any sessions that still show a random ID, using their first message as the name.
|
||||
Only unnamed sessions are affected — existing names are left alone.
|
||||
</p>
|
||||
<button type="button" id="backfill-names-btn" class="btn-secondary">Auto-name old sessions</button>
|
||||
<span id="backfill-names-ok"
|
||||
class="ml-3 text-xs hidden"
|
||||
style="color:#4ade80"></span>
|
||||
</div>
|
||||
|
||||
<!-- Personas -->
|
||||
<div class="section">
|
||||
<h2>Personas</h2>
|
||||
<ul class="persona-list">
|
||||
{{ persona_items }}
|
||||
</ul>
|
||||
<a href="/setup/persona"
|
||||
class="inline-block mt-3 text-xs text-pg-muted hover:text-pg-accent transition-colors">
|
||||
+ Add new persona
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Password confirmation check
|
||||
document.getElementById('password-form').addEventListener('submit', e => {
|
||||
const np = document.getElementById('new_password').value;
|
||||
const cfm = document.getElementById('confirm_password').value;
|
||||
if (np !== cfm) {
|
||||
e.preventDefault();
|
||||
alert('New passwords do not match.');
|
||||
}
|
||||
});
|
||||
|
||||
// Username rename toggle
|
||||
document.getElementById('show-rename-user').addEventListener('click', () => {
|
||||
document.getElementById('show-rename-user').style.display = 'none';
|
||||
document.getElementById('rename-user-form').style.display = 'block';
|
||||
document.getElementById('new_username').focus();
|
||||
});
|
||||
document.getElementById('cancel-rename-user').addEventListener('click', () => {
|
||||
document.getElementById('rename-user-form').style.display = 'none';
|
||||
document.getElementById('show-rename-user').style.display = '';
|
||||
});
|
||||
|
||||
// Clear localStorage (keeps JWT cookie — no sign-out)
|
||||
document.getElementById('clear-ls-btn').addEventListener('click', () => {
|
||||
localStorage.clear();
|
||||
document.getElementById('clear-ls-ok').style.display = 'inline';
|
||||
});
|
||||
|
||||
// Show OpenRouter quick-start card if no model is configured
|
||||
(async () => {
|
||||
try {
|
||||
const d = await fetch('/backend').then(r => r.json());
|
||||
if ((d.available_roles || []).length === 0) {
|
||||
const el = document.getElementById('openrouter-quickstart');
|
||||
el.classList.remove('hidden');
|
||||
el.style.display = 'block';
|
||||
}
|
||||
} catch (_) {}
|
||||
})();
|
||||
|
||||
// Usage summary table
|
||||
(async () => {
|
||||
const wrap = document.getElementById('usage-table-wrap');
|
||||
try {
|
||||
const resp = await fetch('/api/usage/summary');
|
||||
if (!resp.ok) throw new Error(resp.statusText);
|
||||
const rows_data = await resp.json();
|
||||
if (!rows_data.length) {
|
||||
wrap.innerHTML = '<p class="section-note">No usage recorded yet.</p>';
|
||||
return;
|
||||
}
|
||||
const fmt = n => n >= 1000 ? (n / 1000).toFixed(1) + 'k' : String(n);
|
||||
const rows = rows_data.map(d => {
|
||||
const labelCell = d.label !== d.key
|
||||
? `<span title="${d.key}">${d.label}</span>`
|
||||
: `<span>${d.key}</span>`;
|
||||
return `<tr>
|
||||
<td>${labelCell}</td>
|
||||
<td>${d.calls}</td>
|
||||
<td>${fmt(d.prompt_tokens)}</td>
|
||||
<td>${fmt(d.completion_tokens)}</td>
|
||||
<td>${fmt(d.total_tokens)}</td>
|
||||
</tr>`;
|
||||
}).join('');
|
||||
wrap.innerHTML = `<table class="usage-table">
|
||||
<thead><tr>
|
||||
<th style="text-align:left">Model</th>
|
||||
<th>Calls</th><th>Prompt</th><th>Output</th><th>Total</th>
|
||||
</tr></thead>
|
||||
<tbody>${rows}</tbody>
|
||||
</table>`;
|
||||
} catch (e) {
|
||||
wrap.innerHTML = '<p class="section-note">Could not load usage data.</p>';
|
||||
}
|
||||
})();
|
||||
|
||||
// Auto-name old sessions backfill
|
||||
document.getElementById('backfill-names-btn').addEventListener('click', async () => {
|
||||
const btn = document.getElementById('backfill-names-btn');
|
||||
const ok = document.getElementById('backfill-names-ok');
|
||||
btn.disabled = true;
|
||||
btn.textContent = 'Working…';
|
||||
try {
|
||||
const params = new URLSearchParams(window.location.search);
|
||||
const user = params.get('user') || document.querySelector('input[value]')?.value || '';
|
||||
const persona = params.get('persona') || '';
|
||||
const qs = user ? `?user=${encodeURIComponent(user)}&persona=${encodeURIComponent(persona)}` : '';
|
||||
const res = await fetch(`/api/sessions/backfill-names${qs}`, { method: 'POST' });
|
||||
const data = await res.json();
|
||||
if (!res.ok) throw new Error(data.detail || res.statusText);
|
||||
const n = data.named ?? 0;
|
||||
ok.textContent = `Named ${n} session${n !== 1 ? 's' : ''}.`;
|
||||
ok.style.display = 'inline';
|
||||
ok.classList.remove('hidden');
|
||||
} catch (e) {
|
||||
ok.textContent = 'Error — check console.';
|
||||
ok.style.color = '#f87171';
|
||||
ok.style.display = 'inline';
|
||||
ok.classList.remove('hidden');
|
||||
}
|
||||
btn.textContent = 'Auto-name old sessions';
|
||||
btn.disabled = false;
|
||||
});
|
||||
|
||||
// Persona rename toggle
|
||||
document.querySelectorAll('.persona-rename-toggle').forEach(btn => {
|
||||
btn.addEventListener('click', () => {
|
||||
const p = btn.dataset.persona;
|
||||
const form = document.querySelector(`.persona-rename-form[data-persona="${p}"]`);
|
||||
btn.style.display = 'none';
|
||||
form.style.display = 'flex';
|
||||
form.querySelector('input[type="text"]').focus();
|
||||
});
|
||||
});
|
||||
document.querySelectorAll('.persona-rename-cancel').forEach(btn => {
|
||||
btn.addEventListener('click', () => {
|
||||
const form = btn.closest('.persona-rename-form');
|
||||
const p = form.dataset.persona;
|
||||
const toggle = document.querySelector(`.persona-rename-toggle[data-persona="${p}"]`);
|
||||
form.style.display = 'none';
|
||||
toggle.style.display = '';
|
||||
});
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -4,6 +4,9 @@
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Cortex — Setup</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
|
||||
|
||||
@@ -13,7 +16,10 @@
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
background: #0f1117;
|
||||
font-family: system-ui, -apple-system, sans-serif;
|
||||
font-family: 'Inter', system-ui, -apple-system, sans-serif;
|
||||
font-weight: 450;
|
||||
-webkit-font-smoothing: antialiased;
|
||||
-moz-osx-font-smoothing: grayscale;
|
||||
color: #e2e8f0;
|
||||
padding: 1.5rem;
|
||||
}
|
||||
@@ -33,7 +39,7 @@
|
||||
}
|
||||
|
||||
.logo h1 { font-size: 1.6rem; font-weight: 700; letter-spacing: 0.05em; color: #a78bfa; }
|
||||
.logo p { font-size: 0.8rem; color: #64748b; margin-top: 0.25rem; }
|
||||
.logo p { font-size: 0.8rem; color: #94a3b8; margin-top: 0.25rem; }
|
||||
|
||||
h2 {
|
||||
font-size: 1rem;
|
||||
@@ -52,7 +58,7 @@
|
||||
margin-bottom: 0.4rem;
|
||||
}
|
||||
|
||||
label small { font-weight: 400; color: #475569; }
|
||||
label small { font-weight: 400; color: #94a3b8; }
|
||||
|
||||
input, select {
|
||||
width: 100%;
|
||||
@@ -71,7 +77,7 @@
|
||||
|
||||
.field { margin-bottom: 1rem; }
|
||||
|
||||
.hint { font-size: 0.75rem; color: #475569; margin-top: 0.3rem; }
|
||||
.hint { font-size: 0.75rem; color: #94a3b8; margin-top: 0.3rem; }
|
||||
|
||||
button[type="submit"] {
|
||||
width: 100%;
|
||||
@@ -98,7 +104,7 @@
|
||||
|
||||
.step-label {
|
||||
font-size: 0.7rem;
|
||||
color: #475569;
|
||||
color: #94a3b8;
|
||||
text-align: right;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
@@ -121,6 +127,36 @@
|
||||
|
||||
.emoji-opt.selected { border-color: #7c3aed; background: #2d1f52; }
|
||||
#emoji-hidden { display: none; }
|
||||
|
||||
.provider-badge {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 0.4rem;
|
||||
background: #2d1f52;
|
||||
border: 1px solid #7c3aed;
|
||||
border-radius: 6px;
|
||||
padding: 0.3rem 0.6rem;
|
||||
font-size: 0.78rem;
|
||||
color: #a78bfa;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.skip-link {
|
||||
display: block;
|
||||
text-align: center;
|
||||
margin-top: 1rem;
|
||||
font-size: 0.8rem;
|
||||
color: #64748b;
|
||||
text-decoration: none;
|
||||
}
|
||||
.skip-link:hover { color: #94a3b8; }
|
||||
|
||||
.model-hint {
|
||||
font-size: 0.72rem;
|
||||
color: #64748b;
|
||||
margin-top: 0.75rem;
|
||||
text-align: center;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
@@ -131,10 +167,11 @@
|
||||
</div>
|
||||
|
||||
<!-- ERROR -->
|
||||
<!-- ERROR_MODEL -->
|
||||
|
||||
<!-- ── Step 1: password ───────────────────────────────────────── -->
|
||||
<div id="step-password">
|
||||
<div class="step-label">Step 1 of 2</div>
|
||||
<div class="step-label">Step 1 of 3</div>
|
||||
<h2>Set your password</h2>
|
||||
<form method="POST" action="" id="password-form">
|
||||
<input type="hidden" name="step" value="password">
|
||||
@@ -155,7 +192,7 @@
|
||||
|
||||
<!-- ── Step 2: persona ────────────────────────────────────────── -->
|
||||
<div id="step-persona" style="display:none">
|
||||
<div class="step-label">Step 2 of 2</div>
|
||||
<div class="step-label">Step 2 of 3</div>
|
||||
<h2>Create your persona</h2>
|
||||
<form method="POST" action="" id="persona-form">
|
||||
<input type="hidden" name="step" value="persona">
|
||||
@@ -197,6 +234,39 @@
|
||||
<button type="submit">Create my persona →</button>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<!-- ── Step 3: model connect ─────────────────────────────────── -->
|
||||
<div id="step-model" style="display:none">
|
||||
<div class="step-label"><!-- SETUP_STEP3_LABEL --></div>
|
||||
<h2>Connect an AI model</h2>
|
||||
<div class="provider-badge">⚡ Recommended: OpenRouter</div>
|
||||
<p style="font-size:0.82rem;color:#94a3b8;margin-bottom:1rem;">
|
||||
One API key gives you access to Claude, Gemini, Llama, and dozens of other models.
|
||||
Get a free key at <a href="https://openrouter.ai/keys" target="_blank" style="color:#a78bfa;">openrouter.ai/keys</a>.
|
||||
</p>
|
||||
<form method="POST" action="/setup/model" id="model-form">
|
||||
<div class="field">
|
||||
<label for="api_key">OpenRouter API key</label>
|
||||
<input type="password" id="api_key" name="api_key"
|
||||
autocomplete="off" placeholder="sk-or-v1-..." required>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label for="model_name">Starting model</label>
|
||||
<select id="model_name" name="model_name">
|
||||
<option value="anthropic/claude-3-5-haiku-20241022">Claude 3.5 Haiku — Fast & affordable</option>
|
||||
<option value="anthropic/claude-3-7-sonnet-20250219">Claude 3.7 Sonnet — Smarter Claude</option>
|
||||
<option value="google/gemini-2.0-flash-001">Gemini 2.0 Flash — Fast Google model</option>
|
||||
<option value="meta-llama/llama-3.3-70b-instruct">Llama 3.3 70B — Open source</option>
|
||||
</select>
|
||||
<p class="hint">You can add more models or switch anytime in Account → Model Registry.</p>
|
||||
</div>
|
||||
<button type="submit">Connect & start chatting →</button>
|
||||
</form>
|
||||
<p class="model-hint">
|
||||
Using Ollama, a local model, or something else?
|
||||
<a href="#" id="skip-model-link" style="color:#64748b;">Skip this step →</a>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
@@ -226,6 +296,11 @@
|
||||
document.getElementById('step-password').style.display = 'none';
|
||||
document.getElementById('step-persona').style.display = 'block';
|
||||
}
|
||||
if (params.get('step') === '3') {
|
||||
document.getElementById('step-password').style.display = 'none';
|
||||
document.getElementById('step-persona').style.display = 'none';
|
||||
document.getElementById('step-model').style.display = 'block';
|
||||
}
|
||||
|
||||
// ── Client-side confirm password check ───────────────────────────
|
||||
document.getElementById('password-form').addEventListener('submit', e => {
|
||||
@@ -237,6 +312,15 @@
|
||||
}
|
||||
});
|
||||
|
||||
// ── Skip model setup — navigate to user home ─────────────────────
|
||||
document.getElementById('skip-model-link')?.addEventListener('click', e => {
|
||||
e.preventDefault();
|
||||
// Ask server for skip target (the cx_setup_persona cookie has the path)
|
||||
fetch('/setup/model/skip', { method: 'POST', credentials: 'same-origin' })
|
||||
.then(r => { if (r.redirected) location.href = r.url; else location.href = '/'; })
|
||||
.catch(() => { location.href = '/'; });
|
||||
});
|
||||
|
||||
// ── Auto-generate persona slug from display name ─────────────────
|
||||
document.getElementById('display_name').addEventListener('input', function() {
|
||||
const slugField = document.getElementById('persona_name');
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
106
cortex/static/sw.js
Normal file
106
cortex/static/sw.js
Normal file
@@ -0,0 +1,106 @@
|
||||
const CACHE = 'cortex-v2';
|
||||
|
||||
const PRECACHE = [
|
||||
'/static/style.css',
|
||||
'/static/app.js',
|
||||
'/static/marked.min.js',
|
||||
'/static/icon-192.png',
|
||||
'/static/icon-512.png',
|
||||
'/static/icon.svg',
|
||||
'/static/manifest.json',
|
||||
];
|
||||
|
||||
self.addEventListener('install', evt => {
|
||||
evt.waitUntil(
|
||||
caches.open(CACHE)
|
||||
.then(c => c.addAll(PRECACHE))
|
||||
.then(() => self.skipWaiting())
|
||||
);
|
||||
});
|
||||
|
||||
self.addEventListener('activate', evt => {
|
||||
evt.waitUntil(
|
||||
caches.keys()
|
||||
.then(keys => Promise.all(
|
||||
keys.filter(k => k !== CACHE).map(k => caches.delete(k))
|
||||
))
|
||||
.then(() => self.clients.claim())
|
||||
);
|
||||
});
|
||||
|
||||
self.addEventListener('push', evt => {
|
||||
let data = { title: 'Cortex', body: '', url: '/' };
|
||||
if (evt.data) {
|
||||
try { data = { ...data, ...evt.data.json() }; } catch (_) {}
|
||||
}
|
||||
evt.waitUntil(
|
||||
self.registration.showNotification(data.title, {
|
||||
body: data.body,
|
||||
icon: '/static/icon-192.png',
|
||||
badge: '/static/icon-192.png',
|
||||
data: { url: data.url },
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
self.addEventListener('notificationclick', evt => {
|
||||
evt.notification.close();
|
||||
const url = evt.notification.data?.url || '/';
|
||||
evt.waitUntil(
|
||||
clients.matchAll({ type: 'window', includeUncontrolled: true }).then(list => {
|
||||
for (const c of list) {
|
||||
if (c.url.includes(self.location.origin) && 'focus' in c) {
|
||||
c.navigate(url);
|
||||
return c.focus();
|
||||
}
|
||||
}
|
||||
if (clients.openWindow) return clients.openWindow(url);
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
self.addEventListener('fetch', evt => {
|
||||
const url = new URL(evt.request.url);
|
||||
|
||||
// Only handle same-origin GETs
|
||||
if (evt.request.method !== 'GET' || url.origin !== self.location.origin) return;
|
||||
|
||||
// Never intercept streaming or API calls
|
||||
if (
|
||||
url.pathname.startsWith('/chat') ||
|
||||
url.pathname.startsWith('/orchestrate') ||
|
||||
url.pathname.startsWith('/api/') ||
|
||||
url.pathname.startsWith('/distill') ||
|
||||
url.pathname.startsWith('/webhook') ||
|
||||
url.pathname.startsWith('/auth/')
|
||||
) return;
|
||||
|
||||
// Static assets — cache first, refresh in background (stale-while-revalidate)
|
||||
if (url.pathname.startsWith('/static/')) {
|
||||
evt.respondWith(
|
||||
caches.open(CACHE).then(cache =>
|
||||
cache.match(evt.request).then(cached => {
|
||||
const network = fetch(evt.request).then(resp => {
|
||||
if (resp.ok) cache.put(evt.request, resp.clone());
|
||||
return resp;
|
||||
});
|
||||
return cached || network;
|
||||
})
|
||||
)
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
// HTML pages — network first, cached shell fallback
|
||||
evt.respondWith(
|
||||
fetch(evt.request)
|
||||
.then(resp => {
|
||||
if (resp.ok) {
|
||||
const clone = resp.clone();
|
||||
caches.open(CACHE).then(c => c.put(evt.request, clone));
|
||||
}
|
||||
return resp;
|
||||
})
|
||||
.catch(() => caches.match(evt.request))
|
||||
);
|
||||
});
|
||||
213
cortex/static/tools_settings.html
Normal file
213
cortex/static/tools_settings.html
Normal file
@@ -0,0 +1,213 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Tool Settings — Cortex</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@100..900&display=swap" rel="stylesheet">
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
corePlugins: { preflight: false },
|
||||
darkMode: ['selector', '[data-theme="dark"]'],
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
pg: {
|
||||
bg: 'var(--pg-bg)',
|
||||
surface: 'var(--pg-surface)',
|
||||
border: 'var(--pg-border)',
|
||||
text: 'var(--pg-text)',
|
||||
muted: 'var(--pg-muted)',
|
||||
dim: 'var(--pg-dim)',
|
||||
dimmer: 'var(--pg-dimmer)',
|
||||
bright: 'var(--pg-bright)',
|
||||
accent: 'var(--pg-accent)',
|
||||
action: 'var(--pg-action)',
|
||||
}
|
||||
},
|
||||
fontFamily: { sans: ['Inter', 'system-ui', 'sans-serif'] }
|
||||
}
|
||||
}
|
||||
}
|
||||
</script>
|
||||
<link rel="stylesheet" href="/static/pg.css">
|
||||
<script>(function(){var t=localStorage.getItem('theme')||(window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light');document.documentElement.setAttribute('data-theme',t);})();</script>
|
||||
<style>
|
||||
/* ── Server-generated tool table ── */
|
||||
.table-section-label {
|
||||
font-size: 0.7rem; font-weight: 700; letter-spacing: 0.08em;
|
||||
text-transform: uppercase; color: var(--pg-dimmer);
|
||||
margin: 1.75rem 0 0.6rem;
|
||||
}
|
||||
.tool-table {
|
||||
width: 100%; border-collapse: collapse;
|
||||
background: var(--pg-surface); border: 1px solid var(--pg-border);
|
||||
border-radius: 0.75rem; overflow: hidden; margin-bottom: 0.5rem;
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
.tool-table th {
|
||||
text-align: left; padding: 0.55rem 0.9rem;
|
||||
border-bottom: 1px solid var(--pg-border);
|
||||
color: var(--pg-muted); font-weight: 600; font-size: 0.78rem;
|
||||
text-transform: uppercase; letter-spacing: 0.04em;
|
||||
}
|
||||
.tool-table td { padding: 0.5rem 0.9rem; border-bottom: 1px solid var(--pg-border); vertical-align: middle; }
|
||||
.tool-table tr:last-child td { border-bottom: none; }
|
||||
.tool-table tr:hover td { background: rgba(124,58,237,0.04); }
|
||||
.tool-name { font-family: monospace; font-size: 0.82rem; }
|
||||
|
||||
/* Risk badges (server-generated) */
|
||||
.risk { display: inline-block; font-size: 0.7rem; font-weight: 700;
|
||||
padding: 0.15rem 0.45rem; border-radius: 9999px; letter-spacing: 0.04em; }
|
||||
.risk-low { background: rgba(34,197,94,0.12); color: #4ade80; }
|
||||
.risk-medium { background: rgba(234,179,8,0.12); color: #fbbf24; }
|
||||
.risk-high { background: rgba(239,68,68,0.12); color: #f87171; }
|
||||
[data-theme="light"] .risk-low { background: rgba(34,197,94,0.15); color: #16a34a; }
|
||||
[data-theme="light"] .risk-medium { background: rgba(234,179,8,0.15); color: #ca8a04; }
|
||||
[data-theme="light"] .risk-high { background: rgba(239,68,68,0.15); color: #dc2626; }
|
||||
|
||||
/* Auto-status pill (server-generated, updated by JS) */
|
||||
.auto-pill {
|
||||
display: inline-block; font-size: 0.68rem; font-weight: 600;
|
||||
padding: 0.12rem 0.4rem; border-radius: 9999px;
|
||||
}
|
||||
.auto-on { background: rgba(124,58,237,0.12); color: #a78bfa; }
|
||||
.auto-off { background: rgba(148,163,184,0.12); color: var(--pg-dimmer); }
|
||||
[data-theme="light"] .auto-on { color: #7c3aed; }
|
||||
|
||||
/* Override select (server-generated) */
|
||||
.override-sel {
|
||||
font-size: 0.78rem; padding: 0.25rem 0.5rem;
|
||||
border-radius: 0.3rem; min-width: 7rem; width: auto;
|
||||
}
|
||||
.override-sel.forced-on { border-color: #7c3aed; color: #7c3aed; }
|
||||
.override-sel.forced-off { border-color: #dc2626; color: #dc2626; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<nav class="page-nav">
|
||||
<a href="{{ back_href }}" class="nav-link">← Chat</a>
|
||||
<a href="{{ help_href }}" class="nav-link">Help</a>
|
||||
<a href="/settings" class="nav-link">Settings</a>
|
||||
<a href="/settings/models" class="nav-link">Models</a>
|
||||
<a href="/settings/notifications" class="nav-link">Notifications</a>
|
||||
<a href="/settings/tools" class="nav-link active">Tools</a>
|
||||
<a href="/settings/crons" class="nav-link">Schedules</a>
|
||||
{{ integrations_nav }}
|
||||
<span class="nav-spacer"></span>
|
||||
<a href="/logout" class="nav-link nav-logout">Sign out</a>
|
||||
</nav>
|
||||
|
||||
<div class="page-wrap">
|
||||
<h1 class="page-title">Tool Settings</h1>
|
||||
<p class="page-subtitle">
|
||||
Control which orchestrator tools are available. The risk level sets an automatic threshold;
|
||||
whitelist and blacklist let you fine-tune individual tools beyond that.
|
||||
</p>
|
||||
|
||||
<!-- SUCCESS -->
|
||||
<!-- ERROR -->
|
||||
|
||||
<form method="POST" action="/settings/tools" id="tools-form">
|
||||
|
||||
<!-- Risk policy card -->
|
||||
<div class="rounded-xl border border-pg-border bg-pg-surface p-5 mb-5">
|
||||
<h2 class="text-sm font-semibold text-pg-bright mb-4">Risk Policy</h2>
|
||||
<div class="flex items-center gap-4 flex-wrap mb-3">
|
||||
<span class="text-sm font-medium text-pg-text min-w-[6rem]">Max risk level</span>
|
||||
<select name="max_risk" id="max-risk-sel" class="w-auto">
|
||||
<option value="" {{ sel_none }}>No filter — use all role-permitted tools</option>
|
||||
<option value="low" {{ sel_low }}>Low — read-only and sandboxed tools only</option>
|
||||
<option value="medium" {{ sel_medium }}>Medium — low + medium risk (recommended)</option>
|
||||
<option value="high" {{ sel_high }}>High — all tools including destructive ones</option>
|
||||
</select>
|
||||
</div>
|
||||
<p class="text-xs text-pg-muted leading-relaxed mb-2">
|
||||
<strong class="text-pg-text">Low</strong> tools are read-only and sandboxed (web search, project file reads, HA status checks).<br>
|
||||
<strong class="text-pg-text">Medium</strong> tools write to local data or send notifications to you (cron jobs, scratch, task management).<br>
|
||||
<strong class="text-pg-text">High</strong> tools affect external systems or the host (shell exec, email, device control, service restart).
|
||||
</p>
|
||||
<p class="text-xs text-pg-muted leading-relaxed">
|
||||
The <em>Auto</em> column below shows each tool's status at your current max risk level.
|
||||
Use the override column to force-include or force-exclude individual tools.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<!-- Legend -->
|
||||
<div class="flex gap-5 flex-wrap mb-4 text-xs text-pg-muted">
|
||||
<span><span class="inline-block w-2 h-2 rounded-full bg-[#a78bfa] mr-1.5"></span>Auto-included by risk level</span>
|
||||
<span><span class="inline-block w-2 h-2 rounded-full bg-pg-dimmer mr-1.5"></span>Auto-excluded by risk level</span>
|
||||
</div>
|
||||
|
||||
<!-- Tool table (server-generated) -->
|
||||
{{ tool_table_html }}
|
||||
|
||||
<!-- Confirmation gate card -->
|
||||
<div class="rounded-xl border border-pg-border bg-pg-surface p-5 mt-5 mb-5">
|
||||
<h2 class="text-sm font-semibold text-pg-bright mb-2">Confirmation Gate</h2>
|
||||
<p class="text-xs text-pg-muted leading-relaxed mb-4">
|
||||
Some tools require explicit confirmation before executing. Override the defaults here.<br>
|
||||
Tools requiring confirmation by default: <code class="font-mono text-pg-accent bg-pg-bg border border-pg-border rounded px-1">{{ confirm_required_tools }}</code>
|
||||
</p>
|
||||
<div class="flex gap-6 flex-wrap items-start">
|
||||
<div class="flex-1 min-w-[200px]">
|
||||
<label class="block text-xs font-semibold text-pg-muted mb-1">Allow list — bypass confirmation</label>
|
||||
<textarea name="allow_list" rows="4"
|
||||
placeholder="reminders_clear cron_remove"
|
||||
autocomplete="off" spellcheck="false">{{ tool_allow }}</textarea>
|
||||
<p class="hint">One tool name per line. These tools skip the confirmation prompt.</p>
|
||||
</div>
|
||||
<div class="flex-1 min-w-[200px]">
|
||||
<label class="block text-xs font-semibold text-pg-muted mb-1">Deny list — always block</label>
|
||||
<textarea name="deny_list" rows="4"
|
||||
placeholder="shell_exec file_write"
|
||||
autocomplete="off" spellcheck="false">{{ tool_deny }}</textarea>
|
||||
<p class="hint">These tools are always blocked regardless of risk policy.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="mt-4">
|
||||
<button type="submit" class="btn-submit w-full md:w-96">Save tool settings</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const riskRank = { "": 99, "low": 0, "medium": 1, "high": 2 };
|
||||
const toolRisk = {{ tool_risk_json }};
|
||||
|
||||
const sel = document.getElementById('max-risk-sel');
|
||||
|
||||
function updateAutoPills() {
|
||||
const maxRank = riskRank[sel.value] ?? 99;
|
||||
document.querySelectorAll('[data-tool-risk]').forEach(row => {
|
||||
const risk = row.dataset.toolRisk;
|
||||
const pill = row.querySelector('.auto-pill');
|
||||
const isAuto = riskRank[risk] <= maxRank;
|
||||
pill.textContent = isAuto ? 'auto ✓' : 'excluded';
|
||||
pill.className = 'auto-pill ' + (isAuto ? 'auto-on' : 'auto-off');
|
||||
});
|
||||
}
|
||||
|
||||
sel.addEventListener('change', updateAutoPills);
|
||||
updateAutoPills();
|
||||
|
||||
// Color the override selects
|
||||
document.querySelectorAll('.override-sel').forEach(s => {
|
||||
function refresh() {
|
||||
s.className = 'override-sel';
|
||||
if (s.value === 'whitelist') s.classList.add('forced-on');
|
||||
if (s.value === 'blacklist') s.classList.add('forced-off');
|
||||
}
|
||||
s.addEventListener('change', refresh);
|
||||
refresh();
|
||||
});
|
||||
</script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
876
cortex/tests/test_agent_manager.py
Normal file
876
cortex/tests/test_agent_manager.py
Normal file
@@ -0,0 +1,876 @@
|
||||
"""
|
||||
Tests for agent_manager.py and the spawn_agent / aider_run background paths.
|
||||
|
||||
Run with:
|
||||
cd cortex && .venv/bin/python -m pytest tests/test_agent_manager.py -v
|
||||
|
||||
No browser, no LLM calls, no Cortex service needed. All LLM interactions are mocked.
|
||||
The agent_manager tests need no mocks at all — the module is pure asyncio.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from datetime import datetime, timedelta
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _make_mock_result(response: str = "Agent done."):
|
||||
"""Build a mock OrchestratorResult returned by openai_orchestrator.run."""
|
||||
r = MagicMock()
|
||||
r.checkpoint = None
|
||||
r.response = response
|
||||
return r
|
||||
|
||||
|
||||
def _mock_spawn_deps(
|
||||
model_type: str = "local_openai",
|
||||
user_role: str = "admin",
|
||||
tool_policy: dict | None = None,
|
||||
role_tools: list | None = None,
|
||||
):
|
||||
"""Return a context-manager stack that patches all spawn_agent external deps."""
|
||||
if tool_policy is None:
|
||||
tool_policy = {"allow": [], "deny": []}
|
||||
model_cfg = {
|
||||
"type": model_type,
|
||||
"api_url": "http://localhost:3000",
|
||||
"model_name": "test-model",
|
||||
"api_key": "x",
|
||||
}
|
||||
role_cfg = {
|
||||
"tools": role_tools,
|
||||
"system_append": "",
|
||||
"inject_datetime": True,
|
||||
"inject_mode": True,
|
||||
}
|
||||
|
||||
class _Stack:
|
||||
def __enter__(self_):
|
||||
self_._patches = [
|
||||
patch("model_registry.get_role_config", return_value=role_cfg),
|
||||
patch("model_registry.get_model_for_role", return_value=model_cfg),
|
||||
patch("model_registry.get_registry", return_value={"hosts": []}),
|
||||
patch("context_loader.load_context", return_value="Test system prompt"),
|
||||
patch("auth_utils.get_user_role", return_value=user_role),
|
||||
patch("auth_utils.get_tool_policy", return_value=tool_policy),
|
||||
patch("persona.get_user", return_value="scott"),
|
||||
]
|
||||
for p in self_._patches:
|
||||
p.start()
|
||||
return self_
|
||||
|
||||
def __exit__(self_, *args):
|
||||
for p in self_._patches:
|
||||
p.stop()
|
||||
|
||||
return _Stack()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fixture — reset agent_manager state between tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clear_agent_registry():
|
||||
"""Wipe the in-process agent registry before each test."""
|
||||
import agent_manager
|
||||
agent_manager._agents.clear()
|
||||
yield
|
||||
agent_manager._agents.clear()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# agent_manager — core CRUD
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAgentManagerCore:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_register_creates_record(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(
|
||||
user="scott", role="research", task="Investigate topic X", level=2
|
||||
)
|
||||
assert rec.agent_id in agent_manager._agents
|
||||
assert rec.status == "running"
|
||||
assert rec.level == 2
|
||||
assert rec.role == "research"
|
||||
assert rec.task == "Investigate topic X"
|
||||
assert rec.user == "scott"
|
||||
assert rec.finished is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_register_truncates_long_task(self):
|
||||
import agent_manager
|
||||
long_task = "x" * 500
|
||||
rec = await agent_manager.register(user="scott", role="chat", task=long_task, level=2)
|
||||
assert len(rec.task) == 200
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_finish_updates_record(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
await agent_manager.finish(rec.agent_id, "All done!", "done")
|
||||
|
||||
updated = agent_manager.get(rec.agent_id)
|
||||
assert updated.status == "done"
|
||||
assert updated.result == "All done!"
|
||||
assert updated.finished is not None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_finish_truncates_result(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
await agent_manager.finish(rec.agent_id, "y" * 2000)
|
||||
|
||||
updated = agent_manager.get(rec.agent_id)
|
||||
assert len(updated.result) <= agent_manager._RESULT_PREVIEW_CHARS
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_finish_failed_status(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
await agent_manager.finish(rec.agent_id, "Boom", "failed")
|
||||
assert agent_manager.get(rec.agent_id).status == "failed"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cancel_own_agent(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
msg = await agent_manager.cancel_agent(rec.agent_id, "scott")
|
||||
assert "cancelled" in msg
|
||||
assert agent_manager.get(rec.agent_id).status == "cancelled"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cancel_wrong_user_denied(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
msg = await agent_manager.cancel_agent(rec.agent_id, "holly")
|
||||
assert "denied" in msg.lower()
|
||||
assert agent_manager.get(rec.agent_id).status == "running"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cancel_nonexistent_agent(self):
|
||||
import agent_manager
|
||||
msg = await agent_manager.cancel_agent("does-not-exist", "scott")
|
||||
assert "No agent found" in msg
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cancel_already_done(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
await agent_manager.finish(rec.agent_id, "done", "done")
|
||||
msg = await agent_manager.cancel_agent(rec.agent_id, "scott")
|
||||
assert "already" in msg or "done" in msg
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cancel_kills_real_task(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
|
||||
sleep_task = asyncio.create_task(asyncio.sleep(60))
|
||||
agent_manager.set_task_ref(rec.agent_id, sleep_task)
|
||||
|
||||
await agent_manager.cancel_agent(rec.agent_id, "scott")
|
||||
await asyncio.sleep(0) # let the event loop process the cancellation
|
||||
|
||||
assert sleep_task.cancelled() or sleep_task.done()
|
||||
|
||||
def test_list_agents_returns_users_agents(self):
|
||||
import agent_manager
|
||||
# Manually populate the registry
|
||||
agent_manager._agents["a1"] = _make_record("a1", "scott", "running")
|
||||
agent_manager._agents["a2"] = _make_record("a2", "scott", "done")
|
||||
agent_manager._agents["a3"] = _make_record("a3", "holly", "running")
|
||||
|
||||
records = agent_manager.list_agents("scott")
|
||||
ids = {r.agent_id for r in records}
|
||||
assert "a1" in ids
|
||||
assert "a2" in ids
|
||||
assert "a3" not in ids
|
||||
|
||||
def test_list_agents_filters_by_status(self):
|
||||
import agent_manager
|
||||
agent_manager._agents["a1"] = _make_record("a1", "scott", "running")
|
||||
agent_manager._agents["a2"] = _make_record("a2", "scott", "done")
|
||||
|
||||
running = agent_manager.list_agents("scott", status="running")
|
||||
assert len(running) == 1
|
||||
assert running[0].agent_id == "a1"
|
||||
|
||||
def test_list_agents_respects_limit(self):
|
||||
import agent_manager
|
||||
for i in range(20):
|
||||
agent_manager._agents[f"a{i}"] = _make_record(f"a{i}", "scott", "done")
|
||||
|
||||
records = agent_manager.list_agents("scott", limit=5)
|
||||
assert len(records) == 5
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_prune_removes_old_completed(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
await agent_manager.finish(rec.agent_id, "done")
|
||||
|
||||
# Manually backdate the finished time past the prune threshold
|
||||
agent_manager._agents[rec.agent_id].finished = (
|
||||
datetime.now() - agent_manager._PRUNE_AFTER - timedelta(seconds=1)
|
||||
)
|
||||
|
||||
# Trigger pruning via a new registration
|
||||
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
|
||||
|
||||
assert agent_manager.get(rec.agent_id) is None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_prune_keeps_running_agents(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
# Running agent — finished is None so it should never be pruned
|
||||
assert rec.agent_id in agent_manager._agents
|
||||
|
||||
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
|
||||
assert agent_manager.get(rec.agent_id) is not None
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_finish_unknown_agent_is_noop(self):
|
||||
import agent_manager
|
||||
# Should not raise
|
||||
await agent_manager.finish("ghost-id", "result", "done")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# agent_manager — notification hook
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAgentManagerNotify:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_notify_called_on_done(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(
|
||||
user="scott", role="chat", task="t", level=2, notify=True
|
||||
)
|
||||
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
|
||||
await agent_manager.finish(rec.agent_id, "All good", "done")
|
||||
mock_notify.assert_called_once()
|
||||
call_args = mock_notify.call_args
|
||||
assert call_args[0][0] == "scott" # user
|
||||
assert "✅" in call_args[0][1] # success emoji
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_notify_called_on_failed(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(
|
||||
user="scott", role="chat", task="t", level=2, notify=True
|
||||
)
|
||||
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
|
||||
await agent_manager.finish(rec.agent_id, "Oops", "failed")
|
||||
mock_notify.assert_called_once()
|
||||
assert "⚠️" in mock_notify.call_args[0][1]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_notify_when_cancelled(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(
|
||||
user="scott", role="chat", task="t", level=2, notify=True
|
||||
)
|
||||
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
|
||||
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
|
||||
mock_notify.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_notify_when_flag_false(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(
|
||||
user="scott", role="chat", task="t", level=2, notify=False
|
||||
)
|
||||
with patch("notification.notify", new_callable=AsyncMock) as mock_notify:
|
||||
await agent_manager.finish(rec.agent_id, "Done", "done")
|
||||
mock_notify.assert_not_called()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# spawn_agent — background mode
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestSpawnAgentBackground:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_returns_agent_id_immediately(self):
|
||||
import agent_manager
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
mock_result = _make_mock_result("Research complete.")
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
|
||||
result = await spawn_agent(
|
||||
task="Test background research",
|
||||
role="research",
|
||||
background=True,
|
||||
)
|
||||
|
||||
assert "Agent started in background" in result
|
||||
assert "ID:" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_registers_agent(self):
|
||||
import agent_manager
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
mock_result = _make_mock_result()
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
|
||||
await spawn_agent(task="Background task", background=True)
|
||||
|
||||
agents = agent_manager.list_agents("scott")
|
||||
assert len(agents) >= 1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_agent_eventually_completes(self):
|
||||
import agent_manager
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
mock_result = _make_mock_result("Task done!")
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
|
||||
result = await spawn_agent(task="Quick task", background=True)
|
||||
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
|
||||
|
||||
# Poll while patches are still active
|
||||
for _ in range(40):
|
||||
rec = agent_manager.get(agent_id)
|
||||
if rec and rec.status != "running":
|
||||
break
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
rec = agent_manager.get(agent_id)
|
||||
assert rec is not None
|
||||
assert rec.status == "done"
|
||||
assert "Task done!" in (rec.result or "")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_sync_path_unchanged(self):
|
||||
"""Verify that background=False still blocks and returns the result string."""
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
mock_result = _make_mock_result("Sync result here.")
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", new_callable=AsyncMock, return_value=mock_result):
|
||||
result = await spawn_agent(task="Sync task", background=False)
|
||||
|
||||
assert result == "Sync result here."
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_agent_timeout(self):
|
||||
import agent_manager
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
async def _slow(*args, **kwargs):
|
||||
await asyncio.sleep(60)
|
||||
return _make_mock_result()
|
||||
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", side_effect=_slow):
|
||||
result = await spawn_agent(task="Slow task", background=True, timeout=1)
|
||||
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
|
||||
|
||||
# Poll while patches are still active (timeout=1s so this completes quickly)
|
||||
for _ in range(60):
|
||||
rec = agent_manager.get(agent_id)
|
||||
if rec and rec.status != "running":
|
||||
break
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
rec = agent_manager.get(agent_id)
|
||||
assert rec.status == "timeout"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_agent_failure(self):
|
||||
import agent_manager
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", new_callable=AsyncMock, side_effect=RuntimeError("Boom")):
|
||||
result = await spawn_agent(task="Failing task", background=True)
|
||||
|
||||
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
|
||||
|
||||
for _ in range(20):
|
||||
rec = agent_manager.get(agent_id)
|
||||
if rec and rec.status != "running":
|
||||
break
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
assert agent_manager.get(agent_id).status == "failed"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# spawn_agent — level enforcement
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestLevelEnforcement:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_l2_parent_denies_spawn_in_l3_child(self):
|
||||
"""Level 2 agent spawning a child: spawn_agent and aider_run must be denied."""
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
captured_kwargs = {}
|
||||
|
||||
async def _capture_run(**kwargs):
|
||||
captured_kwargs.update(kwargs)
|
||||
return _make_mock_result()
|
||||
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", side_effect=_capture_run):
|
||||
await spawn_agent(
|
||||
task="Test L3 enforcement",
|
||||
background=False,
|
||||
_agent_level=2, # this agent is Level 2; its child would be Level 3
|
||||
)
|
||||
|
||||
# The orchestrator should have received spawn_agent and aider_run in confirm_deny
|
||||
confirm_deny = captured_kwargs.get("confirm_deny", set())
|
||||
assert "spawn_agent" in confirm_deny, "spawn_agent must be blocked for L3 children"
|
||||
assert "aider_run" in confirm_deny, "aider_run must be blocked for L3 children"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_l1_parent_does_not_deny_spawn(self):
|
||||
"""Level 1 agent (persona) spawning a Level 2 child: no extra denies."""
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
captured_kwargs = {}
|
||||
|
||||
async def _capture_run(**kwargs):
|
||||
captured_kwargs.update(kwargs)
|
||||
return _make_mock_result()
|
||||
|
||||
with _mock_spawn_deps():
|
||||
with patch("openai_orchestrator.run", side_effect=_capture_run):
|
||||
await spawn_agent(
|
||||
task="Test L2 spawn",
|
||||
background=False,
|
||||
_agent_level=1, # persona is Level 1; child would be Level 2
|
||||
)
|
||||
|
||||
confirm_deny = captured_kwargs.get("confirm_deny", set())
|
||||
assert "spawn_agent" not in confirm_deny, "L2 agents must be allowed to spawn"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_l2_deny_intersected_with_tool_list(self):
|
||||
"""When the role has an explicit tool_list, L3 deny removes from list directly."""
|
||||
from tools.agents import spawn_agent
|
||||
|
||||
captured_kwargs = {}
|
||||
|
||||
async def _capture_run(**kwargs):
|
||||
captured_kwargs.update(kwargs)
|
||||
return _make_mock_result()
|
||||
|
||||
# Role has an explicit tool_list that includes spawn_agent
|
||||
with _mock_spawn_deps(role_tools=["web_search", "spawn_agent", "aider_run"]):
|
||||
with patch("openai_orchestrator.run", side_effect=_capture_run):
|
||||
await spawn_agent(
|
||||
task="Test",
|
||||
background=False,
|
||||
_agent_level=2,
|
||||
)
|
||||
|
||||
# spawn_agent and aider_run must be absent from the tool_list passed to orchestrator
|
||||
tool_list = captured_kwargs.get("tool_list", [])
|
||||
assert "spawn_agent" not in tool_list
|
||||
assert "aider_run" not in tool_list
|
||||
assert "web_search" in tool_list # unrelated tools must survive
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Agent lifecycle tools — output formatting
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAgentLifecycleTools:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_status_running(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="research", task="Do research", level=2)
|
||||
|
||||
with patch("persona.get_user", return_value="scott"):
|
||||
from tools.agents import agent_status
|
||||
output = await agent_status(rec.agent_id)
|
||||
|
||||
assert "running" in output
|
||||
assert "research" in output
|
||||
assert rec.agent_id[:8] in output
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_status_done(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="Task", level=2)
|
||||
await agent_manager.finish(rec.agent_id, "The result text", "done")
|
||||
|
||||
with patch("persona.get_user", return_value="scott"):
|
||||
from tools.agents import agent_status
|
||||
output = await agent_status(rec.agent_id)
|
||||
|
||||
assert "done" in output
|
||||
assert "The result text" in output
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_status_wrong_user(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
|
||||
with patch("persona.get_user", return_value="holly"):
|
||||
from tools.agents import agent_status
|
||||
output = await agent_status(rec.agent_id)
|
||||
|
||||
assert "denied" in output.lower()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_status_not_found(self):
|
||||
with patch("persona.get_user", return_value="scott"):
|
||||
from tools.agents import agent_status
|
||||
output = await agent_status("nonexistent-id")
|
||||
|
||||
assert "No agent found" in output
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_list_shows_running(self):
|
||||
import agent_manager
|
||||
await agent_manager.register(user="scott", role="research", task="Research X", level=2)
|
||||
await agent_manager.register(user="scott", role="coder", task="Fix bug", level=2)
|
||||
|
||||
with patch("persona.get_user", return_value="scott"):
|
||||
from tools.agents import agent_list
|
||||
output = await agent_list()
|
||||
|
||||
assert "2 agent(s)" in output
|
||||
assert "research" in output
|
||||
assert "coder" in output
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_list_status_filter(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
await agent_manager.finish(rec.agent_id, "done", "done")
|
||||
await agent_manager.register(user="scott", role="chat", task="t2", level=2)
|
||||
|
||||
with patch("persona.get_user", return_value="scott"):
|
||||
from tools.agents import agent_list
|
||||
output = await agent_list(status="running")
|
||||
|
||||
assert "1 agent(s)" in output
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_list_empty(self):
|
||||
with patch("persona.get_user", return_value="scott"):
|
||||
from tools.agents import agent_list
|
||||
output = await agent_list()
|
||||
|
||||
assert "No agents found" in output
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_agent_cancel_tool(self):
|
||||
import agent_manager
|
||||
rec = await agent_manager.register(user="scott", role="chat", task="t", level=2)
|
||||
|
||||
with patch("persona.get_user", return_value="scott"):
|
||||
from tools.agents import agent_cancel
|
||||
output = await agent_cancel(rec.agent_id)
|
||||
|
||||
assert "cancelled" in output
|
||||
assert agent_manager.get(rec.agent_id).status == "cancelled"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# aider_run — background mode
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAiderRunBackground:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_returns_agent_id(self):
|
||||
import agent_manager
|
||||
|
||||
async def _fake_proc(*args, **kwargs):
|
||||
mock_proc = MagicMock()
|
||||
mock_proc.communicate = AsyncMock(return_value=(b"All changes applied.", b""))
|
||||
mock_proc.returncode = 0
|
||||
return mock_proc
|
||||
|
||||
with (
|
||||
patch("persona.get_user", return_value="scott"),
|
||||
patch("model_registry.get_registry", return_value={"hosts": []}),
|
||||
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
|
||||
):
|
||||
from tools.aider import aider_run
|
||||
result = await aider_run(
|
||||
project=str(_CORTEX_DIR.parent), # use actual project root (exists)
|
||||
task="Test background task",
|
||||
background=True,
|
||||
)
|
||||
|
||||
assert "Aider task started in background" in result
|
||||
assert "ID:" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_background_agent_completes(self):
|
||||
import agent_manager
|
||||
|
||||
async def _fake_proc(*args, **kwargs):
|
||||
mock_proc = MagicMock()
|
||||
mock_proc.communicate = AsyncMock(return_value=(b"Edits applied.", b""))
|
||||
mock_proc.returncode = 0
|
||||
return mock_proc
|
||||
|
||||
from tools.aider import aider_run
|
||||
with (
|
||||
patch("persona.get_user", return_value="scott"),
|
||||
patch("model_registry.get_registry", return_value={"hosts": []}),
|
||||
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
|
||||
):
|
||||
result = await aider_run(
|
||||
project=str(_CORTEX_DIR.parent),
|
||||
task="Test",
|
||||
background=True,
|
||||
)
|
||||
agent_id = result.split("ID: ")[1].split("\n")[0].strip()
|
||||
|
||||
# Poll while patches are still active
|
||||
for _ in range(40):
|
||||
rec = agent_manager.get(agent_id)
|
||||
if rec and rec.status != "running":
|
||||
break
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
rec = agent_manager.get(agent_id)
|
||||
assert rec.status == "done"
|
||||
assert "Edits applied" in (rec.result or "")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_invalid_project_directory(self):
|
||||
from tools.aider import aider_run
|
||||
result = await aider_run(project="/this/does/not/exist", task="Test")
|
||||
assert "does not exist" in result
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sync_path_still_works(self):
|
||||
async def _fake_proc(*args, **kwargs):
|
||||
mock_proc = MagicMock()
|
||||
mock_proc.communicate = AsyncMock(return_value=(b"Done.", b""))
|
||||
mock_proc.returncode = 0
|
||||
return mock_proc
|
||||
|
||||
with (
|
||||
patch("persona.get_user", return_value="scott"),
|
||||
patch("model_registry.get_registry", return_value={"hosts": []}),
|
||||
patch("asyncio.create_subprocess_exec", side_effect=_fake_proc),
|
||||
):
|
||||
from tools.aider import aider_run
|
||||
result = await aider_run(
|
||||
project=str(_CORTEX_DIR.parent),
|
||||
task="Sync test",
|
||||
background=False,
|
||||
)
|
||||
|
||||
assert "Done." in result
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# aider_run — credential resolver (_resolve_credentials)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAiderCredentialResolver:
|
||||
"""Pure unit tests for _resolve_credentials — no subprocess, no registry I/O."""
|
||||
|
||||
def _registry(self, hosts=None, anthropic_key=None):
|
||||
reg = {"hosts": hosts or [], "providers": {}}
|
||||
if anthropic_key:
|
||||
reg["providers"]["anthropic"] = {
|
||||
"credentials": [{"api_key": anthropic_key}]
|
||||
}
|
||||
return reg
|
||||
|
||||
def _host(self, label, api_url, api_key="sk-test", host_type="openai"):
|
||||
return {"id": "x", "label": label, "api_url": api_url,
|
||||
"api_key": api_key, "host_type": host_type}
|
||||
|
||||
# --- Provider detection ---
|
||||
|
||||
def test_openrouter_host_gets_api_key_flag(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
|
||||
])
|
||||
flags, model = _resolve_credentials(reg, None, None)
|
||||
assert "--api-key" in flags
|
||||
assert "openrouter=or-key" in flags
|
||||
|
||||
def test_anthropic_model_hint_uses_provider_key(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(
|
||||
hosts=[self._host("OpenRouter", "https://openrouter.ai/api/v1")],
|
||||
anthropic_key="ant-key",
|
||||
)
|
||||
flags, model = _resolve_credentials(reg, "claude-3-5-sonnet-20241022", None)
|
||||
assert "anthropic=ant-key" in flags
|
||||
assert model == "claude-3-5-sonnet-20241022"
|
||||
|
||||
def test_anthropic_slash_prefix_hint(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(anthropic_key="ant-key")
|
||||
flags, _ = _resolve_credentials(reg, "anthropic/claude-opus-4", None)
|
||||
assert "anthropic=ant-key" in flags
|
||||
|
||||
def test_local_openwebui_host_gets_base_url(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("Local", "http://192.168.32.19:3000", "localkey", host_type="openwebui"),
|
||||
])
|
||||
flags, model = _resolve_credentials(reg, None, None)
|
||||
assert "--openai-api-base" in flags
|
||||
base = flags[flags.index("--openai-api-base") + 1]
|
||||
assert base == "http://192.168.32.19:3000/api"
|
||||
assert "--openai-api-key" in flags
|
||||
|
||||
def test_local_host_appends_api_suffix_for_openwebui(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("OpenWebUI", "http://localhost:3000", host_type="openwebui"),
|
||||
])
|
||||
flags, _ = _resolve_credentials(reg, None, None)
|
||||
base = flags[flags.index("--openai-api-base") + 1]
|
||||
assert base.endswith("/api")
|
||||
|
||||
def test_generic_openai_host_no_api_suffix(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("Custom", "http://localhost:8080/v1", host_type="openai"),
|
||||
])
|
||||
flags, _ = _resolve_credentials(reg, None, None)
|
||||
base = flags[flags.index("--openai-api-base") + 1]
|
||||
assert not base.endswith("/api")
|
||||
assert base == "http://localhost:8080/v1"
|
||||
|
||||
# --- Model name adjustment ---
|
||||
|
||||
def test_local_host_prefixes_model_without_slash(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("Local", "http://localhost:3000", host_type="openwebui"),
|
||||
])
|
||||
_, model = _resolve_credentials(reg, "gemma-4-27b-it", None)
|
||||
assert model == "openai/gemma-4-27b-it"
|
||||
|
||||
def test_local_host_leaves_model_with_slash(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("Local", "http://localhost:3000", host_type="openwebui"),
|
||||
])
|
||||
_, model = _resolve_credentials(reg, "ollama/gemma4", None)
|
||||
assert model == "ollama/gemma4" # already prefixed, don't touch
|
||||
|
||||
def test_cloud_provider_does_not_prefix_model(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("OpenRouter", "https://openrouter.ai/api/v1"),
|
||||
])
|
||||
_, model = _resolve_credentials(reg, "google/gemma-3-27b-it", None)
|
||||
assert model == "google/gemma-3-27b-it"
|
||||
|
||||
# --- Host label override ---
|
||||
|
||||
def test_host_label_selects_local_over_openrouter(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
|
||||
self._host("Local RTX", "http://192.168.32.19:3000", "local-key", host_type="openwebui"),
|
||||
])
|
||||
flags, _ = _resolve_credentials(reg, None, "Local")
|
||||
assert "--openai-api-base" in flags
|
||||
assert "--api-key" not in flags
|
||||
|
||||
def test_host_label_case_insensitive(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
|
||||
])
|
||||
flags, _ = _resolve_credentials(reg, None, "openrouter")
|
||||
assert "openrouter=or-key" in flags
|
||||
|
||||
# --- Model prefix routing ---
|
||||
|
||||
def test_model_openrouter_prefix_routes_to_openrouter(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("Local", "http://localhost:3000", host_type="openwebui"),
|
||||
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
|
||||
])
|
||||
flags, model = _resolve_credentials(reg, "openrouter/google/gemma-3-27b-it", None)
|
||||
assert "openrouter=or-key" in flags
|
||||
assert model == "openrouter/google/gemma-3-27b-it"
|
||||
|
||||
def test_model_groq_prefix_routes_to_groq_host(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("Groq", "https://api.groq.com/openai/v1", "groq-key"),
|
||||
])
|
||||
flags, _ = _resolve_credentials(reg, "groq/llama-3.3-70b", None)
|
||||
assert "groq=groq-key" in flags
|
||||
|
||||
# --- Default fallback priority ---
|
||||
|
||||
def test_prefers_openrouter_over_local_when_no_hint(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(hosts=[
|
||||
self._host("Local", "http://localhost:3000", host_type="openwebui"),
|
||||
self._host("OpenRouter", "https://openrouter.ai/api/v1", "or-key"),
|
||||
])
|
||||
flags, _ = _resolve_credentials(reg, None, None)
|
||||
assert "openrouter=or-key" in flags
|
||||
|
||||
def test_prefers_anthropic_over_local_when_no_openrouter(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
reg = self._registry(
|
||||
hosts=[self._host("Local", "http://localhost:3000", host_type="openwebui")],
|
||||
anthropic_key="ant-key",
|
||||
)
|
||||
flags, _ = _resolve_credentials(reg, None, None)
|
||||
assert "anthropic=ant-key" in flags
|
||||
|
||||
def test_empty_registry_returns_no_flags(self):
|
||||
from tools.aider import _resolve_credentials
|
||||
flags, model = _resolve_credentials({}, "gemma-4", None)
|
||||
assert flags == []
|
||||
assert model == "gemma-4"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers for manual test record creation (used in list tests above)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
import agent_manager as _am
|
||||
|
||||
_CORTEX_DIR = _am.__file__ and _am and __import__("pathlib").Path(_am.__file__).parent
|
||||
|
||||
|
||||
def _make_record(agent_id: str, user: str, status: str) -> "_am.AgentRecord":
|
||||
from datetime import datetime
|
||||
import agent_manager
|
||||
rec = agent_manager.AgentRecord(
|
||||
agent_id=agent_id,
|
||||
level=2,
|
||||
role="chat",
|
||||
task="test task",
|
||||
status=status,
|
||||
started=datetime.now(),
|
||||
user=user,
|
||||
finished=datetime.now() if status != "running" else None,
|
||||
)
|
||||
return rec
|
||||
@@ -25,7 +25,10 @@ async def test_files_get_allowed(client):
|
||||
@pytest.mark.anyio
|
||||
async def test_files_get_not_in_allowed(client):
|
||||
"""Files outside the ALLOWED set should return 404, not the file content."""
|
||||
for name in ("TASKS.json", "CRONS.json", "SCRATCH.md", "../config.py", ".env"):
|
||||
# Note: paths with '..' are normalized at the ASGI layer (e.g. /files/../config.py
|
||||
# becomes /config.py which hits the /{username} UI catch-all, not the files router).
|
||||
# Only test paths that stay within the files router's scope.
|
||||
for name in ("TASKS.json", "CRONS.json", "SCRATCH.md", ".env"):
|
||||
r = await client.get(f"/files/{name}")
|
||||
assert r.status_code == 404, f"Expected 404 for {name}, got {r.status_code}"
|
||||
|
||||
|
||||
@@ -30,5 +30,7 @@ async def test_distill_status(client):
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_unknown_route_404(client):
|
||||
r = await client.get("/does-not-exist")
|
||||
# Single-segment paths hit the /{username} persona-picker catch-all (302 redirect).
|
||||
# Three-segment paths don't match any route pattern → genuine 404.
|
||||
r = await client.get("/totally/unknown/deep-path")
|
||||
assert r.status_code == 404
|
||||
|
||||
805
cortex/tests/test_model_registry.py
Normal file
805
cortex/tests/test_model_registry.py
Normal file
@@ -0,0 +1,805 @@
|
||||
"""
|
||||
Unit tests for model_registry.py — no HTTP, no LLM calls, no running service.
|
||||
|
||||
All file I/O is redirected to tmp_path via patch.object(config.settings, "home_dir", ...).
|
||||
|
||||
Coverage:
|
||||
- Empty registry (no files)
|
||||
- Save/load round-trip
|
||||
- Migration from local_llm.json (v0 flat and v1 hosts/models)
|
||||
- Host CRUD
|
||||
- Model CRUD (including role reference cleanup on remove)
|
||||
- Role assignment (set_role, validation)
|
||||
- Model resolution (_resolve_model: built-ins, local_openai, missing host/model)
|
||||
- get_model_for_role: registry chain → .env fallback → hardcoded fallback
|
||||
- get_best_local_model: role chain, first-local fallback, no-local case
|
||||
- Backup chain: skips missing models, returns next valid
|
||||
"""
|
||||
|
||||
import json
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _home(tmp_path: Path) -> Path:
|
||||
"""Create a minimal home directory and return the root."""
|
||||
root = tmp_path / "home"
|
||||
root.mkdir()
|
||||
return root
|
||||
|
||||
|
||||
def _user_dir(home: Path, username: str = "scott") -> Path:
|
||||
d = home / username
|
||||
d.mkdir(exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def _write_registry(home: Path, data: dict, username: str = "scott") -> Path:
|
||||
_user_dir(home, username)
|
||||
path = home / username / "model_registry.json"
|
||||
path.write_text(json.dumps(data))
|
||||
return path
|
||||
|
||||
|
||||
def _write_local_llm(home: Path, data: dict, username: str = "scott") -> Path:
|
||||
_user_dir(home, username)
|
||||
path = home / username / "local_llm.json"
|
||||
path.write_text(json.dumps(data))
|
||||
return path
|
||||
|
||||
|
||||
def _read_registry(home: Path, username: str = "scott") -> dict:
|
||||
path = home / username / "model_registry.json"
|
||||
return json.loads(path.read_text())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Empty / fresh state
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_empty_registry_no_files(tmp_path):
|
||||
"""With no files, _load returns an empty structure."""
|
||||
home = _home(tmp_path)
|
||||
_user_dir(home)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
data = reg._load("scott")
|
||||
assert data["version"] == 2
|
||||
assert data["hosts"] == []
|
||||
assert data["models"] == []
|
||||
assert data["roles"] == {}
|
||||
|
||||
|
||||
def test_empty_registry_missing_user_dir(tmp_path):
|
||||
"""Even with no user dir, _load returns an empty structure gracefully."""
|
||||
home = _home(tmp_path)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
data = reg._load("nobody")
|
||||
assert data["hosts"] == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Save / load round-trip
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_save_and_load(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_user_dir(home)
|
||||
import config
|
||||
import model_registry as reg
|
||||
|
||||
registry = {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "ML Box", "api_url": "http://10.0.0.1:3000", "api_key": "sk-test"}],
|
||||
"models": [{"id": "m1", "type": "local_openai", "label": "Gemma Small",
|
||||
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": ["fast"]}],
|
||||
"roles": {"chat": {"primary": "m1"}},
|
||||
}
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
reg._save("scott", registry)
|
||||
loaded = reg._load("scott")
|
||||
|
||||
assert loaded["hosts"][0]["label"] == "ML Box"
|
||||
assert loaded["models"][0]["model_name"] == "gemma4:e4b"
|
||||
assert loaded["roles"]["chat"]["primary"] == "m1"
|
||||
|
||||
|
||||
def test_corrupt_registry_falls_back_to_empty(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
path = _user_dir(home) / "model_registry.json"
|
||||
path.write_text("{bad json{{")
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
data = reg._load("scott")
|
||||
assert data["hosts"] == []
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Migration from local_llm.json
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_migrate_v1_hosts_models(tmp_path):
|
||||
"""v1 local_llm.json (hosts/models/active_model_id) migrates correctly."""
|
||||
home = _home(tmp_path)
|
||||
_write_local_llm(home, {
|
||||
"hosts": [{"id": "h1", "label": "Home", "api_url": "http://10.0.0.1:3000", "api_key": "sk-1"}],
|
||||
"models": [
|
||||
{"id": "m1", "host_id": "h1", "label": "Gemma Small", "model_name": "gemma4:e4b"},
|
||||
{"id": "m2", "host_id": "h1", "label": "Gemma Med", "model_name": "gemma4:26b"},
|
||||
],
|
||||
"active_model_id": "m1",
|
||||
})
|
||||
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
data = reg._load("scott")
|
||||
|
||||
assert len(data["hosts"]) == 1
|
||||
assert data["hosts"][0]["api_url"] == "http://10.0.0.1:3000"
|
||||
assert len(data["models"]) == 2
|
||||
assert all(m["type"] == "local_openai" for m in data["models"])
|
||||
# active_model_id → roles.chat.primary
|
||||
assert data["roles"].get("chat", {}).get("primary") == "m1"
|
||||
|
||||
|
||||
def test_migrate_v1_no_active_model(tmp_path):
|
||||
"""Migration with active_model_id=null: chat role stays unset."""
|
||||
home = _home(tmp_path)
|
||||
_write_local_llm(home, {
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m1", "host_id": "h1", "label": "Model", "model_name": "llama3"}],
|
||||
"active_model_id": None,
|
||||
})
|
||||
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
data = reg._load("scott")
|
||||
|
||||
assert "chat" not in data["roles"] or data["roles"]["chat"].get("primary") is None
|
||||
|
||||
|
||||
def test_migrate_v0_flat_format(tmp_path):
|
||||
"""v0 flat local_llm.json is wrapped into hosts/models structure."""
|
||||
home = _home(tmp_path)
|
||||
_write_local_llm(home, {
|
||||
"api_url": "http://10.0.0.2:3000",
|
||||
"api_key": "sk-flat",
|
||||
"model": "qwen3:8b",
|
||||
})
|
||||
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
data = reg._load("scott")
|
||||
|
||||
assert len(data["hosts"]) == 1
|
||||
assert data["hosts"][0]["api_url"] == "http://10.0.0.2:3000"
|
||||
assert len(data["models"]) == 1
|
||||
assert data["models"][0]["model_name"] == "qwen3:8b"
|
||||
|
||||
|
||||
def test_migrate_v0_empty_url_returns_empty(tmp_path):
|
||||
"""v0 with no api_url and no .env fallback → nothing to migrate, empty registry."""
|
||||
home = _home(tmp_path)
|
||||
_write_local_llm(home, {"api_url": "", "api_key": "", "model": ""})
|
||||
|
||||
import config
|
||||
import model_registry as reg
|
||||
with (
|
||||
patch.object(config.settings, "home_dir", home),
|
||||
patch.object(config.settings, "local_api_url", ""), # ensure no .env fallback
|
||||
patch.object(config.settings, "local_model", ""),
|
||||
):
|
||||
data = reg._load("scott")
|
||||
|
||||
assert data["hosts"] == []
|
||||
assert data["models"] == []
|
||||
|
||||
|
||||
def test_migrate_v1_distill_local_sets_role(tmp_path):
|
||||
"""When DISTILL_BACKEND_MID=local and active model exists, distill role is set."""
|
||||
home = _home(tmp_path)
|
||||
_write_local_llm(home, {
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m1", "host_id": "h1", "label": "G", "model_name": "gemma4:e4b"}],
|
||||
"active_model_id": "m1",
|
||||
})
|
||||
|
||||
import config
|
||||
import model_registry as reg
|
||||
with (
|
||||
patch.object(config.settings, "home_dir", home),
|
||||
patch.object(config.settings, "distill_backend_mid", "local"),
|
||||
):
|
||||
data = reg._load("scott")
|
||||
|
||||
assert data["roles"].get("distill", {}).get("primary") == "m1"
|
||||
|
||||
|
||||
def test_migration_saves_registry_file(tmp_path):
|
||||
"""After migration, model_registry.json is written so next load skips migration."""
|
||||
home = _home(tmp_path)
|
||||
_write_local_llm(home, {
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [],
|
||||
"active_model_id": None,
|
||||
})
|
||||
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
reg._load("scott") # triggers migration + save
|
||||
# Second load should read model_registry.json, not re-run migration
|
||||
data2 = reg._load("scott")
|
||||
|
||||
assert (home / "scott" / "model_registry.json").exists()
|
||||
assert data2["version"] == 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Built-in model resolution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_builtin_claude_cli(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_user_dir(home)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg._resolve_model(reg._empty(), "claude_cli")
|
||||
assert result is not None
|
||||
assert result["type"] == "claude_cli"
|
||||
assert result["id"] == "claude_cli"
|
||||
|
||||
|
||||
def test_builtin_gemini_api(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_user_dir(home)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg._resolve_model(reg._empty(), "gemini_api")
|
||||
assert result["type"] == "gemini_api"
|
||||
|
||||
|
||||
def test_builtin_gemini_cli(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg._resolve_model(reg._empty(), "gemini_cli")
|
||||
assert result["type"] == "gemini_cli"
|
||||
|
||||
|
||||
def test_builtin_unknown_returns_none(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg._resolve_model(reg._empty(), "does_not_exist")
|
||||
assert result is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# User model resolution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_resolve_local_openai_merges_host(tmp_path):
|
||||
"""local_openai model gets api_url and api_key merged from its host."""
|
||||
home = _home(tmp_path)
|
||||
registry = {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": "sk-test"}],
|
||||
"models": [{"id": "m1", "type": "local_openai", "label": "G", "model_name": "gemma4:e4b",
|
||||
"host_id": "h1", "context_k": 72, "tags": []}],
|
||||
"roles": {},
|
||||
}
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg._resolve_model(registry, "m1")
|
||||
assert result["api_url"] == "http://10.0.0.1:3000"
|
||||
assert result["api_key"] == "sk-test"
|
||||
assert result["model_name"] == "gemma4:e4b"
|
||||
|
||||
|
||||
def test_resolve_local_openai_missing_host_returns_none(tmp_path):
|
||||
"""A model pointing to a non-existent host_id returns None."""
|
||||
home = _home(tmp_path)
|
||||
registry = {
|
||||
"version": 1, "hosts": [], "roles": {},
|
||||
"models": [{"id": "m1", "type": "local_openai", "host_id": "missing",
|
||||
"label": "X", "model_name": "x", "context_k": 0, "tags": []}],
|
||||
}
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg._resolve_model(registry, "m1")
|
||||
assert result is None
|
||||
|
||||
|
||||
def test_resolve_unknown_model_id_returns_none(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg._resolve_model(reg._empty(), "no_such_model")
|
||||
assert result is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# get_model_for_role
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_get_model_for_role_uses_registry(tmp_path):
|
||||
"""Registry primary assignment is returned first."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m1", "type": "local_openai", "label": "G",
|
||||
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
|
||||
"roles": {"chat": {"primary": "m1"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg.get_model_for_role("scott", "chat")
|
||||
assert result["model_name"] == "gemma4:e4b"
|
||||
assert result["api_url"] == "http://10.0.0.1:3000"
|
||||
|
||||
|
||||
def test_get_model_for_role_uses_builtin_from_registry(tmp_path):
|
||||
"""Registry can assign built-in IDs (claude_cli, gemini_api, etc.)."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1, "hosts": [], "models": [],
|
||||
"roles": {"chat": {"primary": "claude_cli"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg.get_model_for_role("scott", "chat")
|
||||
assert result["type"] == "claude_cli"
|
||||
|
||||
|
||||
def test_get_model_for_role_skips_missing_primary(tmp_path):
|
||||
"""If primary model_id is not found, falls through to backup_1."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m2", "type": "local_openai", "label": "Backup",
|
||||
"model_name": "llama3:8b", "host_id": "h1", "context_k": 8, "tags": []}],
|
||||
"roles": {"chat": {"primary": "gone", "backup_1": "m2"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg.get_model_for_role("scott", "chat")
|
||||
assert result["model_name"] == "llama3:8b"
|
||||
|
||||
|
||||
def test_get_model_for_role_env_fallback(tmp_path):
|
||||
"""No registry entry for role → falls back to .env setting."""
|
||||
home = _home(tmp_path)
|
||||
_user_dir(home)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with (
|
||||
patch.object(config.settings, "home_dir", home),
|
||||
patch.object(config.settings, "role_chat", "gemini_cli"),
|
||||
):
|
||||
result = reg.get_model_for_role("scott", "chat")
|
||||
assert result["type"] == "gemini_cli"
|
||||
|
||||
|
||||
def test_get_model_for_role_hardcoded_fallback(tmp_path):
|
||||
"""No registry + no .env for role → hardcoded last resort."""
|
||||
home = _home(tmp_path)
|
||||
_user_dir(home)
|
||||
import config
|
||||
import model_registry as reg
|
||||
# Clear the .env default for 'chat' to simulate unset
|
||||
with (
|
||||
patch.object(config.settings, "home_dir", home),
|
||||
patch.object(config.settings, "role_chat", ""),
|
||||
):
|
||||
result = reg.get_model_for_role("scott", "chat")
|
||||
# claude_cli is the hardcoded last resort for 'chat'
|
||||
assert result["type"] == "claude_cli"
|
||||
|
||||
|
||||
def test_get_model_for_role_custom_role(tmp_path):
|
||||
"""Custom roles not in DEFINED_ROLES can still be assigned and resolved."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1, "hosts": [], "models": [],
|
||||
"roles": {"therapy": {"primary": "gemini_api"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg.get_model_for_role("scott", "therapy")
|
||||
assert result["type"] == "gemini_api"
|
||||
|
||||
|
||||
def test_get_model_for_role_full_backup_chain(tmp_path):
|
||||
"""Walks the entire priority chain before falling back."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m4", "type": "local_openai", "label": "Last",
|
||||
"model_name": "tiny:1b", "host_id": "h1", "context_k": 4, "tags": []}],
|
||||
"roles": {"chat": {
|
||||
"primary": "gone1",
|
||||
"backup_1": "gone2",
|
||||
"backup_2": "gone3",
|
||||
"backup_3": "gone4",
|
||||
"backup_4": "m4",
|
||||
}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg.get_model_for_role("scott", "chat")
|
||||
assert result["model_name"] == "tiny:1b"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# get_best_local_model
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_get_best_local_prefers_role_chain(tmp_path):
|
||||
"""Returns the first local_openai model in the chat role chain."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [
|
||||
{"id": "m1", "type": "local_openai", "label": "Preferred",
|
||||
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []},
|
||||
],
|
||||
"roles": {"chat": {"primary": "claude_cli", "backup_1": "m1"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
# primary is claude_cli (not local), backup_1 is m1 (local)
|
||||
result = reg.get_best_local_model("scott", "chat")
|
||||
assert result["model_name"] == "gemma4:e4b"
|
||||
|
||||
|
||||
def test_get_best_local_falls_back_to_first_model(tmp_path):
|
||||
"""No local model in role chain → returns first configured local model."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [
|
||||
{"id": "m1", "type": "local_openai", "label": "G",
|
||||
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []},
|
||||
],
|
||||
"roles": {}, # no chat role assigned
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg.get_best_local_model("scott", "chat")
|
||||
assert result["model_name"] == "gemma4:e4b"
|
||||
|
||||
|
||||
def test_get_best_local_returns_none_when_no_local_models(tmp_path):
|
||||
"""No local_openai models configured → returns None."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1, "hosts": [], "models": [],
|
||||
"roles": {"chat": {"primary": "claude_cli"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
result = reg.get_best_local_model("scott", "chat")
|
||||
assert result is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Host CRUD
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_save_host_creates_new(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_user_dir(home)
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
host_id = reg.save_host("scott", None, "ML Box", "http://10.0.0.1:3000", "sk-abc")
|
||||
data = reg._load("scott")
|
||||
assert len(data["hosts"]) == 1
|
||||
assert data["hosts"][0]["id"] == host_id
|
||||
assert data["hosts"][0]["label"] == "ML Box"
|
||||
assert data["hosts"][0]["api_key"] == "sk-abc"
|
||||
|
||||
|
||||
def test_save_host_updates_existing(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Old Label", "api_url": "http://10.0.0.1:3000", "api_key": "sk-old"}],
|
||||
"models": [], "roles": {},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
reg.save_host("scott", "h1", "New Label", "http://10.0.0.2:3000", "")
|
||||
data = reg._load("scott")
|
||||
assert len(data["hosts"]) == 1
|
||||
assert data["hosts"][0]["label"] == "New Label"
|
||||
assert data["hosts"][0]["api_url"] == "http://10.0.0.2:3000"
|
||||
# Empty api_key → existing key preserved
|
||||
assert data["hosts"][0]["api_key"] == "sk-old"
|
||||
|
||||
|
||||
def test_save_host_unknown_id_creates_new(tmp_path):
|
||||
"""Passing a host_id that doesn't exist creates a new host instead of crashing."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
reg.save_host("scott", "ghost-id", "New", "http://10.0.0.3:3000", "")
|
||||
data = reg._load("scott")
|
||||
assert len(data["hosts"]) == 1
|
||||
|
||||
|
||||
def test_remove_host_also_removes_models(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m1", "type": "local_openai", "host_id": "h1",
|
||||
"label": "G", "model_name": "gemma4:e4b", "context_k": 72, "tags": []}],
|
||||
"roles": {"chat": {"primary": "m1"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
found = reg.remove_host("scott", "h1")
|
||||
data = reg._load("scott")
|
||||
assert found is True
|
||||
assert data["hosts"] == []
|
||||
assert data["models"] == []
|
||||
|
||||
|
||||
def test_remove_host_not_found_returns_false(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
found = reg.remove_host("scott", "nope")
|
||||
assert found is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Model CRUD
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_save_model_creates(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [], "roles": {},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
model_id = reg.save_model("scott", None, "h1", "Gemma Small", "gemma4:e4b", 72, ["fast", "distill"])
|
||||
data = reg._load("scott")
|
||||
assert len(data["models"]) == 1
|
||||
assert data["models"][0]["id"] == model_id
|
||||
assert data["models"][0]["context_k"] == 72
|
||||
assert data["models"][0]["tags"] == ["fast", "distill"]
|
||||
assert data["models"][0]["type"] == "local_openai"
|
||||
|
||||
|
||||
def test_save_model_updates_existing(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m1", "type": "local_openai", "label": "Old",
|
||||
"model_name": "llama3", "host_id": "h1", "context_k": 8, "tags": []}],
|
||||
"roles": {},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
reg.save_model("scott", "m1", "h1", "New Label", "llama3:latest", 128, ["updated"])
|
||||
data = reg._load("scott")
|
||||
assert len(data["models"]) == 1
|
||||
assert data["models"][0]["label"] == "New Label"
|
||||
assert data["models"][0]["context_k"] == 128
|
||||
|
||||
|
||||
def test_remove_model_clears_role_refs(tmp_path):
|
||||
"""Removing a model clears it from any role assignments."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m1", "type": "local_openai", "label": "G",
|
||||
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
|
||||
"roles": {
|
||||
"chat": {"primary": "m1", "backup_1": "m1"},
|
||||
"distill": {"primary": "claude_cli", "backup_1": "m1"},
|
||||
},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
found = reg.remove_model("scott", "m1")
|
||||
data = reg._load("scott")
|
||||
assert found is True
|
||||
assert data["models"] == []
|
||||
assert data["roles"]["chat"].get("primary") is None
|
||||
assert data["roles"]["chat"].get("backup_1") is None
|
||||
assert data["roles"]["distill"].get("backup_1") is None
|
||||
# claude_cli assignment preserved
|
||||
assert data["roles"]["distill"]["primary"] == "claude_cli"
|
||||
|
||||
|
||||
def test_remove_model_not_found_returns_false(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
found = reg.remove_model("scott", "ghost")
|
||||
assert found is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# set_role
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_set_role_assigns_model(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1,
|
||||
"hosts": [{"id": "h1", "label": "Box", "api_url": "http://10.0.0.1:3000", "api_key": ""}],
|
||||
"models": [{"id": "m1", "type": "local_openai", "label": "G",
|
||||
"model_name": "gemma4:e4b", "host_id": "h1", "context_k": 72, "tags": []}],
|
||||
"roles": {},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
ok = reg.set_role("scott", "chat", "primary", "m1")
|
||||
data = reg._load("scott")
|
||||
assert ok is True
|
||||
assert data["roles"]["chat"]["primary"] == "m1"
|
||||
|
||||
|
||||
def test_set_role_assigns_builtin(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
ok = reg.set_role("scott", "orchestrator", "primary", "gemini_api")
|
||||
data = reg._load("scott")
|
||||
assert ok is True
|
||||
assert data["roles"]["orchestrator"]["primary"] == "gemini_api"
|
||||
|
||||
|
||||
def test_set_role_clears_with_none(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1, "hosts": [], "models": [],
|
||||
"roles": {"chat": {"primary": "claude_cli"}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
ok = reg.set_role("scott", "chat", "primary", None)
|
||||
data = reg._load("scott")
|
||||
assert ok is True
|
||||
assert data["roles"]["chat"]["primary"] is None
|
||||
|
||||
|
||||
def test_set_role_invalid_slot_returns_false(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
ok = reg.set_role("scott", "chat", "backup_99", "claude_cli")
|
||||
assert ok is False
|
||||
|
||||
|
||||
def test_set_role_unknown_model_id_returns_false(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
ok = reg.set_role("scott", "chat", "primary", "nonexistent_model")
|
||||
assert ok is False
|
||||
|
||||
|
||||
def test_set_role_creates_role_key_if_missing(tmp_path):
|
||||
"""set_role on a role that isn't in roles{} yet creates it."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
reg.set_role("scott", "medical", "primary", "claude_cli")
|
||||
data = reg._load("scott")
|
||||
assert data["roles"]["medical"]["primary"] == "claude_cli"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# get_defined_roles
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_get_defined_roles_returns_registry_roles(tmp_path):
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {
|
||||
"version": 1, "hosts": [], "models": [],
|
||||
"roles": {"chat": {"primary": "claude_cli"}, "distill": {}},
|
||||
})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
roles = reg.get_defined_roles("scott")
|
||||
# Should include all settings.defined_roles, filling gaps with {}
|
||||
for role in config.settings.get_defined_roles():
|
||||
assert role in roles
|
||||
|
||||
|
||||
def test_get_defined_roles_fills_gaps(tmp_path):
|
||||
"""Roles in settings.defined_roles that aren't in registry get empty dicts."""
|
||||
home = _home(tmp_path)
|
||||
_write_registry(home, {"version": 1, "hosts": [], "models": [], "roles": {}})
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
roles = reg.get_defined_roles("scott")
|
||||
assert "chat" in roles
|
||||
assert roles["chat"] == {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Multi-user isolation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_registries_are_isolated_per_user(tmp_path):
|
||||
"""Each user has their own registry file — changes don't bleed across users."""
|
||||
home = _home(tmp_path)
|
||||
(home / "scott").mkdir()
|
||||
(home / "holly").mkdir()
|
||||
|
||||
import config
|
||||
import model_registry as reg
|
||||
with patch.object(config.settings, "home_dir", home):
|
||||
reg.save_host("scott", None, "Scott Host", "http://10.0.0.1:3000", "")
|
||||
scott_data = reg._load("scott")
|
||||
holly_data = reg._load("holly")
|
||||
|
||||
assert len(scott_data["hosts"]) == 1
|
||||
assert holly_data["hosts"] == []
|
||||
@@ -69,10 +69,11 @@ async def test_nct_replayed_request_rejected(client):
|
||||
payload = json.dumps({"type": "Create", "actor": {}, "object": {}, "target": {}}).encode()
|
||||
# Use wrong secret to generate sig
|
||||
wrong_sig = hmac_lib.new(b"wrong-secret", b"abc123" + payload, hashlib.sha256).hexdigest()
|
||||
_channels = {"nextcloud": {"bot_secret": "correct-secret", "url": "https://nc.example.com"}}
|
||||
from unittest.mock import patch
|
||||
with patch("config.settings.nextcloud_talk_bot_secret", "correct-secret"):
|
||||
with patch("routers.nextcloud_talk.get_user_channels", return_value=_channels):
|
||||
r = await client.post(
|
||||
"/inara-nextcloud-talk-webhook",
|
||||
"/webhook/nextcloud/scott",
|
||||
content=payload,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
@@ -118,9 +119,11 @@ async def test_known_gap__gchat_no_audience_bypass(client, mock_llm):
|
||||
LLM responses without a valid token.
|
||||
Fix: make audience required; fail loudly if not set.
|
||||
"""
|
||||
# Channel config with no audience — JWT check is skipped (the known gap).
|
||||
_channels = {"google_chat": {"persona": "inara"}}
|
||||
from unittest.mock import patch
|
||||
with patch("config.settings.google_chat_audience", ""):
|
||||
r = await client.post("/channels/google-chat", json={
|
||||
with patch("routers.google_chat.get_user_channels", return_value=_channels):
|
||||
r = await client.post("/channels/google-chat/scott", json={
|
||||
"chat": {
|
||||
"messagePayload": {
|
||||
"message": {"text": "Exploit"},
|
||||
|
||||
@@ -101,19 +101,19 @@ class TestTasks:
|
||||
|
||||
def test_list_empty(self):
|
||||
from tools.tasks import _task_list
|
||||
assert "No tasks" in _task_list(status=None)
|
||||
assert "No tasks" in _task_list(status=None, priority=None)
|
||||
|
||||
def test_create_and_list(self):
|
||||
from tools.tasks import _task_list
|
||||
self._mk("Buy coffee", description="Dark roast", priority="high")
|
||||
result = _task_list(status=None)
|
||||
result = _task_list(status=None, priority=None)
|
||||
assert "Buy coffee" in result
|
||||
assert "[high]" in result
|
||||
|
||||
def test_create_bad_priority_defaults_to_normal(self):
|
||||
from tools.tasks import _task_list
|
||||
self._mk("Test task", priority="urgent") # invalid — becomes "normal"
|
||||
result = _task_list(status=None)
|
||||
result = _task_list(status=None, priority=None)
|
||||
assert "Test task" in result
|
||||
assert "[normal]" not in result # normal priority not shown in brackets
|
||||
|
||||
@@ -121,20 +121,20 @@ class TestTasks:
|
||||
from tools.tasks import _task_update, _task_list
|
||||
tid = self._id(self._mk("Work item"))
|
||||
_task_update(tid, status="in_progress", title=None, description=None, priority=None)
|
||||
assert "Work item" in _task_list(status="in_progress")
|
||||
assert "Work item" in _task_list(status="in_progress", priority=None)
|
||||
|
||||
def test_complete(self):
|
||||
from tools.tasks import _task_complete, _task_list
|
||||
tid = self._id(self._mk("Finish this"))
|
||||
_task_complete(tid)
|
||||
assert "Finish this" in _task_list(status="done")
|
||||
assert "Finish this" not in _task_list(status="todo")
|
||||
assert "Finish this" in _task_list(status="done", priority=None)
|
||||
assert "Finish this" not in _task_list(status="todo", priority=None)
|
||||
|
||||
def test_filter_by_status(self):
|
||||
from tools.tasks import _task_list
|
||||
self._mk("A task")
|
||||
assert "A task" in _task_list(status="todo")
|
||||
assert "A task" not in _task_list(status="done")
|
||||
assert "A task" in _task_list(status="todo", priority=None)
|
||||
assert "A task" not in _task_list(status="done", priority=None)
|
||||
|
||||
def test_update_unknown_id(self):
|
||||
from tools.tasks import _task_update
|
||||
@@ -231,7 +231,8 @@ class TestCronTools:
|
||||
|
||||
def _extract_id(self, result: str) -> str:
|
||||
import re
|
||||
m = re.search(r'c_\w+', result)
|
||||
# token_urlsafe can include '-'; use [\w-]+ to capture the full ID
|
||||
m = re.search(r'c_[\w-]+', result)
|
||||
assert m, f"No cron ID in: {result}"
|
||||
return m.group()
|
||||
|
||||
|
||||
@@ -2,6 +2,10 @@
|
||||
Webhook auth tests — NC Talk HMAC, Google Chat JWT.
|
||||
|
||||
These tests verify that auth is enforced, not that full LLM responses work.
|
||||
|
||||
Architecture note: channel config (secrets, audience) lives in per-user channels.json,
|
||||
not in settings. Tests mock get_user_channels() rather than patching settings fields.
|
||||
Endpoints are per-user: /webhook/nextcloud/{username} and /channels/google-chat/{username}.
|
||||
"""
|
||||
import hashlib
|
||||
import hmac
|
||||
@@ -26,6 +30,14 @@ _VALID_NC_PAYLOAD = {
|
||||
"target": {"id": "abc123token"},
|
||||
}
|
||||
|
||||
_NCT_CHANNELS = {
|
||||
"nextcloud": {
|
||||
"bot_secret": _NC_SECRET,
|
||||
"notification_room": "abc123token",
|
||||
"url": "https://nc.example.com",
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def _nc_headers(body: bytes, secret: str) -> dict:
|
||||
random_str = "abc123"
|
||||
@@ -43,11 +55,11 @@ def _nc_headers(body: bytes, secret: str) -> dict:
|
||||
@pytest.mark.anyio
|
||||
async def test_nct_valid_signature(client, mock_llm):
|
||||
body = json.dumps(_VALID_NC_PAYLOAD).encode()
|
||||
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
|
||||
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
|
||||
with patch("routers.nextcloud_talk._send_reply", new_callable=AsyncMock):
|
||||
headers = _nc_headers(body, _NC_SECRET)
|
||||
r = await client.post(
|
||||
"/inara-nextcloud-talk-webhook",
|
||||
"/webhook/nextcloud/scott",
|
||||
content=body,
|
||||
headers={**headers, "Content-Type": "application/json"},
|
||||
)
|
||||
@@ -57,9 +69,9 @@ async def test_nct_valid_signature(client, mock_llm):
|
||||
@pytest.mark.anyio
|
||||
async def test_nct_wrong_signature(client):
|
||||
body = json.dumps(_VALID_NC_PAYLOAD).encode()
|
||||
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
|
||||
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
|
||||
r = await client.post(
|
||||
"/inara-nextcloud-talk-webhook",
|
||||
"/webhook/nextcloud/scott",
|
||||
content=body,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
@@ -73,9 +85,9 @@ async def test_nct_wrong_signature(client):
|
||||
@pytest.mark.anyio
|
||||
async def test_nct_missing_signature(client):
|
||||
body = json.dumps(_VALID_NC_PAYLOAD).encode()
|
||||
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
|
||||
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
|
||||
r = await client.post(
|
||||
"/inara-nextcloud-talk-webhook",
|
||||
"/webhook/nextcloud/scott",
|
||||
content=body,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
@@ -84,11 +96,13 @@ async def test_nct_missing_signature(client):
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_nct_no_secret_configured(client):
|
||||
"""Service should return 500 if secret is not set, not process the message."""
|
||||
"""Service should return 500 if bot_secret is missing, not process the message."""
|
||||
body = json.dumps(_VALID_NC_PAYLOAD).encode()
|
||||
with patch("config.settings.nextcloud_talk_bot_secret", ""):
|
||||
# cfg must be non-empty (truthy) to get past the 404 guard; missing bot_secret → 500
|
||||
empty_cfg = {"nextcloud": {"url": "https://nc.example.com"}}
|
||||
with patch("routers.nextcloud_talk.get_user_channels", return_value=empty_cfg):
|
||||
r = await client.post(
|
||||
"/inara-nextcloud-talk-webhook",
|
||||
"/webhook/nextcloud/scott",
|
||||
content=body,
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
@@ -100,10 +114,10 @@ async def test_nct_bot_message_ignored(client):
|
||||
"""Messages from other bots should be silently ignored (not processed)."""
|
||||
payload = {**_VALID_NC_PAYLOAD, "actor": {"type": "bots", "id": "otherbot", "name": "Bot"}}
|
||||
body = json.dumps(payload).encode()
|
||||
with patch("config.settings.nextcloud_talk_bot_secret", _NC_SECRET):
|
||||
with patch("routers.nextcloud_talk.get_user_channels", return_value=_NCT_CHANNELS):
|
||||
headers = _nc_headers(body, _NC_SECRET)
|
||||
r = await client.post(
|
||||
"/inara-nextcloud-talk-webhook",
|
||||
"/webhook/nextcloud/scott",
|
||||
content=body,
|
||||
headers={**headers, "Content-Type": "application/json"},
|
||||
)
|
||||
@@ -124,21 +138,29 @@ _GCHAT_PAYLOAD = {
|
||||
}
|
||||
}
|
||||
|
||||
_GCHAT_CHANNELS_NO_AUDIENCE = {
|
||||
# cfg must be non-empty (truthy) to pass the 404 guard; no audience → JWT skipped
|
||||
"google_chat": {"persona": "inara"}
|
||||
}
|
||||
|
||||
_GCHAT_CHANNELS_WITH_AUDIENCE = {
|
||||
"google_chat": {"audience": "123456789"}
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_gchat_no_audience_configured(client, mock_llm):
|
||||
"""When audience is not set, JWT check is skipped (current behaviour — documented bypass)."""
|
||||
with patch("config.settings.google_chat_audience", ""):
|
||||
r = await client.post("/channels/google-chat", json=_GCHAT_PAYLOAD)
|
||||
# Should process the message (no auth enforcement when audience is empty)
|
||||
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_NO_AUDIENCE):
|
||||
r = await client.post("/channels/google-chat/scott", json=_GCHAT_PAYLOAD)
|
||||
assert r.status_code == 200
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_gchat_missing_token_with_audience(client):
|
||||
"""When audience IS configured, requests without a token must be rejected."""
|
||||
with patch("config.settings.google_chat_audience", "123456789"):
|
||||
r = await client.post("/channels/google-chat", json=_GCHAT_PAYLOAD)
|
||||
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_WITH_AUDIENCE):
|
||||
r = await client.post("/channels/google-chat/scott", json=_GCHAT_PAYLOAD)
|
||||
assert r.status_code == 401
|
||||
|
||||
|
||||
@@ -149,8 +171,8 @@ async def test_gchat_invalid_token_with_audience(client):
|
||||
**_GCHAT_PAYLOAD,
|
||||
"authorizationEventObject": {"systemIdToken": "not.a.valid.jwt"},
|
||||
}
|
||||
with patch("config.settings.google_chat_audience", "123456789"):
|
||||
r = await client.post("/channels/google-chat", json=payload_with_token)
|
||||
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_WITH_AUDIENCE):
|
||||
r = await client.post("/channels/google-chat/scott", json=payload_with_token)
|
||||
assert r.status_code == 401
|
||||
|
||||
|
||||
@@ -158,7 +180,7 @@ async def test_gchat_invalid_token_with_audience(client):
|
||||
async def test_gchat_added_to_space(client, mock_llm):
|
||||
"""Bot added to a space — should return a greeting, no auth when audience empty."""
|
||||
payload = {"chat": {"addedToSpacePayload": {"space": {"type": "ROOM"}}}}
|
||||
with patch("config.settings.google_chat_audience", ""):
|
||||
r = await client.post("/channels/google-chat", json=payload)
|
||||
with patch("routers.google_chat.get_user_channels", return_value=_GCHAT_CHANNELS_NO_AUDIENCE):
|
||||
r = await client.post("/channels/google-chat/scott", json=payload)
|
||||
assert r.status_code == 200
|
||||
assert "hostAppDataAction" in r.json()
|
||||
|
||||
156
cortex/tool_audit.py
Normal file
156
cortex/tool_audit.py
Normal file
@@ -0,0 +1,156 @@
|
||||
"""
|
||||
Tool call audit log.
|
||||
|
||||
One JSONL file per user per day:
|
||||
home/{user}/tool_audit/YYYY-MM-DD.jsonl
|
||||
|
||||
Each line is a JSON object:
|
||||
ts ISO timestamp (seconds)
|
||||
user username
|
||||
tool tool name
|
||||
args call arguments (string values truncated at ARG_MAX chars)
|
||||
status "ok" | "error" | "denied"
|
||||
result_chars length of full result string
|
||||
result_snippet first SNIPPET_MAX chars of result
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from contextvars import ContextVar
|
||||
from datetime import datetime, date
|
||||
from pathlib import Path
|
||||
|
||||
from config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_ARG_MAX = 500 # truncate individual arg string values longer than this
|
||||
_SNIPPET_MAX = 300 # chars of result to keep as snippet
|
||||
|
||||
# Per-file write locks — prevents interleaved lines under concurrent tool calls
|
||||
_locks: dict[str, asyncio.Lock] = {}
|
||||
|
||||
# ContextVars set by orchestrators before their tool loop runs
|
||||
_audit_engine: ContextVar[str] = ContextVar("_audit_engine", default="")
|
||||
_audit_model: ContextVar[str] = ContextVar("_audit_model", default="")
|
||||
|
||||
|
||||
def set_context(engine: str, model: str) -> None:
|
||||
"""Call at the start of each orchestrator run to tag subsequent tool calls."""
|
||||
_audit_engine.set(engine)
|
||||
_audit_model.set(model)
|
||||
|
||||
|
||||
def _truncate_args(args: dict) -> dict:
|
||||
out = {}
|
||||
for k, v in args.items():
|
||||
if isinstance(v, str) and len(v) > _ARG_MAX:
|
||||
out[k] = v[:_ARG_MAX] + f" …[{len(v)} chars total]"
|
||||
else:
|
||||
out[k] = v
|
||||
return out
|
||||
|
||||
|
||||
def _audit_path(user: str, day: date | None = None) -> Path:
|
||||
d = day or date.today()
|
||||
audit_dir = settings.home_root() / user / "tool_audit"
|
||||
audit_dir.mkdir(parents=True, exist_ok=True)
|
||||
return audit_dir / f"{d.isoformat()}.jsonl"
|
||||
|
||||
|
||||
async def record(
|
||||
user: str,
|
||||
tool: str,
|
||||
args: dict,
|
||||
status: str, # "ok" | "error" | "denied"
|
||||
result: str = "",
|
||||
) -> None:
|
||||
"""Append one audit entry. Fire with asyncio.create_task — never awaited directly."""
|
||||
path = _audit_path(user)
|
||||
key = str(path)
|
||||
if key not in _locks:
|
||||
_locks[key] = asyncio.Lock()
|
||||
|
||||
entry = {
|
||||
"ts": datetime.now().isoformat(timespec="seconds"),
|
||||
"user": user,
|
||||
"engine": _audit_engine.get(),
|
||||
"model": _audit_model.get(),
|
||||
"tool": tool,
|
||||
"args": _truncate_args(args),
|
||||
"status": status,
|
||||
"result_chars": len(result),
|
||||
"result_snippet": result[:_SNIPPET_MAX],
|
||||
}
|
||||
|
||||
async with _locks[key]:
|
||||
try:
|
||||
with path.open("a", encoding="utf-8") as f:
|
||||
f.write(json.dumps(entry) + "\n")
|
||||
except Exception as e:
|
||||
logger.warning("audit log write failed for %s: %s", user, e)
|
||||
|
||||
|
||||
def read_recent(user: str, days: int = 7, limit: int = 200) -> list[dict]:
|
||||
"""Read the most recent `limit` entries across the last `days` days.
|
||||
|
||||
Returns entries sorted newest-first (by ts field, file order within a day).
|
||||
"""
|
||||
from datetime import timedelta
|
||||
today = date.today()
|
||||
entries: list[dict] = []
|
||||
|
||||
for offset in range(days):
|
||||
day = today - timedelta(days=offset)
|
||||
path = settings.home_root() / user / "tool_audit" / f"{day.isoformat()}.jsonl"
|
||||
if not path.exists():
|
||||
continue
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8").splitlines()
|
||||
except Exception:
|
||||
continue
|
||||
day_entries = []
|
||||
for line in lines:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
day_entries.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
# Newest within the day first
|
||||
entries.extend(reversed(day_entries))
|
||||
if len(entries) >= limit:
|
||||
break
|
||||
|
||||
return entries[:limit]
|
||||
|
||||
|
||||
def read_day(user: str, day_str: str) -> list[dict]:
|
||||
"""Read all entries for a specific date string (YYYY-MM-DD), chronological order."""
|
||||
path = settings.home_root() / user / "tool_audit" / f"{day_str}.jsonl"
|
||||
if not path.exists():
|
||||
return []
|
||||
entries = []
|
||||
try:
|
||||
for line in path.read_text(encoding="utf-8").splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
entries.append(json.loads(line))
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
return entries
|
||||
|
||||
|
||||
def read_recent_all_users(days: int = 7, limit: int = 500) -> list[dict]:
|
||||
"""Read recent entries across all users, sorted newest-first."""
|
||||
from persona import list_users
|
||||
all_entries: list[dict] = []
|
||||
for user in list_users():
|
||||
all_entries.extend(read_recent(user, days=days, limit=limit))
|
||||
all_entries.sort(key=lambda e: e.get("ts", ""), reverse=True)
|
||||
return all_entries[:limit]
|
||||
File diff suppressed because it is too large
Load Diff
253
cortex/tools/ae_database.py
Normal file
253
cortex/tools/ae_database.py
Normal file
@@ -0,0 +1,253 @@
|
||||
"""
|
||||
Aether MariaDB tools — SELECT-only access to the Aether Platform database.
|
||||
|
||||
Credentials are read from the current user's channels.json:
|
||||
"aether_db": {
|
||||
"host": "192.168.64.5",
|
||||
"port": 3306,
|
||||
"name": "aether_dev",
|
||||
"user": "aether_dev",
|
||||
"password": "..."
|
||||
}
|
||||
|
||||
Configure per-user in Settings → Notifications (or edit channels.json directly).
|
||||
Only SELECT, SHOW, DESCRIBE, and EXPLAIN statements are permitted — no writes possible.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
|
||||
from google.genai import types
|
||||
|
||||
from auth_utils import get_user_channels
|
||||
from persona import get_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_MAX_ROWS = 200
|
||||
_MAX_CELL = 120
|
||||
_ALLOWED = {"select", "show", "describe", "desc", "explain"}
|
||||
_SAFE_ID = re.compile(r'^[a-zA-Z0-9_]+$')
|
||||
|
||||
|
||||
def _get_db_cfg() -> tuple[dict, str | None]:
|
||||
"""Return (cfg_dict, error_string). cfg is empty dict on error."""
|
||||
channels = get_user_channels(get_user())
|
||||
cfg = channels.get("aether_db") or {}
|
||||
if not cfg.get("host") or not cfg.get("user"):
|
||||
return {}, (
|
||||
"Aether DB not configured for this user. "
|
||||
"Add an 'aether_db' block to channels.json: "
|
||||
'{"host": "...", "port": 3306, "name": "aether_dev", "user": "...", "password": "..."}'
|
||||
)
|
||||
return cfg, None
|
||||
|
||||
|
||||
def _is_read_only(sql: str) -> bool:
|
||||
stripped = sql.strip()
|
||||
if not stripped:
|
||||
return False
|
||||
first = stripped.split()[0].lower().rstrip(";")
|
||||
return first in _ALLOWED
|
||||
|
||||
|
||||
def _fmt(columns: list[str], rows: list[tuple]) -> str:
|
||||
if not rows:
|
||||
return f"({len(columns)} column{'s' if len(columns) != 1 else ''}, 0 rows)"
|
||||
|
||||
str_rows = [
|
||||
[("NULL" if v is None else str(v))[:_MAX_CELL] for v in row]
|
||||
for row in rows
|
||||
]
|
||||
|
||||
widths = [
|
||||
max([len(col)] + [len(r[i]) for r in str_rows])
|
||||
for i, col in enumerate(columns)
|
||||
]
|
||||
|
||||
sep = "+" + "+".join("-" * (w + 2) for w in widths) + "+"
|
||||
header = "|" + "|".join(f" {c:<{w}} " for c, w in zip(columns, widths)) + "|"
|
||||
lines = [sep, header, sep]
|
||||
for row in str_rows:
|
||||
lines.append("|" + "|".join(f" {v:<{w}} " for v, w in zip(row, widths)) + "|")
|
||||
lines.append(sep)
|
||||
|
||||
note = " — results truncated at limit" if len(rows) == _MAX_ROWS else ""
|
||||
lines.append(f"({len(rows)} row{'s' if len(rows) != 1 else ''}{note})")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _connect(cfg: dict):
|
||||
import pymysql
|
||||
import pymysql.cursors
|
||||
return pymysql.connect(
|
||||
host=cfg["host"],
|
||||
port=int(cfg.get("port", 3306)),
|
||||
user=cfg["user"],
|
||||
password=cfg.get("password", ""),
|
||||
database=cfg.get("name", "aether_dev"),
|
||||
cursorclass=pymysql.cursors.Cursor,
|
||||
connect_timeout=10,
|
||||
)
|
||||
|
||||
|
||||
async def ae_db_query(sql: str) -> str:
|
||||
"""Run a read-only SQL query against the Aether MariaDB and return formatted results."""
|
||||
cfg, err = _get_db_cfg()
|
||||
if err:
|
||||
return err
|
||||
|
||||
if not _is_read_only(sql):
|
||||
first = sql.strip().split()[0] if sql.strip() else "(empty)"
|
||||
return f"Only SELECT, SHOW, DESCRIBE, and EXPLAIN are permitted. Got: {first!r}"
|
||||
|
||||
def _run() -> tuple[list[str], list[tuple]]:
|
||||
conn = _connect(cfg)
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(sql)
|
||||
columns = [d[0] for d in cur.description] if cur.description else []
|
||||
rows = list(cur.fetchmany(_MAX_ROWS))
|
||||
return columns, rows
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
try:
|
||||
columns, rows = await asyncio.to_thread(_run)
|
||||
return _fmt(columns, rows)
|
||||
except Exception as e:
|
||||
logger.warning("ae_db_query error: %s", e)
|
||||
return f"Query error: {e}"
|
||||
|
||||
|
||||
async def ae_db_describe(table: str, detailed: bool = False) -> str:
|
||||
"""Describe the columns of an Aether DB table or view."""
|
||||
cfg, err = _get_db_cfg()
|
||||
if err:
|
||||
return err
|
||||
|
||||
if not _SAFE_ID.match(table):
|
||||
return f"Invalid table name: {table!r}. Only letters, digits, and underscores allowed."
|
||||
|
||||
def _run():
|
||||
conn = _connect(cfg)
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(f"DESCRIBE `{table}`")
|
||||
columns = [d[0] for d in cur.description] if cur.description else []
|
||||
rows = list(cur.fetchall())
|
||||
return columns, rows
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
try:
|
||||
columns, rows = await asyncio.to_thread(_run)
|
||||
if not detailed:
|
||||
fields = [row[0] for row in rows]
|
||||
return f"{table}: " + ", ".join(fields)
|
||||
return _fmt(columns, rows)
|
||||
except Exception as e:
|
||||
logger.warning("ae_db_describe error: %s", e)
|
||||
return f"Describe error: {e}"
|
||||
|
||||
|
||||
async def ae_db_show_view(view_name: str) -> str:
|
||||
"""Return the CREATE VIEW SQL for an Aether DB view."""
|
||||
cfg, err = _get_db_cfg()
|
||||
if err:
|
||||
return err
|
||||
|
||||
if not _SAFE_ID.match(view_name):
|
||||
return f"Invalid view name: {view_name!r}. Only letters, digits, and underscores allowed."
|
||||
|
||||
def _run():
|
||||
conn = _connect(cfg)
|
||||
try:
|
||||
with conn.cursor() as cur:
|
||||
cur.execute(f"SHOW CREATE VIEW `{view_name}`")
|
||||
return cur.fetchone()
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
try:
|
||||
row = await asyncio.to_thread(_run)
|
||||
if not row:
|
||||
return f"View not found: {view_name}"
|
||||
return str(row[1]) if len(row) > 1 else str(row[0])
|
||||
except Exception as e:
|
||||
logger.warning("ae_db_show_view error: %s", e)
|
||||
return f"Show view error: {e}"
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="ae_db_describe",
|
||||
description=(
|
||||
"Describe the columns of an Aether Platform table or view. "
|
||||
"Returns a compact field list by default; pass detailed=true for full schema "
|
||||
"(type, nullability, default, key). Use to understand data structure before "
|
||||
"writing a SELECT query, or to answer 'what fields does X have?'. "
|
||||
"Examples: table='ae_journals'; table='clients'; table='time_entries'."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"table": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Table or view name (letters, digits, underscores only)",
|
||||
),
|
||||
"detailed": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description="Return full schema (type, nullability, key, default) instead of just field names",
|
||||
),
|
||||
},
|
||||
required=["table"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_db_show_view",
|
||||
description=(
|
||||
"Return the CREATE VIEW SQL for an Aether Platform database view. "
|
||||
"Use to understand how a view is constructed before querying it, "
|
||||
"or to debug unexpected results from a view. "
|
||||
"Example: view_name='v_active_journals'."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"view_name": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="View name (letters, digits, underscores only)",
|
||||
),
|
||||
},
|
||||
required=["view_name"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_db_query",
|
||||
description=(
|
||||
"Run a read-only SQL query against the Aether Platform MariaDB. "
|
||||
"Permitted statements: SELECT, SHOW, DESCRIBE, EXPLAIN. No writes are possible. "
|
||||
"Use for debugging: bad data, missing records, broken foreign keys, schema questions. "
|
||||
"Results capped at 200 rows; cells truncated at 120 chars. "
|
||||
"Examples: SELECT * FROM clients WHERE email = 'x@y.com'; "
|
||||
"SELECT COUNT(*) FROM time_entries WHERE billed = 0 AND deleted_at IS NULL; "
|
||||
"SHOW TABLES; DESCRIBE ae_journals; "
|
||||
"SELECT id_random, enable, deleted_at FROM ae_journals WHERE id_random = 'abc123'."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"sql": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"SQL query to run — SELECT, SHOW, DESCRIBE, or EXPLAIN only. "
|
||||
"No semicolons required but harmless if present."
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["sql"],
|
||||
),
|
||||
),
|
||||
]
|
||||
@@ -1,15 +1,17 @@
|
||||
"""
|
||||
Aether Platform knowledge tools — journal search and entry creation.
|
||||
Aether Platform knowledge tools — journal search, listing, and entry management.
|
||||
|
||||
These tools give the orchestrator read/write access to the AE Journals module,
|
||||
which serves as the primary long-term knowledge base.
|
||||
|
||||
Auth: x-aether-api-key + x-account-id headers (same pattern as agents_sync scripts).
|
||||
API: V3 CRUD — POST /v3/crud/journal_entry/search, POST /v3/crud/journal/{id}/journal_entry/
|
||||
PATCH /v3/crud/journal_entry/{entry_id}, GET /v3/crud/journal_entry/{entry_id}
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from google.genai import types
|
||||
from config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -40,36 +42,98 @@ def _check_config() -> str | None:
|
||||
# Tool: ae_journal_search
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_search(query: str, journal_id: str | None = None, max_results: int = 10) -> str:
|
||||
"""Search AE Journal entries by keyword.
|
||||
async def journal_search(
|
||||
query: str = "",
|
||||
journal_id: str = "",
|
||||
tags: str = "",
|
||||
type_code: str = "",
|
||||
topic_code: str = "",
|
||||
date_from: str = "",
|
||||
date_to: str = "",
|
||||
sort_by: str = "updated",
|
||||
sort_order: str = "desc",
|
||||
status: int | None = None,
|
||||
priority: int | None = None,
|
||||
max_results: int = 10,
|
||||
page: int = 1,
|
||||
) -> str:
|
||||
"""Search AE Journal entries.
|
||||
|
||||
Searches across the default_qry_str field (title + content excerpt).
|
||||
Optionally scoped to a specific journal by journal_id (id_random).
|
||||
Returns a markdown-formatted list of matching entries.
|
||||
At least one of query, tags, type_code, topic_code, date_from, or journal_id
|
||||
should be provided. All filters combine with AND.
|
||||
"""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
|
||||
return await asyncio.to_thread(_sync_journal_search, query, journal_id, max_results)
|
||||
return await asyncio.to_thread(
|
||||
_sync_journal_search,
|
||||
query, journal_id, tags, type_code, topic_code,
|
||||
date_from, date_to, sort_by, sort_order,
|
||||
status, priority, max_results, page,
|
||||
)
|
||||
|
||||
|
||||
def _sync_journal_search(query: str, journal_id: str | None, max_results: int) -> str:
|
||||
def _sync_journal_search(
|
||||
query: str,
|
||||
journal_id: str,
|
||||
tags: str,
|
||||
type_code: str,
|
||||
topic_code: str,
|
||||
date_from: str,
|
||||
date_to: str,
|
||||
sort_by: str,
|
||||
sort_order: str,
|
||||
status: int | None,
|
||||
priority: int | None,
|
||||
max_results: int,
|
||||
page: int,
|
||||
) -> str:
|
||||
import requests
|
||||
|
||||
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
|
||||
search_body = {
|
||||
"and_filters": [
|
||||
{"field": "default_qry_str", "op": "icontains", "value": query}
|
||||
],
|
||||
"page_size": max_results,
|
||||
# Build sort field
|
||||
sort_field_map = {
|
||||
"updated": "updated_on",
|
||||
"created": "created_on",
|
||||
"name": "name",
|
||||
"priority": "priority",
|
||||
}
|
||||
sort_field = sort_field_map.get(sort_by, "updated_on")
|
||||
order_by = f"{'-' if sort_order == 'desc' else ''}{sort_field}"
|
||||
|
||||
params = {}
|
||||
search_body: dict = {"page_size": max_results, "page": page, "order_by": order_by}
|
||||
|
||||
# Fulltext keyword — uses MATCH/AGAINST index
|
||||
if query:
|
||||
search_body["query_string"] = query
|
||||
|
||||
# Additional AND filters
|
||||
and_filters: list[dict] = []
|
||||
if tags:
|
||||
and_filters.append({"field": "tags", "op": "icontains", "value": tags})
|
||||
if type_code:
|
||||
and_filters.append({"field": "type_code", "op": "eq", "value": type_code})
|
||||
if topic_code:
|
||||
and_filters.append({"field": "topic_code", "op": "eq", "value": topic_code})
|
||||
if date_from:
|
||||
and_filters.append({"field": "created_on", "op": "gte", "value": date_from})
|
||||
if date_to:
|
||||
and_filters.append({"field": "created_on", "op": "lte", "value": date_to})
|
||||
if status is not None:
|
||||
and_filters.append({"field": "status", "op": "eq", "value": status})
|
||||
if priority is not None:
|
||||
and_filters.append({"field": "priority", "op": "eq", "value": priority})
|
||||
if and_filters:
|
||||
search_body["and"] = and_filters
|
||||
# query_string must be present for `and` filters to apply
|
||||
if "query_string" not in search_body:
|
||||
search_body["query_string"] = "%"
|
||||
|
||||
params: dict = {}
|
||||
if journal_id:
|
||||
params["for_obj_type"] = "journal"
|
||||
params["for_obj_id"] = journal_id
|
||||
|
||||
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
|
||||
try:
|
||||
resp = requests.post(
|
||||
url,
|
||||
@@ -85,33 +149,92 @@ def _sync_journal_search(query: str, journal_id: str | None, max_results: int) -
|
||||
return f"Journal search error: {e}"
|
||||
|
||||
entries = data.get("data", [])
|
||||
if not entries:
|
||||
return f"No journal entries found matching: {query}"
|
||||
total = (data.get("meta") or {}).get("data_list_count") or len(entries)
|
||||
|
||||
if not entries:
|
||||
desc = query or tags or type_code or topic_code or f"journal {journal_id}"
|
||||
return f"No journal entries found for: {desc}"
|
||||
|
||||
label = query or tags or f"{len(entries)} entries"
|
||||
lines = [f"Journal entries — **{label}** ({total} total, page {page}):\n"]
|
||||
|
||||
lines = [f"Journal entries matching **{query}** ({len(entries)} result(s)):\n"]
|
||||
for entry in entries:
|
||||
title = entry.get("name") or "(untitled)"
|
||||
entry_id = entry.get("id_random", "")
|
||||
entry_id = entry.get("journal_entry_id") or entry.get("id") or ""
|
||||
journal_name = entry.get("journal_name") or entry.get("parent_name") or ""
|
||||
summary = entry.get("summary") or ""
|
||||
content_preview = (entry.get("content") or "")[:200].replace("\n", " ")
|
||||
entry_tags = entry.get("tags") or []
|
||||
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:10]
|
||||
content_preview = (entry.get("content") or "")[:400].replace("\n", " ")
|
||||
|
||||
header = f"**{title}**"
|
||||
if journal_name:
|
||||
header += f" ({journal_name})"
|
||||
if entry_id:
|
||||
header += f" — id: `{entry_id}`"
|
||||
|
||||
if updated:
|
||||
header += f" [{updated}]"
|
||||
lines.append(header)
|
||||
if entry_tags:
|
||||
tag_list = entry_tags if isinstance(entry_tags, list) else [t.strip() for t in str(entry_tags).split(",")]
|
||||
lines.append(f" Tags: {', '.join(tag_list)}")
|
||||
if summary:
|
||||
lines.append(f" Summary: {summary}")
|
||||
if content_preview:
|
||||
lines.append(f" {content_preview}…")
|
||||
lines.append(f" {summary}")
|
||||
elif content_preview:
|
||||
lines.append(f" {content_preview}{'…' if len(entry.get('content', '')) > 400 else ''}")
|
||||
lines.append("")
|
||||
|
||||
if total > page * max_results:
|
||||
lines.append(f"(More results — call again with page={page + 1})")
|
||||
|
||||
return "\n".join(lines).strip()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_list
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_list() -> str:
|
||||
"""List all journals accessible to the configured AE account."""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
return await asyncio.to_thread(_sync_journal_list)
|
||||
|
||||
|
||||
def _sync_journal_list() -> str:
|
||||
import requests
|
||||
|
||||
url = f"{settings.ae_api_url}/v3/crud/journal/search"
|
||||
try:
|
||||
resp = requests.post(
|
||||
url,
|
||||
headers=_headers(),
|
||||
json={"page_size": 100},
|
||||
timeout=settings.ae_api_timeout,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except Exception as e:
|
||||
logger.warning("ae_journal_list failed: %s", e)
|
||||
return f"Journal list error: {e}"
|
||||
|
||||
journals = data.get("data", [])
|
||||
if not journals:
|
||||
return "No journals found for this account."
|
||||
|
||||
lines = [f"Journals ({len(journals)}):\n"]
|
||||
for j in journals:
|
||||
jid = j.get("journal_id") or j.get("id_random") or j.get("id") or "?"
|
||||
name = j.get("name") or "(untitled)"
|
||||
desc = j.get("description") or ""
|
||||
line = f"- **{name}** — id: `{jid}`"
|
||||
if desc:
|
||||
line += f"\n {desc}"
|
||||
lines.append(line)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_entry_create
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -170,8 +293,455 @@ def _sync_journal_entry_create(
|
||||
return f"Journal entry creation error: {e}"
|
||||
|
||||
entry_id = (
|
||||
result.get("data", {}).get("id_random")
|
||||
result.get("data", {}).get("journal_entry_id")
|
||||
or result.get("data", {}).get("id_random")
|
||||
or result.get("id_random")
|
||||
or "unknown"
|
||||
)
|
||||
return f"Journal entry created. id: `{entry_id}`, title: \"{title}\", journal: `{journal_id}`"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared helper: fetch a single journal entry by id
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_entry(entry_id: str) -> dict | str:
|
||||
"""Return the entry dict, or an error string on failure."""
|
||||
import requests
|
||||
url = f"{settings.ae_api_url}/v3/crud/journal_entry/{entry_id}"
|
||||
try:
|
||||
resp = requests.get(url, headers=_headers(), timeout=settings.ae_api_timeout)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
entry = data.get("data") or data
|
||||
if not isinstance(entry, dict):
|
||||
return f"Unexpected response shape for entry {entry_id}"
|
||||
return entry
|
||||
except Exception as e:
|
||||
logger.warning("_get_entry %s failed: %s", entry_id, e)
|
||||
return f"Error fetching entry {entry_id}: {e}"
|
||||
|
||||
|
||||
def _patch_entry(entry_id: str, payload: dict) -> str:
|
||||
"""PATCH a journal entry. Returns a success/error string."""
|
||||
import requests
|
||||
url = f"{settings.ae_api_url}/v3/crud/journal_entry/{entry_id}"
|
||||
try:
|
||||
resp = requests.patch(
|
||||
url,
|
||||
headers=_headers(),
|
||||
json=payload,
|
||||
timeout=settings.ae_api_timeout,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return "ok"
|
||||
except Exception as e:
|
||||
logger.warning("_patch_entry %s failed: %s", entry_id, e)
|
||||
return f"Error updating entry {entry_id}: {e}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_entry_read
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_entry_read(entry_id: str, max_content_chars: int = 4000) -> str:
|
||||
"""Return the full content of a single journal entry by its id_random."""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
return await asyncio.to_thread(_sync_journal_entry_read, entry_id, max_content_chars)
|
||||
|
||||
|
||||
def _sync_journal_entry_read(entry_id: str, max_content_chars: int) -> str:
|
||||
entry = _get_entry(entry_id)
|
||||
if isinstance(entry, str):
|
||||
return entry
|
||||
|
||||
title = entry.get("name") or "(untitled)"
|
||||
journal = entry.get("journal_name") or entry.get("parent_name") or ""
|
||||
summary = entry.get("summary") or ""
|
||||
raw_tags = entry.get("tags") or []
|
||||
tags = raw_tags if isinstance(raw_tags, list) else [t.strip() for t in str(raw_tags).split(",") if t.strip()]
|
||||
content = entry.get("content") or ""
|
||||
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:19].replace("T", " ")
|
||||
enabled = entry.get("enable", True)
|
||||
|
||||
lines = [f"# {title}"]
|
||||
meta: list[str] = [f"id: `{entry_id}`"]
|
||||
if journal:
|
||||
meta.append(f"journal: {journal}")
|
||||
if updated:
|
||||
meta.append(f"updated: {updated}")
|
||||
if not enabled:
|
||||
meta.append("**DISABLED**")
|
||||
lines.append(" ".join(meta))
|
||||
if tags:
|
||||
lines.append(f"Tags: {', '.join(tags)}")
|
||||
if summary:
|
||||
lines.append(f"\nSummary: {summary}")
|
||||
lines.append("\n---\n")
|
||||
|
||||
truncated = len(content) > max_content_chars
|
||||
lines.append(content[:max_content_chars])
|
||||
if truncated:
|
||||
lines.append(
|
||||
f"\n\n[Content truncated at {max_content_chars} chars — "
|
||||
f"{len(content)} total. Call again with a higher max_content_chars to read more.]"
|
||||
)
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_entries_list
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_entries_list(journal_id: str, max_results: int = 20, page: int = 1) -> str:
|
||||
"""List entries in a specific journal, newest first."""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
return await asyncio.to_thread(_sync_journal_entries_list, journal_id, max_results, page)
|
||||
|
||||
|
||||
def _sync_journal_entries_list(journal_id: str, max_results: int, page: int) -> str:
|
||||
import requests
|
||||
|
||||
url = f"{settings.ae_api_url}/v3/crud/journal_entry/search"
|
||||
search_body: dict = {
|
||||
"page_size": max_results,
|
||||
"page": page,
|
||||
"order_by": "-updated_on",
|
||||
}
|
||||
params = {"for_obj_type": "journal", "for_obj_id": journal_id}
|
||||
|
||||
try:
|
||||
resp = requests.post(
|
||||
url,
|
||||
headers=_headers(),
|
||||
params=params,
|
||||
json=search_body,
|
||||
timeout=settings.ae_api_timeout,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except Exception as e:
|
||||
logger.warning("ae_journal_entries_list failed: %s", e)
|
||||
return f"Journal entries list error: {e}"
|
||||
|
||||
entries = data.get("data", [])
|
||||
total = (data.get("meta") or {}).get("data_list_count") or len(entries)
|
||||
|
||||
if not entries:
|
||||
return f"No entries found in journal `{journal_id}`."
|
||||
|
||||
offset = (page - 1) * max_results + 1
|
||||
lines = [f"Entries in journal `{journal_id}` — showing {offset}–{offset + len(entries) - 1} of {total}:\n"]
|
||||
for i, entry in enumerate(entries, offset):
|
||||
title = entry.get("name") or "(untitled)"
|
||||
entry_id = entry.get("journal_entry_id") or entry.get("id") or ""
|
||||
raw_tags = entry.get("tags") or []
|
||||
tags = raw_tags if isinstance(raw_tags, list) else [t.strip() for t in str(raw_tags).split(",") if t.strip()]
|
||||
summary = entry.get("summary") or ""
|
||||
updated = (entry.get("updated_on") or entry.get("created_on") or "")[:10]
|
||||
enabled = entry.get("enable", True)
|
||||
|
||||
status = "" if enabled else " [disabled]"
|
||||
date_str = f" [{updated}]" if updated else ""
|
||||
lines.append(f"{i}. **{title}**{status} — id: `{entry_id}`{date_str}")
|
||||
if tags:
|
||||
lines.append(f" Tags: {', '.join(tags)}")
|
||||
if summary:
|
||||
lines.append(f" {summary[:150]}{'…' if len(summary) > 150 else ''}")
|
||||
lines.append("")
|
||||
|
||||
if total > offset + len(entries) - 1:
|
||||
lines.append(f"(More entries available — call again with page={page + 1})")
|
||||
|
||||
return "\n".join(lines).rstrip()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_entry_update
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_entry_update(
|
||||
entry_id: str,
|
||||
title: str = "",
|
||||
content: str = "",
|
||||
summary: str = "",
|
||||
tags: str = "",
|
||||
enable: bool | None = None,
|
||||
) -> str:
|
||||
"""Update fields on an existing journal entry. Only provided fields are changed."""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
return await asyncio.to_thread(_sync_journal_entry_update, entry_id, title, content, summary, tags, enable)
|
||||
|
||||
|
||||
def _sync_journal_entry_update(
|
||||
entry_id: str,
|
||||
title: str,
|
||||
content: str,
|
||||
summary: str,
|
||||
tags: str,
|
||||
enable: bool | None,
|
||||
) -> str:
|
||||
payload: dict = {}
|
||||
if title:
|
||||
payload["name"] = title
|
||||
if content:
|
||||
payload["content"] = content
|
||||
if summary:
|
||||
payload["summary"] = summary
|
||||
if tags:
|
||||
payload["tags"] = [t.strip() for t in tags.split(",") if t.strip()]
|
||||
if enable is not None:
|
||||
payload["enable"] = enable
|
||||
|
||||
if not payload:
|
||||
return "Nothing to update — no fields provided."
|
||||
|
||||
result = _patch_entry(entry_id, payload)
|
||||
if result != "ok":
|
||||
return result
|
||||
|
||||
updated = ", ".join(payload.keys())
|
||||
return f"Journal entry `{entry_id}` updated. Fields changed: {updated}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_entry_disable
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_entry_disable(entry_id: str) -> str:
|
||||
"""Soft-delete a journal entry by setting enable=false."""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
return await asyncio.to_thread(_patch_entry, entry_id, {"enable": False})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_entry_append
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_entry_append(entry_id: str, content: str, heading: str = "") -> str:
|
||||
"""Append a timestamped section to the bottom of a journal entry's content."""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
return await asyncio.to_thread(_sync_journal_entry_append, entry_id, content, heading)
|
||||
|
||||
|
||||
def _sync_journal_entry_append(entry_id: str, content: str, heading: str) -> str:
|
||||
from datetime import datetime, timezone
|
||||
|
||||
entry = _get_entry(entry_id)
|
||||
if isinstance(entry, str):
|
||||
return entry
|
||||
|
||||
existing = (entry.get("content") or "").rstrip()
|
||||
ts = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||
section_heading = heading or ts
|
||||
new_content = f"{existing}\n\n### {section_heading}\n{content.strip()}"
|
||||
|
||||
result = _patch_entry(entry_id, {"content": new_content})
|
||||
if result != "ok":
|
||||
return result
|
||||
return f"Appended to journal entry `{entry_id}` under heading \"{section_heading}\"."
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool: ae_journal_entry_prepend
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def journal_entry_prepend(entry_id: str, content: str, heading: str = "") -> str:
|
||||
"""Prepend a timestamped section to the top of a journal entry's content."""
|
||||
err = _check_config()
|
||||
if err:
|
||||
return err
|
||||
return await asyncio.to_thread(_sync_journal_entry_prepend, entry_id, content, heading)
|
||||
|
||||
|
||||
def _sync_journal_entry_prepend(entry_id: str, content: str, heading: str) -> str:
|
||||
from datetime import datetime, timezone
|
||||
|
||||
entry = _get_entry(entry_id)
|
||||
if isinstance(entry, str):
|
||||
return entry
|
||||
|
||||
existing = (entry.get("content") or "").lstrip()
|
||||
ts = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||
section_heading = heading or ts
|
||||
new_content = f"### {section_heading}\n{content.strip()}\n\n{existing}"
|
||||
|
||||
result = _patch_entry(entry_id, {"content": new_content})
|
||||
if result != "ok":
|
||||
return result
|
||||
return f"Prepended to journal entry `{entry_id}` under heading \"{section_heading}\"."
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_list",
|
||||
description=(
|
||||
"List all Aether Journals available for this account. "
|
||||
"Returns each journal's name and id_random. "
|
||||
"Call this first when you need to write a new entry or scope a search to a specific journal "
|
||||
"and don't already know the journal's id."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_search",
|
||||
description=(
|
||||
"Search Aether Journal entries. All parameters are optional — combine freely. "
|
||||
"Use 'query' for fulltext keyword search (supports boolean: +required -excluded \"phrase\"). "
|
||||
"Use 'tags' to filter by tag substring. Use 'date_from'/'date_to' for date ranges (YYYY-MM-DD). "
|
||||
"Always search before creating a new entry to avoid duplicates."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"query": types.Schema(type=types.Type.STRING, description="Fulltext keyword search. Supports boolean mode: +required -excluded \"exact phrase\"."),
|
||||
"journal_id": types.Schema(type=types.Type.STRING, description="Scope results to a specific journal by its id_random. Omit to search all journals."),
|
||||
"tags": types.Schema(type=types.Type.STRING, description="Filter by tag substring (e.g. 'networking' matches entries tagged 'networking' or 'home-networking')."),
|
||||
"type_code": types.Schema(type=types.Type.STRING, description="Filter by exact type_code (e.g. 'note', 'meeting', 'log')."),
|
||||
"topic_code": types.Schema(type=types.Type.STRING, description="Filter by exact topic_code."),
|
||||
"date_from": types.Schema(type=types.Type.STRING, description="Return entries created on or after this date (YYYY-MM-DD)."),
|
||||
"date_to": types.Schema(type=types.Type.STRING, description="Return entries created on or before this date (YYYY-MM-DD)."),
|
||||
"sort_by": types.Schema(type=types.Type.STRING, description="Sort field: 'updated' (default), 'created', 'name', or 'priority'."),
|
||||
"sort_order": types.Schema(type=types.Type.STRING, description="Sort direction: 'desc' (default, newest first) or 'asc'."),
|
||||
"status": types.Schema(type=types.Type.INTEGER, description="Filter by exact status code."),
|
||||
"priority": types.Schema(type=types.Type.INTEGER, description="Filter by exact priority (1=low, 5=high)."),
|
||||
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of results per page (default 10)."),
|
||||
"page": types.Schema(type=types.Type.INTEGER, description="Page number for pagination (default 1)."),
|
||||
},
|
||||
required=[],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_entry_read",
|
||||
description=(
|
||||
"Fetch the full content of a single journal entry by its id_random. "
|
||||
"Use this when you need to read an entry before editing it, or when search results "
|
||||
"don't show enough content. Returns title, journal, tags, summary, and full content."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"entry_id": types.Schema(type=types.Type.STRING, description="The id_random of the journal entry to read."),
|
||||
"max_content_chars": types.Schema(type=types.Type.INTEGER, description="Maximum characters of content to return (default 4000). Increase for long entries."),
|
||||
},
|
||||
required=["entry_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_entries_list",
|
||||
description=(
|
||||
"List entries in a specific journal, newest first. "
|
||||
"Use this to browse what's in a journal when you don't have a search keyword, "
|
||||
"or to find entries by browsing rather than searching. "
|
||||
"Returns numbered entries with id, title, tags, summary, and date."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"journal_id": types.Schema(type=types.Type.STRING, description="The id_random of the journal to list entries from."),
|
||||
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of entries to return (default 20, max 50)."),
|
||||
"page": types.Schema(type=types.Type.INTEGER, description="Page number for pagination (default 1)."),
|
||||
},
|
||||
required=["journal_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_entry_create",
|
||||
description=(
|
||||
"Create a new entry in an Aether Journal. "
|
||||
"Use this to save notes, summaries, or any content the user wants to store. "
|
||||
"Always call ae_journal_search first to check for existing entries on the same topic."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"journal_id": types.Schema(type=types.Type.STRING, description="The id_random of the target journal. Ask the user which journal to write to if not specified."),
|
||||
"title": types.Schema(type=types.Type.STRING, description="Entry title"),
|
||||
"content": types.Schema(type=types.Type.STRING, description="Full entry content (markdown supported)"),
|
||||
"summary": types.Schema(type=types.Type.STRING, description="Optional short summary (1-2 sentences)"),
|
||||
"tags": types.Schema(type=types.Type.STRING, description="Optional comma-separated tags (e.g. 'wireguard, networking, homelab')"),
|
||||
},
|
||||
required=["journal_id", "title", "content"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_entry_update",
|
||||
description=(
|
||||
"Update fields on an existing journal entry. Only the fields you provide are changed — "
|
||||
"omitted fields are left as-is. Use ae_journal_search to find the entry_id first. "
|
||||
"To soft-delete, use ae_journal_entry_disable instead."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
|
||||
"title": types.Schema(type=types.Type.STRING, description="New title"),
|
||||
"content": types.Schema(type=types.Type.STRING, description="Replacement content (full, markdown supported)"),
|
||||
"summary": types.Schema(type=types.Type.STRING, description="New summary"),
|
||||
"tags": types.Schema(type=types.Type.STRING, description="Replacement comma-separated tags"),
|
||||
"enable": types.Schema(type=types.Type.BOOLEAN, description="Set false to hide/disable the entry"),
|
||||
},
|
||||
required=["entry_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_entry_disable",
|
||||
description=(
|
||||
"Soft-delete a journal entry by setting enable=false. "
|
||||
"The entry is hidden but not permanently removed. "
|
||||
"Use ae_journal_search to find the entry_id first."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
|
||||
},
|
||||
required=["entry_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_entry_append",
|
||||
description=(
|
||||
"Append a new section to the bottom of a journal entry's content. "
|
||||
"Each section gets a UTC timestamp heading unless you provide one. "
|
||||
"Ideal for timestamped logs, running notes, or data logs."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
|
||||
"content": types.Schema(type=types.Type.STRING, description="The text to append (markdown supported)"),
|
||||
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading (defaults to current UTC timestamp)"),
|
||||
},
|
||||
required=["entry_id", "content"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ae_journal_entry_prepend",
|
||||
description=(
|
||||
"Prepend a new section to the top of a journal entry's content. "
|
||||
"Each section gets a UTC timestamp heading unless you provide one. "
|
||||
"Useful for most-recent-first logs."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"entry_id": types.Schema(type=types.Type.STRING, description="Journal entry id_random"),
|
||||
"content": types.Schema(type=types.Type.STRING, description="The text to prepend (markdown supported)"),
|
||||
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading (defaults to current UTC timestamp)"),
|
||||
},
|
||||
required=["entry_id", "content"],
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
@@ -16,6 +16,8 @@ import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Resolved at import time — agents_sync is always at ~/agents_sync on this machine.
|
||||
@@ -98,3 +100,20 @@ def _read_bucket(bucket_dir: Path) -> list[dict]:
|
||||
except Exception as e:
|
||||
logger.warning("Failed to read task file %s: %s", path, e)
|
||||
return tasks
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="ae_task_list",
|
||||
description=(
|
||||
"List tasks from the agents_sync Kanban board (todo and in-progress). "
|
||||
"Use this when asked about current work, pending tasks, or project status."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"include_done": types.Schema(type=types.Type.BOOLEAN, description="If true, also include completed tasks (default false)"),
|
||||
},
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
155
cortex/tools/agent_notes.py
Normal file
155
cortex/tools/agent_notes.py
Normal file
@@ -0,0 +1,155 @@
|
||||
"""
|
||||
Agent private notes — AGENT_NOTES.md.
|
||||
|
||||
A persistent notepad only the orchestrator can write to. The file itself is
|
||||
never exposed in the Files panel or loaded into user-facing context tiers.
|
||||
Up to 3 rolling backups are kept automatically before each write so past
|
||||
versions can be reviewed.
|
||||
|
||||
Use for: observations about the user's patterns, working hypotheses,
|
||||
long-running goals, things to remember across sessions that shouldn't
|
||||
be part of the distilled memory visible to the user.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
from persona import persona_path
|
||||
|
||||
|
||||
_FILENAME = "AGENT_NOTES.md"
|
||||
_N_BACKUPS = 3
|
||||
|
||||
|
||||
def _notes_path() -> Path:
|
||||
return persona_path() / _FILENAME
|
||||
|
||||
|
||||
def _now_label() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||
|
||||
|
||||
def _rotate(path: Path) -> None:
|
||||
"""Rotate up to _N_BACKUPS rolling backups before a write."""
|
||||
if not path.exists():
|
||||
return
|
||||
for i in range(_N_BACKUPS, 1, -1):
|
||||
older = path.parent / f"{path.stem}.bak{i}.md"
|
||||
newer = path.parent / f"{path.stem}.bak{i - 1}.md"
|
||||
if newer.exists():
|
||||
older.write_text(newer.read_text())
|
||||
bak1 = path.parent / f"{path.stem}.bak1.md"
|
||||
bak1.write_text(path.read_text())
|
||||
|
||||
|
||||
# ── Sync implementations ────────────────────────────────────────────────────
|
||||
|
||||
def _agent_notes_read() -> str:
|
||||
p = _notes_path()
|
||||
if not p.exists() or not p.read_text().strip():
|
||||
return "Agent notes are empty."
|
||||
return p.read_text()
|
||||
|
||||
|
||||
def _agent_notes_write(content: str) -> str:
|
||||
p = _notes_path()
|
||||
_rotate(p)
|
||||
p.write_text(content.rstrip() + "\n")
|
||||
return "Agent notes updated."
|
||||
|
||||
|
||||
def _agent_notes_append(content: str, heading: str | None = None) -> str:
|
||||
p = _notes_path()
|
||||
_rotate(p)
|
||||
existing = p.read_text() if p.exists() else ""
|
||||
label = heading or _now_label()
|
||||
section = f"\n## {label}\n\n{content.strip()}\n"
|
||||
p.write_text(existing.rstrip() + "\n" + section)
|
||||
return f"Appended to agent notes: {label}"
|
||||
|
||||
|
||||
def _agent_notes_clear() -> str:
|
||||
p = _notes_path()
|
||||
_rotate(p)
|
||||
p.write_text("")
|
||||
return "Agent notes cleared."
|
||||
|
||||
|
||||
# ── Async wrappers ───────────────────────────────────────────────────────────
|
||||
|
||||
async def agent_notes_read() -> str:
|
||||
return await asyncio.to_thread(_agent_notes_read)
|
||||
|
||||
async def agent_notes_write(content: str) -> str:
|
||||
return await asyncio.to_thread(_agent_notes_write, content)
|
||||
|
||||
async def agent_notes_append(content: str, heading: str | None = None) -> str:
|
||||
return await asyncio.to_thread(_agent_notes_append, content, heading)
|
||||
|
||||
async def agent_notes_clear() -> str:
|
||||
return await asyncio.to_thread(_agent_notes_clear)
|
||||
|
||||
|
||||
# ── Gemini FunctionDeclarations ──────────────────────────────────────────────
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="agent_notes_read",
|
||||
description=(
|
||||
"Read your private agent notes — a persistent notepad only you can write to. "
|
||||
"Use this to recall observations, working hypotheses, long-running goals, or "
|
||||
"anything you want to remember across sessions without surfacing it to the user. "
|
||||
"This file is never shown in the user's Files panel."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="agent_notes_write",
|
||||
description=(
|
||||
"Replace your private agent notes with new content. "
|
||||
"A backup is saved automatically before writing. "
|
||||
"Use agent_notes_append to add without replacing."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"content": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="The new notes content (markdown supported).",
|
||||
),
|
||||
},
|
||||
required=["content"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="agent_notes_append",
|
||||
description=(
|
||||
"Add a new section to your private agent notes without replacing existing content. "
|
||||
"A backup is saved automatically before writing. "
|
||||
"Each section gets a UTC timestamp heading unless you supply one."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"content": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="The content to append (markdown supported).",
|
||||
),
|
||||
"heading": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Optional section heading. Defaults to current UTC timestamp.",
|
||||
),
|
||||
},
|
||||
required=["content"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="agent_notes_clear",
|
||||
description=(
|
||||
"Erase all private agent notes. A backup is saved automatically before clearing."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
]
|
||||
446
cortex/tools/agents.py
Normal file
446
cortex/tools/agents.py
Normal file
@@ -0,0 +1,446 @@
|
||||
"""
|
||||
Agent spawning and lifecycle tools.
|
||||
|
||||
spawn_agent — synchronous or background sub-agent via any configured role model.
|
||||
agent_status / agent_list / agent_cancel — lifecycle management for background agents.
|
||||
|
||||
Sub-agents run using the model and tools assigned to the given role. The three-level
|
||||
hierarchy (Persona → Specialized → Support) is enforced by denying spawn_agent and
|
||||
aider_run at the L2→L3 boundary — Level 3 agents cannot delegate further.
|
||||
|
||||
Supported model types for sub-agents: local_openai, gemini_api.
|
||||
claude_cli / gemini_cli are chat-only and do not support tool-enabled sub-agents.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from datetime import datetime
|
||||
|
||||
from google.genai import types
|
||||
|
||||
import agent_manager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Per-host semaphores — keyed by "host:<host_id>" or "type:<model_type>"
|
||||
# Created lazily on first use; never deleted (module-level singletons)
|
||||
_semaphores: dict[str, asyncio.Semaphore] = {}
|
||||
_sem_lock = asyncio.Lock()
|
||||
|
||||
# Tools denied at the L2→L3 boundary so Level 3 agents cannot delegate further.
|
||||
_L3_DENY_TOOLS = ["spawn_agent", "aider_run"]
|
||||
|
||||
|
||||
async def _get_semaphore(key: str, max_concurrent: int) -> asyncio.Semaphore:
|
||||
"""Return (or create) the semaphore for a given host/type key."""
|
||||
async with _sem_lock:
|
||||
if key not in _semaphores:
|
||||
_semaphores[key] = asyncio.Semaphore(max_concurrent)
|
||||
return _semaphores[key]
|
||||
|
||||
|
||||
async def spawn_agent(
|
||||
task: str,
|
||||
role: str = "chat",
|
||||
tier: int = 1,
|
||||
timeout: int = 120,
|
||||
max_rounds: int | None = None,
|
||||
allow_tools: list[str] | None = None,
|
||||
deny_tools: list[str] | None = None,
|
||||
background: bool = False,
|
||||
notify: bool = False,
|
||||
_agent_level: int = 2,
|
||||
) -> str:
|
||||
"""
|
||||
Spawn a sub-agent to complete a task.
|
||||
|
||||
In synchronous mode (background=False, the default): blocks until done and returns
|
||||
the result string.
|
||||
|
||||
In background mode (background=True): registers the agent, fires it as an asyncio
|
||||
background task, and returns an agent_id string immediately. Use agent_status() to
|
||||
poll, or set notify=True to receive a push notification on completion.
|
||||
|
||||
Level enforcement: this agent (level _agent_level) spawns children at level+1.
|
||||
Children at level 3 automatically have spawn_agent and aider_run denied so they
|
||||
cannot delegate further.
|
||||
"""
|
||||
import model_registry
|
||||
from context_loader import load_context
|
||||
from auth_utils import get_user_role, get_tool_policy
|
||||
from persona import get_user
|
||||
|
||||
user = get_user() or "scott"
|
||||
|
||||
role_cfg = model_registry.get_role_config(user, role)
|
||||
model_cfg = model_registry.get_model_for_role(user, role)
|
||||
|
||||
if not model_cfg:
|
||||
return f"spawn_agent: no model configured for role '{role}'"
|
||||
|
||||
model_type = model_cfg.get("type", "unknown")
|
||||
|
||||
if model_type not in ("local_openai", "gemini_api"):
|
||||
return (
|
||||
f"spawn_agent: model type '{model_type}' does not support tool-enabled sub-agents. "
|
||||
f"Assign a local_openai or gemini_api model to role '{role}'."
|
||||
)
|
||||
|
||||
# Determine concurrency key and semaphore limit
|
||||
host_id = model_cfg.get("host_id")
|
||||
if host_id:
|
||||
registry = model_registry.get_registry(user)
|
||||
host = next((h for h in registry.get("hosts", []) if h["id"] == host_id), None)
|
||||
max_concurrent = (host or {}).get("max_concurrent", 3)
|
||||
sem_key = f"host:{host_id}"
|
||||
else:
|
||||
max_concurrent = 5 if model_type == "gemini_api" else 3
|
||||
sem_key = f"type:{model_type}"
|
||||
|
||||
sem = await _get_semaphore(sem_key, max_concurrent)
|
||||
|
||||
system_prompt = load_context(
|
||||
tier=tier,
|
||||
include_long=(tier >= 2),
|
||||
include_mid=(tier >= 2),
|
||||
include_short=(tier >= 2),
|
||||
role_append=role_cfg.get("system_append", ""),
|
||||
inject_datetime=role_cfg.get("inject_datetime", True),
|
||||
)
|
||||
|
||||
user_role = get_user_role(user)
|
||||
tool_list = role_cfg.get("tools")
|
||||
policy = get_tool_policy(user)
|
||||
confirm_allow = set(policy.get("allow", []))
|
||||
confirm_deny = set(policy.get("deny", []))
|
||||
|
||||
# Per-call tool restrictions — role config remains the authoritative ceiling
|
||||
if allow_tools is not None:
|
||||
if tool_list is not None:
|
||||
tool_list = [t for t in tool_list if t in allow_tools]
|
||||
else:
|
||||
tool_list = list(allow_tools)
|
||||
|
||||
if deny_tools is not None:
|
||||
deny_set = set(deny_tools)
|
||||
if tool_list is not None:
|
||||
tool_list = [t for t in tool_list if t not in deny_set]
|
||||
else:
|
||||
confirm_deny = confirm_deny | deny_set
|
||||
|
||||
# Level enforcement: children of this agent are at level _agent_level + 1.
|
||||
# Level 3 children cannot delegate — auto-deny the spawning tools.
|
||||
child_level = _agent_level + 1
|
||||
if child_level >= 3:
|
||||
l3_deny = set(_L3_DENY_TOOLS)
|
||||
if tool_list is not None:
|
||||
tool_list = [t for t in tool_list if t not in l3_deny]
|
||||
else:
|
||||
confirm_deny = confirm_deny | l3_deny
|
||||
|
||||
if max_rounds is not None:
|
||||
model_cfg = dict(model_cfg)
|
||||
model_cfg["max_rounds"] = max_rounds
|
||||
|
||||
async def _run() -> str:
|
||||
if model_type == "local_openai":
|
||||
import openai_orchestrator
|
||||
result = await openai_orchestrator.run(
|
||||
task=task,
|
||||
system_prompt=system_prompt,
|
||||
model_cfg=model_cfg,
|
||||
respond_with_final=True,
|
||||
user_role=user_role,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=confirm_allow,
|
||||
confirm_deny=confirm_deny,
|
||||
)
|
||||
if result.checkpoint:
|
||||
return (
|
||||
"Sub-agent requires user confirmation — "
|
||||
"confirmation gates are not supported inside spawn_agent. "
|
||||
"Pre-allow the tool in the user's tool policy or use a different role."
|
||||
)
|
||||
return result.response or "(sub-agent returned no output)"
|
||||
|
||||
# gemini_api
|
||||
import orchestrator_engine
|
||||
from auth_utils import get_user_gemini_key
|
||||
gemini_key = model_cfg.get("api_key") or get_user_gemini_key(user)
|
||||
result = await orchestrator_engine.run(
|
||||
task=task,
|
||||
system_prompt=system_prompt,
|
||||
session_messages=None,
|
||||
respond_with_claude=True,
|
||||
gemini_api_key=gemini_key,
|
||||
model_name=model_cfg.get("model_name"),
|
||||
response_role=role,
|
||||
user_role=user_role,
|
||||
tool_list=tool_list,
|
||||
confirm_allow=confirm_allow,
|
||||
confirm_deny=confirm_deny,
|
||||
max_rounds=model_cfg.get("max_rounds"),
|
||||
)
|
||||
if result.checkpoint:
|
||||
return (
|
||||
"Sub-agent requires user confirmation — "
|
||||
"confirmation gates are not supported inside spawn_agent."
|
||||
)
|
||||
return result.response or "(sub-agent returned no output)"
|
||||
|
||||
if background:
|
||||
rec = await agent_manager.register(
|
||||
user=user,
|
||||
role=role,
|
||||
task=task,
|
||||
level=_agent_level,
|
||||
notify=notify,
|
||||
)
|
||||
|
||||
async def _bg_task() -> None:
|
||||
async with sem:
|
||||
try:
|
||||
logger.info(
|
||||
"spawn_agent [bg]: %s role=%s level=%d timeout=%ds",
|
||||
rec.agent_id[:8], role, _agent_level, timeout,
|
||||
)
|
||||
result = await asyncio.wait_for(_run(), timeout=float(timeout))
|
||||
await agent_manager.finish(rec.agent_id, result, "done")
|
||||
logger.info("spawn_agent [bg]: done %s", rec.agent_id[:8])
|
||||
except asyncio.CancelledError:
|
||||
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
|
||||
raise
|
||||
except asyncio.TimeoutError:
|
||||
msg = f"Sub-agent timed out after {timeout}s (role={role})"
|
||||
logger.warning("spawn_agent [bg]: timeout %s", rec.agent_id[:8])
|
||||
await agent_manager.finish(rec.agent_id, msg, "timeout")
|
||||
except Exception as e:
|
||||
logger.exception("spawn_agent [bg]: failed %s", rec.agent_id[:8])
|
||||
await agent_manager.finish(rec.agent_id, str(e), "failed")
|
||||
|
||||
bg = asyncio.create_task(_bg_task())
|
||||
agent_manager.set_task_ref(rec.agent_id, bg)
|
||||
return f"Agent started in background. ID: {rec.agent_id}\nUse agent_status('{rec.agent_id}') to check progress."
|
||||
|
||||
# Synchronous path — unchanged behaviour
|
||||
async with sem:
|
||||
try:
|
||||
logger.info(
|
||||
"spawn_agent: role=%s tier=%d timeout=%ds task=%.80s",
|
||||
role, tier, timeout, task,
|
||||
)
|
||||
response = await asyncio.wait_for(_run(), timeout=float(timeout))
|
||||
logger.info("spawn_agent: done role=%s response=%d chars", role, len(response))
|
||||
return response
|
||||
except asyncio.TimeoutError:
|
||||
logger.warning("spawn_agent: timed out after %ds role=%s", timeout, role)
|
||||
return f"Sub-agent timed out after {timeout}s (role={role})"
|
||||
except Exception as e:
|
||||
logger.exception("spawn_agent: failed role=%s", role)
|
||||
return f"Sub-agent error ({role}): {e}"
|
||||
|
||||
|
||||
# ── Agent lifecycle tools ─────────────────────────────────────────────────────
|
||||
|
||||
async def agent_status(agent_id: str) -> str:
|
||||
"""Return the status and result preview of a background agent."""
|
||||
from persona import get_user
|
||||
user = get_user() or "unknown"
|
||||
rec = agent_manager.get(agent_id)
|
||||
if not rec:
|
||||
return f"No agent found with ID: {agent_id}"
|
||||
if rec.user != user:
|
||||
return "Access denied."
|
||||
|
||||
now = datetime.now()
|
||||
end = rec.finished or now
|
||||
elapsed = int((end - rec.started).total_seconds())
|
||||
|
||||
lines = [
|
||||
f"Agent {rec.agent_id[:8]}…",
|
||||
f" Status: {rec.status}",
|
||||
f" Role: {rec.role} (Level {rec.level})",
|
||||
f" Elapsed: {elapsed}s",
|
||||
f" Started: {rec.started.strftime('%Y-%m-%d %H:%M:%S')}",
|
||||
f" Task: {rec.task}",
|
||||
]
|
||||
if rec.parent_id:
|
||||
lines.append(f" Parent: {rec.parent_id[:8]}…")
|
||||
if rec.result is not None:
|
||||
lines.append(f" Result: {rec.result[:300]}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
async def agent_list(status: str | None = None, limit: int = 10) -> str:
|
||||
"""List background agents for the current user."""
|
||||
from persona import get_user
|
||||
user = get_user() or "unknown"
|
||||
limit = min(max(int(limit), 1), 50)
|
||||
records = agent_manager.list_agents(user, status=status, limit=limit)
|
||||
|
||||
if not records:
|
||||
suffix = f" (filter: status={status})" if status else ""
|
||||
return f"No agents found.{suffix}"
|
||||
|
||||
now = datetime.now()
|
||||
lines = []
|
||||
for rec in records:
|
||||
end = rec.finished or now
|
||||
elapsed = int((end - rec.started).total_seconds())
|
||||
preview = rec.task[:60].replace("\n", " ")
|
||||
result_hint = f" → {rec.result[:50]}" if rec.result else ""
|
||||
lines.append(
|
||||
f"[{rec.agent_id[:8]}] {rec.status:<10s} L{rec.level} "
|
||||
f"{rec.role:<12s} {elapsed:>5}s {preview}{result_hint}"
|
||||
)
|
||||
|
||||
header = f"{len(records)} agent(s)" + (f" (status={status})" if status else "") + ":"
|
||||
return header + "\n" + "\n".join(lines)
|
||||
|
||||
|
||||
async def agent_cancel(agent_id: str) -> str:
|
||||
"""Cancel a running background agent."""
|
||||
from persona import get_user
|
||||
user = get_user() or "unknown"
|
||||
return await agent_manager.cancel_agent(agent_id, user)
|
||||
|
||||
|
||||
# ── Declarations ──────────────────────────────────────────────────────────────
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="spawn_agent",
|
||||
description=(
|
||||
"Spawn a sub-agent to complete a task. "
|
||||
"In synchronous mode (default): blocks until the sub-agent finishes and returns its response. "
|
||||
"In background mode (background=True): fires the agent asynchronously and returns an agent_id "
|
||||
"immediately — use agent_status() to check progress or set notify=True for a completion alert. "
|
||||
"The sub-agent uses the model and tool set assigned to the given role. "
|
||||
"Use for processing pipelines, parallel analysis, or delegating specialized work "
|
||||
"(research, coding, data migration, etc.)."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"task": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="The complete task description for the sub-agent.",
|
||||
),
|
||||
"role": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"Role determining the model and tools. "
|
||||
"E.g. 'research' for web lookups, 'coder' for code tasks, "
|
||||
"'distill' for summarization. Defaults to 'chat'."
|
||||
),
|
||||
),
|
||||
"tier": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description=(
|
||||
"Context tier: 1 = minimal (fast, identity only), "
|
||||
"2 = standard (+ memory), 3 = + last 2 session logs. "
|
||||
"Use 1 for pure processing tasks."
|
||||
),
|
||||
),
|
||||
"timeout": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Max seconds to wait (default 120). Applies in both sync and background mode.",
|
||||
),
|
||||
"max_rounds": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Override max tool-loop iterations for this call.",
|
||||
),
|
||||
"allow_tools": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
items=types.Schema(type=types.Type.STRING),
|
||||
description=(
|
||||
"Restrict the sub-agent to only these tools. "
|
||||
"Intersected with the role's tool set — cannot grant more than the role allows. "
|
||||
"Example: ['web_search', 'web_read'] for a pure research agent."
|
||||
),
|
||||
),
|
||||
"deny_tools": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
items=types.Schema(type=types.Type.STRING),
|
||||
description=(
|
||||
"Block these tools from the sub-agent regardless of role config. "
|
||||
"Example: ['shell_exec', 'file_write', 'cortex_restart']."
|
||||
),
|
||||
),
|
||||
"background": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description=(
|
||||
"Run asynchronously in the background (default: false). "
|
||||
"When true, returns an agent_id immediately instead of blocking for the result. "
|
||||
"Use agent_status(agent_id) to check progress. "
|
||||
"Best for tasks that take more than ~30 seconds."
|
||||
),
|
||||
),
|
||||
"notify": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description=(
|
||||
"Send a push/Talk notification when the background agent completes (default: false). "
|
||||
"Only meaningful when background=true."
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["task"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="agent_status",
|
||||
description=(
|
||||
"Get the current status of a background agent by ID. "
|
||||
"Returns status (running/done/failed/cancelled/timeout), role, elapsed time, "
|
||||
"task description, and result preview."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"agent_id": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="The agent ID returned by spawn_agent(background=True) or aider_run(background=True).",
|
||||
),
|
||||
},
|
||||
required=["agent_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="agent_list",
|
||||
description=(
|
||||
"List background agents for the current user. "
|
||||
"Returns recent agents with ID, status, role, level, elapsed time, and task preview. "
|
||||
"Use to survey what's running or recently completed."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"status": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Filter by status: 'running', 'done', 'failed', 'cancelled', 'timeout'. Omit for all.",
|
||||
),
|
||||
"limit": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Max agents to return (default 10, max 50).",
|
||||
),
|
||||
},
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="agent_cancel",
|
||||
description=(
|
||||
"Cancel a running background agent. ADMIN ONLY. Requires confirmation. "
|
||||
"Use agent_list() to find the agent ID first."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"agent_id": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="The agent ID to cancel.",
|
||||
),
|
||||
},
|
||||
required=["agent_id"],
|
||||
),
|
||||
),
|
||||
]
|
||||
406
cortex/tools/aider.py
Normal file
406
cortex/tools/aider.py
Normal file
@@ -0,0 +1,406 @@
|
||||
"""
|
||||
Aider coding agent tool — invokes Aider AI pair programming as a subprocess.
|
||||
|
||||
Aider handles repo-map generation, file editing, git commits, and linting automatically.
|
||||
It works with any OpenAI-compatible model — point it at DeepSeek, Ollama, OpenRouter, etc.
|
||||
via AIDER_MODEL / AIDER_OPENAI_API_BASE env vars or the project's .aider.conf.yml.
|
||||
|
||||
Credentials are pulled automatically from the Cortex model registry:
|
||||
- Named cloud providers (OpenRouter, OpenAI, Groq, Anthropic, …) → --api-key slug=key
|
||||
- Generic OpenAI-compatible hosts (Open WebUI, Ollama, local) → --openai-api-base + key
|
||||
- Anthropic from providers.anthropic.credentials → --api-key anthropic=key
|
||||
|
||||
background=True runs the subprocess asynchronously and returns an agent_id immediately.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
|
||||
import agent_manager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_CORTEX_DIR = Path(__file__).parent # .../Cortex_and_Inara_dev/cortex/
|
||||
_PROJECT_ROOT = _CORTEX_DIR.parent # .../Cortex_and_Inara_dev/
|
||||
|
||||
# Known project aliases — expand before passing to subprocess
|
||||
_PROJECT_ALIASES: dict[str, str] = {
|
||||
"cortex": str(_PROJECT_ROOT),
|
||||
"aether_api": "~/OSIT_dev/aether_api_fastapi",
|
||||
"aether_frontend": "~/OSIT_dev/aether_app_sveltekit",
|
||||
"aether_container": "~/OSIT_dev/aether_container_env",
|
||||
}
|
||||
|
||||
_MAX_OUTPUT_CHARS = 12_000
|
||||
|
||||
# Maps URL fragments → Aider --api-key provider slug.
|
||||
# Order matters: more specific patterns first.
|
||||
_CLOUD_PROVIDER_URL_MAP: list[tuple[str, str]] = [
|
||||
("openrouter.ai", "openrouter"),
|
||||
("api.openai.com", "openai"),
|
||||
("groq.com", "groq"),
|
||||
("api.together.xyz", "togetherai"),
|
||||
("fireworks.ai", "fireworks"),
|
||||
("api.x.ai", "xai"),
|
||||
("api.deepseek.com", "deepseek"),
|
||||
("api.mistral.ai", "mistral"),
|
||||
]
|
||||
|
||||
|
||||
def _provider_slug(api_url: str) -> str | None:
|
||||
"""Return the Aider --api-key provider slug for a known cloud URL, None for generic."""
|
||||
url_lower = api_url.lower()
|
||||
for fragment, slug in _CLOUD_PROVIDER_URL_MAP:
|
||||
if fragment in url_lower:
|
||||
return slug
|
||||
return None
|
||||
|
||||
|
||||
def _host_flags(host: dict, model: str | None) -> tuple[list[str], str | None]:
|
||||
"""Build Aider credential flags for a specific host entry.
|
||||
|
||||
Returns (extra_args, adjusted_model). For generic (local) endpoints the model
|
||||
name may be prefixed with 'openai/' so Aider routes through the OpenAI client.
|
||||
"""
|
||||
api_url = (host.get("api_url") or "").rstrip("/")
|
||||
api_key = host.get("api_key") or "none"
|
||||
host_type = host.get("host_type", "openai")
|
||||
slug = _provider_slug(api_url)
|
||||
|
||||
if slug:
|
||||
# Named cloud provider — Aider maps --api-key slug=key → SLUG_API_KEY env var
|
||||
flags = ["--api-key", f"{slug}={api_key}"] if api_key and api_key != "none" else []
|
||||
return flags, model
|
||||
|
||||
# Generic OpenAI-compatible (local Open WebUI, Ollama, custom)
|
||||
base_url = api_url
|
||||
if host_type == "openwebui":
|
||||
# Open WebUI serves the chat endpoint at /api/chat/completions
|
||||
base_url = base_url + "/api"
|
||||
|
||||
flags = ["--openai-api-base", base_url, "--openai-api-key", api_key]
|
||||
|
||||
# Prefix model with 'openai/' for generic endpoints when no provider prefix is set
|
||||
adj_model = model
|
||||
if model and "/" not in model:
|
||||
adj_model = f"openai/{model}"
|
||||
|
||||
return flags, adj_model
|
||||
|
||||
|
||||
def _resolve_credentials(
|
||||
registry: dict,
|
||||
model: str | None,
|
||||
host_label: str | None,
|
||||
) -> tuple[list[str], str | None]:
|
||||
"""Determine Aider credential flags and (possibly adjusted) model name.
|
||||
|
||||
Resolution order:
|
||||
1. Anthropic model hint (claude-* / anthropic/*) → Anthropic API key
|
||||
2. Explicit host_label → that host's credentials
|
||||
3. Model prefix hint (openrouter/*, groq/*, …) → matching host
|
||||
4. Default priority: OpenRouter → Anthropic → any keyed cloud host → local host
|
||||
|
||||
Returns (extra_args, adjusted_model).
|
||||
"""
|
||||
hosts = registry.get("hosts", [])
|
||||
|
||||
# Extract Anthropic key from providers.anthropic.credentials (not a host entry)
|
||||
anthropic_key = None
|
||||
for cred in registry.get("providers", {}).get("anthropic", {}).get("credentials", []):
|
||||
if cred.get("api_key"):
|
||||
anthropic_key = cred["api_key"]
|
||||
break
|
||||
|
||||
# ── 1. Anthropic model hint ────────────────────────────────────────────────
|
||||
if model and any(h in model.lower() for h in ("claude-", "anthropic/")):
|
||||
if anthropic_key:
|
||||
logger.debug("aider: Anthropic model detected — using Anthropic API key")
|
||||
return ["--api-key", f"anthropic={anthropic_key}"], model
|
||||
|
||||
# ── 2. Explicit host_label override ───────────────────────────────────────
|
||||
if host_label:
|
||||
ll = host_label.lower()
|
||||
host = next((h for h in hosts if ll in h.get("label", "").lower()), None)
|
||||
if host:
|
||||
logger.debug("aider: using explicitly requested host '%s'", host.get("label"))
|
||||
return _host_flags(host, model)
|
||||
|
||||
# ── 3. Model prefix hints ─────────────────────────────────────────────────
|
||||
if model:
|
||||
ml = model.lower()
|
||||
for fragment, slug in _CLOUD_PROVIDER_URL_MAP:
|
||||
if ml.startswith(slug + "/") or ml.startswith(fragment):
|
||||
host = next(
|
||||
(h for h in hosts if fragment in h.get("api_url", "").lower()), None
|
||||
)
|
||||
if host:
|
||||
logger.debug("aider: model prefix '%s' → host '%s'", slug, host.get("label"))
|
||||
return _host_flags(host, model)
|
||||
|
||||
# ── 4. Default priority ───────────────────────────────────────────────────
|
||||
# OpenRouter first (most model coverage)
|
||||
or_host = next((h for h in hosts if "openrouter.ai" in h.get("api_url", "")), None)
|
||||
if or_host and or_host.get("api_key"):
|
||||
logger.debug("aider: defaulting to OpenRouter")
|
||||
return _host_flags(or_host, model)
|
||||
|
||||
# Anthropic API key (no model hint but it's configured)
|
||||
if anthropic_key:
|
||||
logger.debug("aider: defaulting to Anthropic API key")
|
||||
return ["--api-key", f"anthropic={anthropic_key}"], model
|
||||
|
||||
# Any other keyed cloud host
|
||||
for host in hosts:
|
||||
slug = _provider_slug(host.get("api_url", ""))
|
||||
if slug and host.get("api_key"):
|
||||
logger.debug("aider: using keyed cloud host '%s'", host.get("label"))
|
||||
return _host_flags(host, model)
|
||||
|
||||
# Generic / local host (no key or unknown provider)
|
||||
for host in hosts:
|
||||
flags, adj_model = _host_flags(host, model)
|
||||
if flags:
|
||||
logger.debug("aider: using local host '%s'", host.get("label"))
|
||||
return flags, adj_model
|
||||
|
||||
logger.debug("aider: no credentials found in registry — relying on env vars / .aider.conf.yml")
|
||||
return [], model
|
||||
|
||||
|
||||
async def aider_run(
|
||||
project: str,
|
||||
task: str,
|
||||
files: list[str] | None = None,
|
||||
model: str | None = None,
|
||||
host_label: str | None = None,
|
||||
auto_commit: bool = True,
|
||||
timeout: int = 300,
|
||||
background: bool = False,
|
||||
notify: bool = False,
|
||||
) -> str:
|
||||
"""Run Aider with a single task in a project directory, then exit.
|
||||
|
||||
Credentials are resolved automatically from the Cortex model registry. Use
|
||||
host_label to pick a specific configured host (e.g. 'OpenRouter', 'Local').
|
||||
|
||||
When background=True, fires the subprocess asynchronously and returns an agent_id
|
||||
immediately. Use agent_status(agent_id) to check progress; set notify=True to
|
||||
receive a push/Talk notification on completion.
|
||||
"""
|
||||
resolved = _PROJECT_ALIASES.get(project, project)
|
||||
cwd = Path(os.path.expanduser(resolved))
|
||||
|
||||
if not cwd.is_dir():
|
||||
return f"Error: project directory '{resolved}' does not exist."
|
||||
|
||||
timeout = min(max(int(timeout), 10), 600)
|
||||
|
||||
# Resolve credentials before building the command (model name may be adjusted)
|
||||
user = "scott"
|
||||
extra_cred_flags: list[str] = []
|
||||
try:
|
||||
import model_registry
|
||||
from persona import get_user
|
||||
user = get_user() or "scott"
|
||||
registry = model_registry.get_registry(user)
|
||||
extra_cred_flags, model = _resolve_credentials(registry, model, host_label)
|
||||
except Exception as e:
|
||||
logger.debug("aider: credential resolution failed (%s) — relying on env", e)
|
||||
|
||||
cmd: list[str] = [
|
||||
"aider",
|
||||
"--message", task,
|
||||
"--yes-always",
|
||||
"--no-pretty",
|
||||
"--no-stream",
|
||||
"--no-check-update",
|
||||
"--no-detect-urls",
|
||||
"--auto-commits" if auto_commit else "--no-auto-commits",
|
||||
]
|
||||
|
||||
cmd += extra_cred_flags
|
||||
|
||||
if model:
|
||||
cmd += ["--model", model]
|
||||
|
||||
for f in (files or []):
|
||||
cmd += ["--file", f]
|
||||
|
||||
logger.info(
|
||||
"aider_run: project=%s model=%s host_label=%s auto_commit=%s background=%s task=%.120s",
|
||||
project, model, host_label, auto_commit, background, task,
|
||||
)
|
||||
|
||||
async def _run() -> str:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
cwd=str(cwd),
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=float(timeout))
|
||||
|
||||
out = stdout.decode(errors="replace").strip()
|
||||
err = stderr.decode(errors="replace").strip()
|
||||
|
||||
parts = []
|
||||
if out:
|
||||
parts.append(out)
|
||||
if err:
|
||||
parts.append(f"[stderr]\n{err}")
|
||||
combined = "\n".join(parts) if parts else "(no output)"
|
||||
|
||||
if len(combined) > _MAX_OUTPUT_CHARS:
|
||||
half = _MAX_OUTPUT_CHARS // 2
|
||||
combined = (
|
||||
combined[:half]
|
||||
+ f"\n\n[... {len(combined) - _MAX_OUTPUT_CHARS} chars trimmed ...]\n\n"
|
||||
+ combined[-half:]
|
||||
)
|
||||
|
||||
if proc.returncode not in (0, 1):
|
||||
return f"[exit {proc.returncode}]\n{combined}"
|
||||
return combined
|
||||
|
||||
if background:
|
||||
rec = await agent_manager.register(
|
||||
user=user,
|
||||
role="aider",
|
||||
task=task,
|
||||
level=2,
|
||||
notify=notify,
|
||||
)
|
||||
|
||||
async def _bg_task() -> None:
|
||||
try:
|
||||
result = await _run()
|
||||
await agent_manager.finish(rec.agent_id, result, "done")
|
||||
logger.info("aider_run [bg]: done %s", rec.agent_id[:8])
|
||||
except asyncio.CancelledError:
|
||||
await agent_manager.finish(rec.agent_id, "Cancelled.", "cancelled")
|
||||
raise
|
||||
except asyncio.TimeoutError:
|
||||
msg = f"Aider timed out after {timeout}s"
|
||||
logger.warning("aider_run [bg]: timeout %s", rec.agent_id[:8])
|
||||
await agent_manager.finish(rec.agent_id, msg, "timeout")
|
||||
except FileNotFoundError:
|
||||
msg = "Error: 'aider' not found in PATH — run: pip install aider-chat"
|
||||
await agent_manager.finish(rec.agent_id, msg, "failed")
|
||||
except Exception as e:
|
||||
logger.error("aider_run [bg]: failed %s: %s", rec.agent_id[:8], e)
|
||||
await agent_manager.finish(rec.agent_id, str(e), "failed")
|
||||
|
||||
bg = asyncio.create_task(_bg_task())
|
||||
agent_manager.set_task_ref(rec.agent_id, bg)
|
||||
return (
|
||||
f"Aider task started in background. ID: {rec.agent_id}\n"
|
||||
f"Use agent_status('{rec.agent_id}') to monitor progress."
|
||||
)
|
||||
|
||||
# Synchronous path
|
||||
try:
|
||||
return await _run()
|
||||
except asyncio.TimeoutError:
|
||||
return f"Error: aider timed out after {timeout}s"
|
||||
except FileNotFoundError:
|
||||
return "Error: 'aider' not found in PATH — run: pip install aider-chat"
|
||||
except Exception as e:
|
||||
logger.error("aider_run error: %s", e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="aider_run",
|
||||
description=(
|
||||
"Run the Aider AI coding agent on a project with a single task, then exit. "
|
||||
"Aider maps the repo, edits files, runs lint checks, and optionally commits. "
|
||||
"Credentials are resolved automatically from the Cortex model registry — "
|
||||
"OpenRouter, local Open WebUI/Ollama, Anthropic API, and other configured hosts "
|
||||
"are all supported. Use host_label to pick a specific host. "
|
||||
"Set background=True for long tasks — returns an agent_id immediately and sends "
|
||||
"a notification when done. ADMIN ONLY. Requires confirmation."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"project": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"Project alias or absolute path. Known aliases: "
|
||||
"'cortex' (this project), 'aether_api', 'aether_frontend', "
|
||||
"'aether_container'. Or provide an absolute path."
|
||||
),
|
||||
),
|
||||
"task": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"Full task description sent to Aider as --message. "
|
||||
"Be specific — include file names, what to change, and why."
|
||||
),
|
||||
),
|
||||
"files": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
items=types.Schema(type=types.Type.STRING),
|
||||
description=(
|
||||
"Optional files to add explicitly to the editing context "
|
||||
"(paths relative to project root). Aider builds a repo map "
|
||||
"automatically — these get priority."
|
||||
),
|
||||
),
|
||||
"model": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"Optional model override. Format depends on the provider: "
|
||||
"'openrouter/anthropic/claude-3-5-haiku-20241022' (OpenRouter), "
|
||||
"'claude-3-5-sonnet-20241022' (Anthropic direct), "
|
||||
"'gemma-4-27b-it' or 'openai/gemma-4-27b-it' (local Open WebUI), "
|
||||
"'deepseek/deepseek-chat' (DeepSeek via OpenRouter). "
|
||||
"Defaults to the project's .aider.conf.yml model or AIDER_MODEL env var."
|
||||
),
|
||||
),
|
||||
"host_label": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"Pick a specific configured host by label (partial match, case-insensitive). "
|
||||
"Examples: 'OpenRouter', 'Local', 'scott-lt-i7-rtx'. "
|
||||
"Overrides automatic credential resolution. "
|
||||
"Omit to let credentials be chosen automatically."
|
||||
),
|
||||
),
|
||||
"auto_commit": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description=(
|
||||
"Auto-commit changes after edits (default: true). "
|
||||
"Set to false to review diffs before committing manually."
|
||||
),
|
||||
),
|
||||
"timeout": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Max seconds to wait for Aider to finish (default 300, max 600).",
|
||||
),
|
||||
"background": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description=(
|
||||
"Run asynchronously in the background (default: false). "
|
||||
"Returns an agent_id immediately; use agent_status(agent_id) to monitor. "
|
||||
"Recommended for tasks expected to take more than ~60 seconds."
|
||||
),
|
||||
),
|
||||
"notify": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description=(
|
||||
"Send a push/Talk notification when the background task completes "
|
||||
"(default: false). Only applies when background=true."
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["project", "task"],
|
||||
),
|
||||
)
|
||||
]
|
||||
@@ -17,6 +17,7 @@ import secrets
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
from persona import persona_path, get_user, get_persona
|
||||
from cron_runner import load_crons, save_crons, parse_schedule
|
||||
|
||||
@@ -57,8 +58,9 @@ def _cron_add(label: str, schedule: str, job_type: str, payload: str) -> str:
|
||||
except ValueError as e:
|
||||
return f"Bad schedule: {e}"
|
||||
|
||||
if job_type not in ("remind", "note"):
|
||||
return "Bad type: must be 'remind' or 'note'."
|
||||
_VALID_TYPES = ("remind", "note", "message", "brief", "task")
|
||||
if job_type not in _VALID_TYPES:
|
||||
return f"Bad type: must be one of {', '.join(_VALID_TYPES)}."
|
||||
|
||||
current_user = get_user()
|
||||
current_persona = get_persona()
|
||||
@@ -194,3 +196,73 @@ async def cron_toggle(cron_id: str) -> str:
|
||||
|
||||
async def reminders_clear() -> str:
|
||||
return await asyncio.to_thread(_reminders_clear)
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="cron_list",
|
||||
description=(
|
||||
"List all scheduled cron jobs — their ID, label, schedule, type, and last run time. "
|
||||
"Use this to see what's scheduled before adding or removing jobs."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="cron_add",
|
||||
description=(
|
||||
"Create a new scheduled cron job and register it immediately (no restart needed). "
|
||||
"Job types: "
|
||||
"'remind' — appends to REMINDERS.md, auto-surfaced in chat context at tier 2+; "
|
||||
"'note' — appends to SCRATCH.md, read on demand; "
|
||||
"'message' — sends payload text directly to the user's notification channel; "
|
||||
"'brief' — calls the LLM (no tools) with payload as the prompt, sends the response; "
|
||||
"'task' — runs the full orchestrator tool loop with payload as the request, sends "
|
||||
"Claude's response to the notification channel (use for agentic scheduled work: "
|
||||
"research, checks, file updates, summaries that need tool access). "
|
||||
"Schedule formats: 'hourly' | 'daily' | 'daily:HH:MM' | 'weekly:DOW' | 'weekly:DOW:HH:MM' | "
|
||||
"'monthly' | 'monthly:DD' | 'monthly:DD:HH:MM' | 'yearly:MM:DD' | 'yearly:MM:DD:HH:MM'. "
|
||||
"Examples: schedule='weekly:mon:08:00' for Monday briefings; "
|
||||
"schedule='monthly:1:09:00' for a first-of-month review; "
|
||||
"schedule='yearly:03:15' for a March 15 birthday reminder."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"label": types.Schema(type=types.Type.STRING, description="Short human-readable name for this job (e.g. 'Monday task summary')"),
|
||||
"schedule": types.Schema(type=types.Type.STRING, description="When to run: hourly | daily | daily:HH:MM | weekly:DOW | weekly:DOW:HH:MM | monthly | monthly:DD | monthly:DD:HH:MM | yearly:MM:DD | yearly:MM:DD:HH:MM"),
|
||||
"job_type": types.Schema(type=types.Type.STRING, description="remind | note | message | brief | task"),
|
||||
"payload": types.Schema(type=types.Type.STRING, description="The text/prompt to use when the job fires"),
|
||||
},
|
||||
required=["label", "schedule", "job_type", "payload"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="cron_remove",
|
||||
description=(
|
||||
"Permanently delete a scheduled cron job. Use cron_list first to get the ID. "
|
||||
"To temporarily disable without deleting, use cron_toggle instead."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"cron_id": types.Schema(type=types.Type.STRING, description="Job ID (e.g. c_abc123) — get from cron_list"),
|
||||
},
|
||||
required=["cron_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="cron_toggle",
|
||||
description=(
|
||||
"Pause a running cron job, or resume a paused one. "
|
||||
"The job stays in the list and can be re-enabled later. "
|
||||
"Use cron_list to see current enabled/paused state."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"cron_id": types.Schema(type=types.Type.STRING, description="Job ID (e.g. c_abc123) — get from cron_list"),
|
||||
},
|
||||
required=["cron_id"],
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
@@ -1,108 +1,78 @@
|
||||
"""
|
||||
File read tool — restricted to known-safe directory roots.
|
||||
File read/write/search tools — two access scopes.
|
||||
|
||||
Lets the orchestrator read local files (documentation, notes, config references)
|
||||
without exposing arbitrary filesystem access. All paths are resolved and checked
|
||||
against an allowlist of roots before any read is performed.
|
||||
Project scope (no admin required):
|
||||
project_file_read — read a file with optional line-range (offset)
|
||||
project_file_list — list a directory with sizes + timestamps
|
||||
file_stat — size, modified time, line count for a path
|
||||
file_grep — regex search with context lines; up to 50 matches
|
||||
file_syntax_check — py_compile (.py) or json.loads (.json) check
|
||||
|
||||
System scope (admin-only):
|
||||
file_read — read a file from ~/agents_sync/, ~/OSIT_dev/, etc.
|
||||
file_list — list a directory (same roots)
|
||||
file_write — write/append (~/agents_sync/ + Cortex home/)
|
||||
|
||||
Session tools (user-level, persona-isolated):
|
||||
session_read — read a session log by date
|
||||
session_search — keyword search across session logs
|
||||
|
||||
All project-scope tools are restricted to the Cortex project root:
|
||||
~/agents_sync/projects/Cortex_and_Inara_dev/
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import subprocess
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Directories the orchestrator is allowed to read from.
|
||||
# Paths are resolved (symlinks followed, ~ expanded) at import time.
|
||||
_ALLOWED_ROOTS: list[Path] = [
|
||||
# ── Access roots ──────────────────────────────────────────────────────────────
|
||||
|
||||
# Project root: two levels up from cortex/tools/files.py → Cortex_and_Inara_dev/
|
||||
_PROJECT_ROOT: Path = Path(__file__).parent.parent.parent.resolve()
|
||||
|
||||
# System-wide read roots
|
||||
def _build_allowed_roots() -> list[Path]:
|
||||
roots = [
|
||||
Path.home() / "agents_sync",
|
||||
Path.home() / "OSIT_dev",
|
||||
Path.home() / "DgrZone_Nextcloud",
|
||||
Path.home() / "OSIT_Nextcloud",
|
||||
]
|
||||
]
|
||||
try:
|
||||
from config import settings
|
||||
roots.append(settings.home_root())
|
||||
except Exception:
|
||||
pass
|
||||
return roots
|
||||
|
||||
# Hard cap on file size to prevent accidental context blowout
|
||||
_MAX_BYTES = 50_000 # ~50 KB
|
||||
_ALLOWED_ROOTS: list[Path] = _build_allowed_roots()
|
||||
|
||||
# Write is tighter
|
||||
_WRITE_ROOTS: list[Path] = [Path.home() / "agents_sync"]
|
||||
|
||||
# Size limits
|
||||
_MAX_BYTES = 50_000
|
||||
_MAX_LINES = 500
|
||||
_MAX_GREP_MATCHES = 50
|
||||
|
||||
|
||||
async def file_read(path: str, max_lines: int | None = None) -> str:
|
||||
"""Read a local file and return its contents as a string.
|
||||
|
||||
Only files within allowed directories can be read:
|
||||
~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, ~/OSIT_Nextcloud/
|
||||
|
||||
Args:
|
||||
path: Absolute or home-relative path to the file (e.g. ~/agents_sync/CLAUDE.md).
|
||||
max_lines: Optional line limit (default 500, hard cap). Use for large files.
|
||||
|
||||
Returns the file contents (truncated if over the size limit), or an error message.
|
||||
"""
|
||||
return await asyncio.to_thread(_sync_file_read, path, max_lines)
|
||||
|
||||
|
||||
def _sync_file_read(path: str, max_lines: int | None) -> str:
|
||||
# Expand ~ and resolve to absolute path
|
||||
def _is_project_allowed(resolved: Path) -> bool:
|
||||
try:
|
||||
resolved = Path(path).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
# Security check — must be under an allowed root
|
||||
if not _is_allowed(resolved):
|
||||
allowed_str = ", ".join(str(r) for r in _ALLOWED_ROOTS)
|
||||
return (
|
||||
f"Access denied: {resolved}\n"
|
||||
f"Allowed directories: {allowed_str}"
|
||||
)
|
||||
|
||||
if not resolved.exists():
|
||||
return f"File not found: {resolved}"
|
||||
|
||||
if not resolved.is_file():
|
||||
# If it's a directory, list its contents instead
|
||||
try:
|
||||
entries = sorted(resolved.iterdir())
|
||||
names = [e.name + ("/" if e.is_dir() else "") for e in entries[:100]]
|
||||
return f"Directory listing for {resolved}:\n" + "\n".join(names)
|
||||
except Exception as e:
|
||||
return f"Cannot list directory: {e}"
|
||||
|
||||
# Read the file
|
||||
try:
|
||||
raw = resolved.read_bytes()
|
||||
except Exception as e:
|
||||
return f"Read error: {e}"
|
||||
|
||||
# Binary files
|
||||
try:
|
||||
text = raw.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return f"Binary file (not readable as text): {resolved} [{len(raw)} bytes]"
|
||||
|
||||
# Apply line limit
|
||||
limit = min(max_lines or _MAX_LINES, _MAX_LINES)
|
||||
lines = text.splitlines()
|
||||
truncated = False
|
||||
|
||||
if len(lines) > limit:
|
||||
lines = lines[:limit]
|
||||
truncated = True
|
||||
|
||||
# Apply byte cap as a final safety net
|
||||
result = "\n".join(lines)
|
||||
if len(result) > _MAX_BYTES:
|
||||
result = result[:_MAX_BYTES]
|
||||
truncated = True
|
||||
|
||||
if truncated:
|
||||
result += f"\n\n… [truncated — file has {len(text.splitlines())} lines total]"
|
||||
|
||||
return result
|
||||
resolved.relative_to(_PROJECT_ROOT)
|
||||
return True
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
def _is_allowed(resolved: Path) -> bool:
|
||||
"""Check that resolved path is under one of the allowed roots."""
|
||||
for root in _ALLOWED_ROOTS:
|
||||
try:
|
||||
resolved.relative_to(root)
|
||||
@@ -110,3 +80,725 @@ def _is_allowed(resolved: Path) -> bool:
|
||||
except ValueError:
|
||||
continue
|
||||
return False
|
||||
|
||||
|
||||
def _is_write_allowed(resolved: Path) -> bool:
|
||||
for root in _WRITE_ROOTS:
|
||||
try:
|
||||
resolved.relative_to(root)
|
||||
return True
|
||||
except ValueError:
|
||||
continue
|
||||
try:
|
||||
from config import settings
|
||||
resolved.relative_to(settings.home_root())
|
||||
return True
|
||||
except (ValueError, Exception):
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
# ── Shared implementations ────────────────────────────────────────────────────
|
||||
|
||||
def _read_impl(path_str: str, offset: int | None, max_lines: int | None, is_allowed_fn) -> str:
|
||||
try:
|
||||
resolved = Path(path_str).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
if not is_allowed_fn(resolved):
|
||||
return f"Access denied: {resolved}"
|
||||
|
||||
if not resolved.exists():
|
||||
return f"File not found: {resolved}"
|
||||
|
||||
if not resolved.is_file():
|
||||
try:
|
||||
entries = sorted(resolved.iterdir())
|
||||
names = [e.name + ("/" if e.is_dir() else "") for e in entries[:100]]
|
||||
return f"Directory listing for {resolved}:\n" + "\n".join(names)
|
||||
except Exception as e:
|
||||
return f"Cannot list directory: {e}"
|
||||
|
||||
try:
|
||||
raw = resolved.read_bytes()
|
||||
except Exception as e:
|
||||
return f"Read error: {e}"
|
||||
|
||||
try:
|
||||
text = raw.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
return f"Binary file (not readable as text): {resolved} [{len(raw)} bytes]"
|
||||
|
||||
all_lines = text.splitlines()
|
||||
total = len(all_lines)
|
||||
|
||||
# offset is 1-based; default = start of file
|
||||
start = max(0, (offset or 1) - 1)
|
||||
working = all_lines[start:]
|
||||
|
||||
limit = min(max_lines or _MAX_LINES, _MAX_LINES)
|
||||
truncated = False
|
||||
if len(working) > limit:
|
||||
working = working[:limit]
|
||||
truncated = True
|
||||
|
||||
result = "\n".join(working)
|
||||
if len(result) > _MAX_BYTES:
|
||||
result = result[:_MAX_BYTES]
|
||||
truncated = True
|
||||
|
||||
end_line = start + len(working)
|
||||
header = f"[Lines {start + 1}–{end_line} of {total}]\n" if (start > 0 or truncated) else ""
|
||||
trailer = f"\n\n… [truncated — file has {total} lines; use offset={end_line + 1} to read more]" if truncated else ""
|
||||
|
||||
return header + result + trailer
|
||||
|
||||
|
||||
def _list_impl(path_str: str, is_allowed_fn) -> str:
|
||||
try:
|
||||
resolved = Path(path_str).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
if not is_allowed_fn(resolved):
|
||||
return f"Access denied: {resolved}"
|
||||
|
||||
if not resolved.exists():
|
||||
return f"Path not found: {resolved}"
|
||||
|
||||
if resolved.is_file():
|
||||
return f"{resolved} is a file. Use file_read / project_file_read to read it."
|
||||
|
||||
try:
|
||||
entries = sorted(resolved.iterdir(), key=lambda e: (e.is_file(), e.name.lower()))
|
||||
lines = []
|
||||
for e in entries[:200]:
|
||||
if e.is_dir():
|
||||
suffix = "/"
|
||||
else:
|
||||
try:
|
||||
st = e.stat()
|
||||
mtime = datetime.fromtimestamp(st.st_mtime).strftime("%Y-%m-%d %H:%M")
|
||||
suffix = f" ({st.st_size:,} B, {mtime})"
|
||||
except Exception:
|
||||
suffix = ""
|
||||
lines.append(f"{e.name}{suffix}")
|
||||
result = "\n".join(lines)
|
||||
if len(entries) > 200:
|
||||
result += f"\n… ({len(entries) - 200} more not shown)"
|
||||
return f"Contents of {resolved}:\n\n{result}"
|
||||
except Exception as e:
|
||||
return f"Cannot list directory: {e}"
|
||||
|
||||
|
||||
# ── Project-scoped tools ──────────────────────────────────────────────────────
|
||||
|
||||
async def project_file_read(path: str, offset: int | None = None, max_lines: int | None = None) -> str:
|
||||
"""Read a file within the Cortex project directory, with optional line range."""
|
||||
return await asyncio.to_thread(_read_impl, path, offset, max_lines, _is_project_allowed)
|
||||
|
||||
|
||||
async def project_file_list(path: str) -> str:
|
||||
"""List directory contents within the Cortex project directory, with sizes and timestamps."""
|
||||
return await asyncio.to_thread(_list_impl, path, _is_project_allowed)
|
||||
|
||||
|
||||
async def file_stat(path: str) -> str:
|
||||
"""Return metadata for a file or directory: type, size, modified time, line count."""
|
||||
return await asyncio.to_thread(_sync_file_stat, path)
|
||||
|
||||
|
||||
def _sync_file_stat(path_str: str) -> str:
|
||||
try:
|
||||
resolved = Path(path_str).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
if not _is_project_allowed(resolved):
|
||||
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
|
||||
|
||||
if not resolved.exists():
|
||||
return f"Path not found: {resolved}"
|
||||
|
||||
try:
|
||||
st = resolved.stat()
|
||||
except Exception as e:
|
||||
return f"Cannot stat: {e}"
|
||||
|
||||
modified = datetime.fromtimestamp(st.st_mtime).strftime("%Y-%m-%d %H:%M:%S")
|
||||
lines = [
|
||||
f"Path: {resolved}",
|
||||
f"Type: {'directory' if resolved.is_dir() else 'file'}",
|
||||
f"Size: {st.st_size:,} bytes",
|
||||
f"Modified: {modified}",
|
||||
]
|
||||
|
||||
if resolved.is_file():
|
||||
try:
|
||||
raw = resolved.read_bytes()
|
||||
if b'\x00' not in raw[:1024]:
|
||||
lines.append(f"Lines: {len(raw.decode('utf-8', errors='replace').splitlines())}")
|
||||
except Exception:
|
||||
pass
|
||||
elif resolved.is_dir():
|
||||
try:
|
||||
entries = list(resolved.iterdir())
|
||||
n_files = sum(1 for e in entries if e.is_file())
|
||||
n_dirs = sum(1 for e in entries if e.is_dir())
|
||||
lines.append(f"Contents: {n_files} file(s), {n_dirs} subdirector{'y' if n_dirs == 1 else 'ies'}")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
async def file_grep(path: str, pattern: str, context_lines: int = 2, recursive: bool = True) -> str:
|
||||
"""Search for a regex pattern in a file or directory, returning matching lines with context."""
|
||||
return await asyncio.to_thread(_sync_file_grep, path, pattern, context_lines, recursive)
|
||||
|
||||
|
||||
def _sync_file_grep(path_str: str, pattern: str, context_lines: int, recursive: bool) -> str:
|
||||
try:
|
||||
resolved = Path(path_str).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
if not _is_project_allowed(resolved):
|
||||
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
|
||||
|
||||
if not resolved.exists():
|
||||
return f"Path not found: {resolved}"
|
||||
|
||||
try:
|
||||
regex = re.compile(pattern, re.IGNORECASE)
|
||||
except re.error as e:
|
||||
return f"Invalid regex pattern: {e}"
|
||||
|
||||
ctx = max(0, min(context_lines, 5))
|
||||
|
||||
if resolved.is_file():
|
||||
files_to_search = [resolved]
|
||||
elif recursive:
|
||||
files_to_search = sorted(f for f in resolved.rglob("*") if f.is_file())
|
||||
else:
|
||||
files_to_search = sorted(f for f in resolved.iterdir() if f.is_file())
|
||||
|
||||
total_matches = 0
|
||||
sections: list[str] = []
|
||||
capped = False
|
||||
|
||||
for fp in files_to_search:
|
||||
if total_matches >= _MAX_GREP_MATCHES:
|
||||
capped = True
|
||||
break
|
||||
try:
|
||||
raw = fp.read_bytes()
|
||||
except OSError:
|
||||
continue
|
||||
if b'\x00' in raw[:1024]:
|
||||
continue # skip binary
|
||||
try:
|
||||
text = raw.decode("utf-8", errors="replace")
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
file_lines = text.splitlines()
|
||||
match_indices = [i for i, line in enumerate(file_lines) if regex.search(line)]
|
||||
if not match_indices:
|
||||
continue
|
||||
|
||||
total_matches += len(match_indices)
|
||||
|
||||
try:
|
||||
label = str(fp.relative_to(_PROJECT_ROOT))
|
||||
except ValueError:
|
||||
label = str(fp)
|
||||
|
||||
file_output = [f"── {label} ──"]
|
||||
printed: set[int] = set()
|
||||
|
||||
for mi in match_indices:
|
||||
start = max(0, mi - ctx)
|
||||
end = min(len(file_lines), mi + ctx + 1)
|
||||
if printed and start > max(printed) + 1:
|
||||
file_output.append(" ···")
|
||||
for j in range(start, end):
|
||||
if j not in printed:
|
||||
marker = "►" if j == mi else " "
|
||||
file_output.append(f" {j + 1:4d}{marker} {file_lines[j]}")
|
||||
printed.add(j)
|
||||
|
||||
sections.append("\n".join(file_output))
|
||||
|
||||
if not sections:
|
||||
return f"No matches for '{pattern}' in {resolved}"
|
||||
|
||||
cap_note = f" (capped at {_MAX_GREP_MATCHES})" if capped else ""
|
||||
header = f"grep '{pattern}' — {total_matches} match(es){cap_note}:"
|
||||
return header + "\n\n" + "\n\n".join(sections)
|
||||
|
||||
|
||||
async def file_diff(path_a: str, path_b: str) -> str:
|
||||
"""Compare two files and return a unified diff."""
|
||||
return await asyncio.to_thread(_sync_file_diff, path_a, path_b)
|
||||
|
||||
|
||||
def _sync_file_diff(path_a: str, path_b: str) -> str:
|
||||
try:
|
||||
resolved_a = Path(path_a).expanduser().resolve()
|
||||
resolved_b = Path(path_b).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
for resolved in (resolved_a, resolved_b):
|
||||
if not _is_project_allowed(resolved):
|
||||
return f"Access denied: {resolved}"
|
||||
if not resolved.exists():
|
||||
return f"File not found: {resolved}"
|
||||
if not resolved.is_file():
|
||||
return f"Not a file: {resolved}"
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["diff", "-u", str(resolved_a), str(resolved_b)],
|
||||
capture_output=True, text=True, timeout=15,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return f"Files are identical: {resolved_a.name} vs {resolved_b.name}"
|
||||
output = result.stdout
|
||||
if not output:
|
||||
return f"diff returned no output (exit {result.returncode}): {result.stderr}"
|
||||
if len(output) > _MAX_BYTES:
|
||||
output = output[:_MAX_BYTES] + "\n… [truncated]"
|
||||
return output
|
||||
except subprocess.TimeoutExpired:
|
||||
return "Timeout running diff"
|
||||
except Exception as e:
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
async def file_syntax_check(path: str) -> str:
|
||||
"""Check syntax of a Python (.py) or JSON (.json) file."""
|
||||
return await asyncio.to_thread(_sync_file_syntax_check, path)
|
||||
|
||||
|
||||
def _sync_file_syntax_check(path_str: str) -> str:
|
||||
try:
|
||||
resolved = Path(path_str).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
if not _is_project_allowed(resolved):
|
||||
return f"Access denied: {resolved}\nProject root: {_PROJECT_ROOT}"
|
||||
|
||||
if not resolved.exists():
|
||||
return f"File not found: {resolved}"
|
||||
|
||||
if not resolved.is_file():
|
||||
return f"Not a file: {resolved}"
|
||||
|
||||
suffix = resolved.suffix.lower()
|
||||
|
||||
if suffix == ".py":
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["python3", "-m", "py_compile", str(resolved)],
|
||||
capture_output=True, text=True, timeout=15,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return f"OK — {resolved.name}: syntax valid"
|
||||
err = (result.stderr or result.stdout).strip()
|
||||
return f"Syntax error in {resolved.name}:\n{err}"
|
||||
except subprocess.TimeoutExpired:
|
||||
return f"Timeout running py_compile on {resolved.name}"
|
||||
except Exception as e:
|
||||
return f"Error: {e}"
|
||||
|
||||
elif suffix == ".json":
|
||||
try:
|
||||
text = resolved.read_text(encoding="utf-8")
|
||||
json.loads(text)
|
||||
return f"OK — {resolved.name}: valid JSON"
|
||||
except json.JSONDecodeError as e:
|
||||
return f"JSON error in {resolved.name}: {e}"
|
||||
except Exception as e:
|
||||
return f"Error reading {resolved.name}: {e}"
|
||||
|
||||
else:
|
||||
return f"Syntax check not supported for '{suffix}' files. Supported: .py, .json"
|
||||
|
||||
|
||||
# ── System-scoped tools ───────────────────────────────────────────────────────
|
||||
|
||||
async def file_read(path: str, offset: int | None = None, max_lines: int | None = None) -> str:
|
||||
"""Read a local file from the broader system. Allowed: ~/agents_sync/, ~/OSIT_dev/, etc. ADMIN ONLY."""
|
||||
return await asyncio.to_thread(_read_impl, path, offset, max_lines, _is_allowed)
|
||||
|
||||
|
||||
async def file_list(path: str) -> str:
|
||||
"""List directory contents from the broader system. ADMIN ONLY."""
|
||||
return await asyncio.to_thread(_list_impl, path, _is_allowed)
|
||||
|
||||
|
||||
async def file_write(path: str, content: str, mode: str = "overwrite") -> str:
|
||||
"""Write or append content to a file. Write roots: ~/agents_sync/ and Cortex home/. ADMIN ONLY."""
|
||||
return await asyncio.to_thread(_sync_file_write, path, content, mode)
|
||||
|
||||
|
||||
def _sync_file_write(path: str, content: str, mode: str) -> str:
|
||||
try:
|
||||
resolved = Path(path).expanduser().resolve()
|
||||
except Exception as e:
|
||||
return f"Invalid path: {e}"
|
||||
|
||||
if not _is_write_allowed(resolved):
|
||||
return (
|
||||
f"Write access denied: {resolved}\n"
|
||||
f"Allowed write roots: ~/agents_sync/ and the Cortex home/ directory."
|
||||
)
|
||||
|
||||
if mode not in ("overwrite", "append"):
|
||||
return f"Invalid mode '{mode}' — use 'overwrite' or 'append'."
|
||||
|
||||
try:
|
||||
resolved.parent.mkdir(parents=True, exist_ok=True)
|
||||
if mode == "append":
|
||||
with resolved.open("a", encoding="utf-8") as f:
|
||||
f.write(content)
|
||||
return f"Appended {len(content)} chars to {resolved}"
|
||||
else:
|
||||
resolved.write_text(content, encoding="utf-8")
|
||||
return f"Wrote {len(content)} chars to {resolved}"
|
||||
except Exception as e:
|
||||
logger.error("file_write error for %s: %s", resolved, e)
|
||||
return f"Write error: {e}"
|
||||
|
||||
|
||||
# ── Session tools ─────────────────────────────────────────────────────────────
|
||||
|
||||
_SEARCH_EXCERPT_CHARS = 150
|
||||
|
||||
|
||||
async def session_read(date: str) -> str:
|
||||
"""Read a full session log by date (YYYY-MM-DD)."""
|
||||
return await asyncio.to_thread(_sync_session_read, date.strip())
|
||||
|
||||
|
||||
def _sync_session_read(date: str) -> str:
|
||||
from persona import persona_path
|
||||
sessions_dir = persona_path() / "sessions"
|
||||
if not sessions_dir.exists():
|
||||
return "No session logs found."
|
||||
|
||||
target = sessions_dir / f"{date}.md"
|
||||
if target.exists():
|
||||
content = target.read_text()
|
||||
return f"Session log for {date} ({len(content)} chars):\n\n{content}"
|
||||
|
||||
available = sorted([f.stem for f in sessions_dir.glob("*.md")], reverse=True)
|
||||
if not available:
|
||||
return "No session logs found."
|
||||
recent = "\n".join(f" {d}" for d in available[:15])
|
||||
return f"No session log found for '{date}'. Available dates (most recent first):\n{recent}"
|
||||
|
||||
|
||||
async def session_search(query: str, limit: int = 5) -> str:
|
||||
"""Search past session logs for a keyword or phrase."""
|
||||
return await asyncio.to_thread(_sync_session_search, query, limit)
|
||||
|
||||
|
||||
def _sync_session_search(query: str, limit: int) -> str:
|
||||
from persona import persona_path
|
||||
sessions_dir = persona_path() / "sessions"
|
||||
if not sessions_dir.exists():
|
||||
return "No session logs found."
|
||||
|
||||
limit = max(1, min(limit, 20))
|
||||
pattern = re.compile(re.escape(query), re.IGNORECASE)
|
||||
session_files = sorted(sessions_dir.glob("*.md"), reverse=True)
|
||||
|
||||
matches = []
|
||||
for sf in session_files:
|
||||
if len(matches) >= limit:
|
||||
break
|
||||
try:
|
||||
text = sf.read_text()
|
||||
except OSError:
|
||||
continue
|
||||
for m in pattern.finditer(text):
|
||||
if len(matches) >= limit:
|
||||
break
|
||||
start = max(0, m.start() - _SEARCH_EXCERPT_CHARS)
|
||||
end = min(len(text), m.end() + _SEARCH_EXCERPT_CHARS)
|
||||
excerpt = text[start:end].strip()
|
||||
if start > 0:
|
||||
excerpt = "…" + excerpt
|
||||
if end < len(text):
|
||||
excerpt = excerpt + "…"
|
||||
matches.append(f"[{sf.stem}] {excerpt}")
|
||||
|
||||
if not matches:
|
||||
return f"No matches for '{query}' across {len(session_files)} session logs."
|
||||
header = f"Session search: '{query}' — {len(matches)} match(es) across {len(session_files)} logs\n"
|
||||
return header + "\n\n".join(matches)
|
||||
|
||||
|
||||
# ── Declarations ──────────────────────────────────────────────────────────────
|
||||
|
||||
DECLARATIONS = [
|
||||
# Project-scoped
|
||||
types.FunctionDeclaration(
|
||||
name="project_file_read",
|
||||
description=(
|
||||
"Read a file within the Cortex project directory (source code, docs, config, persona files). "
|
||||
"Supports reading a specific line range via offset — use to page through large files "
|
||||
"without re-reading from the top. If given a directory path, returns a listing instead. "
|
||||
"Project root: ~/agents_sync/projects/Cortex_and_Inara_dev/"
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Absolute or ~/... path to the file",
|
||||
),
|
||||
"offset": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Start reading from this line number (1-based). Omit to read from the top.",
|
||||
),
|
||||
"max_lines": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Maximum lines to return (default 500)",
|
||||
),
|
||||
},
|
||||
required=["path"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="project_file_list",
|
||||
description=(
|
||||
"List files and subdirectories within the Cortex project directory. "
|
||||
"Shows file sizes and modified timestamps. "
|
||||
"Project root: ~/agents_sync/projects/Cortex_and_Inara_dev/"
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Absolute or ~/... path to the directory",
|
||||
),
|
||||
},
|
||||
required=["path"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="file_stat",
|
||||
description=(
|
||||
"Get metadata for a file or directory: type, size, modified timestamp, line count (for text files) "
|
||||
"or entry counts (for directories). Use before reading to check recency or size. "
|
||||
"Restricted to the Cortex project directory."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Absolute or ~/... path to the file or directory",
|
||||
),
|
||||
},
|
||||
required=["path"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="file_grep",
|
||||
description=(
|
||||
"Search for a regex pattern in a file or directory, returning matching lines with surrounding "
|
||||
"context. Much more efficient than reading an entire source file — use this to find function "
|
||||
"definitions, variable names, TODO comments, imports, error strings, etc. "
|
||||
"Searches recursively by default. Capped at 50 matches. Skips binary files. "
|
||||
"Case-insensitive. Restricted to the Cortex project directory."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="File or directory to search (e.g. ~/agents_sync/projects/Cortex_and_Inara_dev/cortex/)",
|
||||
),
|
||||
"pattern": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Regex pattern to search for (case-insensitive). Examples: 'def ha_', 'import httpx', 'TODO'",
|
||||
),
|
||||
"context_lines": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Lines of context before/after each match (default 2, max 5)",
|
||||
),
|
||||
"recursive": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description="Search subdirectories recursively (default true)",
|
||||
),
|
||||
},
|
||||
required=["path", "pattern"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="file_diff",
|
||||
description=(
|
||||
"Compare two files and return a unified diff (diff -u). "
|
||||
"Use for code review, verifying what changed between two versions of a file, "
|
||||
"or comparing config files side-by-side. "
|
||||
"Returns 'Files are identical' if there are no differences. "
|
||||
"Restricted to the Cortex project directory."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path_a": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Path to the first file (the 'before' or reference file)",
|
||||
),
|
||||
"path_b": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Path to the second file (the 'after' or comparison file)",
|
||||
),
|
||||
},
|
||||
required=["path_a", "path_b"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="file_syntax_check",
|
||||
description=(
|
||||
"Check the syntax of a Python (.py) or JSON (.json) file without executing it. "
|
||||
"Returns OK or the error with line number. "
|
||||
"Use after editing a file before restarting Cortex. "
|
||||
"Restricted to the Cortex project directory."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Path to the .py or .json file to check",
|
||||
),
|
||||
},
|
||||
required=["path"],
|
||||
),
|
||||
),
|
||||
# System-scoped
|
||||
types.FunctionDeclaration(
|
||||
name="file_read",
|
||||
description=(
|
||||
"Read a local file from the broader system (~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, "
|
||||
"~/OSIT_Nextcloud/, Cortex home/). Supports offset for reading specific line ranges. "
|
||||
"For files within the Cortex project, prefer project_file_read instead. "
|
||||
"ADMIN ONLY."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Absolute or ~/... path to the file",
|
||||
),
|
||||
"offset": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Start reading from this line number (1-based)",
|
||||
),
|
||||
"max_lines": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Maximum lines to return (default 500)",
|
||||
),
|
||||
},
|
||||
required=["path"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="file_list",
|
||||
description=(
|
||||
"List files and subdirectories from the broader system. "
|
||||
"Shows sizes and modified timestamps. "
|
||||
"Allowed: ~/agents_sync/, ~/OSIT_dev/, ~/DgrZone_Nextcloud/, ~/OSIT_Nextcloud/. "
|
||||
"ADMIN ONLY."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Absolute or ~/... path to the directory",
|
||||
),
|
||||
},
|
||||
required=["path"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="file_write",
|
||||
description=(
|
||||
"Write or append content to a file. "
|
||||
"Write-allowed paths: ~/agents_sync/ and the Cortex home/ directory. "
|
||||
"Creates parent directories if needed. "
|
||||
"ADMIN ONLY. Requires user confirmation before executing."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Absolute or ~/... path to write to",
|
||||
),
|
||||
"content": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Content to write",
|
||||
),
|
||||
"mode": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="'overwrite' (default, replaces file) or 'append' (adds to end)",
|
||||
),
|
||||
},
|
||||
required=["path", "content"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="session_read",
|
||||
description=(
|
||||
"Read a full conversation session log by date (YYYY-MM-DD). "
|
||||
"Useful for continuity and recalling past decisions. "
|
||||
"If the date is not found, lists available dates. "
|
||||
"Only reads this user's own sessions."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"date": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Date in YYYY-MM-DD format (e.g. '2026-05-08')",
|
||||
),
|
||||
},
|
||||
required=["date"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="session_search",
|
||||
description=(
|
||||
"Search past conversation session logs for a keyword or phrase. "
|
||||
"Returns matching excerpts with session dates, newest first. "
|
||||
"Only searches this user's own sessions."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"query": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Keyword or phrase to search for",
|
||||
),
|
||||
"limit": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Max results to return (default 5, max 20)",
|
||||
),
|
||||
},
|
||||
required=["query"],
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
158
cortex/tools/git.py
Normal file
158
cortex/tools/git.py
Normal file
@@ -0,0 +1,158 @@
|
||||
"""
|
||||
Git inspection tools — project-scoped, read-only.
|
||||
|
||||
git_status — working tree status (staged, unstaged, untracked changes)
|
||||
git_log — recent commit history with optional path filter
|
||||
git_diff — diff between commits, branches, or working tree vs HEAD
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_PROJECT_ROOT: Path = Path(__file__).parent.parent.parent.resolve()
|
||||
_MAX_OUTPUT = 50_000
|
||||
|
||||
|
||||
async def _git(*args: str, timeout: int = 15) -> tuple[int, str]:
|
||||
"""Run a git command in the project root. Returns (returncode, output)."""
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
"git", "-C", str(_PROJECT_ROOT), *args,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
try:
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
|
||||
except asyncio.TimeoutError:
|
||||
proc.kill()
|
||||
return 1, "git command timed out"
|
||||
out = (stdout or b"").decode(errors="replace").strip()
|
||||
err = (stderr or b"").decode(errors="replace").strip()
|
||||
combined = out if out else err
|
||||
return proc.returncode, combined
|
||||
|
||||
|
||||
def _cap(text: str) -> str:
|
||||
if len(text) > _MAX_OUTPUT:
|
||||
return text[:_MAX_OUTPUT] + "\n… [truncated]"
|
||||
return text
|
||||
|
||||
|
||||
async def git_status() -> str:
|
||||
"""Return the current git working tree status."""
|
||||
rc, out = await _git("status")
|
||||
if rc != 0:
|
||||
return f"git status failed: {out}"
|
||||
return out or "Working tree clean — nothing to report."
|
||||
|
||||
|
||||
async def git_log(n: int = 20, path: str = "", oneline: bool = True) -> str:
|
||||
"""Return recent git commit history."""
|
||||
args = ["log"]
|
||||
if oneline:
|
||||
args += ["--oneline"]
|
||||
else:
|
||||
args += ["--format=%H %as %an%n %s", "--date=short"]
|
||||
args += [f"-{max(1, min(n, 200))}"]
|
||||
if path:
|
||||
args += ["--", path]
|
||||
rc, out = await _git(*args)
|
||||
if rc != 0:
|
||||
return f"git log failed: {out}"
|
||||
return _cap(out) or "No commits found."
|
||||
|
||||
|
||||
async def git_diff(ref_a: str = "", ref_b: str = "", path: str = "", stat_only: bool = False) -> str:
|
||||
"""Show a git diff. Defaults to working tree vs HEAD (unstaged changes)."""
|
||||
args = ["diff"]
|
||||
if stat_only:
|
||||
args += ["--stat"]
|
||||
if ref_a and ref_b:
|
||||
args += [f"{ref_a}..{ref_b}"]
|
||||
elif ref_a:
|
||||
args += [ref_a]
|
||||
if path:
|
||||
args += ["--", path]
|
||||
rc, out = await _git(*args)
|
||||
# diff exits 1 when there are differences — that's normal
|
||||
if rc not in (0, 1):
|
||||
return f"git diff failed: {out}"
|
||||
return _cap(out) or "No differences found."
|
||||
|
||||
|
||||
# ── Declarations ──────────────────────────────────────────────────────────────
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="git_status",
|
||||
description=(
|
||||
"Show the current git working tree status for the Cortex project: "
|
||||
"staged changes, unstaged modifications, and untracked files. "
|
||||
"Use to check whether there are uncommitted changes before restarting or deploying."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={},
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="git_log",
|
||||
description=(
|
||||
"Show recent git commit history for the Cortex project. "
|
||||
"Returns commit hashes, dates, and messages. "
|
||||
"Optionally filter to a specific file or directory path."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"n": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Number of commits to return (default 20, max 200)",
|
||||
),
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Optional file or directory path to filter commits by",
|
||||
),
|
||||
"oneline": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description="Use compact one-line format (default true). Set false for more detail.",
|
||||
),
|
||||
},
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="git_diff",
|
||||
description=(
|
||||
"Show a git diff for the Cortex project. "
|
||||
"With no arguments: shows unstaged working tree changes vs HEAD. "
|
||||
"With ref_a only: shows changes between that ref and HEAD. "
|
||||
"With ref_a and ref_b: shows changes between the two refs (commits, branches, or tags). "
|
||||
"Use stat_only to get a summary of changed files instead of full patch output."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"ref_a": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="First ref (commit hash, branch name, or tag). Omit for working tree diff.",
|
||||
),
|
||||
"ref_b": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Second ref. When provided with ref_a, shows diff between the two.",
|
||||
),
|
||||
"path": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Optional file or directory path to restrict the diff to",
|
||||
),
|
||||
"stat_only": types.Schema(
|
||||
type=types.Type.BOOLEAN,
|
||||
description="Return only a file-change summary (--stat) instead of the full diff",
|
||||
),
|
||||
},
|
||||
),
|
||||
),
|
||||
]
|
||||
277
cortex/tools/homeassistant.py
Normal file
277
cortex/tools/homeassistant.py
Normal file
@@ -0,0 +1,277 @@
|
||||
"""
|
||||
Home Assistant tools — read device states and call services.
|
||||
|
||||
Credentials are read automatically from the current user's channels.json:
|
||||
"homeassistant": {"url": "https://ha.example.com", "token": "<long-lived-token>"}
|
||||
|
||||
Configure in Settings → Notifications → Home Assistant.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
from google.genai import types
|
||||
|
||||
from auth_utils import get_user_channels
|
||||
from persona import get_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_TIMEOUT = 10
|
||||
|
||||
# Attributes that are internal/noisy and not useful to show
|
||||
_SKIP_ATTRS = {
|
||||
"friendly_name", "icon", "entity_picture", "supported_features",
|
||||
"supported_color_modes", "color_mode", "min_color_temp_kelvin",
|
||||
"max_color_temp_kelvin", "min_mireds", "max_mireds",
|
||||
"assumed_state", "attribution",
|
||||
}
|
||||
|
||||
|
||||
def _get_ha_cfg() -> tuple[str, str]:
|
||||
"""Return (base_url, token) from the current user's channels.json."""
|
||||
channels = get_user_channels(get_user())
|
||||
ha = channels.get("homeassistant") or {}
|
||||
url = (ha.get("url") or "").rstrip("/")
|
||||
token = ha.get("token") or ""
|
||||
if not url or not token:
|
||||
raise ValueError(
|
||||
"Home Assistant not configured — add URL and token in Settings → Notifications."
|
||||
)
|
||||
return url, token
|
||||
|
||||
|
||||
def _auth(token: str) -> dict:
|
||||
return {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
|
||||
|
||||
|
||||
def _fmt_state(s: dict) -> str:
|
||||
"""Format a single HA state dict as a compact readable line."""
|
||||
entity_id = s.get("entity_id", "")
|
||||
state = s.get("state", "unknown")
|
||||
attrs = s.get("attributes", {})
|
||||
name = attrs.get("friendly_name", entity_id)
|
||||
|
||||
label = f"{name} ({entity_id})" if name != entity_id else entity_id
|
||||
useful = {k: v for k, v in attrs.items() if k not in _SKIP_ATTRS}
|
||||
|
||||
extra = ""
|
||||
if useful:
|
||||
parts = []
|
||||
for k, v in list(useful.items())[:6]: # cap at 6 attrs per entity
|
||||
parts.append(f"{k}: {v}")
|
||||
extra = " [" + ", ".join(parts) + "]"
|
||||
|
||||
return f"{label}: {state}{extra}"
|
||||
|
||||
|
||||
async def ha_get_state(entity_id: str) -> str:
|
||||
"""Return the current state and attributes of a single Home Assistant entity."""
|
||||
try:
|
||||
url, token = _get_ha_cfg()
|
||||
except ValueError as e:
|
||||
return str(e)
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.get(f"{url}/api/states/{entity_id}", headers=_auth(token))
|
||||
|
||||
if resp.status_code == 404:
|
||||
return f"Entity not found: {entity_id}"
|
||||
if resp.status_code != 200:
|
||||
return f"HA API error {resp.status_code}: {resp.text[:400]}"
|
||||
|
||||
s = resp.json()
|
||||
attrs = s.get("attributes", {})
|
||||
lines = [
|
||||
f"**{attrs.get('friendly_name', entity_id)}** (`{entity_id}`)",
|
||||
f"State: **{s.get('state', 'unknown')}**",
|
||||
]
|
||||
changed = (s.get("last_changed") or "")[:19].replace("T", " ")
|
||||
if changed:
|
||||
lines.append(f"Last changed: {changed} UTC")
|
||||
|
||||
useful = {k: v for k, v in attrs.items() if k not in _SKIP_ATTRS}
|
||||
if useful:
|
||||
lines.append("Attributes:")
|
||||
for k, v in useful.items():
|
||||
lines.append(f" {k}: {v}")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
return f"Connection error: {e}"
|
||||
except Exception as e:
|
||||
logger.warning("ha_get_state error: %s", e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
async def ha_get_states(domain: str = "", area: str = "") -> str:
|
||||
"""List HA entity states, optionally filtered by domain (e.g. 'light') or area name."""
|
||||
try:
|
||||
url, token = _get_ha_cfg()
|
||||
except ValueError as e:
|
||||
return str(e)
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.get(f"{url}/api/states", headers=_auth(token))
|
||||
|
||||
if resp.status_code != 200:
|
||||
return f"HA API error {resp.status_code}: {resp.text[:400]}"
|
||||
|
||||
states = resp.json()
|
||||
|
||||
if domain:
|
||||
states = [s for s in states if s.get("entity_id", "").startswith(f"{domain}.")]
|
||||
if area:
|
||||
al = area.lower()
|
||||
states = [s for s in states
|
||||
if al in (s.get("attributes", {}).get("friendly_name") or "").lower()]
|
||||
|
||||
if not states:
|
||||
filters = [f"domain={domain}"] * bool(domain) + [f"area={area}"] * bool(area)
|
||||
return "No entities found" + (f" ({', '.join(filters)})" if filters else "")
|
||||
|
||||
lines = [f"{len(states)} entit{'y' if len(states) == 1 else 'ies'}:"]
|
||||
for s in sorted(states, key=lambda x: x.get("entity_id", "")):
|
||||
lines.append(_fmt_state(s))
|
||||
return "\n".join(lines)
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
return f"Connection error: {e}"
|
||||
except Exception as e:
|
||||
logger.warning("ha_get_states error: %s", e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
async def ha_call_service(
|
||||
domain: str,
|
||||
service: str,
|
||||
entity_id: str = "",
|
||||
data: str = "",
|
||||
) -> str:
|
||||
"""Call a Home Assistant service (turn on/off lights, set thermostat, lock doors, etc.)."""
|
||||
try:
|
||||
url, token = _get_ha_cfg()
|
||||
except ValueError as e:
|
||||
return str(e)
|
||||
|
||||
payload: dict = {}
|
||||
if entity_id:
|
||||
payload["entity_id"] = entity_id
|
||||
if data:
|
||||
try:
|
||||
extra = json.loads(data)
|
||||
if isinstance(extra, dict):
|
||||
payload.update(extra)
|
||||
except json.JSONDecodeError:
|
||||
return f"Invalid JSON in data: {data}"
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.post(
|
||||
f"{url}/api/services/{domain}/{service}",
|
||||
headers=_auth(token),
|
||||
json=payload,
|
||||
)
|
||||
|
||||
if resp.status_code not in (200, 201):
|
||||
return f"HA API error {resp.status_code}: {resp.text[:400]}"
|
||||
|
||||
changed = resp.json()
|
||||
if not changed:
|
||||
return f"✓ {domain}.{service} called (no state changes reported)."
|
||||
|
||||
lines = [f"✓ {domain}.{service} — {len(changed)} entity state(s) updated:"]
|
||||
for s in changed:
|
||||
lines.append(f" {s.get('entity_id', '')}: {s.get('state', '')}")
|
||||
return "\n".join(lines)
|
||||
|
||||
except httpx.HTTPError as e:
|
||||
return f"Connection error: {e}"
|
||||
except Exception as e:
|
||||
logger.warning("ha_call_service error: %s", e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="ha_get_state",
|
||||
description=(
|
||||
"Get the current state and attributes of a single Home Assistant entity. "
|
||||
"Use to check if a light is on, read a thermostat temperature, check a "
|
||||
"door/window sensor, battery level, HVAC mode, etc. "
|
||||
"entity_id format: domain.name — e.g. light.living_room, switch.garage, "
|
||||
"climate.ecobee, binary_sensor.front_door, sensor.outdoor_temp."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"entity_id": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Full entity ID, e.g. light.living_room or climate.ecobee_main",
|
||||
),
|
||||
},
|
||||
required=["entity_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ha_get_states",
|
||||
description=(
|
||||
"List Home Assistant entity states, optionally filtered by domain or area. "
|
||||
"Use to survey what devices exist or check multiple entities at once. "
|
||||
"Domain examples: light, switch, sensor, climate, binary_sensor, lock, cover, "
|
||||
"media_player, input_boolean. Leave both blank to list everything (can be large)."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"domain": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Filter to this domain, e.g. 'light' or 'switch' (optional)",
|
||||
),
|
||||
"area": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Filter by area name substring match on friendly name (optional)",
|
||||
),
|
||||
},
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ha_call_service",
|
||||
description=(
|
||||
"Call a Home Assistant service to control a device or trigger an automation. "
|
||||
"Requires user confirmation before executing. Common examples: "
|
||||
"domain=light service=turn_on entity_id=light.living_room; "
|
||||
"domain=light service=turn_off entity_id=light.all; "
|
||||
"domain=switch service=toggle entity_id=switch.garage; "
|
||||
"domain=climate service=set_temperature data={\"temperature\":72}; "
|
||||
"domain=lock service=lock entity_id=lock.front_door; "
|
||||
"domain=script service=turn_on entity_id=script.goodnight."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"domain": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Service domain: light, switch, climate, lock, cover, script, automation, etc.",
|
||||
),
|
||||
"service": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Service name: turn_on, turn_off, toggle, set_temperature, lock, unlock, open, close, etc.",
|
||||
),
|
||||
"entity_id": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="Target entity ID — omit for services that don't target a specific entity",
|
||||
),
|
||||
"data": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description='Extra service data as JSON string, e.g. {"temperature": 72, "hvac_mode": "heat"}',
|
||||
),
|
||||
},
|
||||
required=["domain", "service"],
|
||||
),
|
||||
),
|
||||
]
|
||||
234
cortex/tools/notify.py
Normal file
234
cortex/tools/notify.py
Normal file
@@ -0,0 +1,234 @@
|
||||
"""
|
||||
Notification tools — proactively send messages to user channels.
|
||||
|
||||
nc_talk_send routes through notification.py → channels.json.
|
||||
email_send uses the server SMTP config from .env (smtp_server, smtp_from_*).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
|
||||
import httpx
|
||||
from google.genai import types
|
||||
from config import settings
|
||||
from persona import get_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _load_allowlist(username: str) -> list[str]:
|
||||
"""Load the per-user email allowlist. Returns empty list if not configured."""
|
||||
path = settings.home_root() / username / "email_allowlist.json"
|
||||
try:
|
||||
return [str(p).strip() for p in json.loads(path.read_text()) if str(p).strip()]
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
except Exception as e:
|
||||
logger.warning("failed to read email_allowlist.json for %s: %s", username, e)
|
||||
return []
|
||||
|
||||
|
||||
def _email_allowed(address: str, patterns: list[str]) -> bool:
|
||||
"""Return True if address matches any pattern (regex, case-insensitive full match)."""
|
||||
addr = address.strip()
|
||||
for pattern in patterns:
|
||||
try:
|
||||
if re.fullmatch(pattern, addr, re.IGNORECASE):
|
||||
return True
|
||||
except re.error:
|
||||
logger.warning("invalid regex in email allowlist: %r", pattern)
|
||||
return False
|
||||
|
||||
|
||||
async def email_send(to: str, subject: str, body: str) -> str:
|
||||
"""Send an email via the server's configured SMTP account."""
|
||||
username = get_user()
|
||||
allowlist = _load_allowlist(username)
|
||||
|
||||
if not allowlist:
|
||||
return (
|
||||
"Email blocked — no allowlist configured. "
|
||||
f"Add allowed patterns to home/{username}/email_allowlist.json as a JSON array."
|
||||
)
|
||||
if not _email_allowed(to, allowlist):
|
||||
return f"Email blocked — {to} does not match any allowed pattern for {username}."
|
||||
|
||||
from email_utils import send_email
|
||||
ok = await asyncio.to_thread(
|
||||
send_email,
|
||||
to_email=to,
|
||||
subject=subject,
|
||||
body_text=body,
|
||||
body_html=body.replace("\n", "<br>"),
|
||||
)
|
||||
if ok:
|
||||
return f"Email sent to {to}."
|
||||
return "Failed to send email — check SMTP configuration in .env."
|
||||
|
||||
|
||||
async def web_push(title: str, body: str, url: str = "") -> str:
|
||||
"""Send a browser push notification to the current user's registered devices."""
|
||||
import push_utils
|
||||
username = get_user()
|
||||
result = await push_utils.send_push(username, title, body, url)
|
||||
if "error" in result:
|
||||
return f"Push failed: {result['error']}"
|
||||
return f"Push sent to {result['sent']} device(s) for {username} (pruned {result['pruned']} stale)."
|
||||
|
||||
|
||||
async def nc_talk_history(conversation_token: str = "", limit: int = 20) -> str:
|
||||
"""Read recent messages from a Nextcloud Talk conversation.
|
||||
|
||||
Requires nc_username and nc_app_password in channels.json under 'nextcloud'.
|
||||
conversation_token defaults to notification_room if not specified.
|
||||
"""
|
||||
from auth_utils import get_user_channels
|
||||
username = get_user()
|
||||
channels = get_user_channels(username)
|
||||
nct = channels.get("nextcloud", {})
|
||||
|
||||
url = nct.get("url", "").rstrip("/")
|
||||
nc_username = nct.get("nc_username", "").strip()
|
||||
nc_app_password = nct.get("nc_app_password", "").strip()
|
||||
token = conversation_token.strip() or nct.get("notification_room", "").strip()
|
||||
|
||||
if not url or not nc_username or not nc_app_password:
|
||||
return (
|
||||
"nc_talk_history requires nc_username and nc_app_password in channels.json "
|
||||
f"(under 'nextcloud'). Add these to home/{username}/channels.json to enable message reading."
|
||||
)
|
||||
if not token:
|
||||
return "No conversation token provided and no notification_room set in channels.json."
|
||||
|
||||
limit = min(max(int(limit), 1), 200)
|
||||
return await asyncio.to_thread(_sync_nc_talk_history, url, nc_username, nc_app_password, token, limit)
|
||||
|
||||
|
||||
def _sync_nc_talk_history(url: str, nc_user: str, nc_pass: str, token: str, limit: int) -> str:
|
||||
from datetime import datetime, timezone
|
||||
endpoint = f"{url}/ocs/v2.php/apps/spreed/api/v4/chat/{token}"
|
||||
try:
|
||||
resp = httpx.get(
|
||||
endpoint,
|
||||
params={"limit": limit, "lookIntoFuture": 0, "setReadMarker": 0, "noStatusUpdate": 1},
|
||||
auth=(nc_user, nc_pass),
|
||||
headers={"OCS-APIRequest": "true", "Accept": "application/json"},
|
||||
timeout=15,
|
||||
)
|
||||
except Exception as e:
|
||||
return f"NC Talk API error: {e}"
|
||||
|
||||
if resp.status_code != 200:
|
||||
return f"NC Talk API returned HTTP {resp.status_code}: {resp.text[:200]}"
|
||||
|
||||
try:
|
||||
messages = resp.json().get("ocs", {}).get("data", [])
|
||||
except Exception as e:
|
||||
return f"Failed to parse NC Talk response: {e}"
|
||||
|
||||
if not messages:
|
||||
return "No messages found in this conversation."
|
||||
|
||||
# NC Talk returns newest-first; reverse to chronological order
|
||||
lines = [f"Last {len(messages)} messages from {token}:\n"]
|
||||
for msg in reversed(messages):
|
||||
sender = msg.get("actorDisplayName") or msg.get("actorId") or "Unknown"
|
||||
ts = msg.get("timestamp", 0)
|
||||
time_str = datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||
text = msg.get("message", "")
|
||||
if msg.get("messageType") == "system":
|
||||
lines.append(f"[system {time_str}] {text}")
|
||||
else:
|
||||
lines.append(f"{sender} ({time_str}): {text}")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
async def nc_talk_send(message: str) -> str:
|
||||
"""Send a message to the user via their configured notification channel.
|
||||
|
||||
Channel is resolved from the user's channels.json (notification_channel key).
|
||||
Falls back to Nextcloud Talk if configured. No-op if no channel is set.
|
||||
"""
|
||||
from notification import notify
|
||||
username = get_user()
|
||||
try:
|
||||
await notify(username, message)
|
||||
return f"Message sent to {username}'s notification channel."
|
||||
except Exception as e:
|
||||
logger.warning("nc_talk_send error for %s: %s", username, e)
|
||||
return f"Failed to send notification: {e}"
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="web_push",
|
||||
description=(
|
||||
"Send a browser push notification to the current user. Works even when the "
|
||||
"Cortex tab is not open. Use for completing long tasks, reminders that fire "
|
||||
"in the background, or anything the user should see immediately. "
|
||||
"url is optional — if set, clicking the notification opens that URL."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"title": types.Schema(type=types.Type.STRING, description="Notification title (short)"),
|
||||
"body": types.Schema(type=types.Type.STRING, description="Notification body text"),
|
||||
"url": types.Schema(type=types.Type.STRING, description="Optional URL to open on click"),
|
||||
},
|
||||
required=["title", "body"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="email_send",
|
||||
description=(
|
||||
"Send an email from the server's configured SMTP account. Use for delivering "
|
||||
"summaries, reports, reminders, or any content the user wants emailed. "
|
||||
"body is plain text; newlines are preserved."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"to": types.Schema(type=types.Type.STRING, description="Recipient email address"),
|
||||
"subject": types.Schema(type=types.Type.STRING, description="Email subject line"),
|
||||
"body": types.Schema(type=types.Type.STRING, description="Plain-text email body"),
|
||||
},
|
||||
required=["to", "subject", "body"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="nc_talk_send",
|
||||
description=(
|
||||
"Send a proactive message to the user via their configured notification channel "
|
||||
"(Nextcloud Talk by default). Use this to notify the user of completed background "
|
||||
"tasks, important events, or anything they should know between sessions. "
|
||||
"Requires notification_channel and notification_room set in channels.json."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"message": types.Schema(type=types.Type.STRING, description="The message to send to the user"),
|
||||
},
|
||||
required=["message"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="nc_talk_history",
|
||||
description=(
|
||||
"Read recent messages from a Nextcloud Talk conversation. Useful for checking "
|
||||
"what was said in a room before composing a reply, or reviewing recent context. "
|
||||
"Requires nc_username and nc_app_password in channels.json under 'nextcloud'. "
|
||||
"conversation_token defaults to notification_room if not provided."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"conversation_token": types.Schema(type=types.Type.STRING, description="NC Talk room token (defaults to notification_room from channels.json)"),
|
||||
"limit": types.Schema(type=types.Type.INTEGER, description="Number of messages to return (default 20, max 200)"),
|
||||
},
|
||||
required=[],
|
||||
),
|
||||
),
|
||||
]
|
||||
255
cortex/tools/reminders.py
Normal file
255
cortex/tools/reminders.py
Normal file
@@ -0,0 +1,255 @@
|
||||
"""
|
||||
Reminders tools.
|
||||
|
||||
Reminders are stored in persona/REMINDERS.md and automatically surfaced
|
||||
in the system prompt at Tier 2+. Each reminder can have an optional due date —
|
||||
only due or undated reminders surface in context; future-dated ones are stored
|
||||
but invisible until their date arrives.
|
||||
|
||||
Operations:
|
||||
reminders_add — append a new reminder, optional due date (YYYY-MM-DD)
|
||||
reminders_list — return all reminders with due status (including future)
|
||||
reminders_remove — remove a single reminder by number
|
||||
reminders_clear — erase all reminders
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
from datetime import datetime, timezone, date as _date
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
from persona import persona_path
|
||||
|
||||
|
||||
def _reminders_path() -> Path:
|
||||
return persona_path() / "REMINDERS.md"
|
||||
|
||||
|
||||
def _now_label() -> str:
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
|
||||
|
||||
|
||||
def _parse_sections(text: str) -> list[tuple[str, str]]:
|
||||
"""Split REMINDERS.md into (heading, body) tuples, one per ## section."""
|
||||
sections: list[tuple[str, str]] = []
|
||||
heading: str | None = None
|
||||
body_lines: list[str] = []
|
||||
for line in text.splitlines():
|
||||
if line.startswith("## "):
|
||||
if heading is not None:
|
||||
sections.append((heading, "\n".join(body_lines).strip()))
|
||||
heading = line[3:].strip()
|
||||
body_lines = []
|
||||
elif heading is not None:
|
||||
body_lines.append(line)
|
||||
if heading is not None:
|
||||
sections.append((heading, "\n".join(body_lines).strip()))
|
||||
return sections
|
||||
|
||||
|
||||
def _sections_to_text(sections: list[tuple[str, str]]) -> str:
|
||||
return "".join(f"\n## {h}\n\n{b}\n" for h, b in sections)
|
||||
|
||||
|
||||
def _parse_due(body: str) -> _date | None:
|
||||
"""Extract due date from a 'due: YYYY-MM-DD' line in the body, if present."""
|
||||
m = re.search(r'^due:\s*(\d{4}-\d{2}-\d{2})', body, re.MULTILINE | re.IGNORECASE)
|
||||
if not m:
|
||||
return None
|
||||
try:
|
||||
return _date.fromisoformat(m.group(1))
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _today() -> _date:
|
||||
return datetime.now().astimezone().date()
|
||||
|
||||
|
||||
def _is_due_or_undated(body: str) -> bool:
|
||||
"""Return True if this reminder has no due date or its due date is today or past."""
|
||||
due = _parse_due(body)
|
||||
return due is None or due <= _today()
|
||||
|
||||
|
||||
def _due_label(body: str) -> str:
|
||||
"""Return a human-readable due status string for reminders_list output."""
|
||||
due = _parse_due(body)
|
||||
if due is None:
|
||||
return ""
|
||||
today = _today()
|
||||
if due < today:
|
||||
days = (today - due).days
|
||||
return f" [OVERDUE by {days} day{'s' if days != 1 else ''} — due {due}]"
|
||||
if due == today:
|
||||
return " [due TODAY]"
|
||||
return f" [due: {due}]"
|
||||
|
||||
|
||||
def _body_without_due(body: str) -> str:
|
||||
"""Strip the due: line from body for display (due status shown in heading line)."""
|
||||
return re.sub(r'^due:\s*\S+\s*\n?', '', body, count=1, flags=re.MULTILINE | re.IGNORECASE).strip()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sync implementations
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _reminders_list() -> str:
|
||||
p = _reminders_path()
|
||||
if not p.exists() or not p.read_text().strip():
|
||||
return "No pending reminders."
|
||||
sections = _parse_sections(p.read_text())
|
||||
if not sections:
|
||||
return "No pending reminders."
|
||||
lines = []
|
||||
for i, (heading, body) in enumerate(sections, 1):
|
||||
status = _due_label(body)
|
||||
lines.append(f"{i}. {heading}{status}")
|
||||
display_body = _body_without_due(body)
|
||||
if display_body:
|
||||
for bline in display_body.splitlines()[:4]:
|
||||
lines.append(f" {bline}")
|
||||
lines.append("")
|
||||
return "\n".join(lines).rstrip()
|
||||
|
||||
|
||||
def _reminders_add(text: str, label: str | None = None, due: str | None = None) -> str:
|
||||
p = _reminders_path()
|
||||
existing = p.read_text() if p.exists() else ""
|
||||
heading = label or _now_label()
|
||||
body = text.strip()
|
||||
if due:
|
||||
body = f"due: {due}\n{body}"
|
||||
section = f"\n## {heading}\n\n{body}\n"
|
||||
p.write_text(existing.rstrip() + "\n" + section)
|
||||
msg = f"Reminder added: {heading}"
|
||||
if due:
|
||||
msg += f" (due: {due})"
|
||||
return msg
|
||||
|
||||
|
||||
def _reminders_remove(index: int) -> str:
|
||||
p = _reminders_path()
|
||||
if not p.exists() or not p.read_text().strip():
|
||||
return "No reminders to remove."
|
||||
sections = _parse_sections(p.read_text())
|
||||
if not sections:
|
||||
return "No reminders to remove."
|
||||
if index < 1 or index > len(sections):
|
||||
return (
|
||||
f"Index {index} is out of range. "
|
||||
f"There {'is' if len(sections) == 1 else 'are'} {len(sections)} "
|
||||
f"reminder{'s' if len(sections) != 1 else ''} (1–{len(sections)}). "
|
||||
"Call reminders_list to see them."
|
||||
)
|
||||
removed_heading = sections[index - 1][0]
|
||||
sections.pop(index - 1)
|
||||
p.write_text(_sections_to_text(sections))
|
||||
return f"Removed reminder {index}: {removed_heading}"
|
||||
|
||||
|
||||
def _reminders_clear() -> str:
|
||||
p = _reminders_path()
|
||||
p.write_text("")
|
||||
return "All reminders cleared."
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public helper for context_loader
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def load_due_reminders() -> str:
|
||||
"""Return REMINDERS.md content filtered to only due and undated sections.
|
||||
|
||||
Called by context_loader at Tier 2+. Future-dated reminders are excluded
|
||||
from the system prompt until their due date arrives.
|
||||
"""
|
||||
p = _reminders_path()
|
||||
if not p.exists():
|
||||
return ""
|
||||
text = p.read_text()
|
||||
if not text.strip():
|
||||
return ""
|
||||
sections = _parse_sections(text)
|
||||
due_sections = [(h, b) for h, b in sections if _is_due_or_undated(b)]
|
||||
if not due_sections:
|
||||
return ""
|
||||
# Strip the raw due: line from body — the date is already part of the heading context
|
||||
cleaned = [(h, _body_without_due(b)) for h, b in due_sections]
|
||||
return _sections_to_text(cleaned).strip()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Async wrappers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def reminders_list() -> str:
|
||||
return await asyncio.to_thread(_reminders_list)
|
||||
|
||||
|
||||
async def reminders_add(text: str, label: str | None = None, due: str | None = None) -> str:
|
||||
return await asyncio.to_thread(_reminders_add, text, label, due)
|
||||
|
||||
|
||||
async def reminders_remove(index: int) -> str:
|
||||
return await asyncio.to_thread(_reminders_remove, index)
|
||||
|
||||
|
||||
async def reminders_clear() -> str:
|
||||
return await asyncio.to_thread(_reminders_clear)
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="reminders_add",
|
||||
description=(
|
||||
"Add a new reminder to REMINDERS.md. Reminders are automatically surfaced "
|
||||
"in context at the start of each session (Tier 2+). "
|
||||
"Use this when the user asks you to remember something or follow up on something. "
|
||||
"Set a due date to suppress the reminder until that date — useful for future tasks "
|
||||
"that would be noise today."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"text": types.Schema(type=types.Type.STRING, description="The reminder text"),
|
||||
"label": types.Schema(type=types.Type.STRING, description="Optional heading (e.g. 'Follow up on NC Talk'). Defaults to current timestamp."),
|
||||
"due": types.Schema(type=types.Type.STRING, description="Optional due date in YYYY-MM-DD format. Reminder is hidden from context until this date arrives. Omit for an always-visible reminder."),
|
||||
},
|
||||
required=["text"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="reminders_list",
|
||||
description=(
|
||||
"Read all pending reminders, including future-dated ones not yet in context. "
|
||||
"Shows due status for each (due today, overdue, or future date). "
|
||||
"Use this before adding to avoid duplicates, or to show the user what's queued."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="reminders_remove",
|
||||
description=(
|
||||
"Remove a single reminder by its number. "
|
||||
"Call reminders_list first to get the numbered list, then pass the number to remove."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"index": types.Schema(type=types.Type.INTEGER, description="The number of the reminder to remove (1 = first in reminders_list output)."),
|
||||
},
|
||||
required=["index"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="reminders_clear",
|
||||
description=(
|
||||
"Erase all pending reminders from REMINDERS.md. "
|
||||
"Use this after you have acknowledged and acted on the reminders shown in your context."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
]
|
||||
@@ -17,6 +17,7 @@ import asyncio
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
from persona import persona_path
|
||||
|
||||
|
||||
@@ -77,3 +78,51 @@ async def scratch_append(content: str, heading: str | None = None) -> str:
|
||||
|
||||
async def scratch_clear() -> str:
|
||||
return await asyncio.to_thread(_scratch_clear)
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="scratch_read",
|
||||
description=(
|
||||
"Read the full contents of the scratchpad. "
|
||||
"Use this to recall working notes, mid-task context, or anything previously jotted down. "
|
||||
"The scratchpad is transient — nothing here is distilled or archived."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="scratch_write",
|
||||
description=(
|
||||
"Replace the entire scratchpad with new content. "
|
||||
"Use this to set a clean working note, replacing whatever was there before. "
|
||||
"For adding without replacing, use scratch_append instead."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"content": types.Schema(type=types.Type.STRING, description="The new scratchpad content (markdown supported)"),
|
||||
},
|
||||
required=["content"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="scratch_append",
|
||||
description=(
|
||||
"Add a new section to the bottom of the scratchpad without replacing existing content. "
|
||||
"Each section gets a timestamp heading unless you supply one."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"content": types.Schema(type=types.Type.STRING, description="The content to append (markdown supported)"),
|
||||
"heading": types.Schema(type=types.Type.STRING, description="Optional section heading. Defaults to current UTC timestamp."),
|
||||
},
|
||||
required=["content"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="scratch_clear",
|
||||
description="Erase everything in the scratchpad. Use when the working notes are no longer needed.",
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
]
|
||||
|
||||
@@ -2,13 +2,23 @@
|
||||
System tools — local machine operations.
|
||||
|
||||
These tools affect the host system directly. Use with care.
|
||||
All tools in this module require the admin role.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Absolute paths — resolved relative to this file so they work regardless of cwd
|
||||
_CORTEX_DIR = Path(__file__).parent # .../Cortex_and_Inara_dev/cortex/
|
||||
_PROJECT_ROOT = _CORTEX_DIR.parent # .../Cortex_and_Inara_dev/
|
||||
|
||||
ALLOW_SCRIPT = "/home/scott/.local/bin/claude-allow-dir"
|
||||
|
||||
|
||||
@@ -42,3 +52,283 @@ async def claude_allow_dir(path: str, mode: str = "rw") -> str:
|
||||
except Exception as e:
|
||||
logger.error("claude_allow_dir error: %s", e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
async def shell_exec(command: str, working_dir: str | None = None, timeout: int = 30) -> str:
|
||||
"""Execute a shell command on the Cortex host and return combined stdout/stderr."""
|
||||
timeout = min(max(timeout, 1), 120)
|
||||
|
||||
cwd = None
|
||||
if working_dir:
|
||||
cwd = os.path.expanduser(working_dir)
|
||||
if not os.path.isdir(cwd):
|
||||
return f"Error: working_dir '{working_dir}' does not exist or is not a directory"
|
||||
|
||||
try:
|
||||
proc = await asyncio.create_subprocess_shell(
|
||||
command,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
cwd=cwd,
|
||||
)
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=timeout)
|
||||
|
||||
out = stdout.decode(errors="replace").strip()
|
||||
err = stderr.decode(errors="replace").strip()
|
||||
|
||||
parts = []
|
||||
if out:
|
||||
parts.append(out)
|
||||
if err:
|
||||
parts.append(f"[stderr]\n{err}")
|
||||
combined = "\n".join(parts) if parts else "(no output)"
|
||||
|
||||
if proc.returncode != 0:
|
||||
return f"Exit {proc.returncode}:\n{combined}"
|
||||
return combined
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
return f"Error: command timed out after {timeout}s"
|
||||
except Exception as e:
|
||||
logger.error("shell_exec error: %s", e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
async def cortex_restart() -> str:
|
||||
"""Schedule a Cortex service restart 5 seconds from now.
|
||||
|
||||
Uses a detached subprocess so the restart survives the current process being
|
||||
terminated by systemd. The calling session will drop — user should refresh.
|
||||
"""
|
||||
subprocess.Popen(
|
||||
["bash", "-c", "sleep 5 && systemctl --user restart cortex"],
|
||||
start_new_session=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
close_fds=True,
|
||||
)
|
||||
logger.info("cortex_restart: restart scheduled in 5 seconds")
|
||||
return (
|
||||
"Cortex restart scheduled in 5 seconds. "
|
||||
"The current connection will drop — please refresh the page after a moment."
|
||||
)
|
||||
|
||||
|
||||
async def cortex_logs(lines: int = 50) -> str:
|
||||
"""Return recent lines from the Cortex systemd journal."""
|
||||
n = min(max(int(lines), 1), 200)
|
||||
try:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
"journalctl", "--user", "-u", "cortex", f"-n{n}", "--no-pager",
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=15)
|
||||
out = stdout.decode(errors="replace").strip()
|
||||
return out or stderr.decode(errors="replace").strip() or "No log output."
|
||||
except asyncio.TimeoutError:
|
||||
return "Error: journalctl timed out"
|
||||
except Exception as e:
|
||||
logger.error("cortex_logs error: %s", e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
async def cortex_status() -> str:
|
||||
"""Return Cortex service status: git branch/commit, ahead/behind remote, and systemctl state."""
|
||||
lines = []
|
||||
|
||||
async def _git(*args: str) -> str:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
"git", "-C", str(_PROJECT_ROOT), *args,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.DEVNULL,
|
||||
)
|
||||
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=10)
|
||||
return stdout.decode(errors="replace").strip()
|
||||
|
||||
try:
|
||||
branch = await _git("rev-parse", "--abbrev-ref", "HEAD")
|
||||
commit = await _git("log", "--oneline", "-1")
|
||||
# fetch quietly so ahead/behind is current
|
||||
await asyncio.create_subprocess_exec(
|
||||
"git", "-C", str(_PROJECT_ROOT), "fetch", "--quiet",
|
||||
stdout=asyncio.subprocess.DEVNULL, stderr=asyncio.subprocess.DEVNULL,
|
||||
)
|
||||
ahead_behind = await _git("rev-list", "--left-right", "--count", f"HEAD...origin/{branch}")
|
||||
ahead, behind = (ahead_behind.split() + ["?", "?"])[:2]
|
||||
|
||||
lines.append(f"**Branch:** {branch} | ahead {ahead} / behind {behind}")
|
||||
lines.append(f"**Commit:** {commit}")
|
||||
except Exception as e:
|
||||
lines.append(f"Git info unavailable: {e}")
|
||||
|
||||
lines.append("")
|
||||
|
||||
try:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
"systemctl", "--user", "status", "cortex", "--no-pager", "-l",
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.STDOUT,
|
||||
)
|
||||
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=10)
|
||||
# First 15 lines of systemctl output is enough — avoids log flood
|
||||
status_lines = stdout.decode(errors="replace").splitlines()[:15]
|
||||
lines.extend(status_lines)
|
||||
except Exception as e:
|
||||
lines.append(f"systemctl status unavailable: {e}")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
async def cortex_update() -> str:
|
||||
"""Pull the latest code from git, syntax-check all Python files, and report.
|
||||
|
||||
Does NOT restart automatically — call cortex_restart separately after reviewing
|
||||
the output if you want to apply changes.
|
||||
"""
|
||||
lines = []
|
||||
|
||||
async def _run(*cmd: str, cwd: Path = _PROJECT_ROOT, timeout: int = 30) -> tuple[int, str]:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*cmd, cwd=str(cwd),
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.STDOUT,
|
||||
)
|
||||
stdout, _ = await asyncio.wait_for(proc.communicate(), timeout=timeout)
|
||||
return proc.returncode, stdout.decode(errors="replace").strip()
|
||||
|
||||
# 1. Check for incoming commits before pulling
|
||||
try:
|
||||
await _run("git", "fetch", "--quiet")
|
||||
rc, incoming = await _run("git", "log", "--oneline", "HEAD..origin/HEAD")
|
||||
if rc == 0 and not incoming:
|
||||
# Double-check with branch name in case origin/HEAD isn't set
|
||||
branch_rc, branch = await _run("git", "rev-parse", "--abbrev-ref", "HEAD")
|
||||
_, incoming = await _run("git", "log", "--oneline", f"HEAD..origin/{branch.strip()}")
|
||||
except asyncio.TimeoutError:
|
||||
return "Error: git fetch timed out — check network connectivity."
|
||||
except Exception as e:
|
||||
return f"Error during git fetch: {e}"
|
||||
|
||||
if not incoming:
|
||||
rc2, current = await _run("git", "log", "--oneline", "-1")
|
||||
return f"Already up to date.\n\nCurrent commit: {current}"
|
||||
|
||||
lines.append(f"**Incoming commits:**\n{incoming}\n")
|
||||
|
||||
# 2. Pull
|
||||
try:
|
||||
rc, pull_out = await _run("git", "pull", "--ff-only")
|
||||
except asyncio.TimeoutError:
|
||||
return "Error: git pull timed out."
|
||||
except Exception as e:
|
||||
return f"Error during git pull: {e}"
|
||||
|
||||
if rc != 0:
|
||||
return f"git pull failed (exit {rc}):\n{pull_out}"
|
||||
|
||||
lines.append(f"**git pull:**\n{pull_out}\n")
|
||||
|
||||
# 3. Syntax check all Python files under cortex/
|
||||
py_files = sorted(_CORTEX_DIR.rglob("*.py"))
|
||||
errors = []
|
||||
for f in py_files:
|
||||
result = subprocess.run(
|
||||
["python3", "-m", "py_compile", str(f)],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
errors.append(f" {f.relative_to(_PROJECT_ROOT)}: {result.stderr.strip()}")
|
||||
|
||||
if errors:
|
||||
lines.append(f"**Syntax errors — do NOT restart until fixed:**")
|
||||
lines.extend(errors)
|
||||
else:
|
||||
lines.append(f"**Syntax check:** {len(py_files)} files — all OK.")
|
||||
lines.append("Call `cortex_restart` to apply the update.")
|
||||
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="shell_exec",
|
||||
description=(
|
||||
"Execute a shell command on the Cortex host machine and return its output. "
|
||||
"Use for system diagnostics: disk usage (df -h), process status (ps aux), "
|
||||
"directory listings (ls), memory (free -h), uptime, network info, log tails, etc. "
|
||||
"Commands run as the Cortex service user. Timeout enforced (default 30s, max 120s). "
|
||||
"Avoid destructive commands — prefer read-only system queries."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"command": types.Schema(type=types.Type.STRING, description="Shell command to run (e.g. 'df -h', 'ls ~/agents_sync/', 'journalctl --user -u cortex -n 50')"),
|
||||
"working_dir": types.Schema(type=types.Type.STRING, description="Optional working directory (e.g. '~/agents_sync/projects'). Defaults to home directory."),
|
||||
"timeout": types.Schema(type=types.Type.INTEGER, description="Timeout in seconds (default 30, max 120)"),
|
||||
},
|
||||
required=["command"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="claude_allow_dir",
|
||||
description=(
|
||||
"Add a directory to Claude Code's auto-allow list so Claude can read or write "
|
||||
"files there without prompting. Edits ~/.claude/settings.json on the local machine. "
|
||||
"Use this when Claude is silently hanging or being blocked from accessing a directory. "
|
||||
"Changes take effect in the next Claude Code session."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"path": types.Schema(type=types.Type.STRING, description="Absolute or home-relative path to the directory (e.g. ~/OSIT_dev/aether_api_fastapi or /home/scott/agents_sync)"),
|
||||
"mode": types.Schema(type=types.Type.STRING, description="Permission mode: 'r' (read-only), 'w' (write-only), or 'rw' (both). Default: rw"),
|
||||
},
|
||||
required=["path"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="cortex_restart",
|
||||
description=(
|
||||
"Restart the Cortex service via systemd. Schedules a restart 5 seconds from now. "
|
||||
"The current connection will drop — inform the user to refresh the page. "
|
||||
"Use after config changes, memory edits, or when the service needs a fresh start. "
|
||||
"ADMIN ONLY."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="cortex_logs",
|
||||
description=(
|
||||
"Fetch recent lines from the Cortex systemd service journal. "
|
||||
"Use for debugging errors, checking startup status, or reviewing recent activity. "
|
||||
"ADMIN ONLY."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"lines": types.Schema(type=types.Type.INTEGER, description="Number of log lines to return (default 50, max 200)"),
|
||||
},
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="cortex_status",
|
||||
description=(
|
||||
"Return Cortex service status: current git branch and commit, how many commits "
|
||||
"ahead/behind the remote, and the systemctl service state. "
|
||||
"Use to check what version is running or whether the service is healthy. "
|
||||
"ADMIN ONLY."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="cortex_update",
|
||||
description=(
|
||||
"Pull the latest code from git, run a syntax check on all Python files, and report "
|
||||
"what changed. Does NOT restart automatically — call cortex_restart separately after "
|
||||
"reviewing the output. Will report syntax errors if the pull introduces broken code. "
|
||||
"ADMIN ONLY. Requires confirmation."
|
||||
),
|
||||
parameters=types.Schema(type=types.Type.OBJECT, properties={}),
|
||||
),
|
||||
]
|
||||
|
||||
@@ -20,6 +20,7 @@ import asyncio
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
from persona import persona_path
|
||||
|
||||
|
||||
@@ -59,13 +60,15 @@ def _format_task(t: dict) -> str:
|
||||
# Sync implementations — called via asyncio.to_thread
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _task_list(status: str | None) -> str:
|
||||
def _task_list(status: str | None, priority: str | None) -> str:
|
||||
tasks = _load()
|
||||
if status:
|
||||
tasks = [t for t in tasks if t["status"] == status]
|
||||
if priority:
|
||||
tasks = [t for t in tasks if t.get("priority") == priority]
|
||||
if not tasks:
|
||||
label = f"No {status} tasks." if status else "No tasks yet."
|
||||
return label
|
||||
filters = " ".join(f for f in [status, priority] if f)
|
||||
return f"No {filters} tasks." if filters else "No tasks yet."
|
||||
lines = [f"Tasks ({len(tasks)}):\n"]
|
||||
for t in tasks:
|
||||
lines.append(_format_task(t))
|
||||
@@ -117,8 +120,8 @@ def _task_complete(task_id: str) -> str:
|
||||
# Async wrappers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def task_list(status: str | None = None) -> str:
|
||||
return await asyncio.to_thread(_task_list, status)
|
||||
async def task_list(status: str | None = None, priority: str | None = None) -> str:
|
||||
return await asyncio.to_thread(_task_list, status, priority)
|
||||
|
||||
|
||||
async def task_create(title: str, description: str | None = None,
|
||||
@@ -133,3 +136,71 @@ async def task_update(task_id: str, status: str | None = None, title: str | None
|
||||
|
||||
async def task_complete(task_id: str) -> str:
|
||||
return await asyncio.to_thread(_task_complete, task_id)
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="task_list",
|
||||
description=(
|
||||
"List personal tasks from Inara's task list. "
|
||||
"Use this to check what's on the list, review pending work, or find a task ID. "
|
||||
"Optionally filter by status: 'todo', 'in_progress', or 'done'."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"status": types.Schema(type=types.Type.STRING, description="Filter by status: 'todo', 'in_progress', or 'done'. Omit to list all."),
|
||||
"priority": types.Schema(type=types.Type.STRING, description="Filter by priority: 'low', 'normal', or 'high'. Omit to list all priorities."),
|
||||
},
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="task_create",
|
||||
description=(
|
||||
"Add a new task to Inara's personal task list. "
|
||||
"Use this when the user asks to remember something, add a to-do, or track a follow-up."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"title": types.Schema(type=types.Type.STRING, description="Short task title"),
|
||||
"description": types.Schema(type=types.Type.STRING, description="Optional longer description or context"),
|
||||
"priority": types.Schema(type=types.Type.STRING, description="Priority: 'low', 'normal', or 'high'. Default: normal."),
|
||||
},
|
||||
required=["title"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="task_update",
|
||||
description=(
|
||||
"Update an existing task. Use task_list first to get the task ID. "
|
||||
"Can update status, title, description, or priority. "
|
||||
"To just mark complete, use task_complete instead."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"task_id": types.Schema(type=types.Type.STRING, description="Task ID (e.g. t_abc123) — get from task_list"),
|
||||
"status": types.Schema(type=types.Type.STRING, description="New status: 'todo', 'in_progress', or 'done'"),
|
||||
"title": types.Schema(type=types.Type.STRING, description="Updated title"),
|
||||
"description": types.Schema(type=types.Type.STRING, description="Updated description"),
|
||||
"priority": types.Schema(type=types.Type.STRING, description="Updated priority: 'low', 'normal', or 'high'"),
|
||||
},
|
||||
required=["task_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="task_complete",
|
||||
description=(
|
||||
"Mark a task as done. Use task_list first to get the task ID. "
|
||||
"Shorthand for task_update with status='done'."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"task_id": types.Schema(type=types.Type.STRING, description="Task ID (e.g. t_abc123) — get from task_list"),
|
||||
},
|
||||
required=["task_id"],
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
@@ -1,13 +1,17 @@
|
||||
"""
|
||||
Web search tool — DuckDuckGo backend.
|
||||
|
||||
Uses the duckduckgo-search library. Set DDG_API_KEY in .env for a paid account
|
||||
(higher rate limits). The free unauthenticated tier works for moderate usage.
|
||||
Web tools — search (DuckDuckGo), direct HTTP fetch, clean content extraction, and HTTP POST.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from urllib.parse import urlparse
|
||||
|
||||
import httpx
|
||||
from google.genai import types
|
||||
|
||||
from config import settings
|
||||
from persona import get_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -48,3 +52,216 @@ def _sync_search(query: str, max_results: int) -> list[dict]:
|
||||
except Exception as e:
|
||||
logger.warning("DuckDuckGo search error: %s", e)
|
||||
return []
|
||||
|
||||
|
||||
async def http_fetch(
|
||||
url: str,
|
||||
method: str = "GET",
|
||||
body: str | None = None,
|
||||
timeout: int = 15,
|
||||
max_chars: int = 8192,
|
||||
) -> str:
|
||||
"""Fetch a URL directly and return the raw response body.
|
||||
|
||||
Unlike web_search, this hits a specific URL — useful for health checks,
|
||||
API probing, JSON endpoints, webhook testing, or reading raw page source.
|
||||
For readable article content, use web_read instead.
|
||||
Response body is capped at max_chars (default 8192, max 32768).
|
||||
"""
|
||||
method = method.upper()
|
||||
timeout = min(max(int(timeout), 1), 60)
|
||||
max_chars = min(max(int(max_chars), 100), 131072)
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
|
||||
resp = await client.request(method, url, content=body)
|
||||
body_text = resp.text[:max_chars]
|
||||
truncated = len(resp.text) > max_chars
|
||||
suffix = f"\n\n[… truncated at {max_chars} chars]" if truncated else ""
|
||||
return f"HTTP {resp.status_code} {resp.url}\n\n{body_text}{suffix}"
|
||||
except httpx.HTTPError as e:
|
||||
return f"HTTP error: {e}"
|
||||
except Exception as e:
|
||||
logger.warning("http_fetch error for %s: %s", url, e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
async def web_read(url: str, max_chars: int = 16000) -> str:
|
||||
"""Fetch a URL and extract clean readable text, stripping ads, navigation, and boilerplate.
|
||||
|
||||
Uses trafilatura to extract the main article content — ideal for blog posts,
|
||||
documentation, news articles, and any page where you want the text without
|
||||
surrounding noise. Returns markdown-formatted output.
|
||||
For raw responses (JSON APIs, health checks), use http_fetch instead.
|
||||
"""
|
||||
max_chars = min(max(int(max_chars), 1000), 131072)
|
||||
return await asyncio.to_thread(_sync_web_read, url, max_chars)
|
||||
|
||||
|
||||
def _sync_web_read(url: str, max_chars: int) -> str:
|
||||
try:
|
||||
import trafilatura
|
||||
except ImportError:
|
||||
return "web_read requires trafilatura — run: pip install trafilatura"
|
||||
|
||||
downloaded = trafilatura.fetch_url(url)
|
||||
if downloaded is None:
|
||||
return f"Failed to download content from: {url}"
|
||||
|
||||
text = trafilatura.extract(downloaded, output_format="markdown", include_links=True, url=url)
|
||||
if not text:
|
||||
text = trafilatura.extract(downloaded, url=url)
|
||||
if not text:
|
||||
return f"Could not extract readable content from: {url}"
|
||||
|
||||
if len(text) > max_chars:
|
||||
text = text[:max_chars] + f"\n\n[… truncated at {max_chars} chars — pass a larger max_chars (up to 131072) to see more]"
|
||||
return f"Content from {url}:\n\n{text}"
|
||||
|
||||
|
||||
def _load_http_allowlist(username: str) -> list[str]:
|
||||
"""Load per-user HTTP POST allowlist (URL prefixes). Empty list = all blocked."""
|
||||
path = settings.home_root() / username / "http_allowlist.json"
|
||||
try:
|
||||
return [str(p).strip() for p in json.loads(path.read_text()) if str(p).strip()]
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
except Exception as e:
|
||||
logger.warning("failed to read http_allowlist.json for %s: %s", username, e)
|
||||
return []
|
||||
|
||||
|
||||
def _http_post_allowed(url: str, allowlist: list[str]) -> bool:
|
||||
"""Return True if url starts with any allowlist entry (prefix match)."""
|
||||
for prefix in allowlist:
|
||||
if url.startswith(prefix):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
async def http_post(
|
||||
url: str,
|
||||
body: str = "",
|
||||
headers: dict | None = None,
|
||||
max_chars: int = 4096,
|
||||
) -> str:
|
||||
"""POST to an external URL. Requires the URL to match home/{user}/http_allowlist.json.
|
||||
|
||||
body may be a JSON string or plain text. If body is valid JSON, Content-Type is set
|
||||
to application/json; otherwise text/plain. Override via the headers param.
|
||||
Response is capped at max_chars (default 4096, max 131072).
|
||||
"""
|
||||
username = get_user()
|
||||
allowlist = _load_http_allowlist(username)
|
||||
if not allowlist:
|
||||
return (
|
||||
f"http_post blocked — no allowlist configured. "
|
||||
f"Add allowed URL prefixes to home/{username}/http_allowlist.json as a JSON array. "
|
||||
f"Example: [\"https://api.example.com\"]"
|
||||
)
|
||||
if not _http_post_allowed(url, allowlist):
|
||||
return (
|
||||
f"http_post blocked — {url} does not match any allowlist entry for {username}. "
|
||||
f"Add the URL prefix to home/{username}/http_allowlist.json."
|
||||
)
|
||||
|
||||
max_chars = min(max(int(max_chars), 100), 131072)
|
||||
|
||||
# Auto-detect content type from body
|
||||
body_str = body if isinstance(body, str) else json.dumps(body)
|
||||
try:
|
||||
json.loads(body_str)
|
||||
content_type = "application/json"
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
content_type = "text/plain"
|
||||
|
||||
req_headers = {"Content-Type": content_type}
|
||||
if headers:
|
||||
req_headers.update(headers)
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30, follow_redirects=True) as client:
|
||||
resp = await client.post(url, content=body_str.encode(), headers=req_headers)
|
||||
body_text = resp.text[:max_chars]
|
||||
truncated = len(resp.text) > max_chars
|
||||
suffix = f"\n\n[… truncated at {max_chars} chars]" if truncated else ""
|
||||
return f"HTTP {resp.status_code} {resp.url}\n\n{body_text}{suffix}"
|
||||
except httpx.HTTPError as e:
|
||||
return f"HTTP error: {e}"
|
||||
except Exception as e:
|
||||
logger.warning("http_post error for %s: %s", url, e)
|
||||
return f"Error: {e}"
|
||||
|
||||
|
||||
DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="web_search",
|
||||
description=(
|
||||
"Search the web for current information. Use this when you need up-to-date "
|
||||
"facts, news, documentation, or anything not in your training data."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"query": types.Schema(type=types.Type.STRING, description="The search query string"),
|
||||
"max_results": types.Schema(type=types.Type.INTEGER, description="Number of results to return (default 5, max 10)"),
|
||||
},
|
||||
required=["query"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="http_fetch",
|
||||
description=(
|
||||
"Fetch a specific URL and return the raw response body. Unlike web_search, this hits "
|
||||
"a direct URL — useful for health checks, JSON API endpoints, webhook testing, "
|
||||
"or inspecting raw page source. For readable article/doc content, use web_read instead. "
|
||||
"Response body is capped at max_chars (default 8192, max 32768)."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"url": types.Schema(type=types.Type.STRING, description="Full URL to fetch"),
|
||||
"method": types.Schema(type=types.Type.STRING, description="HTTP method: GET (default), POST, HEAD"),
|
||||
"body": types.Schema(type=types.Type.STRING, description="Optional request body (for POST requests)"),
|
||||
"timeout": types.Schema(type=types.Type.INTEGER, description="Request timeout in seconds (default 15, max 60)"),
|
||||
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max characters to return (default 8192, max 131072)"),
|
||||
},
|
||||
required=["url"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="web_read",
|
||||
description=(
|
||||
"Fetch a URL and extract clean readable text, stripping ads, navigation, sidebars, "
|
||||
"and other boilerplate. Returns the main article/document content as markdown. "
|
||||
"Use this for blog posts, documentation, news articles, GitHub READMEs, or any page "
|
||||
"where you want the content without surrounding noise. "
|
||||
"For raw HTTP responses (JSON APIs, health checks, source inspection), use http_fetch."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"url": types.Schema(type=types.Type.STRING, description="Full URL to fetch and extract"),
|
||||
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max characters to return (default 16000, max 131072)"),
|
||||
},
|
||||
required=["url"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="http_post",
|
||||
description=(
|
||||
"POST to an external URL. Requires the URL to match the user's http_allowlist.json. "
|
||||
"Use for calling webhooks, triggering automations, posting to APIs, or any HTTP action. "
|
||||
"body is a string — JSON or plain text are both accepted (Content-Type auto-detected). "
|
||||
"Override headers as needed. Response capped at max_chars (default 4096, max 131072)."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"url": types.Schema(type=types.Type.STRING, description="Full URL to POST to"),
|
||||
"body": types.Schema(type=types.Type.STRING, description="Request body — JSON string or plain text"),
|
||||
"max_chars": types.Schema(type=types.Type.INTEGER, description="Max response chars (default 4096, max 131072)"),
|
||||
},
|
||||
required=["url"],
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
75
cortex/usage_tracker.py
Normal file
75
cortex/usage_tracker.py
Normal file
@@ -0,0 +1,75 @@
|
||||
"""
|
||||
API usage and token tracking.
|
||||
|
||||
Writes daily buckets to home/{username}/usage.json:
|
||||
|
||||
{
|
||||
"2026-05-01": {
|
||||
"gemini_api/gemini-2.0-flash": {"calls": 3, "prompt_tokens": 8400, "completion_tokens": 520},
|
||||
"local/llama3.2:latest": {"calls": 2, "prompt_tokens": 1200, "completion_tokens": 310}
|
||||
}
|
||||
}
|
||||
|
||||
Claude CLI and Gemini CLI backends produce no structured token data and are not tracked.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
from config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_LOCK = asyncio.Lock()
|
||||
|
||||
|
||||
def _usage_path(username: str) -> Path:
|
||||
return settings.home_root() / username / "usage.json"
|
||||
|
||||
|
||||
async def record(
|
||||
username: str,
|
||||
backend: str,
|
||||
model_name: str,
|
||||
prompt_tokens: int,
|
||||
completion_tokens: int,
|
||||
) -> None:
|
||||
"""Append one call's token counts to the daily usage log for this user.
|
||||
|
||||
backend — "gemini_api" | "local"
|
||||
model_name — the exact model string (e.g. "gemini-2.0-flash", "llama3.2:latest")
|
||||
"""
|
||||
path = _usage_path(username)
|
||||
today = date.today().isoformat()
|
||||
key = f"{backend}/{model_name}"
|
||||
|
||||
async with _LOCK:
|
||||
try:
|
||||
data: dict = json.loads(path.read_text()) if path.exists() else {}
|
||||
except Exception:
|
||||
data = {}
|
||||
|
||||
entry = data.setdefault(today, {}).setdefault(
|
||||
key, {"calls": 0, "prompt_tokens": 0, "completion_tokens": 0}
|
||||
)
|
||||
entry["calls"] += 1
|
||||
entry["prompt_tokens"] += prompt_tokens
|
||||
entry["completion_tokens"] += completion_tokens
|
||||
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(data, indent=2))
|
||||
except Exception as e:
|
||||
logger.warning("Failed to write usage data to %s: %s", path, e)
|
||||
|
||||
|
||||
def read_usage(username: str) -> dict:
|
||||
"""Return the full usage dict for this user. Empty dict if no file yet."""
|
||||
path = _usage_path(username)
|
||||
try:
|
||||
return json.loads(path.read_text()) if path.exists() else {}
|
||||
except Exception:
|
||||
return {}
|
||||
194
cortex/user_settings.py
Normal file
194
cortex/user_settings.py
Normal file
@@ -0,0 +1,194 @@
|
||||
"""
|
||||
Per-user settings stored in home/{user}/local_llm.json.
|
||||
|
||||
Structure:
|
||||
{
|
||||
"hosts": [{"id", "label", "api_url", "api_key"}, ...],
|
||||
"models": [{"id", "host_id", "label", "model_name"}, ...],
|
||||
"active_model_id": "<model id>" | null
|
||||
}
|
||||
|
||||
Values not configured here fall back to .env server defaults.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import secrets
|
||||
from pathlib import Path
|
||||
|
||||
from config import settings as app_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _llm_path(username: str) -> Path:
|
||||
return app_settings.home_root() / username / "local_llm.json"
|
||||
|
||||
|
||||
def _empty() -> dict:
|
||||
return {"hosts": [], "models": [], "active_model_id": None}
|
||||
|
||||
|
||||
def _load(username: str) -> dict:
|
||||
path = _llm_path(username)
|
||||
if not path.exists():
|
||||
return _empty()
|
||||
try:
|
||||
data = json.loads(path.read_text())
|
||||
except (json.JSONDecodeError, OSError):
|
||||
logger.warning("local_llm.json for %s is unreadable — starting fresh", username)
|
||||
return _empty()
|
||||
|
||||
# Migrate old single-model format {api_url, api_key, model} → new format
|
||||
if "hosts" not in data:
|
||||
return _migrate_v0(data)
|
||||
|
||||
return data
|
||||
|
||||
|
||||
def _migrate_v0(old: dict) -> dict:
|
||||
"""Migrate flat {api_url, api_key, model} → hosts/models structure."""
|
||||
data = _empty()
|
||||
api_url = old.get("api_url") or app_settings.local_api_url
|
||||
api_key = old.get("api_key") or app_settings.local_api_key
|
||||
model_name = old.get("model") or app_settings.local_model
|
||||
|
||||
if not api_url:
|
||||
return data
|
||||
|
||||
host_id = secrets.token_hex(4)
|
||||
data["hosts"].append({
|
||||
"id": host_id,
|
||||
"label": "Local Model Server",
|
||||
"api_url": api_url,
|
||||
"api_key": api_key,
|
||||
})
|
||||
|
||||
if model_name:
|
||||
model_id = secrets.token_hex(4)
|
||||
data["models"].append({
|
||||
"id": model_id,
|
||||
"host_id": host_id,
|
||||
"label": model_name,
|
||||
"model_name": model_name,
|
||||
})
|
||||
data["active_model_id"] = model_id
|
||||
|
||||
logger.info("migrated local_llm.json v0 → v1 for user (host=%s)", host_id)
|
||||
return data
|
||||
|
||||
|
||||
def _save(username: str, data: dict) -> None:
|
||||
_llm_path(username).write_text(json.dumps(data, indent=2))
|
||||
|
||||
|
||||
# ── Public read API ───────────────────────────────────────────────────────────
|
||||
|
||||
def get_config(username: str) -> dict:
|
||||
"""Return the full local LLM config for the user."""
|
||||
return _load(username)
|
||||
|
||||
|
||||
def get_active_local_model(username: str) -> dict | None:
|
||||
"""Return effective {api_url, api_key, model_name, label} for the active model.
|
||||
|
||||
Resolution order:
|
||||
1. User's active model + its host config
|
||||
2. .env server defaults (LOCAL_API_URL / LOCAL_API_KEY / LOCAL_MODEL)
|
||||
3. None — caller should raise a helpful error
|
||||
"""
|
||||
data = _load(username)
|
||||
|
||||
active_id = data.get("active_model_id")
|
||||
model = next((m for m in data["models"] if m["id"] == active_id), None)
|
||||
|
||||
if model:
|
||||
host = next((h for h in data["hosts"] if h["id"] == model["host_id"]), None)
|
||||
if host:
|
||||
return {
|
||||
"api_url": host.get("api_url", ""),
|
||||
"api_key": host.get("api_key", ""),
|
||||
"model_name": model["model_name"],
|
||||
"label": model.get("label") or model["model_name"],
|
||||
}
|
||||
|
||||
# Fall back to .env defaults
|
||||
if app_settings.local_api_url and app_settings.local_model:
|
||||
return {
|
||||
"api_url": app_settings.local_api_url,
|
||||
"api_key": app_settings.local_api_key,
|
||||
"model_name": app_settings.local_model,
|
||||
"label": app_settings.local_model,
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ── Host management ───────────────────────────────────────────────────────────
|
||||
|
||||
def save_host(username: str, host_id: str | None,
|
||||
label: str, api_url: str, api_key: str) -> str:
|
||||
"""Create or update a host. Returns the host ID.
|
||||
|
||||
api_key is only written when non-empty, so submitting a masked placeholder
|
||||
with a blank key field leaves the stored key unchanged.
|
||||
"""
|
||||
data = _load(username)
|
||||
|
||||
if host_id:
|
||||
for h in data["hosts"]:
|
||||
if h["id"] == host_id:
|
||||
h["label"] = label.strip()
|
||||
h["api_url"] = api_url.strip()
|
||||
if api_key.strip():
|
||||
h["api_key"] = api_key.strip()
|
||||
break
|
||||
else:
|
||||
host_id = None # ID not found — fall through to create
|
||||
|
||||
if not host_id:
|
||||
host_id = secrets.token_hex(4)
|
||||
data["hosts"].append({
|
||||
"id": host_id,
|
||||
"label": label.strip(),
|
||||
"api_url": api_url.strip(),
|
||||
"api_key": api_key.strip(),
|
||||
})
|
||||
|
||||
_save(username, data)
|
||||
return host_id
|
||||
|
||||
|
||||
# ── Model management ──────────────────────────────────────────────────────────
|
||||
|
||||
def add_model(username: str, host_id: str, label: str, model_name: str) -> str:
|
||||
"""Add a model entry. Auto-activates if it is the first model. Returns the model ID."""
|
||||
data = _load(username)
|
||||
model_id = secrets.token_hex(4)
|
||||
data["models"].append({
|
||||
"id": model_id,
|
||||
"host_id": host_id,
|
||||
"label": label.strip() or model_name.strip(),
|
||||
"model_name": model_name.strip(),
|
||||
})
|
||||
if not data.get("active_model_id"):
|
||||
data["active_model_id"] = model_id
|
||||
_save(username, data)
|
||||
return model_id
|
||||
|
||||
|
||||
def remove_model(username: str, model_id: str) -> None:
|
||||
data = _load(username)
|
||||
data["models"] = [m for m in data["models"] if m["id"] != model_id]
|
||||
if data.get("active_model_id") == model_id:
|
||||
data["active_model_id"] = data["models"][0]["id"] if data["models"] else None
|
||||
_save(username, data)
|
||||
|
||||
|
||||
def set_active_model(username: str, model_id: str) -> bool:
|
||||
"""Set the active model. Returns False if the model ID is not found."""
|
||||
data = _load(username)
|
||||
if not any(m["id"] == model_id for m in data["models"]):
|
||||
return False
|
||||
data["active_model_id"] = model_id
|
||||
_save(username, data)
|
||||
return True
|
||||
26
dev-restart.sh
Executable file
26
dev-restart.sh
Executable file
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env bash
|
||||
# dev-restart.sh — restart Cortex on the gaming laptop and tail logs
|
||||
# Usage:
|
||||
# ./dev-restart.sh restart and show last 30 log lines
|
||||
# ./dev-restart.sh logs tail live logs (ctrl-c to stop)
|
||||
# ./dev-restart.sh status show service status only
|
||||
|
||||
# "scott-lt-i7-rtx" or "192.168.32.19"
|
||||
CORTEX_HOST="scott-lt-i7-rtx" # hostname or IP of the machine running Cortex
|
||||
SERVICE="cortex"
|
||||
|
||||
case "${1:-restart}" in
|
||||
logs)
|
||||
echo "→ Tailing $SERVICE logs on $CORTEX_HOST (ctrl-c to stop)"
|
||||
ssh "$CORTEX_HOST" "journalctl --user -u $SERVICE -f --no-pager"
|
||||
;;
|
||||
status)
|
||||
ssh "$CORTEX_HOST" "systemctl --user status $SERVICE --no-pager -l"
|
||||
;;
|
||||
restart|*)
|
||||
echo "→ Restarting $SERVICE on $CORTEX_HOST …"
|
||||
ssh "$CORTEX_HOST" "systemctl --user restart $SERVICE"
|
||||
echo "→ Last 30 log lines:"
|
||||
ssh "$CORTEX_HOST" "journalctl --user -u $SERVICE --no-pager -n 30"
|
||||
;;
|
||||
esac
|
||||
100
docs/GOOGLE_CHAT_BOT.md
Normal file
100
docs/GOOGLE_CHAT_BOT.md
Normal file
@@ -0,0 +1,100 @@
|
||||
# Google Chat Bot Integration
|
||||
|
||||
Cortex connects to Google Chat as a **Workspace Add-on** — each Cortex user gets their own webhook endpoint routed to their chosen persona.
|
||||
|
||||
**Status:** Live and confirmed working (2026-03-27)
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A Google Cloud project with **Google Chat API** enabled
|
||||
- The Cortex server reachable at a public HTTPS URL
|
||||
- The user pre-registered in Cortex (`manage_passwords.py invite` or `google-add`)
|
||||
|
||||
---
|
||||
|
||||
## Per-User Setup
|
||||
|
||||
### 1. Create the user's `channels.json`
|
||||
|
||||
Create `home/{username}/channels.json` on the Cortex server:
|
||||
|
||||
```json
|
||||
{
|
||||
"google_chat": {
|
||||
"persona": "inara",
|
||||
"audience": "https://cortex.dgrzone.com/channels/google-chat/{username}",
|
||||
"backend": "claude",
|
||||
"timeout": 25
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- **`persona`** — which persona responds (must exist under `home/{username}/persona/`)
|
||||
- **`audience`** — must exactly match the HTTP endpoint URL you set in Google Cloud Console (Google uses this as the JWT `aud` claim)
|
||||
- **`backend`** — `"claude"` recommended; Google Chat requires a response within 30s
|
||||
- **`timeout`** — keep at 25 (Google's hard limit is 30s; this leaves a 5s buffer)
|
||||
|
||||
### 2. Configure Google Chat API in Google Cloud Console
|
||||
|
||||
1. Go to [console.cloud.google.com](https://console.cloud.google.com) and select the project
|
||||
2. **APIs & Services → Enabled APIs & services → Google Chat API**
|
||||
3. Click the **Configuration** tab
|
||||
4. Fill in **Application info:**
|
||||
- App name: `Cortex` (or your persona name)
|
||||
- Avatar URL: optional
|
||||
- Description: optional
|
||||
5. Under **Interactive features:**
|
||||
- Enable **"Join spaces and group conversations"** if you want the bot in group chats, or leave it off for DM-only
|
||||
6. Under **Connection settings:**
|
||||
- Select **HTTP endpoint URL**
|
||||
- Enter: `https://cortex.dgrzone.com/channels/google-chat/{username}`
|
||||
7. Under **Visibility:**
|
||||
- Add the specific Google accounts that should be able to use this bot
|
||||
- For One Sky IT Workspace users: add individuals or the whole domain
|
||||
8. Click **Save**
|
||||
|
||||
> **Important:** The URL in step 6 must exactly match the `audience` value in `channels.json`. Google includes this URL as the JWT `aud` claim on every request, and Cortex rejects any request where they don't match.
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
1. User sends a message in Google Chat → Google POSTs a signed JSON payload to `/channels/google-chat/{username}`
|
||||
2. Cortex reads the user's `channels.json`, verifies the JWT `systemIdToken` from `authorizationEventObject`
|
||||
3. Sets the persona context, builds the system prompt, calls the LLM
|
||||
4. Returns the response wrapped in `hostAppDataAction → chatDataAction → createMessageAction`
|
||||
|
||||
The response must be returned synchronously (Google Chat does not support async/background replies like NC Talk does). The 25s timeout is a hard constraint.
|
||||
|
||||
---
|
||||
|
||||
## JWT Verification
|
||||
|
||||
Google Chat Workspace Add-ons send a `systemIdToken` in the request body at:
|
||||
`body["authorizationEventObject"]["systemIdToken"]`
|
||||
|
||||
Claims verified by Cortex:
|
||||
- `iss` = `https://accounts.google.com`
|
||||
- `aud` = the value of `audience` in `channels.json`
|
||||
|
||||
If `audience` is empty, verification is skipped (useful for local testing, never in production).
|
||||
|
||||
---
|
||||
|
||||
## Nginx
|
||||
|
||||
The `/channels/` prefix is already public in `auth_middleware.py` — no Nginx changes needed if you're already proxying all traffic to Cortex. Verify the path isn't blocked by basic auth or IP restrictions.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---|---|---|
|
||||
| 404 on the webhook | `channels.json` missing or no `google_chat` key | Create/check `home/{username}/channels.json` |
|
||||
| 401 Invalid token | `audience` in `channels.json` doesn't match the endpoint URL | Make them identical — copy the URL exactly |
|
||||
| 401 Missing token | No `systemIdToken` in request | Bot may not be a Workspace Add-on; check connection settings type |
|
||||
| Timeout / no response | LLM too slow | `backend: "claude"` recommended; reduce context tier if needed |
|
||||
| Bot not receiving messages | Visibility not configured | Add the user's Google account under Visibility in Cloud Console |
|
||||
@@ -1,69 +1,78 @@
|
||||
# Nextcloud Talk Bot Integration
|
||||
|
||||
Inara is registered as a bot in Nextcloud Talk, receiving messages via webhook and replying through the bot API.
|
||||
Cortex connects to Nextcloud Talk as a bot — each Cortex user gets their own webhook endpoint routed to their chosen persona.
|
||||
|
||||
**Status:** Live and confirmed working (2026-03-20)
|
||||
**Status:** Live and confirmed working (2026-03-20); per-user routing added 2026-03-27
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
## Prerequisites
|
||||
|
||||
Run on the Nextcloud server (inside the Docker container):
|
||||
|
||||
```bash
|
||||
docker exec -it --user www-data <nc-app-container> php /var/www/html/occ talk:bot:install \
|
||||
"Inara" \
|
||||
"<secret from cortex .env NEXTCLOUD_TALK_BOT_SECRET>" \
|
||||
"https://cortex.dgrzone.com/inara-nextcloud-talk-webhook" \
|
||||
--feature webhook --feature response --feature reaction
|
||||
```
|
||||
|
||||
After installing, enable the bot in each Talk conversation via the conversation settings UI (three-dot menu → Bots).
|
||||
|
||||
To list installed bots and verify registration:
|
||||
|
||||
```bash
|
||||
docker exec -it --user www-data <nc-app-container> php /var/www/html/occ talk:bot:list
|
||||
```
|
||||
|
||||
To uninstall (if re-registering with a new secret):
|
||||
|
||||
```bash
|
||||
docker exec -it --user www-data <nc-app-container> php /var/www/html/occ talk:bot:remove <bot-id>
|
||||
```
|
||||
- Access to the Nextcloud server (Docker exec or SSH)
|
||||
- The Cortex server reachable at a public HTTPS URL
|
||||
- The user pre-registered in Cortex (`manage_passwords.py invite`)
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
## Per-User Setup
|
||||
|
||||
**`cortex/.env`:**
|
||||
```
|
||||
NEXTCLOUD_URL=https://cloud.dgrzone.com
|
||||
NEXTCLOUD_TALK_BOT_SECRET=<shared secret — must match occ install command>
|
||||
```
|
||||
### 1. Create the user's `channels.json`
|
||||
|
||||
`NEXTCLOUD_URL` defaults to `https://cloud.dgrzone.com` in `config.py`.
|
||||
Create `home/{username}/channels.json` on the Cortex server:
|
||||
|
||||
**Nginx:** The `/inara-nextcloud-talk-webhook` endpoint must be reachable by Nextcloud without basic auth. Add a location block before the default `auth_basic` block:
|
||||
|
||||
```nginx
|
||||
location = /inara-nextcloud-talk-webhook {
|
||||
proxy_pass http://127.0.0.1:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
```json
|
||||
{
|
||||
"nextcloud": {
|
||||
"persona": "inara",
|
||||
"url": "https://cloud.dgrzone.com",
|
||||
"bot_secret": "<a secret you choose — must match the occ install command>",
|
||||
"timeout": 55
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(The `/channels/` prefix is already bypassed for Google Chat — consider moving the webhook path to `/channels/nextcloud` in a future cleanup to unify the nginx config.)
|
||||
- **`persona`** — which persona responds (must exist under `home/{username}/persona/`)
|
||||
- **`url`** — base URL of the Nextcloud instance
|
||||
- **`bot_secret`** — a shared HMAC secret; you choose this value and use it in both `channels.json` and the `occ` install command
|
||||
- **`timeout`** — seconds to wait for the LLM before sending a timeout message (NC Talk is async, so 55s is safe)
|
||||
|
||||
### 2. Register the bot in Nextcloud
|
||||
|
||||
The Nextcloud container for DgrZone is `dgr_zone_nextcloud-app-1`. Substitute your own container name if different.
|
||||
|
||||
First, list existing bots to check if one is already registered (note the bot ID):
|
||||
|
||||
```bash
|
||||
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ talk:bot:list
|
||||
```
|
||||
|
||||
If re-registering (new URL or new secret), uninstall the old bot first:
|
||||
|
||||
```bash
|
||||
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ talk:bot:uninstall <bot-id>
|
||||
```
|
||||
|
||||
Install the bot:
|
||||
|
||||
```bash
|
||||
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ talk:bot:install \
|
||||
"Inara" \
|
||||
"<bot_secret from channels.json>" \
|
||||
"https://cortex.dgrzone.com/webhook/nextcloud/{username}" \
|
||||
--feature webhook --feature response --feature reaction
|
||||
```
|
||||
|
||||
After installing, enable the bot in each Talk conversation: open the conversation → three-dot menu → **Bots** → enable the bot by name.
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
1. User sends a message in Talk → Nextcloud POSTs a signed webhook to `/inara-nextcloud-talk-webhook`
|
||||
2. Cortex verifies the incoming HMAC signature, extracts the message text, runs it through the LLM
|
||||
3. Cortex POSTs the reply to `/ocs/v2.php/apps/spreed/api/v1/bot/{token}/message` with its own HMAC signature
|
||||
4. The webhook handler returns HTTP 200 immediately; the LLM call happens in a `BackgroundTask` (prevents Nextcloud from disabling the bot due to slow response)
|
||||
1. User sends a message in Talk → Nextcloud POSTs a signed webhook to `/webhook/nextcloud/{username}`
|
||||
2. Cortex reads the user's `channels.json`, verifies the incoming HMAC signature
|
||||
3. Sets the persona context, builds the system prompt, runs the LLM in a `BackgroundTask`
|
||||
4. Returns HTTP 200 immediately (prevents Nextcloud from disabling the bot due to slow response)
|
||||
5. Cortex POSTs the reply to `/ocs/v2.php/apps/spreed/api/v1/bot/{token}/message` with its own HMAC signature
|
||||
|
||||
---
|
||||
|
||||
@@ -76,7 +85,6 @@ location = /inara-nextcloud-talk-webhook {
|
||||
Nextcloud signs its outgoing webhook with `HMAC-SHA256(secret, random + raw_body)`:
|
||||
|
||||
```python
|
||||
# _verify_signature in nextcloud_talk.py
|
||||
expected = hmac.new(
|
||||
secret.encode(),
|
||||
(random_header + body.decode("utf-8")).encode(),
|
||||
@@ -89,7 +97,6 @@ expected = hmac.new(
|
||||
When Cortex posts a reply, Nextcloud verifies the signature against the *parsed message string*, not the raw body. This is because `BotController::sendMessage` passes the parsed `$message` parameter to `checksumVerificationService::validateRequest`, not `$request->getContent()`.
|
||||
|
||||
```python
|
||||
# _send_reply in nextcloud_talk.py
|
||||
sig = hmac.new(
|
||||
secret.encode(),
|
||||
(random_str + message).encode("utf-8"), # message text only, NOT json.dumps({"message": ...})
|
||||
@@ -105,35 +112,50 @@ sig = hmac.new(secret.encode(), (random_str + '{"message": "..."}').encode(), ha
|
||||
|
||||
---
|
||||
|
||||
## Multi-User Note
|
||||
## Nginx
|
||||
|
||||
NC Talk currently uses the **default user and persona** (`settings.default_tier`, `load_context()`). All Talk conversations go to Inara regardless of who is messaging. Per-conversation persona routing (e.g., Holly gets Tina) is a future enhancement — would require mapping Nextcloud user IDs or conversation tokens to Cortex users.
|
||||
The `/webhook/` prefix is already public in `auth_middleware.py`. If Nginx applies basic auth or IP restrictions, add a `location` block before the default auth block:
|
||||
|
||||
---
|
||||
|
||||
## Claude CLI Auth in systemd
|
||||
|
||||
The `CLAUDE_CODE_OAUTH_TOKEN` in `.env` goes stale after each `claude auth login` (tokens rotate). Cortex reads the token live from `~/.claude/.credentials.json` on every Claude call (`llm_client._fresh_claude_token()`), so no manual `.env` update is needed after re-authentication.
|
||||
|
||||
Also: never set `ANTHROPIC_API_KEY` to an OAuth token value (`sk-ant-oat01-...`) — the Claude CLI treats it as a direct API key and fails. Only real API keys (`sk-ant-api03-...`) belong in `ANTHROPIC_API_KEY`.
|
||||
```nginx
|
||||
location ^~ /webhook/ {
|
||||
proxy_pass http://127.0.0.1:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Triggering the Bot
|
||||
|
||||
- **@mention** — prefix the message with `@inara` (or whatever `AGENT_NAME` is set to); the prefix is stripped before sending to the LLM
|
||||
- **@mention** — prefix the message with `@{persona_name}`; the prefix is stripped before sending to the LLM
|
||||
- **Any message** in a conversation where the bot is enabled — all messages are forwarded, not just @mentions
|
||||
|
||||
---
|
||||
|
||||
## Logs
|
||||
|
||||
Two log streams are useful when debugging:
|
||||
|
||||
```bash
|
||||
# Nextcloud server logs (bot registration errors, webhook rejections)
|
||||
docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ log:tail
|
||||
|
||||
# Cortex service logs (LLM errors, signature failures, timeouts)
|
||||
journalctl --user -u cortex -f
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---|---|---|
|
||||
| 404 on the webhook | `channels.json` missing or no `nextcloud` key | Create/check `home/{username}/channels.json` |
|
||||
| Webhook not received | Bot not enabled for conversation | Enable in Talk conversation settings (Bots) |
|
||||
| Incoming 401 | Wrong secret in `.env` | Match secret to `occ talk:bot:install` value |
|
||||
| Incoming 401 | `bot_secret` in `channels.json` doesn't match `occ install` secret | Re-register with matching secret |
|
||||
| Reply POST returns 401 (first try) | HMAC computed over wrong data | Sign `random + message_text` only (not raw JSON body) |
|
||||
| Reply POST returns 401 (persistent) | Brute force protection triggered | `occ security:bruteforce:reset <cortex-IP>` |
|
||||
| Bot auto-disabled by Nextcloud | Webhook held open too long | Verify `BackgroundTasks` is used — return 200 immediately |
|
||||
| Claude falls back to Gemini | Stale/wrong auth token | Token is auto-refreshed from `~/.claude/.credentials.json`; run `claude auth login` if expired |
|
||||
| No response at all | Nginx blocking the path with basic auth | Add a `location =` block before the auth block (see Nginx section above) |
|
||||
| Reply POST returns 401 (persistent) | Brute force protection triggered | `docker exec -it --user www-data dgr_zone_nextcloud-app-1 php /var/www/html/occ security:bruteforce:reset <cortex-IP>` |
|
||||
| Bot auto-disabled by Nextcloud | Webhook held open too long | Verify `BackgroundTasks` is used — Cortex returns 200 immediately |
|
||||
| Claude falls back to Gemini | Stale/expired auth token | Run `claude auth login`; token is auto-refreshed from `~/.claude/.credentials.json` |
|
||||
| No response at all | Nginx blocking the path | Add a `location ^~ /webhook/` block before any auth block |
|
||||
|
||||
276
docs/OPEN_WEBUI_API.md
Normal file
276
docs/OPEN_WEBUI_API.md
Normal file
@@ -0,0 +1,276 @@
|
||||
# Open WebUI API Reference for Cortex
|
||||
|
||||
> Last updated: 2026-04-03
|
||||
> Source: https://docs.openwebui.com/reference/api-endpoints/
|
||||
> Host in use: `http://192.168.32.19:3000` (scott_gaming — 8 GB VRAM)
|
||||
|
||||
## Local Model Performance (scott_gaming, 8 GB VRAM)
|
||||
|
||||
| Model | Alias | Speed | Practical Context | Spec Context |
|
||||
|---|---|---|---|---|
|
||||
| Gemma 4 E4B | `agent-support-gemma-small` | ~25 t/s | **72k tokens** | 128k |
|
||||
| Gemma 4 26B A4B (MoE) | `agent-support-gemma-medium` | ~9 t/s | **50k tokens** | 256k |
|
||||
|
||||
Context is VRAM-constrained — spec limits are higher but KV cache fills available VRAM first.
|
||||
Techniques to improve: lower KV cache quantization, flash attention, context length tuning in Ollama.
|
||||
|
||||
**Practical implications for the local orchestrator:**
|
||||
- System prompt + memory (T2) + tool results + history: budget ~40-50k for small, ~35-40k for medium
|
||||
- Medium at 9 t/s is fine for background/async tasks; small at 25 t/s is responsive enough for interactive use
|
||||
- Both are well above what's needed for most tool loop iterations (~2-5k tokens per round)
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
All API calls use a bearer token:
|
||||
|
||||
```
|
||||
Authorization: Bearer sk-<api-key>
|
||||
```
|
||||
|
||||
API keys are managed in Open WebUI → Settings → Account → API Keys.
|
||||
Cortex stores these per-user in `home/{username}/local_llm.json` → `hosts[].api_key`.
|
||||
|
||||
---
|
||||
|
||||
## Core Endpoints Used by Cortex
|
||||
|
||||
### List Available Models
|
||||
|
||||
```
|
||||
GET /api/models
|
||||
Authorization: Bearer sk-...
|
||||
```
|
||||
|
||||
Returns all models (Ollama, OpenAI-proxied, custom functions).
|
||||
Used by `/api/local-llm/fetch-models` in `routers/local_llm.py`.
|
||||
|
||||
Response shape:
|
||||
```json
|
||||
{
|
||||
"data": [
|
||||
{ "id": "gemma4-e4b", "name": "Gemma 4 E4B" },
|
||||
...
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Chat Completions (OpenAI-compatible)
|
||||
|
||||
```
|
||||
POST /api/chat/completions
|
||||
Authorization: Bearer sk-...
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
Standard OpenAI chat format. Supports:
|
||||
- `messages` — standard role/content array
|
||||
- `model` — model ID or workspace alias
|
||||
- `tools` + `tool_choice` — function calling (see Tool Loop below)
|
||||
- `stream: true/false`
|
||||
|
||||
This is the endpoint used by `_local()` in `llm_client.py`.
|
||||
|
||||
### Anthropic Messages API Compatibility
|
||||
|
||||
```
|
||||
POST /api/v1/messages
|
||||
Authorization: Bearer sk-...
|
||||
```
|
||||
|
||||
Open WebUI also accepts Anthropic-format requests and auto-converts them.
|
||||
Could be used to route Claude SDK calls through Open WebUI.
|
||||
Base URL for this mode: `http://192.168.32.19:3000/api`
|
||||
|
||||
### Direct Ollama Proxy
|
||||
|
||||
```
|
||||
GET /ollama/api/tags — list models
|
||||
POST /ollama/api/generate — streaming completions
|
||||
POST /ollama/api/embed — generate embeddings
|
||||
```
|
||||
|
||||
Use these if you need to bypass Open WebUI's filter layer and hit Ollama directly.
|
||||
Ollama is also accessible directly at `http://192.168.32.19:11434`.
|
||||
|
||||
---
|
||||
|
||||
## Tool / Function Calling
|
||||
|
||||
Both Gemma 4 models (E4B and 26B A4B) support function calling via the standard
|
||||
OpenAI `tools` parameter. Open WebUI passes this through to the underlying model.
|
||||
|
||||
### Request Format
|
||||
|
||||
```json
|
||||
POST /api/chat/completions
|
||||
{
|
||||
"model": "gemma4-26b-a4b",
|
||||
"messages": [
|
||||
{ "role": "system", "content": "..." },
|
||||
{ "role": "user", "content": "What's the weather?" }
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "web_search",
|
||||
"description": "Search the web for current information",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": { "type": "string", "description": "Search query" }
|
||||
},
|
||||
"required": ["query"]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"tool_choice": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
### Tool Call Response
|
||||
|
||||
When the model wants to call a tool, it returns `finish_reason: "tool_calls"`:
|
||||
|
||||
```json
|
||||
{
|
||||
"choices": [{
|
||||
"finish_reason": "tool_calls",
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": null,
|
||||
"tool_calls": [{
|
||||
"id": "call_abc123",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "web_search",
|
||||
"arguments": "{\"query\": \"current weather NYC\"}"
|
||||
}
|
||||
}]
|
||||
}
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
### Sending Tool Results Back
|
||||
|
||||
Append the assistant's tool_call message and a tool result message, then re-submit:
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{ "role": "user", "content": "What's the weather?" },
|
||||
{ "role": "assistant", "content": null,
|
||||
"tool_calls": [{ "id": "call_abc123", "function": { "name": "web_search", "arguments": "..." } }] },
|
||||
{ "role": "tool", "tool_call_id": "call_abc123",
|
||||
"content": "Current weather in NYC: 62°F, partly cloudy." }
|
||||
],
|
||||
"tools": [...],
|
||||
"tool_choice": "auto"
|
||||
}
|
||||
```
|
||||
|
||||
Repeat until `finish_reason: "stop"`.
|
||||
|
||||
---
|
||||
|
||||
## RAG (Retrieval Augmented Generation)
|
||||
|
||||
### Upload a File
|
||||
|
||||
```
|
||||
POST /api/v1/files/
|
||||
Authorization: Bearer sk-...
|
||||
Content-Type: multipart/form-data
|
||||
|
||||
file=@/path/to/document.pdf
|
||||
```
|
||||
|
||||
Returns a file ID. Poll `/api/v1/files/{id}/process/status` until `completed`.
|
||||
|
||||
### Knowledge Collections
|
||||
|
||||
```
|
||||
POST /api/v1/knowledge/{collection_id}/file/add
|
||||
{ "file_id": "..." }
|
||||
```
|
||||
|
||||
### Use in Chat
|
||||
|
||||
Reference files or knowledge collections in any chat request:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "gemma4-26b-a4b",
|
||||
"messages": [...],
|
||||
"files": [
|
||||
{ "type": "file", "id": "file-id" },
|
||||
{ "type": "collection", "id": "collection-id" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Process a Web URL into a Collection
|
||||
|
||||
```
|
||||
POST /api/v1/retrieval/process/web
|
||||
{ "url": "https://example.com/article", "collection_id": "..." }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Filter Behavior with Direct API Calls
|
||||
|
||||
Open WebUI supports inlet/outlet filter pipelines. With direct API access:
|
||||
|
||||
| Filter | Runs automatically? |
|
||||
|-----------|---------------------|
|
||||
| `inlet()` | ✅ Yes |
|
||||
| `stream()`| ✅ Yes |
|
||||
| `outlet()`| ❌ Manual only — call `POST /api/chat/completed` after receiving response |
|
||||
|
||||
For Cortex's use case (tool loop orchestration), this is not a concern — we're
|
||||
driving the loop ourselves and don't rely on Open WebUI's filter pipeline.
|
||||
|
||||
---
|
||||
|
||||
## Relevant Cortex Files
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `cortex/llm_client.py` — `_local()` | Current local backend (direct chat only) |
|
||||
| `cortex/routers/local_llm.py` | Local model settings page + fetch-models endpoint |
|
||||
| `cortex/user_settings.py` | Per-user host + model config (`local_llm.json`) |
|
||||
| `cortex/orchestrator_engine.py` | Gemini API tool loop — reference for local version |
|
||||
| `home/{user}/local_llm.json` | Stored host/model config |
|
||||
|
||||
---
|
||||
|
||||
## Planned: Local Orchestrator (`local_orchestrator_engine.py`)
|
||||
|
||||
A local equivalent of `orchestrator_engine.py` that:
|
||||
1. Takes the same tool definitions already registered in `cortex/tools/`
|
||||
2. Converts them to OpenAI `tools` format (already close — minor schema diff from Gemini)
|
||||
3. Runs a ReAct loop against the local model via `/api/chat/completions`
|
||||
4. Falls back gracefully if the model doesn't return a valid tool call
|
||||
|
||||
See `documentation/TODO__Agents.md` — `[Local] Tool-capable local orchestrator`.
|
||||
|
||||
Model recommendation:
|
||||
- **Gemma 4 26B A4B** (256k ctx, MoE — fast for its size) for complex tool tasks
|
||||
- **Gemma 4 E4B** (128k ctx) for lightweight/fast tasks
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- Open WebUI workspace aliases (e.g. `agent-support-gemma-small`) resolve to the
|
||||
underlying Ollama model — use aliases in Cortex for human-friendly model names.
|
||||
- `tool_choice: "auto"` lets the model decide; `"none"` forces plain text response;
|
||||
`{"type": "function", "function": {"name": "..."}}` forces a specific tool.
|
||||
- Gemma 4 models support combined tool use + reasoning (thinking tokens) — useful
|
||||
for complex multi-step tasks.
|
||||
- For embeddings (future RAG work), use `/ollama/api/embed` directly.
|
||||
226
documentation/ARCH__AE_INTEGRATION.md
Normal file
226
documentation/ARCH__AE_INTEGRATION.md
Normal file
@@ -0,0 +1,226 @@
|
||||
# Aether Platform Integration — Cortex Tool Layer
|
||||
|
||||
> Last updated: 2026-04-30
|
||||
> Status: Journal toolset complete — broader AE integration planned
|
||||
|
||||
This doc covers how Cortex/Inara integrates with the Aether Platform API, what's
|
||||
implemented, what the data model looks like, and what's planned next.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Cortex connects to the Aether Platform V3 API to give the orchestrator read/write
|
||||
access to the user's knowledge base (Journals) and task data. Auth uses the same
|
||||
`x-aether-api-key` + `x-account-id` headers as every other Aether client.
|
||||
|
||||
Config lives in `.env`:
|
||||
```
|
||||
AE_API_URL=https://dev-api.oneskyit.com
|
||||
AE_API_KEY=...
|
||||
AE_ACCOUNT_ID=...
|
||||
AE_API_TIMEOUT=15
|
||||
```
|
||||
|
||||
Tool implementation: `cortex/tools/ae_knowledge.py`
|
||||
Tool registrations: `cortex/tools/__init__.py`
|
||||
|
||||
---
|
||||
|
||||
## V3 Search Engine
|
||||
|
||||
### Endpoint
|
||||
```
|
||||
POST /v3/crud/{obj_type}/search
|
||||
```
|
||||
For nested objects (journal_entry scoped to a journal):
|
||||
```
|
||||
POST /v3/crud/journal_entry/search
|
||||
?for_obj_type=journal&for_obj_id={journal_id}
|
||||
```
|
||||
|
||||
### Search body
|
||||
```json
|
||||
{
|
||||
"query_string": "fulltext search term",
|
||||
"and": [
|
||||
{ "field": "tags", "op": "icontains", "value": "networking" },
|
||||
{ "field": "created_on", "op": "gte", "value": "2026-01-01" }
|
||||
],
|
||||
"or": [...],
|
||||
"page_size": 20,
|
||||
"page": 1,
|
||||
"order_by": "-updated_on"
|
||||
}
|
||||
```
|
||||
|
||||
**`query_string` vs `and` filters on `default_qry_str`:**
|
||||
- `query_string` → triggers `MATCH(default_qry_str) AGAINST(... IN BOOLEAN MODE)` — uses the
|
||||
FULLTEXT index. Faster and supports boolean operators (`+word`, `-word`, `"phrase"`).
|
||||
- `and` with `icontains` on `default_qry_str` → plain `LIKE '%term%'`. Slower, no index.
|
||||
|
||||
**Important:** `query_string` must be present for `and`/`or` filters to apply. When using
|
||||
filters without a keyword query, pass `query_string: "%"` as a wildcard to activate the
|
||||
filter path without restricting by keyword.
|
||||
|
||||
### Supported operators
|
||||
| Operator | SQL | Notes |
|
||||
|---|---|---|
|
||||
| `eq` | `=` | exact match |
|
||||
| `ne` | `!=` | not equal |
|
||||
| `gt` / `gte` | `>` / `>=` | numeric, dates |
|
||||
| `lt` / `lte` | `<` / `<=` | numeric, dates |
|
||||
| `contains` / `icontains` | `LIKE '%v%'` | substring; both case-insensitive on MariaDB |
|
||||
| `startswith` / `istartswith` | `LIKE 'v%'` | |
|
||||
| `endswith` / `iendswith` | `LIKE '%v'` | |
|
||||
| `like` | `LIKE` | raw LIKE pattern |
|
||||
| `in` | `IN (...)` | value is a list |
|
||||
| `is_null` / `is_not_null` | `IS NULL` / `IS NOT NULL` | no value needed |
|
||||
|
||||
### Sorting
|
||||
`order_by` accepts any indexed field name. Prefix with `-` for descending:
|
||||
- `-updated_on` (default for listing)
|
||||
- `-created_on`
|
||||
- `name`
|
||||
- `-priority`
|
||||
|
||||
### Pagination
|
||||
`page_size` (default 10, max ~100) + `page` (1-based).
|
||||
Total count is in `response["meta"]["data_list_count"]` — not a top-level key.
|
||||
|
||||
---
|
||||
|
||||
## journal_entry Schema
|
||||
|
||||
Full table schema from `ae_describe journal_entry --detailed`:
|
||||
|
||||
| Field | Type | Indexed | Notes |
|
||||
|---|---|---|---|
|
||||
| `id_random` | varchar(22) | UNI | DB public ID field — but API responses return this as `journal_entry_id` (the Vision ID convention: `{obj_type}_id`). `id_random` key is `None` in responses. |
|
||||
| `journal_id` | int | MUL | FK — use `for_obj_id` param in search |
|
||||
| `name` | varchar(250) | MUL | Entry title |
|
||||
| `short_name` | varchar(25) | | |
|
||||
| `summary` | text | | Short summary (1–2 sentences) |
|
||||
| `content` | text | | Full markdown content |
|
||||
| `content_html` | text | | HTML version |
|
||||
| `content_json` | longtext | | Structured content (editor format) |
|
||||
| `content_encrypted` | longtext | | Optional encrypted content |
|
||||
| `tags` | varchar(255) | MUL | Comma-separated string — filter with `icontains` |
|
||||
| `type` / `type_code` | varchar | | Classification: type |
|
||||
| `topic` / `topic_code` | varchar | | Classification: topic |
|
||||
| `activity` / `activity_code` | varchar | | Classification: activity |
|
||||
| `category_code` | varchar(25) | | Classification: category |
|
||||
| `code` | varchar(20) | | Short entry code |
|
||||
| `start_datetime` | datetime | MUL | Optional event start |
|
||||
| `end_datetime` | datetime | | Optional event end |
|
||||
| `seconds` / `hours` | int/decimal | | Duration |
|
||||
| `priority` | tinyint | MUL | 1=low → 5=high |
|
||||
| `status` | int | MUL | Status code (domain-specific) |
|
||||
| `private` / `public` / `personal` / `professional` | tinyint | MUL | Visibility flags |
|
||||
| `billable` | tinyint | | Billing flag |
|
||||
| `enable` | tinyint NOT NULL | MUL | Soft-delete flag (default 1) |
|
||||
| `hide` | tinyint | MUL | UI hide flag |
|
||||
| `archive` | tinyint | MUL | Archived flag |
|
||||
| `default_qry_str` | text | FULLTEXT | Auto-generated search target (name + content) |
|
||||
| `data_json` | longtext | | Arbitrary structured data |
|
||||
| `notes` | text | | Internal notes |
|
||||
| `created_on` | timestamp NOT NULL | MUL | Auto-set on create |
|
||||
| `updated_on` | timestamp | MUL | Auto-updated on change |
|
||||
|
||||
### journal Schema (top-level)
|
||||
|
||||
| Field | Type | Notes |
|
||||
|---|---|---|
|
||||
| `id_random` | varchar(22) | DB field — returned in API as `journal_id` |
|
||||
| `name` | varchar(250) | Journal name |
|
||||
| `summary` / `description` | text | |
|
||||
| `type_code` | varchar(25) | Journal type |
|
||||
| `enable` | tinyint | |
|
||||
| `created_on` / `updated_on` | timestamp | |
|
||||
|
||||
---
|
||||
|
||||
## Current Tool Inventory
|
||||
|
||||
| Tool | Status | Notes |
|
||||
|---|---|---|
|
||||
| `ae_journal_list` | ✅ | Lists journals with id + name |
|
||||
| `ae_journal_search` | ✅ | Fulltext + tag/date/type/status/priority filters; paginated |
|
||||
| `ae_journal_entry_read` | ✅ | Full content by entry_id; configurable truncation |
|
||||
| `ae_journal_entries_list` | ✅ | Browse a journal newest-first; paginated |
|
||||
| `ae_journal_entry_create` | ✅ | Create with title, content, tags, summary |
|
||||
| `ae_journal_entry_update` | ✅ | Patch any fields (title, content, tags, summary, enable) |
|
||||
| `ae_journal_entry_disable` | ✅ | Soft-delete (enable=false) |
|
||||
| `ae_journal_entry_append` | ✅ | Timestamped append to bottom |
|
||||
| `ae_journal_entry_prepend` | ✅ | Timestamped prepend to top |
|
||||
| `ae_task_list` | ✅ | agents_sync Kanban (admin only) |
|
||||
|
||||
---
|
||||
|
||||
## ae_journal_search — Current Signature
|
||||
|
||||
All filters are optional and combine with AND. At least one should be provided.
|
||||
|
||||
```python
|
||||
ae_journal_search(
|
||||
query: str = "", # fulltext via query_string (MATCH/AGAINST)
|
||||
journal_id: str = "", # scope to a specific journal
|
||||
tags: str = "", # icontains on tags field
|
||||
type_code: str = "", # eq on type_code
|
||||
topic_code: str = "", # eq on topic_code
|
||||
date_from: str = "", # created_on gte (YYYY-MM-DD)
|
||||
date_to: str = "", # created_on lte (YYYY-MM-DD, exclusive of time — use next day to include full day)
|
||||
sort_by: str = "updated", # updated | created | name | priority
|
||||
sort_order: str = "desc",
|
||||
status: int | None = None,
|
||||
priority: int | None = None,
|
||||
max_results: int = 10,
|
||||
page: int = 1,
|
||||
)
|
||||
```
|
||||
|
||||
**date_to boundary note:** `date_to='2026-01-17'` means `<= 2026-01-17 00:00:00`, which
|
||||
excludes entries created later that day. Use `date_to='2026-01-18'` to include all of Jan 17.
|
||||
|
||||
---
|
||||
|
||||
## Planned: Broader AE Platform Integration
|
||||
|
||||
### Phase 1 — Journal Toolset (current)
|
||||
Complete read/write/search for Journals and Journal Entries.
|
||||
|
||||
### Phase 2 — Tasks & Projects
|
||||
- `ae_task_create` / `ae_task_update` / `ae_task_complete` on Aether tasks (not just agents_sync Kanban)
|
||||
- Read project/task hierarchy
|
||||
|
||||
### Phase 3 — Knowledge Import Pipeline
|
||||
- Script to walk markdown dirs, chunk by H2, create Journal entries
|
||||
- Dedup via search-before-create pattern
|
||||
- Tag and classify entries automatically via orchestrator
|
||||
|
||||
### Phase 4 — People & Contacts
|
||||
- Read contact records (person, organization)
|
||||
- Link journal entries to contacts
|
||||
|
||||
### Phase 5 — Calendar / Events
|
||||
- `start_datetime` / `end_datetime` already on journal_entry
|
||||
- Could expose time-scoped journal queries as a calendar view
|
||||
|
||||
---
|
||||
|
||||
## Notes on `tags` field
|
||||
|
||||
`tags` is stored as a raw comma-separated varchar(255), not a JSON array.
|
||||
The API accepts a Python list on write (the `tags` PATCH key takes a list and the backend joins it).
|
||||
On read, it comes back as a **string** (e.g. `"shelterluv, api"`), not a list — normalize before
|
||||
displaying: `[t.strip() for t in tags_str.split(",") if t.strip()]`.
|
||||
For filtering: use `icontains` on `tags` inside the `"and"` list, e.g.:
|
||||
`{"field": "tags", "op": "icontains", "value": "networking"}`.
|
||||
A tag search for "net" matches "networking" AND "subnet" — acceptable for now.
|
||||
True per-tag filtering would require a tags junction table.
|
||||
|
||||
## Notes on `default_qry_str`
|
||||
|
||||
Auto-populated by the backend from `name` + content fields. Do not write to it directly.
|
||||
FULLTEXT index supports boolean mode: `+required -excluded "exact phrase"`.
|
||||
The `query_string` key in the search body triggers this path automatically.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user