Refactor core API helpers and implement System Testing Dashboard
- Unified and hardened get, post, patch, and delete helpers with standardized retry logic, kebab-case headers, and V3 response envelope handling. - Implemented robust 'Bootstrap Paradox' resolution logic across the API stack to handle unauthenticated site domain lookups safely. - Enhanced API helpers to support custom fetch injection, enabling reliable server-side execution in SvelteKit. - Upgraded /testing page into a comprehensive System Testing Dashboard for core helper and V3 search verification. - Updated TODO.md and GEMINI.md with 2026-01-08 session learnings and 'Frontier Journals' vision.
This commit is contained in:
Scott Idem
68
GEMINI.md
68
GEMINI.md
@@ -178,7 +178,22 @@ The crucial next step is to use the **Network** tab in the browser's developer t
|
||||
The activity logging functionality is now working as expected. While the original hypothesis of a circular dependency was a plausible architectural issue, the immediate problem was a more fundamental runtime error exacerbated by hidden console output. The temporary isolation of the activity log function (`src/lib/ae_idaa/idaa_activity_log.ts`) is no longer needed.
|
||||
|
||||
---
|
||||
## Unified Aether AI Agent (UE-AE-01) Transition (2026-01-07)
|
||||
## Aether Ops Extension (Gemini CLI)
|
||||
|
||||
The project utilizes a custom Gemini CLI extension (`aether-ops`) to orchestrate the stack via the Model Context Protocol (MCP).
|
||||
|
||||
### Key Tools
|
||||
- **`verify_api_integrity`**: Executes the `test_ae_api_v3.py` suite. Verifies that the FastAPI backend is reachable and that the "Bootstrap Paradox" unauthenticated bypass is functioning correctly.
|
||||
- **`schema_sync`**: Synchronizes Pydantic models with TypeScript interfaces.
|
||||
- **`sql_query`**: Direct read-only access to the MariaDB for state verification.
|
||||
- **`docker_restart` / `docker_logs`**: Controls the backend and database containers.
|
||||
- **`send_message` / `add_task`**: Coordination tools for multi-agent workflows.
|
||||
|
||||
### Workflow Integration
|
||||
Always run `verify_api_integrity` after making changes to:
|
||||
1. Core API helpers in `src/lib/ae_api/`.
|
||||
2. Backend CRUD routes in the FastAPI project.
|
||||
3. Site domain or account lookup logic.
|
||||
|
||||
### Vision
|
||||
The project is moving towards a single, unified agent (UE-AE-01) with "Total System Awareness" across the entire Aether stack: MariaDB (Remote), FastAPI (Docker), SvelteKit (Local), Nginx (Proxy), and Syncthing (Storage).
|
||||
@@ -241,32 +256,41 @@ The `frontend_svelte` agent provided critical feedback to `backend_fastapi` for
|
||||
- **Search Logic Construction:** When building complex V3 `search_query` objects, avoid including empty `and` or `or` arrays, as some backend parsers may strictly validate their presence or content. Only attach these properties if they contain at least one filter.
|
||||
- **Backend Operator Support:** Always verify supported operators (`like`, `eq`, `gt`, etc.) in the backend FastAPI implementation. Using unsupported operators like `ilike` or `contains` will cause immediate backend `ValueError` crashes.
|
||||
|
||||
### Session Learnings (2026-01-07)
|
||||
### Session Learnings (2026-01-07) - Session 2
|
||||
|
||||
**Context:** Finalized IDAA Bulletin Board V3 migration, implemented global `editable_fields.ts` whitelists, and standardized JWT authentication for CRUD V3. Resolved the "Bootstrap Paradox" for site domain lookups.
|
||||
**Context:** Established formal agent identity, verified synchronization crons, and tested the expanded Agent Sync toolset on the workstation.
|
||||
|
||||
**Key Accomplishments:**
|
||||
- **JWT Authentication:** Standardized JWT usage across all CRUD V3 operations. Updated API helpers to automatically inject `Authorization: Bearer` headers and added secure file download support via `jwt` URL parameters.
|
||||
- **Activity Log Management:** Fully migrated to V3 CRUD. Created a standalone management page at `/core/activity_logs` and integrated filtered activity history into the Person detail view.
|
||||
- **IDAA Bulletin Board V3:** Completed migration to V3 CRUD. Resolved a critical bug where results disappeared after filtering by ensuring `account_id` is injected into processed objects before being saved to IndexedDB.
|
||||
- **Race Condition Resolution:** Identified and fixed a race condition during database refresh by `await`ing Dexie `.clear()` operations.
|
||||
- **Global Editable Field Whitelists:** Successfully created `.editable_fields.ts` whitelist files for all remaining Aether objects (Journals, Events, Sponsorships).
|
||||
- **Bug Fix:** Resolved a critical `ReferenceError` in the POST helper that was causing 500 errors during site lookup.
|
||||
- **Bootstrap Paradox Resolution:** Successfully implemented and verified `lookup_site_domain_v3` using unauthenticated POST `/v3/crud/site_domain/search`. Modified the function to aggressively strip all authentication headers (`Authorization`, `x-account-id`, `jwt`) to satisfy the backend guest-access requirement.
|
||||
- **Enhanced Verification UI:** Upgraded `/testing` page with custom FQDN input, `try...catch` error handling, and robust result visualization to debug V3 site lookups without affecting the root layout.
|
||||
- **Agent Identity & Coordination:** Formally established identity as `frontend_svelte`. Updated workstation crontab to monitor the `frontend_svelte` inbox. Sent coordination and status messages to `backend_fastapi` and `gemini_cli`.
|
||||
- **Tool Verification:**
|
||||
- Verified `docker_control.py` for backend container monitoring.
|
||||
- Verified `sql_inspector.py` for direct database verification during UI development.
|
||||
- **Enhanced Schema Sync:** Identified a database dependency issue in the original `schema_sync.py` when run from the host. Created a robust alternative, `schema_sync_v2.py`, which generates TypeScript interfaces directly from the offline `registry_v3.json`. Successfully verified this by generating the `ActivityLog` interface.
|
||||
- **Workflow Optimization:** Provided feedback for the main `Agents Sync` README to include canonical IDs, execution context tags (Host vs. Docker), and a "First 5 Minutes" onboarding checklist for new agents.
|
||||
|
||||
**Key Learnings:**
|
||||
- **Header Normalization:** When merging headers in API helpers, ensure consistent kebab-case normalization (e.g., `Authorization` instead of `authorization`) to match backend expectations and avoid duplicates.
|
||||
- **Secure Direct Access:** For direct browser-led requests like file downloads, passing the JWT as a URL parameter is a robust alternative to header-based auth which can be difficult to set on standard `<a>` or `<img>` tags.
|
||||
- **IndexedDB Filter Consistency:** When using client-side filtering (e.g., `liveQuery`) on fields like `account_id`, it is vital that the frontend data processors inject these IDs if the API response omits them (common in nested V3 routes).
|
||||
- **Asynchronous DB Operations:** Always `await` database cleanup operations (`.clear()`) before triggering new data loads to prevent stale data or empty lists due to race conditions.
|
||||
- **Bootstrap Auth Isolation:** Guest endpoints like `site_domain/search` are extremely sensitive to any authentication headers. Even an empty or "fake" token can trigger a `403 Forbidden` if the backend doesn't explicitly ignore them.
|
||||
- **API Response Robustness:** The V3 API can return different response envelopes (some with `.data`, some without). Frontend helpers should use `json.data !== undefined ? json.data : json` to be truly resilient.
|
||||
- **Custom Fetch Alignment:** `post_object` must be refactored to use the SvelteKit `fetch` (if provided in `api_cfg`) to ensure consistent behavior across different environments (browser vs server vs test) and to match the implementation in `get_object`.
|
||||
- **Execution Context Matters:** Scripts that import backend API modules (like the original `schema_sync.py`) often trigger database connection attempts, which fail without environment-specific credentials. Registry-based tools are safer and more robust for frontend workstation use.
|
||||
- **Inbox Naming Consistency:** Confirmed that the canonical inbox directory name is `frontend_svelte`, which must be strictly used for crons and messaging to avoid missed communications.
|
||||
- **Registry as "Source of Truth":** The `registry_v3.json` file is a powerful asset for the frontend, enabling automated model generation without needing the backend or database to be actively reachable.
|
||||
|
||||
**Next Steps:**
|
||||
- **Person Management:** Build out dedicated edit forms and finalize the "Linked Activity & Content" section.
|
||||
- **Address/Contact Details:** Implement detail pages for these newly added modules.
|
||||
- **Coordination:** Continue checking `agents_sync/inbox` for API V3 updates from the backend agent.
|
||||
- **API Helper Refactoring:** Implement the identified `post_object` improvements (custom fetch and robust extraction).
|
||||
- **V3 Interface Verification:** Verify and integrate the 59 TypeScript interfaces exported to `agents_sync/technical/aether_interfaces.ts`.
|
||||
- **Journals Audit:** Begin the "Frontier Journals" codebase audit.
|
||||
- **UE-AE-01 Transition:** Continue monitoring the inbox for specific migration tasks from the manager.
|
||||
|
||||
### Session Learnings (2026-01-08)
|
||||
|
||||
**Context:** Refactored the core Aether API helper suite (`get`, `post`, `patch`, `delete`) to ensure architectural consistency, robust error handling, and support for unauthenticated lookups (the "Bootstrap Paradox").
|
||||
|
||||
**Key Accomplishments:**
|
||||
- **API Helper Alignment:** Standardized all primary helpers to support custom `fetch` injection (essential for SvelteKit SSR), kebab-case header standardization, and automated JWT-to-Authorization header injection.
|
||||
- **Bootstrap Paradox Resolution:** Implemented a robust bypass in the header merging logic. By passing `x-no-account-id: null` to the helpers, the default `x-account-id` is stripped, and a non-null dummy value is sent, allowing unauthenticated V3 searches for site domains.
|
||||
- **Non-Mutating Config:** Eliminated direct mutations of the shared `api_cfg` object within the helpers, preventing difficult-to-track side effects across the application.
|
||||
- **System Testing Dashboard:** Upgraded the `/testing` page with a dedicated suite for core helper verification and V3 search validation.
|
||||
- **CLI Verification Suite:** Created `agents_sync/scripts/test_ae_api_v3.py` for rapid, headless verification of the API stack from the command line.
|
||||
|
||||
**Key Learnings:**
|
||||
- **Implicit Header Requirements:** Some backend environments (like the Aether V3 API) strictly require the presence of a "bypass" header even when credentials are omitted. A null value in the client-side header map is often stripped by `fetch`, so a dummy string value (e.g., "Nothing to See Here") is necessary for the header to be successfully transmitted.
|
||||
- **Environment Parity in Testing:** Verifying API helpers in both the browser (via the `/testing` page) and the CLI (via Python scripts) is critical for identifying environment-specific issues like CORS, header formatting, and timeout behavior.
|
||||
- **Barrel File Strategy:** As the API stack grows, a pure barrel file strategy for `api.ts` (exporting from dedicated modules) is superior to a single "God Object" for maintainability and code splitting.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user