From 5ce193d4749a48378e0b91d757635b38250d1f5b Mon Sep 17 00:00:00 2001 From: Scott Idem Date: Fri, 9 Jan 2026 17:10:25 -0500 Subject: [PATCH] Saving more note updates --- GEMINI.md | 53 ++++++++++++++------------- documentation/REFACTOR_API_CRUD_V3.md | 13 +++++++ 2 files changed, 41 insertions(+), 25 deletions(-) diff --git a/GEMINI.md b/GEMINI.md index 40efc73..22887c9 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -21,43 +21,46 @@ I am the **primary orchestrator and main helper** for the development of the **U - **V3 Searchable Fields**: Confirmed that `searchable_fields` are fully implemented in `app/object_definitions/` and utilized by `app/lib_sql_search.py` for dynamic query generation. - **Environment Sync Clarification**: Critical understanding that the `/home/scott/OSIT_dev/` directory is **NOT** a synchronized directory across different machines (e.g., Workstation vs. Laptop). This means virtual environments and installed dependencies are environment-specific and do not propagate via Syncthing. This is crucial for debugging cross-machine environment issues. -## Session Summary & Progress (Jan 8, 2026) - -This session focused on debugging, task management, and communication protocols within the Aether AI Agent network. - -* **Task Management Bug Resolved:** The `add_task` tool bug (duplicate IDs) was identified, reported, and fixed by the Manager/Coordinator agent. The fix introduced microsecond precision for IDs in task management. -* **Agent Bridge Restoration Verified:** `app/routers/agent_bridge.py` was inspected; `psutil` handling was confirmed robust. Docker container logs for FastAPI (green/red) showed no crashes related to `agent_bridge.py` or `psutil`. The task was marked as Done. -* **Docker Registry Sync Audit Completed:** An audit of `ae_api_dev_green` and `ae_api_dev_red` against `docker-compose.yml` confirmed consistency in environment variables, volumes, ports, images, and extra hosts. No critical configuration drift was identified. The task was marked as Done. -* **Frontend Interface Validation Successful:** The `frontend` agent confirmed full validation and successful unified type migration for 59 TypeScript interfaces. The task was marked as Done. -* **AE Person Creation Fixes:** - * Identified that `given_name` was failing due to `null` values violating database `NOT NULL` constraints. - * Added Pydantic validators to `Person_Base` to default `given_name` to `""` and `allow_auth_key` to `True` if `None` is provided. - * Modified `app/routers/api_crud_v3.py` to strip virtual `*_id_random` fields (like `account_id_random`) from payloads before database insertion/update. This prevents "Unknown column" errors when the frontend sends convenience lookup fields. -* **`ae_obj_info` Dependency Resolved:** - * Initial investigation confirmed `pytz` was installed in the Workstation's host venv, contradicting an `mcp_agent` message. - * Further clarification revealed `OSIT_dev` is not synced between Workstation and Laptop, causing environment discrepancies. - * The `mcp_agent` provided a traceback from the Laptop, confirming `pytz` was indeed missing from the *Laptop's* host venv. - * User (Scott Idem) confirmed manual installation of `pytz` in the Laptop's host venv, resolving the issue. The task was marked as Done. - * The source code for `ae_obj_info` is confirmed to be at `agents_sync/scripts/ae_object_info.py`. -* **New MCP Tool Announcements:** - * `read_reference` tool introduced for technical stubs. - * `delete_task` and `git_smart_commit` tools are now live, enhancing task management and version control. -* **Codebase Investigator Report:** The report for Aether extension and journal management is pending `gemini_cli`'s instruction; further review is blocked until notification. Task `155435511` is completed for this phase. +## Session Summary & Progress (Jan 9, 2026) +This session focused on fixing Person creation issues, modularizing the V3 CRUD system, and enhancing Developer Experience (DX) for the frontend. +* **V3 CRUD Modularization (Phase 1-3):** + * Successfully refactored `app/routers/api_crud_v3.py` into a modular architecture. + * **Logic Extraction:** Created `app/lib_api_crud_v3.py` for shared security and filtering logic. + * **Nested Routes:** Created `app/routers/api_crud_v3_nested.py` to isolate relational parent-child endpoints. + * **Schema Extraction:** Created `app/lib_schema_v3.py` for database/model introspection. +* **DX & Error Transparency:** + * **Error Bubbling:** Updated `app/db_sql.py` to capture SQL exceptions and implemented `format_db_error` to provide human-readable details in API responses. + * **Validation Endpoint:** Added `POST /v3/crud/{obj_type}/validate` for dry-run payload verification. + * **Automatic Sanitization:** Backend now automatically strips virtual lookup fields (`*_id_random`) and view-only fields (via `fields_to_exclude_from_db`) from write payloads. +* **Object Model Alignment:** + * **Address & Contact:** Updated models and JSON mappings to include `hide`, `priority`, `sort`, `group`, and `notes`. Fixed missing imports and unsafe validator access. + * **Journal:** Added `fields_to_exclude_from_db` to filter out view-only fields (e.g., `person_full_name`) during updates. + * **Post:** Added `external_person_id` to searchable fields. +* **Test Suite Creation:** Established a `/tests` directory with focused scripts: + * `test_model_validation.py` (Person/Auth key defaults). + * `test_v3_filtering.py` (Journal exclusion lists). + * `test_v3_router_filtering.py` (Payload sanitization). + * `test_v3_error_bubbling.py` (SQL error formatting). +* **System Health & Architecture:** + * Audit confirmed API and AI services (Ollama/WebUI) are stable; ComfyUI is stuck in a restart loop. + * Analyzed and recommended scope expansion to the `/home/scott/OSIT_dev/` root for better platform orchestration. ## Current To-Do List ### 1. High Priority & Urgent (UE-AE-01 Orchestration) -- [ ] **Review codebase investigator report for Aether extension and journal management.** (ID: 155435511) - In Progress (awaiting report from codebase_investigator via gemini_cli). -- [x] **Fix AE Person Creation (Backend/API)** - Completed (Validators added and router filtering implemented). +- [ ] **Review codebase investigator report for Aether extension and journal management.** (ID: 155435511) - In Progress (awaiting report from codebase_investigator). +- [ ] **Implement scope expansion to `/home/scott/OSIT_dev/`.** - Pending registry update. ### 2. Infrastructure & Technical Debt - [ ] **Psutil Container Update**: Rebuilding the API image with C-build tools (GCC) to support native system monitoring. (Deferred / Paused) +- [ ] **Fix ComfyUI Container**: Resolve the restart loop in the Ollama/WebUI environment. ### 3. Other Agent Tasks -- [ ] **[scott_wks] Perform Initial System Health Audit** (ID: 131939030) - [ ] **Review and process new messages from various agents.** (ID: 134908245) - In Progress (Internal gemini_cli task). +- [x] **[scott_wks] Perform Initial System Health Audit** (ID: 131939030) - Done. +- [x] **[API-V3] Implement Error Bubbling and Validation Endpoint** (ID: 163752448) - Done. ### Workflow & Collaboration diff --git a/documentation/REFACTOR_API_CRUD_V3.md b/documentation/REFACTOR_API_CRUD_V3.md index 53c7ff7..23c70c5 100644 --- a/documentation/REFACTOR_API_CRUD_V3.md +++ b/documentation/REFACTOR_API_CRUD_V3.md @@ -26,3 +26,16 @@ The V3 CRUD system is now modularized into: - `app/routers/api_crud_v3_nested.py`: Relational/child object routes. - `app/lib_api_crud_v3.py`: Shared security, filtering, and sanitization logic. - `app/lib_schema_v3.py`: Database and model introspection logic. + +## New DX Features + +### 1. Error Bubbling +Database and Validation errors are no longer swallowed. +- **Validation Errors**: Specific Pydantic validation failures are returned in `meta.details`. +- **Database Errors**: MariaDB constraint violations (e.g., Duplicate Entry, Not Null) are captured and cleaned up via `format_db_error()` and returned in `meta.details`. + +### 2. Validation Endpoint +**Route**: `POST /v3/crud/{obj_type}/validate` +- **Purpose**: Dry-run validation of a payload against the Pydantic model. +- **Behavior**: Returns `data: true` if valid, or a 400 error with specific details if invalid. No database operations are performed. +