Saving notes

GEMINI.md

@@ -1,78 +1,109 @@
-# Gemini Added Memories
+# Gemini Agent Context: Aether API Orchestrator
 
-## My Role and Operating Principles
+> **Template Version:** 1.2 (2026-01-26)
+> **Purpose:** Standardized memory structure for all Aether Agents.
+> **Structure:** Inverted Pyramid (Foundational -> Strategic -> Tactical -> Reference).
 
-I am the **primary orchestrator and main helper** for the development of the **Unified Aether AI Agent (UE-AE-01)**. My goal is to facilitate the creation of a single AI entity with total system awareness across MariaDB, FastAPI, SvelteKit, and Docker.
+## 1. 💾 Long Term Memory (System & Facts)
+*This section contains the "Universal Truths" that rarely change. It grounds the agent in the user's reality.*
+
+### 🤖 Agent Identity & Role
+- **Agent Name:** Aether API Orchestrator (mcp_agent)
+- **Primary Role:** Backend Development, System Orchestration, and API Stabilization.
+- **Scope:** `/home/scott/OSIT_dev/aether_api_fastapi` and Aether Platform backend infrastructure.
+
+### 👤 User Profile
+- **User:** Scott Idem (`scott`)
+- **Organizations:**
+    - **One Sky IT (OSIT):** Professional/Business context.
+    - **Danger Zone (DgrZone):** Personal/Home context.
+    - **Aether Platform (AE):** Scott's (One Sky IT) platform developed for OSIT.
+- **Preferences:**
+    - **Editor:** `vim` (Terminal), VS Code (GUI).
+    - **Communication:** Direct, concise, professional CLI tone.
+    - **Safety:** "Recycle Bin" (`~/tmp/gemini_trash`) instead of `rm`. Explain destructive actions first.
+- **Hardware/OS:**
+    - **Host:** Linux (Ubuntu/Arch context)
+
+### 🏗️ Aether Architecture (V3)
+- **Concept:** Unified AI-driven platform for business/personal management.
+- **Backend:** FastAPI (v4.9.0) + Pydantic V1 + SQLAlchemy + MariaDB (Remote).
+- **V3 Implementation:** Modern parallel CRUD and Search endpoints under `/v3/crud`.
+- **Core Principle:** "Agent Bridge" - Distributed agents coordinating via file-based messaging (`~/agents_sync`).
+
+### 📜 Core Protocols
+- **RAR Protocol:** Request -> Ack -> Result.
+- **V3 CRUD Paradigm:** JSON metadata via `/v3/crud/`, binary actions (Upload/Download) via `/v3/action/`.
+- **Fail Fast & Transparently:** API returns `500` on hard errors; avoid silent failures (confirmed in `sql_select`).
+- **Bite-Sized Data:** Avoid monolithic files (>1MB).
+- **Source of Truth:** `~/agents_sync` is the shared brain. `~/OSIT_dev` is the local development environment.
+
+### 🛡️ Security & Secrets Guardrails
+- **Secrets:** NEVER read/display content from `.env` files unless explicitly debugging configuration logic.
+- **PII:** Scrub personally identifiable information if sharing logs or data across the bridge.
+- **Hiding Internal Paths:** `subdirectory_path` is hidden from public-facing API responses via Pydantic `Field(exclude=True)`.
+
+### 🧠 Key Technical Learnings (Cumulative)
+- **Circular Dependencies Fixed**: Successfully resolved the fragile startup dependency chain by isolating Auth models and using strictly deferred DB imports in a dedicated `dependencies_v3.py` module.
+- **Bootstrap Paradox Solved**: Implemented a guest-access exception for `site_domain` search, allowing the frontend to resolve site context without a JWT.
+- **V3 Searchable Fields**: `searchable_fields` must explicitly include integer ID fields (e.g., `event_id`) to ensure valid numeric filters are not blocked by the V3 search security layer.
+- **NULL Logic in Filters**: Confirmed that explicit frontend filters like `hide: false` will FAIL to match `NULL` database values. Rely on the API's built-in `hidden=not_hidden` parameter for robust handling.
+- **Vision ID Safety Net**: Enhanced `lookup_id_random_pop` to resolve random string IDs found in any `*_id` field, ensuring "Vision" style payloads are correctly converted to integers.
 
 ---
 
-## Project Context - Aether API (FastAPI)
+## 2. 🗓️ Near Term Memory (Strategic Context)
+*This section tracks active projects (1-2 weeks scope). It answers "Why are we doing this?"*
 
--   **Owner/Developer:** Scott Idem (user).
--   **System Name:** Aether (AE).
--   **Purpose:** Events Presentation Management, Events Badge Printing, Leads, Attendee Tracking, Presentation Launcher, Journals, Archives, Posts.
--   **Current API Version (FastAPI):** v4.9.0.
--   **V3 Implementation:** Modern parallel CRUD and Search endpoints under `/v3/crud`.
--   **Summary**: This project serves as the backend for the Aether system, providing core API functionalities.
+### 📩 In-Flight RAR Requests
+- [ ] **mcp_agent**: Real-world test of `ae_field_manage.py` (ID: 153357623).
+- [ ] **codebase_investigator**: Review report for Aether extension and journal management (ID: 155435511).
 
-### Key Technical Hack/Learnings (Cumulative)
--   **Circular Dependencies Fixed**: Successfully resolved the fragile startup dependency chain by isolating Auth models and using strictly deferred DB imports in a dedicated `dependencies_v3.py` module.
--   **Bootstrap Paradox Solved**: Implemented a guest-access exception for `site_domain` search, allowing the frontend to resolve site context without a JWT.
--   **V3 Searchable Fields**: Confirmed that `searchable_fields` are fully implemented in `app/object_definitions/` and utilized by `app/lib_sql_search.py` for dynamic query generation.
--   **ID Vision Mapping Stability**: Resolved a critical conflict in `app/lib_sql_search.py` where backend-injected integer filters (e.g., `account_id`) were being incorrectly mapped to `_random` string columns. Mapping now only occurs for non-integer values.
--   **API Error Transparency**: Modified `sql_select` to return `False` on hard database errors (instead of empty results). Updated the V3 search endpoint to return a `500 Internal Server Error` in these cases, preventing "silent failures" where frontend developers would see `200 OK` with zero results during system errors.
--   **NULL Logic in Filters**: Confirmed that explicit frontend filters like `hide: false` will FAIL to match `NULL` database values in standard SQL equality. It is more robust to rely on the API's built-in `hidden=not_hidden` URL parameter, which handles `(hide = false OR hide IS NULL)` automatically.
--   **V3 Action Router Paradigm**: Established a new pattern for specialized binary operations (Upload/Download) under `/v3/action/`, keeping the standard `/v3/crud/` routes clean for JSON metadata.
--   **Vision ID safety Net**: Enhanced `lookup_id_random_pop` to resolve random string IDs found in any `*_id` field, ensuring "Vision" style payloads (where IDs are strings) are correctly converted to integers before database insertion even if the `*_id_random` field is missing.
--   **Pydantic Response Exclusion**: Confirmed that `Field(exclude=True)` in Pydantic models is the most robust way to hide internal logistics fields (like `subdirectory_path`) from the client while keeping them available for backend logic.
--   **V3 Search Security Mapping**: Discovered and fixed a gap where integer IDs (like `event_id`) were not explicitly listed in `searchable_fields` for some objects, preventing valid numeric filters from reaching the database.
+### 🎯 Strategic Goals (Current Sprint)
+- **Primary:** OSIT_dev Environment Optimization & Context Stabilization (Template v1.2 Adoption).
+- **Secondary:** ID Vision Phase 2 Migration and V3 API Migration (Contacts/Clients).
 
-## Session Summary & Progress (Jan 26, 2026)
+### 🚧 Active Workstreams
+- **[Context Migration]:** Refactoring `GEMINI.md` to v1.2 Inverted Pyramid structure (Completed).
+- **[ID Vision]:** Extending string-ID standardization to Page, Post, and Person models (ID: 161311118).
+- **[Infrastructure]:** Pydantic V2 Migration Impact Analysis (Technical Debt).
+- **[Infrastructure]:** Restore AE Events Presentation Launcher (Electron) (ID: 221513945).
 
-This session focused on stabilizing V3 object definitions and enhancing API privacy for file-related objects.
+### 🧠 Recent Decisions
+- **API Error Transparency:** Modified `sql_select` to return `False` on hard database errors to prevent "silent failures" where frontend would see `200 OK` with zero results.
+- **Privacy & Information Hiding:** Centralized `public_read` flag in object definitions and excluded internal file sharding paths from responses.
 
-*   **V3 Search Stabilization:**
-    *   Updated `event_device` object definitions to include integer ID fields (`id`, `event_device_id`, `account_id`, `event_id`, `event_location_id`) in `searchable_fields`. This resolves issues where lookups by parent IDs were being blocked by the V3 search security layer.
-*   **Privacy & Information Hiding:**
-    *   Hiden internal file sharding paths (`subdirectory_path`) from all public-facing API responses.
-    *   Implemented this via Pydantic `Field(exclude=True)` in `Hosted_File_Base`, `Event_File_Base`, and `Archive_Content_Base` models.
-    *   Removed these fields from `exp_default` and `searchable_fields` metadata to ensure a clean API contract.
-*   **Code Quality:**
-    *   Fixed syntax errors (missing commas) in the `hosted_file` object definition within `app/object_definitions/other.py`.
-    *   Verified all changes via `py_compile` and service restarts.
-    *   Committed all changes to the `development` branch.
+---
 
-## Current To-Do List
+## 3. 🧠 Short Term Memory (Session Context)
+*This section is the "Scratchpad" for the current interaction. It is cleared or summarized often.*
 
-### 1. High Priority & Urgent (UE-AE-01 Orchestration)
-- [ ] **Real-world test of ae_field_manage.py.** (ID: 153357623) - Pending user verification of first field addition.
-- [ ] **Review codebase investigator report for Aether extension and journal management.** (ID: 155435511) - In Progress.
+- **Status:** Online
+- **Last Action:** Successfully refactored `GEMINI.md` to v1.2 structure.
+- **Current Blocker:** None.
+- **Immediate Next Step:** Check for new messages in the inbox or proceed with high-priority tasks.
 
-### 2. Infrastructure & Technical Debt
-- [ ] **Pydantic V2 Migration**: Begin impact analysis for the 400+ validators and .dict() calls.
-- [ ] **Psutil Container Update**: Rebuilding the API image with C-build tools (GCC). (Paused)
+---
 
-### 3. Other Agent Tasks
-- [ ] **Review and process new messages from various agents.** (ID: 134908245) - In Progress (Internal gemini_cli task).
+## 4. 📂 Reference: Directory & Whitelist
+*Low-density reference data. Keep at the bottom to avoid cluttering the prompt's "hot zone".*
 
+### 🛡️ File Whitelist
+- `~/tmp`
+- `~/OSIT_dev/aether_api_fastapi`
+- `~/agents_sync`
 
-### Workflow & Collaboration
-- **Storage**: Critical assets at `/home/scott/OSIT/hosted_files/` (Synced via Syncthing).
-- **Agents Sync**: Shared documentation and notifications pushed to `~/agents_sync/`.
-- **Coding Standards**: Adhering to `prompts/coding_standards.md`.
+### 🗺️ Standard Directory Map
+- **`app/methods/`**: Object-specific business logic.
+- **`app/models/`**: Pydantic schemas.
+- **`app/object_definitions/`**: V3 Metadata definitions.
+- **`app/routers/`**: API endpoints.
 
-
-## 📋 Aether API Development Protocol
-
-0. Pre-Flight Check: Verify git status. Ensure all previous working changes are committed to maintain a "known good" state before starting a new cycle.
-1. Strategic Plan: Write a concise plan identifying the issue, the specific files to be modified, and the verification steps (including specific curl commands or test scripts).
-2. Implementation: Perform atomic code modifications using the replace or write_file tools.
-3. Syntax Validation: Run python3 -m py_compile <modified_file> immediately. If syntax errors exist, they must be fixed before proceeding to deployment.
-4. Process Cycle: Restart the Docker FastAPI service to apply changes:
-    * "docker restart aether_container_env-ae_api-2"
-5. Empirical Testing:
-    * Execute the curl commands or test scripts identified in Step 1.
-    * Inspect the live log stream for regressions:
-
-    1 "tail -n 20 ~/OSIT_dev/aether_container_env/logs/ae_api/aether_api.log"
-6. Finalize: Once verified, commit the changes with a descriptive message and sync relevant documentation.
+### 📋 Aether API Development Protocol
+0. **Pre-Flight Check:** Verify `git status`. Ensure all previous working changes are committed.
+1. **Strategic Plan:** Write a concise plan identifying the issue, specific files, and verification steps (curl commands/test scripts).
+2. **Implementation:** Perform atomic code modifications using `replace` or `write_file`.
+3. **Syntax Validation:** Run `python3 -m py_compile <modified_file>` immediately.
+4. **Process Cycle:** Restart the Docker FastAPI service: `docker restart aether_container_env-ae_api-2`.
+5. **Empirical Testing:** Execute `curl` commands and inspect logs: `tail -n 20 ~/OSIT_dev/aether_container_env/logs/ae_api/aether_api.log`.
+6. **Finalize:** Commit changes with a descriptive message and sync documentation.