OSIT-AE-API-FastAPI/GEMINI.md

# Gemini Added Memories

## My Role and Operating Principles

I am an interactive CLI agent assisting with software engineering tasks for One Sky IT, LLC, primarily on the Aether API project. My core mandates include:
-   Adhering to project conventions and existing code style.
-   **Never assuming library/framework availability; always verifying project usage.**
-   Implementing changes idiomatically and with minimal, high-value comments.
-   Being proactive, including adding tests for new features/fixes.
-   **Confirming ambiguity or actions beyond clear scope with the user.**
-   Prioritizing user control and project conventions.
-   **Strictly adhering to instructions and utilizing available tools effectively.**
-   **Awaiting explicit user instructions for significant architectural changes or critical decisions.**

## Project Context - Aether API (FastAPI)

-   **Owner/Developer:** Scott Idem (user).
-   **System Name:** Aether (AE).
-   **Purpose:** Events Presentation Management, Events Badge Printing, Leads, Attendee Tracking, Presentation Launcher, Journals, Archives, Posts.
-   **Started:** Mid-2018.
-   **Frontend History:** Python Flask -> Svelte (current: SvelteKit).
-   **Current API Version (FastAPI):** v4.9.0.
-   **V3 Implementation:** Modern parallel CRUD and Search endpoints under `/v3/crud`.

### API Versioning & Strategy

-   `/crud` (v1): Legacy, still used by older frontend parts.
-   `/v2/crud` (v2.5): Modern, preferred, and mostly functional endpoint.
-   `/v3/crud`: The goal of this project phase. A new, parallel implementation with a refined structure and advanced search. **Runs alongside v1 and v2.**

### Technical Learnings
- **Startup Errors & Logging:** The "worker failed to boot" error is often an import-time error or a logging configuration failure.
    - **Root Cause:** If `logging.config.dictConfig` fails (e.g., due to missing `/logs` directories in Docker), the entire application crashes.
    - **Prevention:** Always wrap logging config in `try/except` and use `import logging.config` explicitly.
    - **Circular Dependencies:** These are frequently masked as logging errors because `app.log` is imported very early in most files. Breaking these loops by moving imports inside functions (deferred imports) is a primary fix.
- **Circular Dependencies during Refactoring:** Attempting to move base CRUD logic and engine initialization into separate modules can trigger "Worker failed to boot" if not done carefully.
    - **Issue:** Moving `db` and `engine` to a separate file like `db_connection.py` often creates circular loops with `db_sql.py` or `log.py` because they are imported by almost every other file at the module level.
    - **Resolution:** A "Facade Pattern" was used for `db_sql.py`, where helper functions (Search builders, Redis lookups) are moved to `lib_sql_search.py` and `lib_redis_helpers.py`, but the core connection and CRUD stay in the original file to maintain boot order stability.
- **V3 API Dependencies:** Standardized `Response` injection should use plain type hints (e.g., `response: Response`) to avoid router initialization failures.
- **Pydantic Compatibility:** The current environment uses Pydantic v1.10. Avoid v2 features like `computed_field` or `model_validator` to prevent startup crashes.

### V3 Architectural Progress (Jan 2026)

-   **Modular Object Definitions:** Monolithic `ae_obj_types_def.py` refactored into domain-specific files in `app/object_definitions/`.
-   **Granular Dependencies:** Monolithic `Common_Route_Params` replaced with specialized dependencies in `app/lib_general_v3.py` (AccountContext, Pagination, StatusFilter, Serialization, Delay).
-   **Advanced Search (POST):** Implemented `POST /v3/crud/{obj}/search` supporting recursive AND/OR grouping and standardized full-text search via the `q` property.
-   **Security Hardening:** Implemented a 5-level recursion depth limit and a field allowlist (`searchable_fields`) for the Search API.
-   **Non-blocking Concurrency:** Standardized on `asyncio.sleep()` for delay simulation to prevent Gunicorn worker hangs.

## Session Learnings & Progress (Jan 2-7, 2026)

### V3 API Security Hardening (Jan 7, 2026) - MILESTONE
- **Mandatory JWT Authentication**: Successfully implemented strict multi-tenant isolation across all V3 CRUD and Search endpoints.
    - All requests (except context resolution) now require a valid JWT `Authorization: Bearer <token>` or `?jwt=<token>`.
    - **Account Isolation**: results are automatically filtered by `account_id` from the JWT.
    - **Documentation**: Updated `V3_FRONTEND_API_GUIDE.md` with explicit instructions and security requirements for the frontend agent.

### Agent Bridge & Docker Integration
- **Agent Bridge Implementation**: Developed `app/routers/agent_bridge.py` for environment diagnostics.
- **MCP Docker Explorer**: Attempted to run `mcp_docker_explorer.py`, but failed with `ModuleNotFoundError: No module named 'mcp'`.
    - **Lesson**: The system python (`/usr/bin/python3`) does not have the `mcp` package installed. We must use the specific virtual environment `env_mcp` (e.g., `./env_mcp/bin/python`) or ensure the package is installed in the active environment.

### V3 CRUD Infrastructure & Search
- **Modular Object Definitions**: Refactored `ae_obj_types_def.py` into modular domain files in `app/object_definitions/`.
- **Advanced Search Fixes**: 
    - Resolved account listing and search issues by implementing `get_supported_filters` in `api_crud_v3.py`.
    - Improved standardized full-text search (`q` parameter) with fallback logic for missing columns.
- **Data Integrity & Aliasing**: Fixed aliased field population by enabling `allow_population_by_field_name` in Pydantic models.

### Startup Failure Resolution (Jan 7, 2026)
- **Root Cause Identified**: The `app/routers/agent_bridge.py` module was preventing the FastAPI worker from booting, likely due to a missing or incompatible dependency (suspected `psutil` in the Docker environment) or a top-level import issue.
- **Resolution**: Commented out the `agent_bridge` router inclusion in `app/main.py`.
- **Status**: The API server has successfully started.
- **Retrospective**: The previous circular dependency refactoring in `lib_general_v3` and `api_crud_v3` might have been unnecessary or at least wasn't the *primary* blocker, though deferring imports is good practice.

## Current To-Do List

1.  **Frontend Integration (Priority: Urgent)**: Re-implement the `site_domain` lookup exception.
    -   *Constraint*: Must allow searching `site_domain` without an `account_id` or JWT.
    -   *Approach*: Re-apply the `optional` authentication dependency logic to `api_crud_v3.py` and `lib_general_v3.py`, now that the server is stable.
2.  **Docker MCP Integration (Priority: High)**: Re-attempt running the MCP explorer using the correct virtual environment path (`./env_mcp/bin/python`) once the API is stable.
3.  **Routing - Nginx (Priority: Medium)**: Resolve 404 errors on `/v3/` and `/agent/` routes.
4.  **Specialized Endpoints (Priority: Medium)**: Plan modernization of custom logic.
5.  **Agent Bridge Repair (Priority: Low)**: Investigate why `agent_bridge.py` crashes the server (check `psutil` availability).

### Workflow & Collaboration
- **`GEMINI.md` Strategy:** The user is creating `GEMINI.md` files in key project directories. Their understanding is that context flows from the current directory up the tree, with `~/.gemini/GEMINI.md` serving as a global catch-all for general memories.
- **Agents Sync (rsync):** Shared documentation, notes, and architectural updates are pushed to the `agents_sync` directory using `rsync`. This allows real-time coordination between different specialized agents (e.g., FastAPI backend and Svelte frontend agents).
- **Home Server:** The user self-hosts a Proxmox server for services like Nextcloud.