OSIT-AE-API-FastAPI/GEMINI.md

Gemini Added Memories

My Role and Operating Principles

I am an interactive CLI agent assisting with software engineering tasks for One Sky IT, LLC, primarily on the Aether API project. My core mandates include:

• Adhering to project conventions and existing code style.
• Never assuming library/framework availability; always verifying project usage.
• Implementing changes idiomatically and with minimal, high-value comments.
• Being proactive, including adding tests for new features/fixes.
• Confirming ambiguity or actions beyond clear scope with the user.
• Prioritizing user control and project conventions.
• Strictly adhering to instructions and utilizing available tools effectively.
• Awaiting explicit user instructions for significant architectural changes or critical decisions.

Project Context - Aether API (FastAPI)

• Owner/Developer: Scott Idem (user).
• System Name: Aether (AE).
• Purpose: Events Presentation Management, Events Badge Printing, Leads, Attendee Tracking, Presentation Launcher, Journals, Archives, Posts.
• Started: Mid-2018.
• Frontend History: Python Flask -> Svelte (current: SvelteKit).
• Current API Version (FastAPI): v4.9.0.
• V3 Implementation: Modern parallel CRUD and Search endpoints under /v3/crud.

API Versioning & Strategy

• /crud (v1): Legacy, still used by older frontend parts.
• /v2/crud (v2.5): Modern, preferred, and mostly functional endpoint.
• /v3/crud: The goal of this project phase. A new, parallel implementation with a refined structure and advanced search. Runs alongside v1 and v2.

Technical Learnings

• Startup Errors & Logging: The "worker failed to boot" error is often an import-time error or a logging configuration failure.
  • Root Cause: If logging.config.dictConfig fails (e.g., due to missing /logs directories in Docker), the entire application crashes.
  • Prevention: Always wrap logging config in try/except and use import logging.config explicitly.
  • Circular Dependencies: These are frequently masked as logging errors because app.log is imported very early in most files. Breaking these loops by moving imports inside functions (deferred imports) is a primary fix.
• Circular Dependencies during Refactoring: Attempting to move base CRUD logic and engine initialization into separate modules can trigger "Worker failed to boot" if not done carefully.
  • Issue: Moving db and engine to a separate file like db_connection.py often creates circular loops with db_sql.py or log.py because they are imported by almost every other file at the module level.
  • Resolution: A "Facade Pattern" was used for db_sql.py, where helper functions (Search builders, Redis lookups) are moved to lib_sql_search.py and lib_redis_helpers.py, but the core connection and CRUD stay in the original file to maintain boot order stability.
• V3 API Dependencies: Standardized Response injection should use plain type hints (e.g., response: Response) to avoid router initialization failures.
• Pydantic Compatibility: The current environment uses Pydantic v1.10. Avoid v2 features like computed_field or model_validator to prevent startup crashes.

V3 Architectural Progress (Jan 2026)

• Modular Object Definitions: Monolithic ae_obj_types_def.py refactored into domain-specific files in app/object_definitions/.
• Granular Dependencies: Monolithic Common_Route_Params replaced with specialized dependencies in app/lib_general_v3.py (AccountContext, Pagination, StatusFilter, Serialization, Delay).
• Advanced Search (POST): Implemented POST /v3/crud/{obj}/search supporting recursive AND/OR grouping and standardized full-text search via the q property.
• Security Hardening: Implemented a 5-level recursion depth limit and a field allowlist (searchable_fields) for the Search API.
• Non-blocking Concurrency: Standardized on asyncio.sleep() for delay simulation to prevent Gunicorn worker hangs.

Session Learnings & Progress (Jan 2-7, 2026)

V3 API Security Hardening (Jan 7, 2026) - MILESTONE

• Mandatory JWT Authentication: Successfully implemented strict multi-tenant isolation across all V3 CRUD and Search endpoints.
  • All requests (except context resolution) now require a valid JWT Authorization: Bearer <token> or ?jwt=<token>.
  • Account Isolation: results are automatically filtered by account_id from the JWT.
  • Documentation: Updated V3_FRONTEND_API_GUIDE.md with explicit instructions and security requirements for the frontend agent.

Agent Bridge & Docker Integration

• Agent Bridge Implementation: Developed app/routers/agent_bridge.py for environment diagnostics.
• MCP Docker Explorer: Attempted to run mcp_docker_explorer.py, but failed with ModuleNotFoundError: No module named 'mcp'.
  • Lesson: The system python (/usr/bin/python3) does not have the mcp package installed. We must use the specific virtual environment env_mcp (e.g., ./env_mcp/bin/python) or ensure the package is installed in the active environment.

V3 CRUD Infrastructure & Search

• Modular Object Definitions: Refactored ae_obj_types_def.py into modular domain files in app/object_definitions/.
• Advanced Search Fixes:
  • Resolved account listing and search issues by implementing get_supported_filters in api_crud_v3.py.
  • Improved standardized full-text search (q parameter) with fallback logic for missing columns.
• Data Integrity & Aliasing: Fixed aliased field population by enabling allow_population_by_field_name in Pydantic models.

Startup Failure Resolution (Jan 7, 2026)

• Root Cause Identified: The app/routers/agent_bridge.py module was preventing the FastAPI worker from booting, likely due to a missing or incompatible dependency (suspected psutil in the Docker environment) or a top-level import issue.
• Resolution: Commented out the agent_bridge router inclusion in app/main.py.
• Status: The API server has successfully started.
• Retrospective: The previous circular dependency refactoring in lib_general_v3 and api_crud_v3 might have been unnecessary or at least wasn't the primary blocker, though deferring imports is good practice.

Current To-Do List

1. Frontend Integration (Priority: Urgent): Re-implement the site_domain lookup exception.
   • Constraint: Must allow searching site_domain without an account_id or JWT.
   • Approach: Re-apply the optional authentication dependency logic to api_crud_v3.py and lib_general_v3.py, now that the server is stable.
2. Docker MCP Integration (Priority: High): Re-attempt running the MCP explorer using the correct virtual environment path (./env_mcp/bin/python) once the API is stable.
3. Routing - Nginx (Priority: Medium): Resolve 404 errors on /v3/ and /agent/ routes.
4. Specialized Endpoints (Priority: Medium): Plan modernization of custom logic.
5. Agent Bridge Repair (Priority: Low): Investigate why agent_bridge.py crashes the server (check psutil availability).

Workflow & Collaboration

• GEMINI.md Strategy: The user is creating GEMINI.md files in key project directories. Their understanding is that context flows from the current directory up the tree, with ~/.gemini/GEMINI.md serving as a global catch-all for general memories.
• Agents Sync (rsync): Shared documentation, notes, and architectural updates are pushed to the agents_sync directory using rsync. This allows real-time coordination between different specialized agents (e.g., FastAPI backend and Svelte frontend agents).
• Home Server: The user self-hosts a Proxmox server for services like Nextcloud.