OSIT-AE-API-FastAPI/GEMINI.md

Gemini Added Memories

My Role and Operating Principles

I am an interactive CLI agent assisting with software engineering tasks for One Sky IT, LLC, primarily on the Aether API project. My core mandates include:

• Adhering to project conventions and existing code style.
• Never assuming library/framework availability; always verifying project usage.
• Implementing changes idiomatically and with minimal, high-value comments.
• Being proactive, including adding tests for new features/fixes.
• Confirming ambiguity or actions beyond clear scope with the user.
• Prioritizing user control and project conventions.
• Strictly adhering to instructions and utilizing available tools effectively.
• Awaiting explicit user instructions for significant architectural changes or critical decisions.

Project Context - Aether API (FastAPI)

• Owner/Developer: Scott Idem (user).
• System Name: Aether (AE).
• Purpose: Events Presentation Management, Events Badge Printing, Leads, Attendee Tracking, Presentation Launcher, Journals, Archives, Posts.
• Started: Mid-2018.
• Frontend History: Python Flask -> Svelte (current: SvelteKit).
• Current API Version (FastAPI): v4.9.0.
• V3 Implementation: Modern parallel CRUD and Search endpoints under /v3/crud.

API Versioning & Strategy

• /crud (v1): Legacy, still used by older frontend parts.
• /v2/crud (v2.5): Modern, preferred, and mostly functional endpoint.
• /v3/crud: The goal of this project phase. A new, parallel implementation with a refined structure and advanced search. Runs alongside v1 and v2.

Technical Learnings

• Startup Errors & Logging: The "worker failed to boot" error is often an import-time error or a logging configuration failure.
  • Root Cause: If logging.config.dictConfig fails (e.g., due to missing /logs directories in Docker), the entire application crashes.
  • Prevention: Always wrap logging config in try/except and use import logging.config explicitly.
  • Circular Dependencies: These are frequently masked as logging errors because app.log is imported very early in most files. Breaking these loops by moving imports inside functions (deferred imports) is a primary fix.
• Circular Dependencies during Refactoring: Attempting to move base CRUD logic and engine initialization into separate modules can trigger "Worker failed to boot" if not done carefully.
  • Issue: Moving db and engine to a separate file like db_connection.py often creates circular loops with db_sql.py or log.py because they are imported by almost every other file at the module level.
  • Resolution: A "Facade Pattern" was used for db_sql.py, where helper functions (Search builders, Redis lookups) are moved to lib_sql_search.py and lib_redis_helpers.py, but the core connection and CRUD stay in the original file to maintain boot order stability.
• V3 API Dependencies: Standardized Response injection should use plain type hints (e.g., response: Response) to avoid router initialization failures.
• Pydantic Compatibility: The current environment uses Pydantic v1.10. Avoid v2 features like computed_field or model_validator to prevent startup crashes.

V3 Architectural Progress (Jan 2026)

• Modular Object Definitions: Monolithic ae_obj_types_def.py refactored into domain-specific files in app/object_definitions/ (core, events, journals, orders, cms, lookups, membership, other).
• Granular Dependencies: Monolithic Common_Route_Params replaced with specialized dependencies in app/lib_general_v3.py (AccountContext, Pagination, StatusFilter, Serialization, Delay).
• Advanced Search (POST): Implemented POST /v3/crud/{obj}/search supporting recursive AND/OR grouping and standardized full-text search via the q property.
• Security Hardening: Implemented a 5-level recursion depth limit and a field allowlist (searchable_fields) for the Search API.
• Non-blocking Concurrency: Standardized on asyncio.sleep() for delay simulation to prevent Gunicorn worker hangs.

Session Learnings & Progress (Jan 2-7, 2026)

Agent Bridge & Docker Integration (Jan 7, 2026)

• Agent Bridge Implementation: Developed app/routers/agent_bridge.py to provide administrative insight into the Docker runtime environment.
  • Endpoints include /status, /system/usage, /logs, /logs/list, /processes, and /container/metadata.
  • Implemented is_admin helper for consistent security across administrative tools.
  • Added support for listing and tailing various log files, with path sanitization.
• MCP Docker Explorer: Created mcp_docker_explorer.py as a standalone client to test the Model Context Protocol (MCP) Docker server integration. This script uses npx to run @modelcontextprotocol/server-docker.
• Authentication Exceptions:
  • Acknowledged that x_no_account_id is used intentionally in some contexts, though a more robust solution will be needed later.
  • Direct Download Links: Noted that Hosted Files and Event Files often require direct download links that cannot include custom headers, as they are often shared directly with users.

V3 CRUD Infrastructure & Search

• Modular Object Definitions: Refactored ae_obj_types_def.py into modular domain files in app/object_definitions/.
• Advanced Search Fixes:
  • Resolved account listing and search issues by implementing get_supported_filters in api_crud_v3.py. This helper automatically adjusts status_filter (enabled/hidden) based on model field support (e.g., handles tables missing the hide column).
  • Improved standardized full-text search (q parameter) in sql_search_qry_part with a dry-run check for the default_qry_str column. It now falls back to a LIKE search across all searchable fields if the column is missing.
  • Added support for the % wildcard in q, allowing it to bypass filtering and return all records.
• Data Integrity & Aliasing: Fixed a critical issue where aliased fields (like account_id_random) were returned as null by adding allow_population_by_field_name = True to Pydantic models (updated Account_Base, Site_Domain_Base, etc.).
• Consolidated V3 Router: Systematically cleaned up api_crud_v3.py, removing duplicate endpoint definitions and standardizing logic across all CRUD and Search routes.
• Robust Error Handling: Updated V3 routers to return 500 status codes on database failures instead of masking errors with empty result sets.

Database Schema Insights

• Verified schema for core tables (account, event, person, user, data_store, site, site_domain).
• Noted that site_domain lacks a hide column, and hosted_file_link lacks both hide and enable.
• Most major tables now have a standardized hide column (added to account on Jan 6, 2026).

Current To-Do List

1. Docker MCP Integration (Priority: High): Proceed with integrating the Docker MCP server into the Gemini CLI environment for direct container management.
2. Security - Field Allowlists (Priority: Done): Finished populating searchable_fields for all object definitions (Core, CMS, Events, Membership, Orders, Other).
3. Refactoring - Modularize db_sql.py (Priority: Done/Low): Successfully implemented a facade pattern, moving search builders and Redis helpers to modular files. This reduced db_sql.py by nearly 500 lines while preserving stability. Further modularization of core CRUD should only be attempted if stability risks are mitigated.
4. Specialized Endpoints (Priority: Medium): Plan modernization of custom logic (importing, websockets) to match V3 patterns.
5. Security - Authentication (Priority: High): Continue refining and enforcing JWT-based authentication across all V3 endpoints, while respecting the need for header-less direct download links for certain file types.
6. Account ID Handling (Priority: Low): Address the x_no_account_id usage with a more permanent and secure architecture.

Workflow & Collaboration

• GEMINI.md Strategy: The user is creating GEMINI.md files in key project directories. Their understanding is that context flows from the current directory up the tree, with ~/.gemini/GEMINI.md serving as a global catch-all for general memories.
• Agents Sync (rsync): Shared documentation, notes, and architectural updates are pushed to the agents_sync directory using rsync. This allows real-time coordination between different specialized agents (e.g., FastAPI backend and Svelte frontend agents).
• Home Server: The user self-hosts a Proxmox server for services like Nextcloud.