diff --git a/GEMINI.md b/GEMINI.md index 0fa215f..e850d86 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -22,30 +22,22 @@ I am the **primary orchestrator and main helper** for the development of the **U - **Environment Sync Clarification**: Critical understanding that the `/home/scott/OSIT_dev/` directory is **NOT** a synchronized directory across different machines (e.g., Workstation vs. Laptop). This means virtual environments and installed dependencies are environment-specific and do not propagate via Syncthing. This is crucial for debugging cross-machine environment issues. - **Circular Import Sensitivity**: Importing `JSONResponse` from `fastapi` in `app/routers/api.py` triggers a circular dependency crash. It is safer to rely on FastAPI's default dict-to-JSON serialization or use `mk_resp` without explicit `JSONResponse` typing in that specific file. -## Session Summary & Progress (Jan 13, 2026) +## Session Summary & Progress (Jan 15, 2026) -This session focused on resolving startup errors in the `ae_api` container and improving database connectivity testing. +This session focused on a comprehensive modernization of the API's core infrastructure to resolve boot-time circular dependencies and improve maintainability. -* **Bug Fixes & Stability:** - * **Fixed NameError:** Resolved missing `mk_resp` import in `app/main.py`. - * **Fixed AttributeError:** Corrected `response_class=PlainTextResponse` to `JSONResponse` (or default) for dict-returning endpoints. - * **Resolved Circular Dependency:** Identified and fixed a circular import caused by `JSONResponse` usage in `app/routers/api.py`. -* **Infrastructure & Testing:** - * **Safe SQL Test Endpoint:** Relocated and refactored the `sql_test` endpoint from `app/main.py` to `app/routers/api.py` (accessible at `/api/sql_test`). - * **Safety Improvement:** Updated the test query to use `SELECT NOW(), VERSION()` instead of querying the `account` table, ensuring a safe connectivity check without exposing data. -* **V3 CRUD Modularization (Phase 1-3):** - * Successfully refactored `app/routers/api_crud_v3.py` into a modular architecture. - * **Logic Extraction:** Created `app/lib_api_crud_v3.py` for shared security and filtering logic. - * **Nested Routes:** Created `app/routers/api_crud_v3_nested.py` to isolate relational parent-child endpoints. - * **Schema Extraction:** Created `app/lib_schema_v3.py` for database/model introspection. -* **DX & Error Transparency:** - * **Error Bubbling:** Updated `app/db_sql.py` to capture SQL exceptions and implemented `format_db_error` to provide human-readable details in API responses. - * **Validation Endpoint:** Added `POST /v3/crud/{obj_type}/validate` for dry-run payload verification. - * **Automatic Sanitization:** Backend now automatically strips virtual lookup fields (`*_id_random`) and view-only fields (via `fields_to_exclude_from_db`) from write payloads. -* **Object Model Alignment:** - * **Address & Contact:** Updated models and JSON mappings to include `hide`, `priority`, `sort`, `group`, and `notes`. Fixed missing imports and unsafe validator access. - * **Journal:** Added `fields_to_exclude_from_db` to filter out view-only fields (e.g., `person_full_name`) during updates. - * **Post:** Added `external_person_id` to searchable fields. +* **FastAPI Entry Point Modularization:** + * **Registry Pattern:** Created `app/routers/registry.py` to centralize 50+ router registrations, slimming down `main.py`. + * **Lifespan Context:** Integrated the `lifespan` context manager in `app/main.py` to manage startup/shutdown tasks cleanly. + * **Naming Collision Fix:** Resolved a critical conflict between the `app` instance and the `app.middleware` package by using explicit aliases. +* **Database Logic Decoupling:** + * **Layered Architecture:** Split `app/db_sql.py` into `lib_sql_core` (engine/connection) and `lib_sql_crud` (high-level helpers). + * **Dynamic Engine Refresh:** Implemented `reconnect_db()` to allow the SQLAlchemy engine to update credentials after the initial boot. +* **Robust Configuration & Logging:** + * **Logging Sandbox:** Moved `dictConfig` to `app/lib_log_v3.py` and deferred its execution to avoid boot-time traps. + * **Safe Bootstrap:** Created `app/lib_config_v3.py` with a non-fatal fallback mechanism for DB configuration sync. +* **Architecture Documentation:** + * Created `documentation/V3_CORE_ARCHITECTURE.md` to document the new file structure and bootstrap sequence. ## Current To-Do List @@ -56,12 +48,14 @@ This session focused on resolving startup errors in the `ae_api` container and i ### 2. Infrastructure & Technical Debt - [ ] **Psutil Container Update**: Rebuilding the API image with C-build tools (GCC) to support native system monitoring. (Deferred / Paused) - [ ] **Fix ComfyUI Container**: Resolve the restart loop in the Ollama/WebUI environment. +- [ ] **Pydantic V2 Migration**: Begin impact analysis for the 400+ validators and .dict() calls. ### 3. Other Agent Tasks - [ ] **Review and process new messages from various agents.** (ID: 134908245) - In Progress (Internal gemini_cli task). - [x] **[scott_wks] Perform Initial System Health Audit** (ID: 131939030) - Done. - [x] **[API-V3] Implement Error Bubbling and Validation Endpoint** (ID: 163752448) - Done. -- [x] **[API] Fix Startup Errors and Refactor SQL Test Endpoint** - Done. +- [x] **[API] Modularize main.py and extract Core Registries** - Done Jan 15. +- [x] **[API] Stabilize Logging and Configuration Bootstrap** - Done Jan 15. ### Workflow & Collaboration diff --git a/documentation/V3_CORE_ARCHITECTURE.md b/documentation/V3_CORE_ARCHITECTURE.md new file mode 100644 index 0000000..020e313 --- /dev/null +++ b/documentation/V3_CORE_ARCHITECTURE.md @@ -0,0 +1,43 @@ +# Aether API V3 Core Architecture (Refactor 2026-01-15) + +This document describes the modular architecture of the Aether API backend, finalized in January 2026. The goal of this refactor was to resolve boot-time circular dependencies, eliminate naming collisions, and isolate global state from business logic. + +## 1. Application Entry Point (`app/main.py`) + +The `app/main.py` file is now a lightweight orchestrator. Its primary roles are: +- Defining the FastAPI application instance. +- Managing the **lifespan** context manager for startup/shutdown tasks. +- Calling the centralized registries for routers and middleware. + +### The Lifespan Sequence: +1. **Logging Setup**: `setup_logging(settings)` is called first but safely. +2. **Configuration Bootstrap**: `bootstrap_db_config(settings)` queries the `cfg` table. +3. **Database Refresh**: `reconnect_db()` refreshes the SQLAlchemy engine with production credentials. +4. **Validation**: `validate_critical_config(settings)` ensures the environment is production-ready. + +## 2. Centralized Registries + +To keep `main.py` clean, registrations are moved to dedicated modules: +- **`app/routers/registry.py`**: Exports `setup_routers(app)`. Handles the import and inclusion of 50+ application routers. +- **`app/middleware.py`**: A dedicated home for utility middlewares (e.g., `add_process_time_header`). + +## 3. Modular Database Logic + +The monolithic `db_sql.py` has been split into three distinct layers: +- **`app/lib_sql_core.py` (Foundational)**: Manages the global SQLAlchemy `engine`, the `db` connection, and the thread-local error state. It is the "Source of Truth" for the connection. +- **`app/lib_sql_crud.py` (Logic)**: Contains high-level CRUD helpers (`sql_insert`, `sql_select`, etc.) that import the connection from the core. +- **`app/db_sql.py` (Facade)**: Maintains backward compatibility by importing and re-exporting all functions. Existing code should still import from `app.db_sql`. + +## 4. Modernized Logging (`app/lib_log_v3.py`) + +Logging configuration is no longer executed at the module level. +- **`setup_logging(settings)`**: Must be explicitly called during application startup. +- **`logger_reset`**: A decorator used throughout the app to ensure consistent log state. +- **`app/log.py`**: Acts as a facade for backward compatibility. + +## 5. Configuration Sync (`app/lib_config_v3.py`) + +Handles the multi-layered configuration strategy: +1. **Initial**: Values are loaded from Docker `.env` via `app/config.py`. +2. **Bootstrap**: `bootstrap_db_config` pulls production values from the MariaDB `cfg` table. +3. **Safety**: Implements a non-fatal fallback; if the DB sync fails, the app continues using `.env` settings.