Scott.Idem/OSIT-AE-API-FastAPI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: Consolidate V3 standards and cleanup documentation directory

Browse Source
This commit is contained in:
Scott Idem
2026-01-15 17:53:06 -05:00
parent 28d5843d52
commit 68862e4545
7 changed files with 47 additions and 372 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
documentation/archive/API_SIMPLIFICATION_PLAN.md Normal file
View File

@@ -0,0 +1,53 @@
# Aether API V3 Implementation Plan
This document outlines the plan to build the Aether API V3. The goal of V3 is to create a new, modern, and consistent set of API endpoints that will run in parallel with the existing legacy endpoints. No existing endpoints will be modified or deleted.
## V3 Architecture
The V3 API will be centered around a new generic CRUD router that implements a hierarchical, nested URL structure for parent-child object relationships.
- **Endpoint Prefix:** All new endpoints will be under `/v3/crud/`.
- **Top-Level Objects:** Handled via a flat URL structure.
- `GET /v3/crud/journal/`
- `POST /v3/crud/journal/`
- `GET /v3/crud/journal/{journal_id}`
- **Child Objects:** Handled via a nested URL structure that enforces the parent-child relationship.
- `GET /v3/crud/journal/{journal_id}/journal_entry/`
- `POST /v3/crud/journal/{journal_id}/journal_entry/`
- `GET /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
This structure will be implemented in a new `app/routers/api_crud_v3.py` file, which will be inspired by the logic in the existing `api_crud_v2.py` but with updated routing to handle the nesting.
## Implementation Phases
### Phase 1: Create Stub V3 Endpoint (Proof-of-Concept)
The first step is to create a minimal, non-functional "stub" endpoint to prove that the basic routing and application integration works.
1. **Create File:** Create a new file: `app/routers/api_crud_v3.py`.
2. **Add Stub Endpoint:** Add a simple health-check or "hello world" endpoint within this file, for example, a `GET` endpoint at `/v3/crud/health`.
3. **Update Main App:** In `app/main.py`, import and include the new V3 router.
### Phase 2: Implement Top-Level Object CRUD
Using `journal` as the first test case, implement the full set of CRUD operations for a top-level object.
- `GET /v3/crud/journal/`
- `POST /v3/crud/journal/`
- `GET /v3/crud/journal/{journal_id}`
- `PATCH /v3/crud/journal/{journal_id}`
- `DELETE /v3/crud/journal/{journal_id}`
### Phase 3: Implement Nested Object CRUD
Using `journal_entry` as the first test case, implement the full set of CRUD operations for a child object, ensuring the `journal_id` from the URL is used for filtering.
- `GET /v3/crud/journal/{journal_id}/journal_entry/`
- `POST /v3/crud/journal/{journal_id}/journal_entry/`
- `GET /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
- `PATCH /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
- `DELETE /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
### Phase 4: Incremental Rollout
Once the patterns for top-level and nested objects are proven with Journal and Journal Entry, we will apply the same pattern to other Aether objects in batches.

31
documentation/archive/Aether_API_CRUD_V3_beta_recommendations.md Normal file
View File

@@ -0,0 +1,31 @@
# Aether API CRUD V3 Beta Recommendations
Following the initial migration of the Journals module to the V3 CRUD endpoints, the following architectural recommendations are proposed for the FastAPI backend to improve developer experience and frontend efficiency.
---
## 1. "View" Selection (`use_alt_tbl` Replacement)
In V2, the `use_alt_tbl` flag was frequently used to fetch "rich" objects (e.g., including joined data like `person_name`).
- **Recommendation:** Implement a `response_view` (or `view`) query parameter for both GET list and POST search endpoints.
- **Example:** `GET /v3/crud/journal/?view=enriched`
- **Goal:** Allow the client to request a heavier model with joined fields only when necessary, keeping the default response lightweight.
## 2. Hybrid Filtering for Search
Currently, the `/search` endpoint requires a full JSON body even for simple standard filters like `enabled` or `hidden`.
- **Recommendation:** Update the `POST .../search` endpoint to accept standard query parameters that automatically append `AND` conditions to the logic defined in the body.
- **Example:** `POST /v3/crud/journal/search?enabled=true`
- **Goal:** Simplifies frontend code for common toggles while still allowing complex logic in the POST body.
## 3. Standardized Full-Text Search Field
The current migration uses a field named `default_qry_str` with a `match` operator for text search.
- **Recommendation:** Implement a reserved top-level property in the Search Pydantic model (e.g., `_search_` or `query_string`) specifically for global text search.
- **Goal:** Decouples the frontend from knowing specific database column names used for full-text indexing. The backend can then decide which columns (name, description, tags, etc.) to include in the search.
## 4. Explicit "Parent" Filtering in Search
Filtering by parent (e.g., Account or Site) is a primary use case but currently requires manual injection into the `and` array.
- **Recommendation:** Expose `for_obj_type` and `for_obj_id` as top-level arguments in the Search API.
- **Goal:** Standardizes how parent context is passed, allowing the backend to more easily perform ownership and permission validation before executing the search.
---
**Date:** 2026-01-02
**Status:** Beta Feedback

41
documentation/archive/REFACTOR_API_CRUD_V3.md Normal file
View File

@@ -0,0 +1,41 @@
# Refactoring Plan: API CRUD V3
**Goal:** Modularize `app/routers/api_crud_v3.py` to improve maintainability, readability, and reusability. The file currently mixes route definitions, security enforcement, data sanitization, and helper utilities.
## Phase 1: Extract Helpers & Core Logic (Safest) - COMPLETED
**Objective:** Move pure functions and business logic out of the router file.
1. **Create `app/lib_api_crud_v3.py`**: DONE
2. **Update `app/routers/api_crud_v3.py`**: DONE (All endpoints now use `sanitize_payload`).
## Phase 2: Separate Child/Nested Routes - COMPLETED
**Objective:** Reduce file size by splitting standard CRUD from relational CRUD.
1. **Create `app/routers/api_crud_v3_nested.py`**: DONE
2. **Update `app/routers/api_crud_v3.py`**: DONE (Included via `router.include_router`)
## Phase 3: Schema Introspection - COMPLETED
**Objective:** Isolate database introspection logic.
1. **Create `app/lib_schema_v3.py`**: DONE
2. **Update `app/routers/api_crud_v3.py`**: DONE
## Refactoring Summary
The V3 CRUD system is now modularized into:
- `app/routers/api_crud_v3.py`: Top-level object routes.
- `app/routers/api_crud_v3_nested.py`: Relational/child object routes.
- `app/lib_api_crud_v3.py`: Shared security, filtering, and sanitization logic.
- `app/lib_schema_v3.py`: Database and model introspection logic.
## New DX Features
### 1. Error Bubbling
Database and Validation errors are no longer swallowed.
- **Validation Errors**: Specific Pydantic validation failures are returned in `meta.details`.
- **Database Errors**: MariaDB constraint violations (e.g., Duplicate Entry, Not Null) are captured and cleaned up via `format_db_error()` and returned in `meta.details`.
### 2. Validation Endpoint
**Route**: `POST /v3/crud/{obj_type}/validate`
- **Purpose**: Dry-run validation of a payload against the Pydantic model.
- **Behavior**: Returns `data: true` if valid, or a 400 error with specific details if invalid. No database operations are performed.

59
documentation/archive/REWRITE_PLAN.md Normal file
View File

@@ -0,0 +1,59 @@
# Aether API Rewrite Plan
## 1. Goal
The primary goal of this rewrite is to simplify the Aether API, remove legacy code, and adopt modern FastAPI best practices. This will make the project easier to maintain, extend, and understand.
## 2. Analysis of the Current State
The current implementation has several key characteristics that contribute to its complexity:
* **"God" Router:** A single file, `app/routers/api_crud_v2.py`, handles all CRUD operations (GET, POST, PATCH, DELETE) for all object types.
* **Dynamic Object Types:** The API uses URL path segments (`obj_type_l1`, `obj_type_l2`, `obj_type_l3`) to dynamically determine the object type being operated on.
* **Configuration via Dictionary:** The core of the dynamic system is the `obj_type_kv_li` dictionary in `app/ae_obj_types_def.py`. This dictionary maps object names to database tables, Pydantic models, and other configuration. This dictionary is inconsistent and contains legacy code.
* **Complex Parameter Handling:** Endpoints use a large number of query parameters and headers to control their behavior.
* **Mixed Logic:** The code mixes URL parsing, database lookups, data validation, and database operations.
## 3. Proposed New Structure
To address these challenges, I propose the following new structure:
* **Eliminate the "God" Router:** Instead of a single `api_crud_v2.py` file, create a separate router file for each major object type (e.g., `app/routers/sponsorship.py`, `app/routers/event.py`, `app/routers/person.py`). This aligns with standard FastAPI practice and will make the code much more organized.
* **Explicit Endpoints:** Within each router, define explicit endpoints for each operation (e.g., `@router.get("/{event_id}")`, `@router.post("/")`). This is more readable, self-documenting (with OpenAPI), and easier to test than the current dynamic URL structure.
* **Dependency Injection for Configuration:** Instead of the `obj_type_kv_li` dictionary, use FastAPI's dependency injection system (`Depends`) to provide configuration and dependencies to the endpoints. This will make the code more explicit and easier to follow.
* **Centralized Database Logic (CRUD Layer):** Create an `app/crud/` directory to house all database interaction logic. Each file in this directory would correspond to a specific object type (e.g., `app/crud/sponsorship.py`, `app/crud/event.py`). This will separate the API layer from the data access layer, improving separation of concerns.
* **Simplified Models:** Review and simplify the Pydantic models. Consolidate the various `mdl_`, `base_name`, `tbl_`, and `table_name` variations into a more consistent scheme. Use `_in` and `_out` models where necessary, but avoid the proliferation of alternate models.
* **Modern Configuration Management:** Move away from the mixed `config.py` and database configuration system. Adopt Pydantic's `BaseSettings` to load configuration from environment variables or a `.env` file. This is the standard for modern FastAPI applications and will make the configuration more explicit and easier to manage in different environments.
## 4. Migration Plan
A big-bang rewrite is risky. I propose a gradual, step-by-step migration:
1. **Step 1: Create the New Structure:**
* Create the `app/crud/` directory.
* Start with a single object type, for example, `sponsorship`.
* Create `app/routers/sponsorship.py` and `app/crud/sponsorship.py`.
2. **Step 2: Migrate the "Sponsorship" Object Type:**
* In `app/crud/sponsorship.py`, create functions for `get_sponsorship`, `list_sponsorships`, `create_sponsorship`, `update_sponsorship`, and `delete_sponsorship`. These functions will contain the `db_sql.py` calls.
* In `app/routers/sponsorship.py`, create a new `APIRouter`.
* Define the explicit endpoints for sponsorship (e.g., `@router.get("/sponsorships/{sponsorship_id}")`).
* These endpoints will call the corresponding functions in `app/crud/sponsorship.py`.
* Update `app/main.py` to include the new sponsorship router.
3. **Step 3: Test and Verify:**
* Thoroughly test the new `/sponsorships` endpoints to ensure they work as expected.
* You can run the old and new APIs side-by-side to compare results.
4. **Step 4: Repeat for Other Object Types:**
* Repeat the process for each major object type (`event`, `person`, `order`, etc.), one at a time.
5. **Step 5: Preserve and Refactor Core Services:**
* The JWT generation logic in `app/routers/api.py` should be preserved. Consider moving it to a dedicated `app/routers/auth.py` router.
* The `jitsi_token` endpoint should also be moved to its own router, perhaps `app/routers/integrations.py`.
6. **Step 6: Deprecate and Remove Legacy Code:**
* Once all object types have been migrated to the new structure, the old `app/routers/api_crud_v2.py` and `app/ae_obj_types_def.py` files can be safely deprecated and then removed.
* The other legacy routers in `app/routers/` can also be cleaned up.
This plan provides a structured approach to refactoring your project. By migrating one piece at a time, you can minimize risk and ensure a smooth transition to a more modern and maintainable codebase.

60
documentation/archive/REWRITE_PLAN_V2.md Normal file
View File

@@ -0,0 +1,60 @@
# Aether API Rewrite Plan (v2) - Revised
## 1. Core Principles of the Rewrite
1. **`id_random` First:** The `id_random` will be the primary public identifier for all Aether objects. The internal, auto-incrementing `id` will be treated as a private implementation detail and never exposed to the API consumer.
2. **Standardized Core Object:** All Aether objects will share a common set of base properties. This will be enforced through a base Pydantic model that all other object models will inherit from.
3. **Explicit and Organized Routers:** Each Aether object type will have its own dedicated router file, promoting a clean and maintainable codebase.
4. **Clear Separation of Concerns:** The API will be structured with a clear separation between the API/routing layer, the business logic/CRUD layer, and the data access layer.
5. **Modern, Explicit Configuration:** Configuration will be managed through a modern, explicit system (e.g., Pydantic's `BaseSettings`), moving away from the complex and inconsistent `obj_type_kv_li` dictionary.
## 2. Updated Rewrite Plan - Revised for Incremental Progress
### Learnings from Previous Session (December 2, 2025):
* Debugging in a Dockerized environment (where `uvicorn` is symlinked within Docker) is challenging for local troubleshooting. We need to focus on methods that validate code before full application integration.
* The `422 Unprocessable Entity` errors encountered were from legacy `/crud/site/domain` endpoints, highlighting that issues can arise from existing code interactions even when introducing new, theoretically sound patterns.
* Small, atomic changes with immediate, verifiable feedback are critical.
### Phase 1: Foundation and Standardization (Revised Approach)
The goal of this phase is to establish the core architecture without introducing runtime errors, focusing on isolated testing.
1. **Isolated Model Validation (`app/models/core_object.py`, `app/models/journal.py`, `app/models/journal_entry.py`, `app/models/site_domain.py`):**
* **Action:** Re-create the `app/models/core_object.py` file first, ensuring all imports are correct and `Pydantic.BaseModel` is explicitly imported where needed.
* **Action:** Re-create `app/models/journal.py`, `app/models/journal_entry.py`, and `app/models/site_domain.py`, ensuring they correctly inherit from `CoreObject` and have all necessary imports.
* **Verification:** Create and run an isolated Python script (e.g., `model_test.py`) that imports *only* these new Pydantic models (and their dependencies like `datetime`, `typing`, `pydantic`) and attempts to instantiate them with sample data. This script *must not* import `app.log` or `app.config` to avoid environment-specific issues and allow local execution. This step will validate model definitions in isolation.
* **Success Criteria:** The `model_test.py` script runs without any `ImportError` or Pydantic validation errors.
2. **Isolated AE Object Registry Validation (`app/ae_object_registry.py`):**
* **Action:** Re-create `app/ae_object_registry.py`, ensuring it correctly defines `AEObjectConfig` with the `children` field and uses singular prefixes. The `journal_entry_config` will be defined as a child of `journal_config`, and `site_domain_config` will be a top-level entry.
* **Verification:** Extend the `model_test.py` script (or create a new one) to import `app.ae_object_registry` and access the `AE_OBJECT_REGISTRY` dictionary, verifying its structure and the object configurations.
* **Success Criteria:** The script runs without errors, and the registry structure is as expected.
3. **Dummy CRUD Layer (`app/crud/base.py`):**
* **Action:** Re-create `app/crud/base.py`. Implement placeholder CRUD functions (`create_object`, `get_object`, `list_objects`, `update_object`, `delete_object`) that accept `AEObjectConfig`, `data`, `id_random`, `parent_id_random`, and `parent_table_name`. These functions will *not* interact with the database; they will only print their arguments and return dummy data that conforms to the expected Pydantic models (e.g., `config.model` or `list[config.model]`).
* **Verification:** Write a dedicated script (`crud_test.py`) that imports `app.crud.base` and `app.ae_object_registry`, then calls these dummy CRUD functions with sample data, including nested object scenarios.
* **Success Criteria:** The `crud_test.py` script runs without errors, and the print statements confirm correct argument passing.
4. **Router Factory Verification (`app/routers/factory.py`):**
* **Action:** Re-create `app/routers/factory.py` with the corrected `create_crud_router` function that handles both top-level and nested router generation, ensuring `parent_config` is passed correctly to child calls and `parent_table_name` is passed to CRUD functions.
* **Verification:** This step is harder to verify in isolation without FastAPI's full context. The primary verification will come in the next step when integrating with `main.py`.
5. **Integration with `main.py`:**
* **Action:** Modify `app/main.py` to include the router factory. Uncomment the loop that iterates through `AE_OBJECT_REGISTRY` and includes the routers generated by `create_crud_router`.
* **Verification:**
* Confirm the FastAPI application loads successfully within the Docker container (by observing logs).
* Use `curl` or a simple Python `requests` script to manually test the new `/journal` and `/journal_entry` endpoints (e.g., `POST /journal`, `GET /journal/{id_random}`, `POST /journal/{id_random}/journal_entry`).
* **Success Criteria:** The application starts without errors, and the new endpoints respond correctly (returning dummy data from `crud/base.py`).
### Phase 2: Implement Real Database Logic for Journals
1. **Database Connection:** Integrate the database connection logic into `app/crud/base.py` (or a more specific `app/crud/journals.py`).
2. **SQL Operations:** Replace dummy CRUD logic with actual `sql_insert`, `sql_select`, `sql_update`, and `sql_delete` calls, ensuring `id_random` to `id` lookups are correctly handled using Redis.
3. **Verification:** Thoroughly test the new endpoints using integration tests or manual `curl` requests against the Dockerized environment, confirming data persistence and retrieval.
### Phase 3: Gradual Migration of Other Aether Objects
(As originally outlined in `REWRITE_PLAN_V2.md`)
This revised plan prioritizes stability and verifiable progress, ensuring each component works in isolation before integration.