Scott.Idem/OSIT-AE-API-FastAPI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

A quick version update for the god like catch all CRUD endpoints. Version 3 will be even better!

Browse Source
This commit is contained in:
Scott Idem
2025-12-03 17:58:58 -05:00
parent 0e41205472
commit b1d05c7e66
6 changed files with 63 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
documentation/API_SIMPLIFICATION_PLAN.md Normal file
View File

@@ -0,0 +1,53 @@
# Aether API V3 Implementation Plan
This document outlines the plan to build the Aether API V3. The goal of V3 is to create a new, modern, and consistent set of API endpoints that will run in parallel with the existing legacy endpoints. No existing endpoints will be modified or deleted.
## V3 Architecture
The V3 API will be centered around a new generic CRUD router that implements a hierarchical, nested URL structure for parent-child object relationships.
- **Endpoint Prefix:** All new endpoints will be under `/v3/crud/`.
- **Top-Level Objects:** Handled via a flat URL structure.
- `GET /v3/crud/journal/`
- `POST /v3/crud/journal/`
- `GET /v3/crud/journal/{journal_id}`
- **Child Objects:** Handled via a nested URL structure that enforces the parent-child relationship.
- `GET /v3/crud/journal/{journal_id}/journal_entry/`
- `POST /v3/crud/journal/{journal_id}/journal_entry/`
- `GET /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
This structure will be implemented in a new `app/routers/api_crud_v3.py` file, which will be inspired by the logic in the existing `api_crud_v2.py` but with updated routing to handle the nesting.
## Implementation Phases
### Phase 1: Create Stub V3 Endpoint (Proof-of-Concept)
The first step is to create a minimal, non-functional "stub" endpoint to prove that the basic routing and application integration works.
1. **Create File:** Create a new file: `app/routers/api_crud_v3.py`.
2. **Add Stub Endpoint:** Add a simple health-check or "hello world" endpoint within this file, for example, a `GET` endpoint at `/v3/crud/health`.
3. **Update Main App:** In `app/main.py`, import and include the new V3 router.
### Phase 2: Implement Top-Level Object CRUD
Using `journal` as the first test case, implement the full set of CRUD operations for a top-level object.
- `GET /v3/crud/journal/`
- `POST /v3/crud/journal/`
- `GET /v3/crud/journal/{journal_id}`
- `PATCH /v3/crud/journal/{journal_id}`
- `DELETE /v3/crud/journal/{journal_id}`
### Phase 3: Implement Nested Object CRUD
Using `journal_entry` as the first test case, implement the full set of CRUD operations for a child object, ensuring the `journal_id` from the URL is used for filtering.
- `GET /v3/crud/journal/{journal_id}/journal_entry/`
- `POST /v3/crud/journal/{journal_id}/journal_entry/`
- `GET /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
- `PATCH /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
- `DELETE /v3/crud/journal/{journal_id}/journal_entry/{entry_id}`
### Phase 4: Incremental Rollout
Once the patterns for top-level and nested objects are proven with Journal and Journal Entry, we will apply the same pattern to other Aether objects in batches.

86
documentation/JOURNALS_API_MIGRATION.md Normal file
View File

@@ -0,0 +1,86 @@
# Journals API Migration Plan
This document outlines the plan for migrating the Journals API to the new, standardized Aether API architecture.
## 1. Current API Analysis
The Journals module consists of two main objects: `journal` and `journal_entry`. Currently, these objects are primarily accessed through the generic CRUD endpoints.
### How the Front-End Likely Interacts with the API:
* **Create a Journal:**
* `POST /v2/crud/journal`
* **Body:** A JSON object with all the fields from the `Journal_Base` model.
* **Get a Journal:**
* `GET /v2/crud/journal/{journal_id_random}`
* **List Journals:**
* `GET /v2/crud/journal/list`
* This would also likely use the `for_obj_type` and `for_obj_id` query parameters to get journals for a specific person or account (e.g., `GET /v2/crud/journal/list?for_obj_type=person&for_obj_id={person_id_random}`).
* **Update a Journal:**
* `PATCH /v2/crud/journal/{journal_id_random}`
* **Body:** A JSON object with the fields to be updated.
* **Delete a Journal:**
* `DELETE /v2/crud/journal/{journal_id_random}`
The same patterns apply to the `journal_entry` object.
## 2. Proposed New API
The new Journals API will be more explicit, organized, and easier to use. It will follow the new architectural principles of the Aether API rewrite.
### New URL Structure:
* `/journals/`
* `/journals/{journal_id_random}`
* `/journals/{journal_id_random}/entries/`
* `/journals/{journal_id_random}/entries/{entry_id_random}`
### New Models:
We will create new, simplified models for `Journal` and `JournalEntry` that inherit from the `CoreObject` base model.
* `app/models/journal.py`:
* `JournalBase(CoreObject)`
* `Journal(JournalBase)`
* `JournalInDB(Journal, CoreObjectInDB)`
* `JournalCreate(JournalBase)`
* `JournalUpdate(JournalBase)`
* `app/models/journal_entry.py`:
* `JournalEntryBase(CoreObject)`
* `JournalEntry(JournalEntryBase)`
* `JournalEntryInDB(JournalEntry, CoreObjectInDB)`
* `JournalEntryCreate(JournalEntryBase)`
* `JournalEntryUpdate(JournalEntryBase)`
### New Endpoints:
A new `app/routers/journals.py` router will be created with the following explicit endpoints:
* `POST /journals/`: Create a new journal.
* `GET /journals/`: List all journals (with query params for filtering).
* `GET /journals/{journal_id_random}`: Get a single journal by its `id_random`.
* `PATCH /journals/{journal_id_random}`: Update a journal.
* `DELETE /journals/{journal_id_random}`: Delete a journal.
* `POST /journals/{journal_id_random}/entries/`: Create a new journal entry for a specific journal.
* `GET /journals/{journal_id_random}/entries/`: List all entries for a specific journal.
* `GET /journals/{journal_id_random}/entries/{entry_id_random}`: Get a single journal entry.
* `PATCH /journals/{journal_id_random}/entries/{entry_id_random}`: Update a journal entry.
* `DELETE /journals/{journal_id_random}/entries/{entry_id_random}`: Delete a journal entry.
## 3. Side-by-Side Comparison
| Operation | Current API Call | Proposed New API Call |
| --- | --- | --- |
| Create Journal | `POST /v2/crud/journal` | `POST /journals/` |
| Get Journal | `GET /v2/crud/journal/{id}` | `GET /journals/{id}` |
| List Journals | `GET /v2/crud/journal/list` | `GET /journals/` |
| Update Journal | `PATCH /v2/crud/journal/{id}` | `PATCH /journals/{id}` |
| Delete Journal | `DELETE /v2/crud/journal/{id}` | `DELETE /journals/{id}` |
| Create Entry | `POST /v2/crud/journal_entry` (with `journal_id_random` in body) | `POST /journals/{journal_id}/entries/` |
| List Entries | `GET /v2/crud/journal_entry/list?for_obj_type=journal&for_obj_id={id}` | `GET /journals/{journal_id}/entries/` |
This plan provides a clear path forward for refactoring the Journals module. By following this plan, we can create a more maintainable and user-friendly API while ensuring a smooth transition for the front-end.

59
documentation/REWRITE_PLAN.md Normal file
View File

@@ -0,0 +1,59 @@
# Aether API Rewrite Plan
## 1. Goal
The primary goal of this rewrite is to simplify the Aether API, remove legacy code, and adopt modern FastAPI best practices. This will make the project easier to maintain, extend, and understand.
## 2. Analysis of the Current State
The current implementation has several key characteristics that contribute to its complexity:
* **"God" Router:** A single file, `app/routers/api_crud_v2.py`, handles all CRUD operations (GET, POST, PATCH, DELETE) for all object types.
* **Dynamic Object Types:** The API uses URL path segments (`obj_type_l1`, `obj_type_l2`, `obj_type_l3`) to dynamically determine the object type being operated on.
* **Configuration via Dictionary:** The core of the dynamic system is the `obj_type_kv_li` dictionary in `app/ae_obj_types_def.py`. This dictionary maps object names to database tables, Pydantic models, and other configuration. This dictionary is inconsistent and contains legacy code.
* **Complex Parameter Handling:** Endpoints use a large number of query parameters and headers to control their behavior.
* **Mixed Logic:** The code mixes URL parsing, database lookups, data validation, and database operations.
## 3. Proposed New Structure
To address these challenges, I propose the following new structure:
* **Eliminate the "God" Router:** Instead of a single `api_crud_v2.py` file, create a separate router file for each major object type (e.g., `app/routers/sponsorship.py`, `app/routers/event.py`, `app/routers/person.py`). This aligns with standard FastAPI practice and will make the code much more organized.
* **Explicit Endpoints:** Within each router, define explicit endpoints for each operation (e.g., `@router.get("/{event_id}")`, `@router.post("/")`). This is more readable, self-documenting (with OpenAPI), and easier to test than the current dynamic URL structure.
* **Dependency Injection for Configuration:** Instead of the `obj_type_kv_li` dictionary, use FastAPI's dependency injection system (`Depends`) to provide configuration and dependencies to the endpoints. This will make the code more explicit and easier to follow.
* **Centralized Database Logic (CRUD Layer):** Create an `app/crud/` directory to house all database interaction logic. Each file in this directory would correspond to a specific object type (e.g., `app/crud/sponsorship.py`, `app/crud/event.py`). This will separate the API layer from the data access layer, improving separation of concerns.
* **Simplified Models:** Review and simplify the Pydantic models. Consolidate the various `mdl_`, `base_name`, `tbl_`, and `table_name` variations into a more consistent scheme. Use `_in` and `_out` models where necessary, but avoid the proliferation of alternate models.
* **Modern Configuration Management:** Move away from the mixed `config.py` and database configuration system. Adopt Pydantic's `BaseSettings` to load configuration from environment variables or a `.env` file. This is the standard for modern FastAPI applications and will make the configuration more explicit and easier to manage in different environments.
## 4. Migration Plan
A big-bang rewrite is risky. I propose a gradual, step-by-step migration:
1. **Step 1: Create the New Structure:**
* Create the `app/crud/` directory.
* Start with a single object type, for example, `sponsorship`.
* Create `app/routers/sponsorship.py` and `app/crud/sponsorship.py`.
2. **Step 2: Migrate the "Sponsorship" Object Type:**
* In `app/crud/sponsorship.py`, create functions for `get_sponsorship`, `list_sponsorships`, `create_sponsorship`, `update_sponsorship`, and `delete_sponsorship`. These functions will contain the `db_sql.py` calls.
* In `app/routers/sponsorship.py`, create a new `APIRouter`.
* Define the explicit endpoints for sponsorship (e.g., `@router.get("/sponsorships/{sponsorship_id}")`).
* These endpoints will call the corresponding functions in `app/crud/sponsorship.py`.
* Update `app/main.py` to include the new sponsorship router.
3. **Step 3: Test and Verify:**
* Thoroughly test the new `/sponsorships` endpoints to ensure they work as expected.
* You can run the old and new APIs side-by-side to compare results.
4. **Step 4: Repeat for Other Object Types:**
* Repeat the process for each major object type (`event`, `person`, `order`, etc.), one at a time.
5. **Step 5: Preserve and Refactor Core Services:**
* The JWT generation logic in `app/routers/api.py` should be preserved. Consider moving it to a dedicated `app/routers/auth.py` router.
* The `jitsi_token` endpoint should also be moved to its own router, perhaps `app/routers/integrations.py`.
6. **Step 6: Deprecate and Remove Legacy Code:**
* Once all object types have been migrated to the new structure, the old `app/routers/api_crud_v2.py` and `app/ae_obj_types_def.py` files can be safely deprecated and then removed.
* The other legacy routers in `app/routers/` can also be cleaned up.
This plan provides a structured approach to refactoring your project. By migrating one piece at a time, you can minimize risk and ensure a smooth transition to a more modern and maintainable codebase.

60
documentation/REWRITE_PLAN_V2.md Normal file
View File

@@ -0,0 +1,60 @@
# Aether API Rewrite Plan (v2) - Revised
## 1. Core Principles of the Rewrite
1. **`id_random` First:** The `id_random` will be the primary public identifier for all Aether objects. The internal, auto-incrementing `id` will be treated as a private implementation detail and never exposed to the API consumer.
2. **Standardized Core Object:** All Aether objects will share a common set of base properties. This will be enforced through a base Pydantic model that all other object models will inherit from.
3. **Explicit and Organized Routers:** Each Aether object type will have its own dedicated router file, promoting a clean and maintainable codebase.
4. **Clear Separation of Concerns:** The API will be structured with a clear separation between the API/routing layer, the business logic/CRUD layer, and the data access layer.
5. **Modern, Explicit Configuration:** Configuration will be managed through a modern, explicit system (e.g., Pydantic's `BaseSettings`), moving away from the complex and inconsistent `obj_type_kv_li` dictionary.
## 2. Updated Rewrite Plan - Revised for Incremental Progress
### Learnings from Previous Session (December 2, 2025):
* Debugging in a Dockerized environment (where `uvicorn` is symlinked within Docker) is challenging for local troubleshooting. We need to focus on methods that validate code before full application integration.
* The `422 Unprocessable Entity` errors encountered were from legacy `/crud/site/domain` endpoints, highlighting that issues can arise from existing code interactions even when introducing new, theoretically sound patterns.
* Small, atomic changes with immediate, verifiable feedback are critical.
### Phase 1: Foundation and Standardization (Revised Approach)
The goal of this phase is to establish the core architecture without introducing runtime errors, focusing on isolated testing.
1. **Isolated Model Validation (`app/models/core_object.py`, `app/models/journal.py`, `app/models/journal_entry.py`, `app/models/site_domain.py`):**
* **Action:** Re-create the `app/models/core_object.py` file first, ensuring all imports are correct and `Pydantic.BaseModel` is explicitly imported where needed.
* **Action:** Re-create `app/models/journal.py`, `app/models/journal_entry.py`, and `app/models/site_domain.py`, ensuring they correctly inherit from `CoreObject` and have all necessary imports.
* **Verification:** Create and run an isolated Python script (e.g., `model_test.py`) that imports *only* these new Pydantic models (and their dependencies like `datetime`, `typing`, `pydantic`) and attempts to instantiate them with sample data. This script *must not* import `app.log` or `app.config` to avoid environment-specific issues and allow local execution. This step will validate model definitions in isolation.
* **Success Criteria:** The `model_test.py` script runs without any `ImportError` or Pydantic validation errors.
2. **Isolated AE Object Registry Validation (`app/ae_object_registry.py`):**
* **Action:** Re-create `app/ae_object_registry.py`, ensuring it correctly defines `AEObjectConfig` with the `children` field and uses singular prefixes. The `journal_entry_config` will be defined as a child of `journal_config`, and `site_domain_config` will be a top-level entry.
* **Verification:** Extend the `model_test.py` script (or create a new one) to import `app.ae_object_registry` and access the `AE_OBJECT_REGISTRY` dictionary, verifying its structure and the object configurations.
* **Success Criteria:** The script runs without errors, and the registry structure is as expected.
3. **Dummy CRUD Layer (`app/crud/base.py`):**
* **Action:** Re-create `app/crud/base.py`. Implement placeholder CRUD functions (`create_object`, `get_object`, `list_objects`, `update_object`, `delete_object`) that accept `AEObjectConfig`, `data`, `id_random`, `parent_id_random`, and `parent_table_name`. These functions will *not* interact with the database; they will only print their arguments and return dummy data that conforms to the expected Pydantic models (e.g., `config.model` or `list[config.model]`).
* **Verification:** Write a dedicated script (`crud_test.py`) that imports `app.crud.base` and `app.ae_object_registry`, then calls these dummy CRUD functions with sample data, including nested object scenarios.
* **Success Criteria:** The `crud_test.py` script runs without errors, and the print statements confirm correct argument passing.
4. **Router Factory Verification (`app/routers/factory.py`):**
* **Action:** Re-create `app/routers/factory.py` with the corrected `create_crud_router` function that handles both top-level and nested router generation, ensuring `parent_config` is passed correctly to child calls and `parent_table_name` is passed to CRUD functions.
* **Verification:** This step is harder to verify in isolation without FastAPI's full context. The primary verification will come in the next step when integrating with `main.py`.
5. **Integration with `main.py`:**
* **Action:** Modify `app/main.py` to include the router factory. Uncomment the loop that iterates through `AE_OBJECT_REGISTRY` and includes the routers generated by `create_crud_router`.
* **Verification:**
* Confirm the FastAPI application loads successfully within the Docker container (by observing logs).
* Use `curl` or a simple Python `requests` script to manually test the new `/journal` and `/journal_entry` endpoints (e.g., `POST /journal`, `GET /journal/{id_random}`, `POST /journal/{id_random}/journal_entry`).
* **Success Criteria:** The application starts without errors, and the new endpoints respond correctly (returning dummy data from `crud/base.py`).
### Phase 2: Implement Real Database Logic for Journals
1. **Database Connection:** Integrate the database connection logic into `app/crud/base.py` (or a more specific `app/crud/journals.py`).
2. **SQL Operations:** Replace dummy CRUD logic with actual `sql_insert`, `sql_select`, `sql_update`, and `sql_delete` calls, ensuring `id_random` to `id` lookups are correctly handled using Redis.
3. **Verification:** Thoroughly test the new endpoints using integration tests or manual `curl` requests against the Dockerized environment, confirming data persistence and retrieval.
### Phase 3: Gradual Migration of Other Aether Objects
(As originally outlined in `REWRITE_PLAN_V2.md`)
This revised plan prioritizes stability and verifiable progress, ensuring each component works in isolation before integration.