OSIT-AE-API-FastAPI/documentation/REWRITE_PLAN_V2.md

Aether API Rewrite Plan (v2) - Revised

1. Core Principles of the Rewrite

1. id_random First: The id_random will be the primary public identifier for all Aether objects. The internal, auto-incrementing id will be treated as a private implementation detail and never exposed to the API consumer.
2. Standardized Core Object: All Aether objects will share a common set of base properties. This will be enforced through a base Pydantic model that all other object models will inherit from.
3. Explicit and Organized Routers: Each Aether object type will have its own dedicated router file, promoting a clean and maintainable codebase.
4. Clear Separation of Concerns: The API will be structured with a clear separation between the API/routing layer, the business logic/CRUD layer, and the data access layer.
5. Modern, Explicit Configuration: Configuration will be managed through a modern, explicit system (e.g., Pydantic's BaseSettings), moving away from the complex and inconsistent obj_type_kv_li dictionary.

2. Updated Rewrite Plan - Revised for Incremental Progress

Learnings from Previous Session (December 2, 2025):

• Debugging in a Dockerized environment (where uvicorn is symlinked within Docker) is challenging for local troubleshooting. We need to focus on methods that validate code before full application integration.
• The 422 Unprocessable Entity errors encountered were from legacy /crud/site/domain endpoints, highlighting that issues can arise from existing code interactions even when introducing new, theoretically sound patterns.
• Small, atomic changes with immediate, verifiable feedback are critical.

Phase 1: Foundation and Standardization (Revised Approach)

The goal of this phase is to establish the core architecture without introducing runtime errors, focusing on isolated testing.

1. Isolated Model Validation (app/models/core_object.py, app/models/journal.py, app/models/journal_entry.py, app/models/site_domain.py):

   • Action: Re-create the app/models/core_object.py file first, ensuring all imports are correct and Pydantic.BaseModel is explicitly imported where needed.
   • Action: Re-create app/models/journal.py, app/models/journal_entry.py, and app/models/site_domain.py, ensuring they correctly inherit from CoreObject and have all necessary imports.
   • Verification: Create and run an isolated Python script (e.g., model_test.py) that imports only these new Pydantic models (and their dependencies like datetime, typing, pydantic) and attempts to instantiate them with sample data. This script must not import app.log or app.config to avoid environment-specific issues and allow local execution. This step will validate model definitions in isolation.
   • Success Criteria: The model_test.py script runs without any ImportError or Pydantic validation errors.

2. Isolated AE Object Registry Validation (app/ae_object_registry.py):

   • Action: Re-create app/ae_object_registry.py, ensuring it correctly defines AEObjectConfig with the children field and uses singular prefixes. The journal_entry_config will be defined as a child of journal_config, and site_domain_config will be a top-level entry.
   • Verification: Extend the model_test.py script (or create a new one) to import app.ae_object_registry and access the AE_OBJECT_REGISTRY dictionary, verifying its structure and the object configurations.
   • Success Criteria: The script runs without errors, and the registry structure is as expected.

3. Dummy CRUD Layer (app/crud/base.py):

   • Action: Re-create app/crud/base.py. Implement placeholder CRUD functions (create_object, get_object, list_objects, update_object, delete_object) that accept AEObjectConfig, data, id_random, parent_id_random, and parent_table_name. These functions will not interact with the database; they will only print their arguments and return dummy data that conforms to the expected Pydantic models (e.g., config.model or list[config.model]).
   • Verification: Write a dedicated script (crud_test.py) that imports app.crud.base and app.ae_object_registry, then calls these dummy CRUD functions with sample data, including nested object scenarios.
   • Success Criteria: The crud_test.py script runs without errors, and the print statements confirm correct argument passing.

4. Router Factory Verification (app/routers/factory.py):

   • Action: Re-create app/routers/factory.py with the corrected create_crud_router function that handles both top-level and nested router generation, ensuring parent_config is passed correctly to child calls and parent_table_name is passed to CRUD functions.
   • Verification: This step is harder to verify in isolation without FastAPI's full context. The primary verification will come in the next step when integrating with main.py.

5. Integration with main.py:

   • Action: Modify app/main.py to include the router factory. Uncomment the loop that iterates through AE_OBJECT_REGISTRY and includes the routers generated by create_crud_router.
   • Verification:
     • Confirm the FastAPI application loads successfully within the Docker container (by observing logs).
     • Use curl or a simple Python requests script to manually test the new /journal and /journal_entry endpoints (e.g., POST /journal, GET /journal/{id_random}, POST /journal/{id_random}/journal_entry).
   • Success Criteria: The application starts without errors, and the new endpoints respond correctly (returning dummy data from crud/base.py).

Phase 2: Implement Real Database Logic for Journals

1. Database Connection: Integrate the database connection logic into app/crud/base.py (or a more specific app/crud/journals.py).
2. SQL Operations: Replace dummy CRUD logic with actual sql_insert, sql_select, sql_update, and sql_delete calls, ensuring id_random to id lookups are correctly handled using Redis.
3. Verification: Thoroughly test the new endpoints using integration tests or manual curl requests against the Dockerized environment, confirming data persistence and retrieval.

Phase 3: Gradual Migration of Other Aether Objects

(As originally outlined in REWRITE_PLAN_V2.md)

This revised plan prioritizes stability and verifiable progress, ensuring each component works in isolation before integration.