OSIT-AE-API-FastAPI/documentation/ARCH__V3_DEVELOPMENT_STANDARDS.md

Aether V3 Development Standards & Strategy

This document serves as the master guide for the Aether API V3. It combines core architectural principles with the finalized infrastructure standards established in January 2026.

1. Core Principles

• id_random Primary: id_random (URL-safe string) is the only identifier exposed to clients. Internal integer id is a private implementation detail.
• Singular naming: All object types and prefixes use singular form (e.g., journal, not journals).
• Inheritance: All models must inherit from CoreObject to ensure consistent base fields.
• Separation of Concerns: Business logic lives in app/methods/, CRUD helpers in app/lib_sql_crud.py, and routing in app/routers/.

2. Infrastructure Standards

Finalized Jan 15, 2026, to ensure boot stability.

Application Entry Point (app/main.py)

• Registry Pattern: All routers are registered via app/routers/registry.py.
• Lifespan Management: All startup tasks (logging setup, DB bootstrap, engine refresh) MUST reside within the @contextlib.asynccontextmanager lifespan.
• No Top-Level Logic: No SQL queries or heavy initialization should execute at the module level.

Database Layering

• lib_sql_core.py: The foundational "Source of Truth" for the SQLAlchemy engine and global db connection.
• lib_sql_crud.py: High-level logic for sql_insert, sql_select, etc.
• db_sql.py: A backward-compatible facade. code should import from here.

Logging

• Explicit Setup: setup_logging(settings) must be called during the lifespan startup.
• lib_log_v3.py: New home for logging configuration and the logger_reset decorator.

3. V3 CRUD Strategy

URL Structure

• Top-Level: /v3/crud/{obj_type}/
• Nested: /v3/crud/{parent_type}/{parent_id_random}/{obj_type}/ (Enforces parent ownership).

Search API

• POST Based: Complex filtering is handled via POST /search with a JSON body containing and, or, and not logic.
• Hybrid Filtering: (Proposed) Query parameters should append simple standard filters (e.g., ?enabled=true) to the complex body logic.

Field Evolution Checklist

When a table or view gains, loses, or renames fields, keep the API contract and search registry in sync:

1. Update the Pydantic model in app/models/ first so CRUD serialization matches the new shape.
2. Update the SQL view or table projection so GET and SEARCH responses actually return the field.
3. Update searchable_fields in app/object_definitions/ only for fields that should be searchable.
4. Add write-only, virtual, or view-only fields to fields_to_exclude_from_db when they must not be persisted.
5. Run the schema/search E2E tests that cover the object type before handing the change off.
6. Restart the Docker API containers (docker compose restart ae_api) — Python file changes inside containers are not picked up until restart.

For archive_content, the public field set now includes external_id and code, and future additions should follow the same order of operations.

Alt-view fields (fields only in tbl_alt)

Some objects have a richer alternate SQL view (tbl_alt) that adds JOINed/computed columns absent from the default view (tbl_default). For example, event_session uses v_event_session_w_file_count as its alt view (triggered by ?view=alt on search, or ?inc_file_count=true on GET).

• Fields from tbl_alt must still be declared in the Pydantic model and in searchable_fields — Pydantic strips undeclared fields, and the search whitelist rejects unknown field names regardless of the view.
• When adding such a field, add a comment noting which view provides it (e.g., # from v_event_session_w_file_count).
• Searching by an alt-view field on the default endpoint returns 400 Unknown column — this is correct behaviour. Clients must pass ?view=alt to use those fields in a search.
• Known alt-view fields restored May 2026: event_presentation_li_qry_str, event_presenter_li_qry_str (event_session); account_name, account_code, and related convenience fields (site_domain).

Response Views (?view= parameter)

• The nested search router (api_crud_v3_nested.py) already supports ?view=<key> to switch between registered views. view=default uses tbl_default; view=alt uses tbl_alt; additional named views can be added to the object registry as tbl_<name> / mdl_<name>.
• Flat search (api_crud_v3.py) does not yet support ?view= — it always uses tbl_default.

4. V3 Dependency Injection Reference

All V3 endpoints use granular, composable Depends() from app/lib_general_v3.py:

Dependency	Purpose
get_account_context → AccountContext	Resolves account_id with precedence: Header → Query Token → Bypass Header. Raises 403 on guest/missing context.
PaginationParams	Standardizes limit and offset.
StatusFilterParams	Handles enabled and hidden filtering.
SerializationParams	Controls Pydantic serialization (by_alias, exclude_unset).
DelayParams	Optional latency simulation (?delay=N) via await asyncio.sleep().

AccountContext also carries administrator, manager, and super flags, populated by a deferred DB lookup when a JWT is present. These flags control whether account isolation is bypassed for support tasks.

5. Security and Data Isolation

Fail-Closed Strategy

If account_id or auth context is missing, the API defaults to a blocking filter (account_id IS NULL) — it does NOT fall back to returning all records. Never relax this.

Multi-Tenant Isolation

• Forced account filtering: apply_forced_account_filter injects an account_id WHERE clause into every list/search query for non-super users.
• Post-retrieval verification: Single-object GET, PATCH, DELETE include a secondary ownership check (check_account_access). A mismatch returns 403.
• Hierarchical verification: Nested endpoints verify parent ownership before allowing operations on children.
• Creation guard: On POST, the user's account_id is automatically forced onto the new record.

IDAA Privacy Baseline

No IDAA object (Events, Files, Posts, Meetings) is public by default. All routes require x-account-id context. The sole exception is site_domain (used for site bootstrapping). This is a Sev-1 class constraint — violating it has happened before.

Bypass / Admin Access

• x-no-account-id: bypass → grants super access, resolves to account_id=1 (One Sky IT Demo). Use only in internal/development utilities; do not expand its use.
• JWT query parameter (?jwt=...) is supported for download links and share URLs where custom headers cannot be provided.

6. FastAPI and Pydantic Gotchas

• response: Response injection: Use it as a direct type hint in function signature. Depends(Response) is not valid and causes router initialization failures.
• Parameter order: In function signatures, arguments without defaults must come before Depends() arguments.
• asyncio.sleep() not time.sleep(): Blocking the event loop in an async endpoint causes worker timeouts and 502 Bad Gateway under load.
• Pydantic V1 only: Do not use V2-only features (computed_field, model_validator, etc.). The migration is a separate planned project — see strategic goals in TODO__Agents.md.
• obj_type_kv_li in ae_obj_types_def.py: Supports both modern keys (tbl, mdl) and legacy keys (table_name, base_name). Legacy V2 endpoints depend on the legacy keys — do not remove them until V1/V2 are fully retired.

7. Stability Rules

1. Baby Step Testing: Restart Docker and verify root health after every modular change.
2. Avoid Shadowing: Never name a module part of the app. package the same as a common instance variable (e.g., avoid app.middleware package if you use app = FastAPI()).
3. Deferred Imports: Use from app.db_sql import ... inside functions in library modules to prevent circular dependency traps.
4. Model changes require container restart: Editing Python files on the host does not hot-reload inside Docker. Always run docker compose restart ae_api after model or object-definition changes, then re-run E2E tests.