OSIT-AE-API-FastAPI/documentation/ARCH__V3_DEVELOPMENT_STANDARDS.md

# Aether V3 Development Standards & Strategy

This document serves as the master guide for the Aether API V3. It combines core architectural principles with the finalized infrastructure standards established in January 2026.

## 1. Core Principles

-   **`id_random` Primary**: `id_random` (URL-safe string) is the only identifier exposed to clients. Internal integer `id` is a private implementation detail.
-   **Singular naming**: All object types and prefixes use singular form (e.g., `journal`, not `journals`).
-   **Inheritance**: All models must inherit from `CoreObject` to ensure consistent base fields.
-   **Separation of Concerns**: Business logic lives in `app/methods/`, CRUD helpers in `app/lib_sql_crud.py`, and routing in `app/routers/`.

## 2. Infrastructure Standards

Finalized Jan 15, 2026, to ensure boot stability.

### Application Entry Point (`app/main.py`)
-   **Registry Pattern**: All routers are registered via `app/routers/registry.py`.
-   **Lifespan Management**: All startup tasks (logging setup, DB bootstrap, engine refresh) MUST reside within the `@contextlib.asynccontextmanager` lifespan.
-   **No Top-Level Logic**: No SQL queries or heavy initialization should execute at the module level.

### Database Layering
-   **`lib_sql_core.py`**: The foundational "Source of Truth" for the SQLAlchemy engine and global `db` connection.
-   **`lib_sql_crud.py`**: High-level logic for `sql_insert`, `sql_select`, etc.
-   **`db_sql.py`**: A backward-compatible facade. code should import from here.

### Logging
-   **Explicit Setup**: `setup_logging(settings)` must be called during the lifespan startup.
-   **`lib_log_v3.py`**: New home for logging configuration and the `logger_reset` decorator.

## 3. V3 CRUD Strategy

### URL Structure
-   **Top-Level**: `/v3/crud/{obj_type}/`
-   **Nested**: `/v3/crud/{parent_type}/{parent_id_random}/{obj_type}/` (Enforces parent ownership).

### Search API
-   **POST Based**: Complex filtering is handled via `POST /search` with a JSON body containing `and`, `or`, and `not` logic.
-   **Hybrid Filtering**: (Proposed) Query parameters should append simple standard filters (e.g., `?enabled=true`) to the complex body logic.

### Field Evolution Checklist
When a table or view gains, loses, or renames fields, keep the API contract and search registry in sync:

1. Update the Pydantic model in `app/models/` first so CRUD serialization matches the new shape.
2. Update the SQL view or table projection so `GET` and `SEARCH` responses actually return the field.
3. Update `searchable_fields` in `app/object_definitions/` only for fields that should be searchable.
4. Add write-only, virtual, or view-only fields to `fields_to_exclude_from_db` when they must not be persisted.
5. Run the schema/search E2E tests that cover the object type before handing the change off.
6. **Restart the Docker API containers** (`docker compose restart ae_api`) — Python file changes inside containers are not picked up until restart.

For `archive_content`, the public field set now includes `external_id` and `code`, and future additions should follow the same order of operations.

#### Alt-view fields (fields only in `tbl_alt`)
Some objects have a richer alternate SQL view (`tbl_alt`) that adds JOINed/computed columns absent from the default view (`tbl_default`). For example, `event_session` uses `v_event_session_w_file_count` as its alt view (triggered by `?view=alt` on search, or `?inc_file_count=true` on GET).

- Fields from `tbl_alt` **must still be declared in the Pydantic model** and in `searchable_fields` — Pydantic strips undeclared fields, and the search whitelist rejects unknown field names regardless of the view.
- When adding such a field, add a comment noting which view provides it (e.g., `# from v_event_session_w_file_count`).
- Searching by an alt-view field on the default endpoint returns `400 Unknown column` — this is correct behaviour. Clients must pass `?view=alt` to use those fields in a search.
- **Known alt-view fields restored May 2026:** `event_presentation_li_qry_str`, `event_presenter_li_qry_str` (event_session); `account_name`, `account_code`, and related convenience fields (site_domain).

### Response Views (`?view=` parameter)
-   The nested search router (`api_crud_v3_nested.py`) already supports `?view=<key>` to switch between registered views. `view=default` uses `tbl_default`; `view=alt` uses `tbl_alt`; additional named views can be added to the object registry as `tbl_<name>` / `mdl_<name>`.
-   Flat search (`api_crud_v3.py`) does not yet support `?view=` — it always uses `tbl_default`.

## 4. V3 Dependency Injection Reference

All V3 endpoints use granular, composable `Depends()` from `app/lib_general_v3.py`:

| Dependency | Purpose |
|---|---|
| `get_account_context` → `AccountContext` | Resolves `account_id` with precedence: Header → Query Token → Bypass Header. Raises 403 on guest/missing context. |
| `PaginationParams` | Standardizes `limit` and `offset`. |
| `StatusFilterParams` | Handles `enabled` and `hidden` filtering. |
| `SerializationParams` | Controls Pydantic serialization (`by_alias`, `exclude_unset`). |
| `DelayParams` | Optional latency simulation (`?delay=N`) via `await asyncio.sleep()`. |

`AccountContext` also carries `administrator`, `manager`, and `super` flags, populated by a deferred DB lookup when a JWT is present. These flags control whether account isolation is bypassed for support tasks.

## 5. Security and Data Isolation

### Fail-Closed Strategy
If `account_id` or auth context is missing, the API defaults to a blocking filter (`account_id IS NULL`) — it does NOT fall back to returning all records. Never relax this.

### Multi-Tenant Isolation
- **Forced account filtering**: `apply_forced_account_filter` injects an `account_id` WHERE clause into every list/search query for non-super users.
- **Post-retrieval verification**: Single-object GET, PATCH, DELETE include a secondary ownership check (`check_account_access`). A mismatch returns 403.
- **Hierarchical verification**: Nested endpoints verify parent ownership before allowing operations on children.
- **Creation guard**: On POST, the user's `account_id` is automatically forced onto the new record.

### IDAA Privacy Baseline
No IDAA object (Events, Files, Posts, Meetings) is public by default. All routes require `x-account-id` context. The sole exception is `site_domain` (used for site bootstrapping). This is a **Sev-1 class constraint** — violating it has happened before.

### Bypass / Admin Access
- `x-no-account-id: bypass` → grants super access, resolves to `account_id=1` (One Sky IT Demo). Use only in internal/development utilities; do not expand its use.
- JWT query parameter (`?jwt=...`) is supported for download links and share URLs where custom headers cannot be provided.

## 6. FastAPI and Pydantic Gotchas

- **`response: Response` injection**: Use it as a direct type hint in function signature. `Depends(Response)` is not valid and causes router initialization failures.
- **Parameter order**: In function signatures, arguments without defaults must come before `Depends()` arguments.
- **`asyncio.sleep()` not `time.sleep()`**: Blocking the event loop in an async endpoint causes worker timeouts and `502 Bad Gateway` under load.
- **Pydantic V1 only**: Do not use V2-only features (`computed_field`, `model_validator`, etc.). The migration is a separate planned project — see strategic goals in `TODO__Agents.md`.
- **`obj_type_kv_li` in `ae_obj_types_def.py`**: Supports both modern keys (`tbl`, `mdl`) and legacy keys (`table_name`, `base_name`). Legacy V2 endpoints depend on the legacy keys — do not remove them until V1/V2 are fully retired.

## 7. Stability Rules

1.  **Baby Step Testing**: Restart Docker and verify root health after *every* modular change.
2.  **Avoid Shadowing**: Never name a module part of the `app.` package the same as a common instance variable (e.g., avoid `app.middleware` package if you use `app = FastAPI()`).
3.  **Deferred Imports**: Use `from app.db_sql import ...` *inside* functions in library modules to prevent circular dependency traps.
4.  **Model changes require container restart**: Editing Python files on the host does not hot-reload inside Docker. Always run `docker compose restart ae_api` after model or object-definition changes, then re-run E2E tests.