From 061c153061f90d62a9c2cc6814f96d364ba67fe2 Mon Sep 17 00:00:00 2001 From: Scott Idem Date: Thu, 22 Jan 2026 19:05:22 -0500 Subject: [PATCH] Saving updated notes. --- GEMINI.md | 25 ++-- documentation/V3_FRONTEND_API_GUIDE.md | 179 ++++++++----------------- 2 files changed, 70 insertions(+), 134 deletions(-) diff --git a/GEMINI.md b/GEMINI.md index 69c1bf1..a00ec64 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -22,19 +22,22 @@ I am the **primary orchestrator and main helper** for the development of the **U - **ID Vision Mapping Stability**: Resolved a critical conflict in `app/lib_sql_search.py` where backend-injected integer filters (e.g., `account_id`) were being incorrectly mapped to `_random` string columns. Mapping now only occurs for non-integer values. - **API Error Transparency**: Modified `sql_select` to return `False` on hard database errors (instead of empty results). Updated the V3 search endpoint to return a `500 Internal Server Error` in these cases, preventing "silent failures" where frontend developers would see `200 OK` with zero results during system errors. - **NULL Logic in Filters**: Confirmed that explicit frontend filters like `hide: false` will FAIL to match `NULL` database values in standard SQL equality. It is more robust to rely on the API's built-in `hidden=not_hidden` URL parameter, which handles `(hide = false OR hide IS NULL)` automatically. +- **V3 Action Router Paradigm**: Established a new pattern for specialized binary operations (Upload/Download) under `/v3/action/`, keeping the standard `/v3/crud/` routes clean for JSON metadata. +- **Vision ID safety Net**: Enhanced `lookup_id_random_pop` to resolve random string IDs found in any `*_id` field, ensuring "Vision" style payloads (where IDs are strings) are correctly converted to integers before database insertion even if the `*_id_random` field is missing. -## Session Summary & Progress (Jan 21, 2026) +## Session Summary & Progress (Jan 22, 2026) -This session focused on stabilizing the V3 search engine and enhancing the "ID Vision" developer experience. +This session focused on migrating and stabilizing the Hosted File management system into the V3 architecture. -* **V3 Search Engine Enhancements:** - * **Comprehensive ID Vision:** Expanded `vision_fields` mapping to include almost all major Aether objects (Event, Journal, Order, Product, etc.), allowing clean names in frontend filters. - * **Error Handling Refactor:** Standardized SQL error bubbling to return `500` status codes with DB details instead of misleading `200` successes. - * **Status Filter Consistency:** Added `not_enabled` alias for the `enabled` status parameter to align with `not_hidden`. -* **Object Definitions:** - * Whitelisted clean ID names (e.g., `event_id`, `account_id`) in `searchable_fields` for Events, Badges, and Journals to support the new mapping logic. -* **Database Synchronization:** - * Verified and resolved `v_journal_entry` missing column errors after user updated the view to include `account_id`. +* **V3 Hosted File Migration (Complete):** + * **Stabilized Models:** Updated `Hosted_File_Base` and `Hosted_File_Link_Base` to prioritize integer IDs internally while ensuring random string mapping for the frontend. + * **New Action Router:** Implemented `app/routers/api_v3_actions_hosted_file.py` containing specialized logic for Upload, Download (Streaming), and Relational Cleanup. + * **Multi-File Upload:** Ported and enhanced upload logic to support `List[UploadFile]` with automatic SHA256 deduplication and relational linking. + * **Intelligent Deletion:** Implemented orphan check logic; files are only physically removed if `rm_orphan=true` and no remaining `hosted_file_link` records exist. + * **Frontend Guide:** Updated `V3_FRONTEND_API_GUIDE.md` with full documentation for the new V3 Action endpoints and synced it to `agents_sync`. +* **Database Integrity:** + * Verified and corrected `hosted_file.account_id` to `int(11)`, resolving a legacy schema mistake. + * Verified successful creation and cleanup of test records via new E2E test suite. ## Current To-Do List @@ -69,4 +72,4 @@ This session focused on stabilizing the V3 search engine and enhancing the "ID V * Inspect the live log stream for regressions: 1 "tail -n 20 ~/OSIT_dev/aether_container_env/logs/ae_api/aether_api.log" -6. Finalize: Once verified, commit the changes with a descriptive message and sync relevant documentation. +6. Finalize: Once verified, commit the changes with a descriptive message and sync relevant documentation. \ No newline at end of file diff --git a/documentation/V3_FRONTEND_API_GUIDE.md b/documentation/V3_FRONTEND_API_GUIDE.md index 84ebe5f..48aa953 100644 --- a/documentation/V3_FRONTEND_API_GUIDE.md +++ b/documentation/V3_FRONTEND_API_GUIDE.md @@ -1,6 +1,6 @@ # Aether API V3 Frontend Integration Guide (Svelte/TypeScript) -This guide explains how to update or create frontend functions to interact with the new **Aether API V3 CRUD** endpoints. V3 introduces a nested URL structure, a powerful POST-based search, and improved performance. +This guide explains how to update or create frontend functions to interact with the new **Aether API V3 CRUD** and **Action** endpoints. V3 introduces a nested URL structure, a powerful POST-based search, and specialized "Action" routes for binary data. --- @@ -37,33 +37,13 @@ Once the API Key is validated, you must specify the context of your request. 3. **Guest / Anonymous Access**: Provide a **Safe Guest JWT**. * **Header:** `x-aether-api-key: ` (No Account Header) * **Query Param:** `?jwt=` - * **Benefit:** Allows the backend to track cryptographically signed session state (e.g., site context) without granting full account access. - -### C. Public Read Whitelist (Unauthenticated) -The V3 architecture allows unauthenticated (**guest**) access to specific objects, provided a valid API Key is present. - -* **Whitelisted Objects:** `site_domain`, `event`, `event_session`, `event_presentation`, `event_presenter`, `post`, `post_comment`, `archive`, `archive_content`, `hosted_file`. -* **Behavior:** Requests to these objects will succeed without a user-level JWT, but are strictly limited to **Read-Only** operations (GET/Search). - -### D. Passcode Authentication (Machine/Kiosk Logins) -For guest machines or specialized kiosks, use the dedicated passcode endpoint to receive a scoped JWT. - -* **Endpoint:** `POST /api/authenticate_passcode` -* **Payload:** `{ "site_id": "", "passcode": "" }` -* **Response:** Returns a JWT containing the site's `account_id` and resolved role flags (`super`, `manager`, `administrator`). - -### E. Fail-Fast on Auth Failures (401/403) -The frontend implements a **Fail-Fast** policy for 401/403 errors. If the API returns these codes, the frontend **must not retry** automatically. It should stop and redirect the user or display an error. --- ## 3. Implementing V3 CRUD Functions ### A. List & Single Object (GET) -Support for view selection allows fetching richer data models when needed. - ```ts -// Example: Get enriched journal details // GET /v3/crud/journal/{id}?view=enriched export async function get_ae_obj_v3({ api_cfg, obj_type, obj_id, view = 'default' }) { const endpoint = `/v3/crud/${obj_type}/${obj_id}`; @@ -72,135 +52,88 @@ export async function get_ae_obj_v3({ api_cfg, obj_type, obj_id, view = 'default ``` ### B. Advanced & Hybrid Search (POST) -The `/search` endpoint combines complex logical bodies with simple query parameters. - ```ts -export async function search_ae_obj_v3({ - api_cfg, - obj_type, - search_query, // { q: "search term", and: [...] } - enabled = 'enabled', - view = 'default' -}) { +// POST /v3/crud/{obj_type}/search +export async function search_ae_obj_v3({ api_cfg, obj_type, search_query, enabled = 'enabled' }) { const endpoint = `/v3/crud/${obj_type}/search`; - const params = { enabled, view }; - - return await post_object({ api_cfg, endpoint, params, data: search_query }); -} -``` - -### C. Standardized Global Search (`q`) -Use the `q` property in your search body for a keyword search. -- **Wildcard**: `q: "%"` returns all records (respecting logical filters). -- **Fallback**: Performs a `LIKE` search across all searchable fields if no index exists. - -```json -{ - "q": "Annual Meeting", - "and": [{ "field": "enable", "op": "eq", "value": true }] + return await post_object({ api_cfg, endpoint, params: { enabled }, data: search_query }); } ``` --- -## 4. Create, Update, & Delete +## 4. Hosted File Management (Modern V3 Actions) -### A. Create (POST) -V3 validates incoming JSON against the `mdl_in` Pydantic model. +V3 uses specialized **"Action"** routes for binary operations to separate processing logic from standard metadata CRUD. -```ts -// POST /v3/crud/{obj_type}/ -export async function create_ae_obj_v3({ api_cfg, obj_type, data }) { - const endpoint = `/v3/crud/${obj_type}/`; - return await post_object({ api_cfg, endpoint, data }); -} -``` +### A. Upload Action +**Path**: `POST /v3/action/hosted_file/upload` +**Format**: `multipart/form-data` -### B. Update (PATCH) -V3 uses `PATCH` for partial updates. Only provided fields are modified. +| Field | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| `file_list` | File[] | Yes | One or more files to upload. | +| `account_id` | String | Yes | Random ID of the owner account. | +| `link_to_type`| String | Yes | Object type to link (e.g., `archive_content`). | +| `link_to_id` | String | Yes | Random ID of the object to link. | -```ts -// PATCH /v3/crud/{obj_type}/{obj_id} -export async function update_ae_obj_v3({ api_cfg, obj_type, obj_id, data }) { - const endpoint = `/v3/crud/${obj_type}/${obj_id}`; - return await patch_object({ api_cfg, endpoint, data }); -} -``` +**Query Parameters:** +- `allowed_extensions`: Whitelist check (e.g., `?allowed_extensions=png&allowed_extensions=jpg`). +- `delay_ms`: Simulate network delay for UI testing. -### C. Delete (DELETE) -Supports soft-delete (`method=hide`) or hard-delete (`method=delete`). +**Features:** +- **Automatic Deduplication:** API checks SHA256 hashes. If a file exists on the server, it reuses the record and creates a new link instead of a duplicate upload. +- **Relational Integrity:** Automatically creates `hosted_file_link` records. -```ts -// DELETE /v3/crud/{obj_type}/{obj_id}?method=hide -export async function delete_ae_obj_v3({ api_cfg, obj_type, obj_id, method = 'delete' }) { - const endpoint = `/v3/crud/${obj_type}/${obj_id}`; - return await delete_object({ api_cfg, endpoint, params: { method } }); -} -``` +### B. Download & Streaming Action +**Path**: `GET /v3/action/hosted_file/{id}/download` + +**Features:** +- **Streaming:** Supports standard `Range` headers for large files and video seeking. +- **Auth Bypass:** Use `?site_key=` to download without an API Key header or JWT (useful for public kiosks). +- **Testing:** Supports `delay_ms` query parameter. + +### C. Deletion & Cleanup Action +**Path**: `DELETE /v3/action/hosted_file/{id}` + +| Parameter | Type | Default | Description | +| :--- | :--- | :--- | :--- | +| `link_to_type`| Query | null | The type of the link to remove. | +| `link_to_id` | Query | null | The random ID of the link to remove. | +| `rm_orphan` | Query | false | If true, physically delete file if no links remain. | +| `method` | Query | `hide` | Cleanup method: `hide`, `disable`, or `delete` (hard). | +| `fake_delete` | Query | false | **Testing Mode:** Verifies file/record existence without modification. | --- -## 5. Specialized Endpoints +## 5. Hosted File Management (Legacy) -### A. Schema Discovery -**Path**: `GET /v3/crud/{obj_type}/schema` -Returns DB column definitions and Pydantic field rules for dynamic UI builders. +The following endpoints are maintained for backward compatibility but should be migrated to V3 Actions. -### B. Secure File Downloads (JWT in URL) -For browsers downloading files, you can pass the JWT directly in the query string. -- `GET /v3/crud/hosted_file/{id}/?jwt={token}` +| Method | Endpoint | Description | +| :--- | :--- | :--- | +| `POST` | `/hosted_file/upload_files` | Legacy multi-upload. | +| `GET` | `/hosted_file/{id}/download` | Legacy download. | +| `GET` | `/hosted_file/{id}/stream` | Legacy buffered streamer. | +| `DELETE` | `/hosted_file/{id}` | Legacy deletion. | --- ## 6. The "ID Vision" Standard (2026) -V3 uses a **String-Only ID Vision**. The frontend and external agents NEVER handle or store internal database integers. +V3 uses a **String-Only ID Vision**. The frontend NEVER handles or stores database integers. -### A. Automatic ID Mapping (Output) -Internal `id_random` fields are renamed to clean, short names in the JSON response. -- **`id`**: Unique random identifier for the record. -- **`account_id`**: Unique random identifier for the parent account. - -### B. Intelligent Mapping (Input/Search) -You can send string IDs back to the API using the same clean names. -- **Search**: Filter by `account_id` using the random string value. -- **Save**: Send `account_id: "nqOzejLCDXM"` in a payload; the API resolves it to an integer. +1. **Automatic Mapping:** Internal `id_random` fields are mapped to `id` and `account_id` in JSON responses. +2. **Intelligent Resolution:** You can send random string IDs back to the API in any `*_id` field; the API resolves them to integers before database insertion. --- -## 7. Standard Patterns & Pitfalls +## 7. Structured Error Handling -### A. Permissive Update Mode (Available) -To simplify state management, V3 supports a **Permissive Update Mode** which ignores unknown extra fields in the payload. -- **Header:** `x-ae-ignore-extra-fields: true` -- **Use when:** Sending back objects that contain read-only technical metadata. - -### B. Structured Error Handling (Rich Bubbling) -V3 returns machine-readable error objects in `meta.details` for failures (400/500). - -**Example Error Response:** -```json -{ - "meta": { - "success": false, - "status_code": 400, - "details": { - "category": "database_schema", - "code": 1054, - "message": "Unknown column 'notes' in 'SET'", - "recoverable": false - } - } -} -``` +V3 returns machine-readable error objects in `meta.details` for failures. **Common Categories:** -- **`database_duplicate`**: Non-unique value (Code 1062). -- **`database_constraint`**: Foreign key violation (Codes 1451, 1452). -- **`database_schema`**: Invalid column name (Codes 1054, 1146). -- **`database_connection`**: Temporary DB issues (`recoverable: true`). -- **`validation`**: Pydantic validation failed. - -### C. Booleans: 'NULL' vs '0' -**Pitfall:** Fields like `hide` can be `NULL` in the DB. -**Standard:** Handle `null`, `undefined`, and `0` identically for boolean checks. +- `database_duplicate`: Non-unique value (Code 1062). +- `database_constraint`: Foreign key violation (Codes 1451, 1452). +- `database_schema`: Invalid column name (Codes 1054, 1146). +- `validation`: Pydantic validation failed (Check `details` for field-specific errors). \ No newline at end of file