V3 Migration Phase 1: Stabilize Hosted File models, IDs, and whitelisting. Added comprehensive verification tests.

documentation/V3_HOSTED_FILE_MIGRATION_PLAN.md

@@ -0,0 +1,74 @@
+# Aether V3: Hosted File System Migration Plan
+
+## 1. Overview
+The goal of this project is to migrate the existing `hosted_file` and `hosted_file_link` logic into the **CRUD V3 Architecture**. This involves splitting the system into a **Standard Record Layer** (metadata) and a **Specialized Action Layer** (binary handling).
+
+## 2. Core Requirements
+- **Relational Integrity:** Fully utilize `hosted_file_link` for all object associations.
+- **Deduplication:** Automatic filesystem and DB hash-checks before creating new records.
+- **Cleanup:** Intelligent "Orphan" removal logic via `rm_orphan` flag.
+- **Flexible Auth:** Support uploads/downloads with JWT, without JWT (Guest), and via URL-key fallback (bypass API Key requirement).
+- **Binary Support:** High-performance streaming, byte-range seeking, and multi-file POST handling.
+- **Developer DX:** Integrated `delay_ms` simulation and extension whitelisting.
+
+---
+
+## 3. Implementation Phases (Bite-Sized Chunks)
+
+### Phase 1: V3 Metadata Baseline
+*Status: Ready to start*
+- Whitelist `hosted_file` and `hosted_file_link` in `obj_type_kv_li`.
+- Verify standard V3 Search works for files (filtering by account, hash, etc.).
+- Enable `PATCH /v3/crud/hosted_file/{id}` for metadata updates (title, description).
+- Implement "Fake Delete" using standard `DELETE ...?method=hide`.
+
+### Phase 2: V3 Action Router Scaffolding
+- Create `app/routers/api_v3_actions_hosted_file.py`.
+- Implement `delay_ms` middleware/logic for action routes.
+- Implement specialized Extension Validator.
+
+### Phase 3: Enhanced Binary Actions
+- **Download Action:** Port the streamer logic to `/v3/action/hosted_file/{id}/download`.
+    - Add URL-param fallback for API Key/Auth bypass.
+- **Upload Action:** Implement `/v3/action/hosted_file/upload`.
+    - Support both single and `List[UploadFile]`.
+    - Implement the Hash-Lookup-Before-Write logic.
+
+### Phase 4: Relational Cleanup & Linking
+- **Relational Delete Logic:**
+    - Implement `DELETE /v3/action/hosted_file/{id}`.
+    - Support `method` parameter: `hide`, `disable`, `delete` (hard).
+    - **Orphan Check:** Logic to count remaining links; if `rm_orphan=True` and count is 0, physically remove file and parent record.
+- **Fake Delete (Test Mode):**
+    - Specialized mode for testing frontend workflows without data loss.
+    - Logic:
+        1. Verify `hosted_file` record existence.
+        2. Verify physical file existence on server.
+        3. Verify `hosted_file_link` existence.
+        4. Return 200 OK success response *without* executing the actual SQL DELETE or `os.unlink`.
+
+---
+
+## 4. Technical Architecture
+
+### Standard CRUD Routes (Metadata)
+| Method | Endpoint | Description |
+| :--- | :--- | :--- |
+| `POST` | `/v3/crud/hosted_file/search` | Find files by hash, name, or account. |
+| `PATCH` | `/v3/crud/hosted_file/{id}` | Update title, description, or notes. |
+| `DELETE` | `/v3/crud/hosted_file/{id}` | Soft-delete (Hide) the file record. |
+
+### Specialized Action Routes (Binary)
+| Method | Endpoint | Description |
+| :--- | :--- | :--- |
+| `POST` | `/v3/action/hosted_file/upload` | Upload 1+ files; handles deduplication. |
+| `GET` | `/v3/action/hosted_file/{id}/download` | Stream binary data; supports range seeking. |
+| `DELETE` | `/v3/action/hosted_file/{id}` | Removes link; optionally deletes orphan file. |
+
+---
+
+## 5. Testing & Verification Strategy
+For every chunk, we will create/update:
+1. **Logic Test:** Unit test for the internal method (e.g., `lookup_file_hash`).
+2. **E2E Test:** Live network test against the dev API to verify real record creation and file persistence.
+3. **Security Test:** Verification of the "Bypass" modes (Site Key / URL Key).