860cf80a4e
- Overhauled README.md to serve as a unified system index and WIP tracker. - Standardized documentation filenames (ARCH__, GUIDE__, PLAN__) for better discoverability. - Archived completed project plans and scopes. - Fixed regressions in unit tests (errors, models, email) caused by V3 architectural updates. - Ensured unit tests remain non-destructive and environment-independent via mocking.
75 lines
3.6 KiB
Markdown
75 lines
3.6 KiB
Markdown
# Aether V3: Hosted File System Migration Plan
|
|
|
|
## 1. Overview
|
|
The goal of this project is to migrate the existing `hosted_file` and `hosted_file_link` logic into the **CRUD V3 Architecture**. This involves splitting the system into a **Standard Record Layer** (metadata) and a **Specialized Action Layer** (binary handling).
|
|
|
|
## 2. Core Requirements
|
|
- **Relational Integrity:** Fully utilize `hosted_file_link` for all object associations.
|
|
- **Deduplication:** Automatic filesystem and DB hash-checks before creating new records.
|
|
- **Cleanup:** Intelligent "Orphan" removal logic via `rm_orphan` flag.
|
|
- **Flexible Auth:** Support uploads/downloads with JWT, without JWT (Guest), and via URL-key fallback (bypass API Key requirement).
|
|
- **Binary Support:** High-performance streaming, byte-range seeking, and multi-file POST handling.
|
|
- **Developer DX:** Integrated `delay_ms` simulation and extension whitelisting.
|
|
|
|
---
|
|
|
|
## 3. Implementation Phases (Bite-Sized Chunks)
|
|
|
|
### Phase 1: V3 Metadata Baseline
|
|
*Status: Ready to start*
|
|
- Whitelist `hosted_file` and `hosted_file_link` in `obj_type_kv_li`.
|
|
- Verify standard V3 Search works for files (filtering by account, hash, etc.).
|
|
- Enable `PATCH /v3/crud/hosted_file/{id}` for metadata updates (title, description).
|
|
- Implement "Fake Delete" using standard `DELETE ...?method=hide`.
|
|
|
|
### Phase 2: V3 Action Router Scaffolding
|
|
- Create `app/routers/api_v3_actions_hosted_file.py`.
|
|
- Implement `delay_ms` middleware/logic for action routes.
|
|
- Implement specialized Extension Validator.
|
|
|
|
### Phase 3: Enhanced Binary Actions
|
|
- **Download Action:** Port the streamer logic to `/v3/action/hosted_file/{id}/download`.
|
|
- Add URL-param fallback for API Key/Auth bypass.
|
|
- **Upload Action:** Implement `/v3/action/hosted_file/upload`.
|
|
- Support both single and `List[UploadFile]`.
|
|
- Implement the Hash-Lookup-Before-Write logic.
|
|
|
|
### Phase 4: Relational Cleanup & Linking
|
|
- **Relational Delete Logic:**
|
|
- Implement `DELETE /v3/action/hosted_file/{id}`.
|
|
- Support `method` parameter: `hide`, `disable`, `delete` (hard).
|
|
- **Orphan Check:** Logic to count remaining links; if `rm_orphan=True` and count is 0, physically remove file and parent record.
|
|
- **Fake Delete (Test Mode):**
|
|
- Specialized mode for testing frontend workflows without data loss.
|
|
- Logic:
|
|
1. Verify `hosted_file` record existence.
|
|
2. Verify physical file existence on server.
|
|
3. Verify `hosted_file_link` existence.
|
|
4. Return 200 OK success response *without* executing the actual SQL DELETE or `os.unlink`.
|
|
|
|
---
|
|
|
|
## 4. Technical Architecture
|
|
|
|
### Standard CRUD Routes (Metadata)
|
|
| Method | Endpoint | Description |
|
|
| :--- | :--- | :--- |
|
|
| `POST` | `/v3/crud/hosted_file/search` | Find files by hash, name, or account. |
|
|
| `PATCH` | `/v3/crud/hosted_file/{id}` | Update title, description, or notes. |
|
|
| `DELETE` | `/v3/crud/hosted_file/{id}` | Soft-delete (Hide) the file record. |
|
|
|
|
### Specialized Action Routes (Binary)
|
|
| Method | Endpoint | Description |
|
|
| :--- | :--- | :--- |
|
|
| `POST` | `/v3/action/hosted_file/upload` | Upload 1+ files; handles deduplication. |
|
|
| `GET` | `/v3/action/hosted_file/{id}/download` | Stream binary data; supports range seeking. |
|
|
| `DELETE` | `/v3/action/hosted_file/{id}` | Removes link; optionally deletes orphan file. |
|
|
|
|
---
|
|
|
|
## 5. Testing & Verification Strategy
|
|
For every chunk, we will create/update:
|
|
1. **Logic Test:** Unit test for the internal method (e.g., `lookup_file_hash`).
|
|
2. **E2E Test:** Live network test against the dev API to verify real record creation and file persistence.
|
|
3. **Security Test:** Verification of the "Bypass" modes (Site Key / URL Key).
|