chore(tests): reorganize test suite and archive redundant scripts

- Moved legacy/redundant tests to tests/archive/.
- Relocated root-level debug scripts to tests/integration/.
- Updated tests/README.md with final organized inventory.
- Cleaned up root directory from one-off reproduction scripts.

tests/README.md

@@ -5,7 +5,7 @@ This directory contains the automated and manual test scripts for the Aether Fas
 ## 📁 Directory Structure
 
 - **`unit/`**: Isolated logic tests. These use heavy mocking to bypass database and network requirements. Fast and safe to run in any environment.
-- **`integration/`**: Local environment tests. These require a connection to the local MariaDB/Redis instance and verify how components interact.
+- **`integration/`**: Local environment tests. These verify component interactions, often requiring a connection to the local MariaDB/Redis instance.
 - **`e2e/` (End-to-End)**: Network-based API tests. these use the `requests` library to call the live API endpoints at `https://dev-api.oneskyit.com`.
 - **`tools/`**: Utility scripts for administrative tasks like registry generation or Docker exploration.
 - **`archive/`**: Legacy or deprecated scripts kept for historical reference.
@@ -22,49 +22,76 @@ This directory contains the automated and manual test scripts for the Aether Fas
 | Script | Description |
 | :--- | :--- |
 | `test_unit_email.py` | Unit tests for email logic with full SMTP mocking. |
-| `test_unit_email_safe.py` | Secondary safe-mode email verification logic. |
 | `test_unit_errors.py` | Verifies the regex-based SQL error string cleanup logic. |
 | `test_unit_filtering.py` | Ensures virtual/view fields are correctly marked for DB exclusion. |
+| `test_unit_hosted_file_logic.py` | Validates Hosted File model aliasing and ID mapping. |
+| `test_unit_hosted_file_resolver.py` | Tests `lookup_id_random_pop` ID resolution logic. |
 | `test_unit_models.py` | Validates custom Pydantic validators (e.g., Person given_name). |
 | `test_unit_payload_sanitization.py` | **Primary Logic Test**: Verifies payload stripping and ID resolution. |
 | `test_unit_router_stripping.py` | Simulates automatic removal of random IDs during updates. |
-| `test_unit_schema_logic.py` | Verifies V3 schema metadata extraction logic with mocked DB rows. |
+| `test_unit_schema_logic.py` | Verifies V3 schema metadata extraction logic. |
-| `test_unit_websockets_v3.py` | Unit tests for the V3 WebSocket manager and message models. |
+| `test_unit_upload_files_flow.py` | Tests the logic flow of multi-file uploads. |
-| `test_unit_websockets_v3_router.py` | Verifies the V3 WebSocket endpoint logic and message routing. |
+| `test_unit_websockets_v3.py` | Unit tests for the V3 WebSocket manager and messages. |
+| `test_unit_websockets_v3_router.py` | Verifies the V3 WebSocket endpoint logic and routing. |
 
 ### Integration Tests (`tests/integration/`)
 | Script | Description |
 | :--- | :--- |
-| `test_int_boot_diagnosis.py` | Progressively imports modules to identify circular dependency traps. |
+| `debug_auth_dependency.py` | Direct test of `get_account_context_optional` logic. |
+| `test_int_boot_diagnosis.py` | Progressively imports modules to identify circular traps. |
 | `test_int_db_connectivity.py` | Direct SQLAlchemy connectivity test (bypasses app config). |
 | `test_int_email_live.py` | Live SMTP connection test (non-mocked). |
+| `test_int_hosted_file_upload.py` | Tests route flow using FastAPI `TestClient`. |
 | `test_int_import_verification.py` | Basic check that all V3 routers are reachable. |
-| `test_int_schema_v3.py` | Verifies the enhanced schema discovery output against the real DB. |
+| `test_int_permissive_mode.py` | Tests `x-ae-ignore-extra-fields` header logic. |
-| `test_int_v3_auth_security.py` | Uses `TestClient` to verify auth bypass rules (Site vs Account). |
+| `test_int_schema_v3.py` | Verifies enhanced schema discovery against real DB. |
-| `test_ws_v3_ping.py` | **Primary Gateway Test**: Verifies WebSocket V3 round-trip through Nginx and Redis. |
+| `test_int_v3_auth_security.py` | Verifies auth bypass rules (Site vs Account). |
-| `test_int_websockets_v3_real.py` | **Isolation Test**: Simulates 3 real clients to verify Group, Direct, and Broadcast routing. |
+| `test_ws_v3_ping.py` | **Primary Gateway Test**: Verifies WebSocket V3 round-trip. |
 
 ### E2E Tests (`tests/e2e/`)
 | Script | Description |
 | :--- | :--- |
-| `test_e2e_agent_bridge.py` | Verifies the `/agent` diagnostics and log streaming endpoints. |
+| `cleanup_test_files.py` | Utility to purge E2E artifacts from storage. |
-| `test_e2e_jwt_guest_auth.py` | **Security Test**: Verifies safe guest token minting and whitelisted access. |
+| `repro_intermittent_errors.py` | Stress test for intermittent 403/Timeout issues. |
-| `test_e2e_legacy_remote_schema.py` | Remote check for legacy schema compatibility. |
+| `test_e2e_agent_bridge.py` | Verifies `/agent` diagnostics and log streaming. |
-| `test_e2e_site_bootstrap.py` | Verifies the unauthenticated FQDN lookup for site initialization. |
+| `test_e2e_jwt_guest_auth.py` | **Security Test**: Verifies safe guest token access. |
-| `test_e2e_v3_accounts.py` | CRUD verification for the Account object via network. |
+| `test_e2e_passcode_auth.py` | **Security Test**: Verifies passcode-to-JWT flow. |
-| `test_e2e_v3_extra_filters.py` | Tests complex filtering (enabled/all) on User and Site objects. |
+| `test_e2e_site_bootstrap.py` | Verifies unauthenticated FQDN lookup. |
-| `test_e2e_v3_schema.py` | Validates the V3 `/schema` endpoint over the network. |
+| `test_e2e_v3_accounts.py` | CRUD verification for Account object. |
-| `test_e2e_v3_search.py` | **Primary API Test**: Verifies all search operators and JWT auth. |
+| `test_e2e_v3_action_delete.py` | Verifies physical/record deletion via V3 Actions. |
-| `test_e2e_v3_security_exceptions.py` | Validates that restricted routes correctly return 403 Forbidden. |
+| `test_e2e_v3_action_download.py` | **Consolidated**: Tests ID Vision, Hash, and Partial DLs. |
+| `test_e2e_v3_action_event_file.py` | Tests atomic event_file upload action. |
+| `test_e2e_v3_action_upload.py` | Tests standard hosted_file upload action. |
+| `test_e2e_v3_data_store_lookup.py` | Verifies code lookup, delay, and POST search parity. |
+| `test_e2e_v3_event_device.py` | Vision ID parity test for Event Device. |
+| `test_e2e_v3_event_session.py` | Vision ID parity test for Event Session. |
+| `test_e2e_v3_extra_filters.py` | Tests complex filtering (enabled/all) on User/Site. |
+| `test_e2e_v3_nested_advanced.py` | Tests POST /search and view param on nested routes. |
+| `test_e2e_v3_registry_verify.py` | Verifies registry whitelist expansion. |
+| `test_e2e_v3_schema.py` | Validates V3 `/schema` endpoint over network. |
+| `test_e2e_v3_search.py` | **Primary API Test**: Verifies all search operators. |
+| `test_e2e_v3_security_exceptions.py` | Validates 403 blocks on restricted routes. |
 
-### Tools (`tests/tools/`)
+### Tools & Utilities (`tests/tools/` or root)
 | Script | Description |
 | :--- | :--- |
-| `tool_generate_registry.py` | Exports the Aether object registry (`obj_type_kv_li`) as JSON. |
+| `gen_test_jwt.py` | Local script to generate test tokens. |
+| `tool_generate_registry.py` | Exports the Aether object registry as JSON. |
 | `tool_mcp_docker_explorer.py` | Model Context Protocol tool for Docker container inspection. |
 
 ---
 
+## 🧹 Maintenance Policy
+
+1.  **Redundant Tests**: If a test is consolidated into a more comprehensive suite (e.g., `test_e2e_v3_hash_download.py` -> `test_e2e_v3_action_download.py`), the older one should be moved to `archive/`.
+2.  **Naming Convention**:
+    *   `test_unit_*`: No database or network.
+    *   `test_int_*`: Uses local DB/Redis or `TestClient`.
+    *   `test_e2e_*`: Full network round-trip.
+3.  **Cleanup**: Always use `tests/e2e/cleanup_test_files.py` after running upload/download tests to keep the dev storage clean.
+
+---
+
 ## 🚀 How to Run
 
 ### Recommended: Use the project virtual environment
@@ -74,27 +101,3 @@ This directory contains the automated and manual test scripts for the Aether Fas
 
 ### Path Requirements
 Always run test scripts from the **project root** directory. Most scripts include `sys.path.append(os.getcwd())` to ensure local imports work correctly.
-
----
-
-## 💡 Best Practices & Reminders
-
-1.  **Check Before Creating**: Always check the **Script Inventory** above to see if a test for your logic already exists, or find a similar one to use as a reference/template.
-2.  **Docker & Service Restarts**: Remember that the Aether Platform runs in Docker. If you modify core application code (e.g., in `app/`), you must restart the FastAPI service for changes to take effect:
-    ```bash
-    docker restart aether_container_env-ae_api-2
-    ```
-    (Note: Restarts are NOT necessary if you are only modifying the test scripts themselves).
-3.  **Clean Up**: Clean up any temporary or debug files created during testing. However, **keep your test scripts**! Refactor them slightly for future use and clarity so they remain valuable assets for the project.
-4.  **Host Header Routing**: When testing through the Nginx gateway (port 5060), ensure your client uses a recognized Host header (e.g., `fastapi.localhost`). Nginx uses this to decide which proxy block to use.
-5.  **Stay Current**: Update this `README.md` when you add new tests or learn something that could help others. This is a living document; keep the **Script Inventory** and tips up to date.
-6.  **Commit Often**: Don't forget to commit your working code and tests before moving on to the next task!
-
----
-
-## 🛠️ Troubleshooting Common Test Errors
-
--   **HTTP 404 (WebSockets)**: Nginx doesn't recognize the path. Check the `location` blocks in the Nginx `.conf` files.
--   **HTTP 400 (WebSockets)**: Usually a Host header mismatch. Nginx is hitting the default "localhost" block instead of the API proxy.
--   **Connection Refused (Redis)**: The script is likely running on the host but trying to hit `localhost:6379` while Redis is only reachable inside the Docker network or mapped to a different port.
--   **Hanging Scripts**: If using `TestClient` with multiple WebSockets, the script may dead-lock. Use an asynchronous library like `websockets` or `httpx` for multi-client tests.