chore(tests): consolidate E2E test suite into standardized primary scripts
- Combined 10+ one-off tests into 4 primary functional suites (Search, Auth, Lifecycle, Vision). - Archived original scripts to tests/archive/. - Updated README with the new standardized inventory. - Applied clean output formatting across the new suite.
This commit is contained in:
Scott Idem
@@ -10,85 +10,35 @@ This directory contains the automated and manual test scripts for the Aether Fas
|
||||
- **`tools/`**: Utility scripts for administrative tasks like registry generation or Docker exploration.
|
||||
- **`archive/`**: Legacy or deprecated scripts kept for historical reference.
|
||||
|
||||
## 🛠️ Shared Helpers
|
||||
## 📜 Standardized E2E Suite (`tests/e2e/`)
|
||||
|
||||
- **`mock_config_helper.py`**: A critical utility that mocks `app.config.settings` before other modules are imported. Use this in unit tests to prevent the application from trying to load real configuration files during import.
|
||||
These consolidated scripts are the primary verification tool for the V3 API.
|
||||
|
||||
| Script | Description |
|
||||
| :--- | :--- |
|
||||
| `test_e2e_v3_search_engine.py` | **Primary Search**: Basic operators, Registry fields, Nested search, and Filter bypass. |
|
||||
| `test_e2e_v3_auth_security.py` | **Primary Auth**: Site bootstrap, Passcode-to-JWT, and permission boundaries. |
|
||||
| `test_e2e_v3_actions_file_lifecycle.py` | **Primary Actions**: Upload, Download (ID/Hash/Streaming), and physical Deletion. |
|
||||
| `test_e2e_v3_data_store_lookup.py` | **V3 Parity**: Verifies code-based lookups and latency simulation. |
|
||||
| `test_e2e_v3_event_vision_parity.py`| **Vision ID**: Verifies string-ID enforcement across event models. |
|
||||
| `test_e2e_v3_accounts.py` | CRUD verification for the core Account object. |
|
||||
| `test_e2e_v3_schema.py` | Network verification of the V3 metadata discovery endpoint. |
|
||||
| `test_e2e_agent_bridge.py` | Verifies container diagnostics and log streaming routes. |
|
||||
| `cleanup_test_files.py` | Utility to purge E2E artifacts from physical storage. |
|
||||
|
||||
---
|
||||
|
||||
## 📜 Script Inventory
|
||||
## 🛠️ Shared Helpers
|
||||
|
||||
### Unit Tests (`tests/unit/`)
|
||||
| Script | Description |
|
||||
| :--- | :--- |
|
||||
| `test_unit_email.py` | Unit tests for email logic with full SMTP mocking. |
|
||||
| `test_unit_errors.py` | Verifies the regex-based SQL error string cleanup logic. |
|
||||
| `test_unit_filtering.py` | Ensures virtual/view fields are correctly marked for DB exclusion. |
|
||||
| `test_unit_hosted_file_logic.py` | Validates Hosted File model aliasing and ID mapping. |
|
||||
| `test_unit_hosted_file_resolver.py` | Tests `lookup_id_random_pop` ID resolution logic. |
|
||||
| `test_unit_models.py` | Validates custom Pydantic validators (e.g., Person given_name). |
|
||||
| `test_unit_payload_sanitization.py` | **Primary Logic Test**: Verifies payload stripping and ID resolution. |
|
||||
| `test_unit_router_stripping.py` | Simulates automatic removal of random IDs during updates. |
|
||||
| `test_unit_schema_logic.py` | Verifies V3 schema metadata extraction logic. |
|
||||
| `test_unit_upload_files_flow.py` | Tests the logic flow of multi-file uploads. |
|
||||
| `test_unit_websockets_v3.py` | Unit tests for the V3 WebSocket manager and messages. |
|
||||
| `test_unit_websockets_v3_router.py` | Verifies the V3 WebSocket endpoint logic and routing. |
|
||||
|
||||
### Integration Tests (`tests/integration/`)
|
||||
| Script | Description |
|
||||
| :--- | :--- |
|
||||
| `debug_auth_dependency.py` | Direct test of `get_account_context_optional` logic. |
|
||||
| `test_int_boot_diagnosis.py` | Progressively imports modules to identify circular traps. |
|
||||
| `test_int_db_connectivity.py` | Direct SQLAlchemy connectivity test (bypasses app config). |
|
||||
| `test_int_email_live.py` | Live SMTP connection test (non-mocked). |
|
||||
| `test_int_hosted_file_upload.py` | Tests route flow using FastAPI `TestClient`. |
|
||||
| `test_int_import_verification.py` | Basic check that all V3 routers are reachable. |
|
||||
| `test_int_permissive_mode.py` | Tests `x-ae-ignore-extra-fields` header logic. |
|
||||
| `test_int_schema_v3.py` | Verifies enhanced schema discovery against real DB. |
|
||||
| `test_int_v3_auth_security.py` | Verifies auth bypass rules (Site vs Account). |
|
||||
| `test_ws_v3_ping.py` | **Primary Gateway Test**: Verifies WebSocket V3 round-trip. |
|
||||
|
||||
### E2E Tests (`tests/e2e/`)
|
||||
| Script | Description |
|
||||
| :--- | :--- |
|
||||
| `cleanup_test_files.py` | Utility to purge E2E artifacts from storage. |
|
||||
| `repro_intermittent_errors.py` | Stress test for intermittent 403/Timeout issues. |
|
||||
| `test_e2e_agent_bridge.py` | Verifies `/agent` diagnostics and log streaming. |
|
||||
| `test_e2e_jwt_guest_auth.py` | **Security Test**: Verifies safe guest token access. |
|
||||
| `test_e2e_passcode_auth.py` | **Security Test**: Verifies passcode-to-JWT flow. |
|
||||
| `test_e2e_site_bootstrap.py` | Verifies unauthenticated FQDN lookup. |
|
||||
| `test_e2e_v3_accounts.py` | CRUD verification for Account object. |
|
||||
| `test_e2e_v3_action_delete.py` | Verifies physical/record deletion via V3 Actions. |
|
||||
| `test_e2e_v3_action_download.py` | **Consolidated**: Tests ID Vision, Hash, and Partial DLs. |
|
||||
| `test_e2e_v3_action_event_file.py` | Tests atomic event_file upload action. |
|
||||
| `test_e2e_v3_action_upload.py` | Tests standard hosted_file upload action. |
|
||||
| `test_e2e_v3_data_store_lookup.py` | Verifies code lookup, delay, and POST search parity. |
|
||||
| `test_e2e_v3_event_device.py` | Vision ID parity test for Event Device. |
|
||||
| `test_e2e_v3_event_session.py` | Vision ID parity test for Event Session. |
|
||||
| `test_e2e_v3_extra_filters.py` | Tests complex filtering (enabled/all) on User/Site. |
|
||||
| `test_e2e_v3_nested_advanced.py` | Tests POST /search and view param on nested routes. |
|
||||
| `test_e2e_v3_registry_verify.py` | Verifies registry whitelist expansion. |
|
||||
| `test_e2e_v3_schema.py` | Validates V3 `/schema` endpoint over network. |
|
||||
| `test_e2e_v3_search.py` | **Primary API Test**: Verifies all search operators. |
|
||||
| `test_e2e_v3_security_exceptions.py` | Validates 403 blocks on restricted routes. |
|
||||
|
||||
### Tools & Utilities (`tests/tools/` or root)
|
||||
| Script | Description |
|
||||
| :--- | :--- |
|
||||
| `gen_test_jwt.py` | Local script to generate test tokens. |
|
||||
| `tool_generate_registry.py` | Exports the Aether object registry as JSON. |
|
||||
| `tool_mcp_docker_explorer.py` | Model Context Protocol tool for Docker container inspection. |
|
||||
- **`mock_config_helper.py`**: A critical utility that mocks `app.config.settings` before other modules are imported. Use this in unit tests.
|
||||
|
||||
---
|
||||
|
||||
## 🧹 Maintenance Policy
|
||||
|
||||
1. **Redundant Tests**: If a test is consolidated into a more comprehensive suite (e.g., `test_e2e_v3_hash_download.py` -> `test_e2e_v3_action_download.py`), the older one should be moved to `archive/`.
|
||||
2. **Naming Convention**:
|
||||
* `test_unit_*`: No database or network.
|
||||
* `test_int_*`: Uses local DB/Redis or `TestClient`.
|
||||
* `test_e2e_*`: Full network round-trip.
|
||||
3. **Cleanup**: Always use `tests/e2e/cleanup_test_files.py` after running upload/download tests to keep the dev storage clean.
|
||||
1. **Standardization**: All E2E tests should use the standard Agent API Key (`PMM4n50teUCaOMMTN8qOJA`) and provide clean `[✅ PASS]` or `[❌ FAIL]` output.
|
||||
2. **Archiving**: When a new specialized test is created, check if it can be combined into one of the "Primary" suites above. If so, combine and move the original to `archive/`.
|
||||
3. **Cleanup**: Always use `tests/e2e/cleanup_test_files.py` after running file lifecycle tests.
|
||||
|
||||
---
|
||||
|
||||
@@ -96,8 +46,8 @@ This directory contains the automated and manual test scripts for the Aether Fas
|
||||
|
||||
### Recommended: Use the project virtual environment
|
||||
```bash
|
||||
./environment/bin/python3 tests/unit/test_unit_payload_sanitization.py
|
||||
./environment/bin/python3 tests/e2e/test_e2e_v3_search_engine.py
|
||||
```
|
||||
|
||||
### Path Requirements
|
||||
Always run test scripts from the **project root** directory. Most scripts include `sys.path.append(os.getcwd())` to ensure local imports work correctly.
|
||||
Always run test scripts from the **project root** directory. Most scripts include `sys.path.append(os.getcwd())` to ensure local imports work correctly.
|
||||
Reference in New Issue
Block a user