From a0767b1c692efad28957effd1a1873e221cce40f Mon Sep 17 00:00:00 2001 From: Scott Idem Date: Wed, 18 Mar 2026 16:16:58 -0400 Subject: [PATCH] Saving documentation updates and clean up. --- documentation/GUIDE__DEPLOYMENT_MANUAL.md | 6 ++ documentation/PLAN__SECURITY_HARDENING.md | 59 ------------- .../PROJECT__Aether_API_Websockets_v3.md | 88 ------------------- 3 files changed, 6 insertions(+), 147 deletions(-) delete mode 100644 documentation/PLAN__SECURITY_HARDENING.md delete mode 100644 documentation/PROJECT__Aether_API_Websockets_v3.md diff --git a/documentation/GUIDE__DEPLOYMENT_MANUAL.md b/documentation/GUIDE__DEPLOYMENT_MANUAL.md index d0279f3..58b0ce8 100644 --- a/documentation/GUIDE__DEPLOYMENT_MANUAL.md +++ b/documentation/GUIDE__DEPLOYMENT_MANUAL.md @@ -1,3 +1,9 @@ + +# DEPRECATED: Manual Server Deployment Guide (Non-Docker) + +> **Notice (March 2026):** +> This manual deployment guide is deprecated. The standard and supported method for deploying the Aether API is now via Docker Compose, as described in the main README and the `aether_container_env` documentation. Use this guide only for legacy or advanced manual setups. + # Manual Server Deployment Guide (Non-Docker) This guide describes the manual process for deploying the Aether API on a Linux server using Nginx, Gunicorn, and Systemd. diff --git a/documentation/PLAN__SECURITY_HARDENING.md b/documentation/PLAN__SECURITY_HARDENING.md deleted file mode 100644 index 710e51c..0000000 --- a/documentation/PLAN__SECURITY_HARDENING.md +++ /dev/null @@ -1,59 +0,0 @@ -# Project: API Security Hardening (V3) - -**Status:** Draft / Planning -**Date:** Jan 18, 2026 -**Owner:** Scott / Aether API Team - -## 1. Executive Summary -This project aims to close a critical security vulnerability in the Aether API V3 dependencies where the `x_no_account_id` header allows unauthorized "Superuser/Bypass" access without validation. Additionally, it addresses the lack of cryptographic verification for JWTs in the V3 CRUD layer. - -## 2. The Vulnerabilities - -### A. The "Bypass Header" (Critical) -* **Issue:** The `get_account_context_optional` dependency in `app/routers/dependencies_v3.py` accepts a header `x_no_account_id`. If present, it sets `auth_method = 'bypass'` and grants full administrative privileges (`super=True`). -* **Note:** This header may be sent as `x_no_account_id` or `x-no-account-id`. FastAPI's `Header` logic should handle the conversion, but validation must be explicit. -* **Current State:** There is **no** validation of this header. Sending `x_no_account_id: anything` works. -* **Risk:** Total system compromise via simple header injection. - -### B. JWT Verification (High) -* **Issue:** The system validates users by looking up the `x_account_id` string in the database (Redis/MariaDB). -* **Current State:** It does **not** cryptographically verify the JWT signature or expiration in the V3 dependency chain. -* **Risk:** Account impersonation if an Account ID is leaked. - -## 3. Implementation Plan - -### Phase 1: Secure the Bypass Header (Immediate) -**Goal:** Restrict "Bypass Mode" to clients possessing a valid, active API Key. - -1. **Refactor Dependencies:** - * Update `get_account_context_optional` to accept `x_aether_api_key`. - * **Logic Change:** If `x_no_account_id` header is detected: - * **Require** `x_aether_api_key`. - * **Lookup** key in `api_key` table. - * **Verify** `enable = 1` and `now()` is between `enable_from` and `enable_to`. - * **Failure Behavior:** If key is invalid/missing, log warning and deny 'bypass' status (fall back to 'guest'). - -2. **Verification:** - * Test `curl` with header only -> Expect 403/Forbidden. - * Test `curl` with header + valid Key -> Expect 200/OK. - * Test Frontend File Download (uses `x_no_account_id_token` param) -> Expect 200/OK (No regression). - -### Phase 2: JWT Verification (Subsequent) -**Goal:** Replace DB-lookup auth with cryptographic JWT validation. - -1. **Audit:** Confirm `app/lib_jwt.py` logic is sound. -2. **Middleware/Dependency:** Integrate `decode_jwt` into the `get_account_context` flow. -3. **Transition:** Allow dual-mode (DB lookup OR JWT) for a transition period if necessary, or cut over if frontend sends valid JWTs. - -## 4. Impact Analysis -* **Frontend (SvelteKit):** - * Audit confirmed no usage of `x_no_account_id` **header** in active source code. - * Usage of `x_no_account_id_token` (Query Param) is **safe** and distinct from the header logic. -* **Scripts/External Tools:** - * Any external scripts using the "Bypass" header must be updated to send a valid API Key. - -## 5. Action Items -- [ ] Create `PROJECT_SECURITY_HARDENING.md` (This document). -- [ ] Refactor `app/routers/dependencies_v3.py`. -- [ ] Verify fix with `curl` tests. -- [ ] Commit changes. diff --git a/documentation/PROJECT__Aether_API_Websockets_v3.md b/documentation/PROJECT__Aether_API_Websockets_v3.md deleted file mode 100644 index 9dd8867..0000000 --- a/documentation/PROJECT__Aether_API_Websockets_v3.md +++ /dev/null @@ -1,88 +0,0 @@ -# Project: Aether API WebSockets V3 - -## 1. Overview -The goal of WebSockets V3 is to provide a high-performance, scalable, and standardized real-time communication layer for the Aether Platform. This version focuses on efficient message routing using Redis granular Pub/Sub, integration with the **Vision ID** (string-based) pattern, and strict data validation via Pydantic. - -The primary use case is **Group Coordination**: allowing a "controller" client to send commands or messages to one or more "worker" clients within the same group. - -## 2. Analysis of Previous Versions - -### V1: `websockets.py` (Memory-Based) -* **Mechanism**: Maintained a list of `WebSocket` objects in a Python list (`active_connections`). -* **Limitation**: Did not scale across multiple Docker containers. Clients on instance A could not communicate with clients on instance B. -* **Feature**: Basic support for `direct`, `group`, and `broadcast`. - -### V2: `websockets_redis.py` (Global Pub/Sub) -* **Mechanism**: Uses `redis.asyncio` to publish all messages to a single `channel:ws`. -* **Limitation**: **"Noisy Neighbor" Problem**. Every API instance receives *every* message sent across the entire platform and must filter them in Python code (`if data.get('target') == 'group'`). This wastes CPU and network bandwidth at scale. -* **Feature**: Solved multi-instance connectivity. - -## 3. V3 Architecture: Granular Pub/Sub - -### Granular Redis Channels -V3 will move filtering from Python to Redis by using specific channel names. A client will subscribe only to the channels relevant to them: -1. **Client Channel**: `ws:client:{client_id_random}` (For Direct Messages) -2. **Group Channel**: `ws:group:{group_id_random}` (For Group Messages) -3. **Global Channel**: `ws:broadcast` (For System-wide Messages) - -### Vision ID Integration -* All IDs in the WebSocket path and payload will be string-based `id_random` values. -* Path format: `/v3/ws/group/{group_id_random}/client/{client_id_random}` - -### Standardized Message Schema -All V3 messages will follow a strict Pydantic model to ensure consistency between different device types. - -```python -class WS_Message_V3(BaseModel): - version: str = "3" - msg_type: str # 'msg', 'cmd', 'heartbeat', 'presence' - target: str # 'direct', 'group', 'broadcast', 'echo' - from_id: str # client_id_random - to_id: Optional[str] # target client_id_random (for direct) - group_id: Optional[str] # target group_id_random (for group) - cmd: Optional[str] # Specific command string - msg: Optional[str] # Human-readable message - payload: Dict[str, Any] # Flexible JSON data - sent_at: datetime -``` - -## 4. Backend Implementation Plan - -### Phase 1: Library Layer (`app/lib_websockets_v3.py`) -* Define the `WS_Message_V3` model. -* Implement `WS_Manager_V3` to handle Redis connections and channel string generation. -* Add presence tracking using Redis Sets (`SADD` on connect, `SREM` on disconnect). - -### Phase 2: Router Layer (`app/routers/websockets_v3.py`) -* Implement the `/v3/ws/...` endpoint. -* **Receiver Loop**: Receives JSON from client -> Validates -> Publishes to correct Redis channel. -* **Sender Loop**: Listens to multiple Redis channels -> Forwards messages to the client. - -### Phase 3: Integration -* Register the router in `app/routers/registry.py`. -* Ensure legacy endpoints (`/ws/group/...`) remain functional in `websockets_redis.py`. - -## 5. Frontend Integration & Changes - -The frontend will need several updates to support the V3 protocol: - -1. **Connection URL**: Update connection logic to use the `/v3/` prefix. - * *Old*: `ws://api.domain.com/ws/group/{id}/client/{id}` - * *New*: `ws://api.domain.com/v3/ws/group/{id}/client/{id}` -2. **Payload Wrapping**: All outgoing messages must be wrapped in the `WS_Message_V3` structure. - * Instead of sending raw text or simple JSON, send the structured object. -3. **Targeting Logic**: - * To send to the group, set `target: "group"`. - * To send to one specific device, set `target: "direct"` and provide `to_id`. -4. **Heartbeats**: The frontend should ideally send a `msg_type: "heartbeat"` every 30-60 seconds to keep the connection alive and update presence in Redis. -5. **Response Handling**: Incoming messages will now have a consistent shape, making it easier to route data to internal app state or components. - -## 6. Security & Safety -* **API Key Verification**: WebSocket handshakes should optionally verify the `X-Aether-API-Key` during the upgrade request. -* **Isolation**: V3 will use its own Redis database or a strict prefixing strategy to ensure messages never bleed into legacy channels. -* **Error Handling**: Standardize the close codes (e.g., 4000 for invalid message schema). - -## 7. Verification Plan -* Create `tests/e2e/test_e2e_v3_websockets.py`. -* Use `websockets` python library to simulate multiple concurrent clients. -* Test cross-instance communication (if possible in the test environment).