Docs: Modernize main README, archive legacy/deprecated guides, and mark completed security/project docs (March 2026 review)

2026-03-18 16:16:20 -04:00
parent 74ad69bc63
commit 356f4b8efc
4 changed files with 432 additions and 58 deletions
--- a/README.md
+++ b/README.md
@@ -1,67 +1,96 @@
-# Aether API v3.00.x (FastAPI)

-The **Aether API** is a high-performance, multi-tenant backend infrastructure built using the Python **FastAPI** framework. It serves as the central data and logic hub for the Aether Platform, supporting both legacy applications and modern V3/V4 standards.
+# Aether API v3.x (FastAPI)
+
+The **Aether API** is a high-performance, multi-tenant backend for the Aether Platform, built on Python **FastAPI**. It powers both legacy and modern (V3/V4) applications, and is now fully containerized for robust, scalable deployment.
+
+---
+

 ---

 ## 🏗️ Architecture Overview

-The API is currently in a transitional state between legacy (V1/V2) patterns and the modern **V3 CRUD Architecture**.
+The API is in transition from legacy (V1/V2) to the modern **V3 CRUD Architecture**. All new development follows V3 standards.

-### **V3 CRUD (Modern)**
+
+### V3 CRUD (Modern)
 - **Path:** `/v3/crud/`
- **Core Principles:**
-    - **`id_random` Primary:** All public communication uses URL-safe string IDs. Internal integer IDs are hidden.
-    - **Nested URL Structure:** Enforces parent-child relationships (e.g., `/v3/crud/journal/{id}/entry/`).
-    - **Nested Advanced Search:** Full support for POST-based search on nested objects.
-    - **Granular Dependencies:** Uses specialized FastAPI dependencies for Account Context, Pagination, Filtering, and Serialization.
-    - **Advanced Search:** POST-based search with recursive logic and standardized operators.
-    - **Schema Discovery:** Dynamic introspection of database and Pydantic models via `/v3/crud/{obj_type}/schema`.
+- **Principles:**
+    - **String IDs:** All public APIs use `id_random` (URL-safe string IDs); internal integer IDs are hidden.
+    - **Nested URLs:** Parent-child relationships enforced in URL structure.
+    - **Advanced Search:** POST-based, recursive, with standardized operators.
+    - **Schema Discovery:** Dynamic model/database introspection at `/v3/crud/{obj_type}/schema`.
+    - **Granular Dependencies:** Specialized FastAPI dependencies for account context, pagination, filtering, serialization.

-### **V3 Actions**
+
+### V3 Actions
 - **Path:** `/v3/action/`
- Handles complex binary operations and atomic business logic separately from standard metadata CRUD.
- **Key Features:**
-    - **Atomic Event Uploads:** Marriage of physical storage and complex event relations in one request.
-    - **Content-Addressable Downloads:** Direct file retrieval by SHA256 hash for high-performance local caching.
-    - **Intelligent ID Resolution:** Standard download endpoints now automatically resolve container IDs (e.g., event_file) to underlying binaries.
+- Handles complex/atomic business logic and binary operations outside standard CRUD.
+- **Features:**
+    - **Atomic Event Uploads:** File storage + event relations in one request.
+    - **Content-Addressable Downloads:** SHA256-based file retrieval for high-performance caching.
+    - **Intelligent ID Resolution:** Download endpoints auto-resolve container IDs.

-### **Legacy API (V1/V2)**
- **Path:** `/`, `/api/`, `/crud/`, `/v2/crud/`
- Maintained for backward compatibility but currently being systematically audited and deprecated.
- **Deprecation System:** Accessing legacy routes triggers a `!!! DEPRECATED ROUTE ACCESSED` warning in logs.
+
+### Legacy API (V1/V2)
+- **Paths:** `/`, `/api/`, `/crud/`, `/v2/crud/`
+- Maintained for backward compatibility, but being systematically deprecated. Accessing legacy routes triggers a warning in logs.

 ---

+
 ## 🛠️ Core Technologies
- **Framework:** FastAPI (v0.95.1)
- **Database:** MariaDB (Remote Master) + SQLAlchemy (v1.4.52)
+- **Framework:** FastAPI (v0.95.1+)
+- **Database:** MariaDB (Docker, shared) + SQLAlchemy (v1.4.52)
 - **Caching/ID Resolution:** Redis
- **Security:** JWT (JSON Web Tokens) + API Key Machine Authorization
- **Logging:** Structured logging with module-level isolation and rotation.
+- **Security:** JWT (JSON Web Tokens), API Key Machine Auth
+- **Logging:** Structured, module-level, with rotation

 ---

-## 🚀 Getting Started

-### **Local Development**
-1.  **Environment:** Requires Python 3.9+.
-2.  **Setup:**
-    ```bash
-    virtualenv environment
-    source environment/bin/activate
-    pip install -r requirements.txt
-    ```
-3.  **Run:**
-    ```bash
-    uvicorn app.main:app --host 0.0.0.0 --port 5005 --reload
-    ```
- **Documentation:** [GUIDE__LOCAL_DEVELOPMENT.md](documentation/GUIDE__LOCAL_DEVELOPMENT.md)
+## 🚀 Quick Start
+
+The Aether API is designed for containerized deployment as part of the unified Aether Docker environment. For full-stack orchestration, see the documentation in the `aether_container_env` project.
+
+### Prerequisites
+- Docker & Docker Compose (for containerized use)
+- Python 3.9+ (for local-only development)
+
+### Local Development (Optional)
+You can run the API locally for debugging:
+```bash
+virtualenv environment
+source environment/bin/activate
+pip install -r requirements.txt
+uvicorn app.main:app --host 0.0.0.0 --port 5005 --reload
+```
+See [GUIDE__LOCAL_DEVELOPMENT.md](documentation/GUIDE__LOCAL_DEVELOPMENT.md) for details.
+
+### Docker Usage
+The API is run and managed via Docker Compose as part of the full Aether stack. Refer to the `aether_container_env` project for orchestration, environment setup, and advanced deployment instructions.
+
+### Service Endpoints (Default Ports)
+- **API Docs:** https://dev-api.oneskyit.com/docs
+- **Frontend:** http://localhost:8888
+- **phpMyAdmin:** http://localhost:8081 (if enabled)
+- **Logs (Dozzle):** http://localhost:8881
+
+
+---
+
+## 🗄️ Database & Backups
+
+All database operations are managed via Docker scripts in `aether_container_env/`:
+- **Backup:** `./backup_db.sh` (saves to `backups/`)
+- **Restore:** `./restore_db.sh [backup_file.gz]`
+- **Export:** `./export_db.sh` (conference-ready backup)
+- **Automated Import:** Drop file in `backups/import/` and run `./check_and_import.sh`
+
+See [GUIDE__DEPLOYMENT_MANUAL.md](documentation/GUIDE__DEPLOYMENT_MANUAL.md) for full deployment and backup/restore instructions.
+
+---

-### **Deployment**
- The API is deployed via **Docker Compose** within the **Aether Docker Environment** (`aether_container_env`).
- **Configuration (Docker)**: All settings (Database, SMTP, Ports) are managed via the master `.env` file in the `aether_container_env/` directory. No local `.env` file is required in this repository.
- **Manual Deployment:** [GUIDE__DEPLOYMENT_MANUAL.md](documentation/GUIDE__DEPLOYMENT_MANUAL.md)

 ---

@@ -82,28 +111,66 @@ The API is currently in a transitional state between legacy (V1/V2) patterns and

 ---

+
 ## 🧪 Testing Suite
-The project maintains an exhaustive test suite under the `tests/` directory.
- **Unit Tests:** `tests/unit/` (Mocked logic).
- **Integration Tests:** `tests/integration/` (Local DB/Redis connectivity).
- **E2E Tests:** `tests/e2e/` (Network-based API validation).
- **Documentation:** [tests/README.md](tests/README.md)
+Tests are under `tests/`:
+- **Unit:** `tests/unit/` (mocked logic)
+- **Integration:** `tests/integration/` (DB/Redis connectivity)
+- **E2E:** `tests/e2e/` (API validation)
+- **Docs:** [tests/README.md](tests/README.md)

 ---

-## 🚧 Current Status & Work in Progress

-### **Active Workstreams**
- **[Backend] API Deprecation:** Systematic pruning of orphaned routers and methods (ID: 111523094).
- **[ID Vision]:** Phase 2 complete. String-ID standardization extended to Page, Post, Person, Journal, Contact, and User models.
- **[V3 Migration]:** Implementation of atomic event actions and hash-based retrieval for high-performance Launcher caching complete.
+## 🚧 Status & Work in Progress
+
+### Active Workstreams
+- **API Deprecation:** Pruning orphaned routers/methods
+- **ID Vision:** String-ID standardization (Phase 2 complete)
+- **V3 Migration:** Atomic event actions, hash-based file retrieval
+
+### Known Issues
+- **Badge Rendering:** Corrupted numeric `id` fields in `event_badge_template` can cause template load failures
+- **Websockets:** Legacy modules need unification and stability improvements
+- **Intermittent Timeouts:** Some E2E tests occasionally reproduce 403s/timeouts on nested GET calls
+
+---

-### **Known Bugs / Issues**
- **Badge Rendering:**Corrupted numeric `id` fields in `event_badge_template` table causing template load failures in Svelte 5 views.
- **Websockets:** Legacy `websockets.py` and `websockets_redis.py` require unification and stability improvements.
- **Intermittent Timeouts:** Some E2E tests occasionally reproduce 403s/Timeouts on nested GET calls (investigating).

 ---

 ## 📜 Release Snapshot
-Current Baseline: **`release/2026-01-28-v3_prod-snapshot`** (Stable v3.0.99).
+Current Baseline: **`release/2026-01-28-v3_prod-snapshot`** (Stable v3.0.99)
+
+---
+
+## 🔐 Security & Access
+- **SSH Required:** All git operations now require SSH (Bitbucket app passwords deprecated June 2026). See your Gitea or Bitbucket account for adding SSH keys.
+- **Never commit secrets:** `.env` and credentials are git-ignored.
+- **JWT Key:** Ensure `AE_API_JWT_KEY` is unique and high-entropy in production.
+- **.env precedence:** API uses `.env` credentials for core infra (SMTP/DB) over DB settings.
+
+---
+
+## 🧑‍💻 Management & Operations
+- **Restart API:** `docker compose restart ae_api`
+- **Restart Frontend:** `docker compose restart ae_app`
+- **Rebuild everything:** `docker compose up -d --build`
+- **Logs:** http://localhost:8881 (Dozzle)
+- **phpMyAdmin:** http://localhost:8081 (if enabled)
+
+---
+
+## 🏠 Directory Map (Key Mounts)
+- `conf/` — Nginx/Gunicorn config templates
+- `logs/` — Centralized logs
+- `srv/` — Data/source code mounts
+- `scripts/` — Automation scripts
+- `backups/` — MariaDB snapshots
+
+---
+
+## 📝 Notes
+- For multi-stack setups, ensure unique `AE_NETWORK_NAME` and `CONTAINER_` prefixes in `.env`.
+- All stacks must connect to `aether_shared_net` for shared DB/Redis.
+- See Docker env README and CHEATSHEET for advanced orchestration and troubleshooting.