docs(event_badge): comment why enable is omitted on import updates

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.gitignore

@@ -142,3 +142,6 @@ myapp/files/
 myapp/file_distribution/
 temp/
 tmp/
+
+# Added 2026-03-23
+gunicorn.ctl

Dockerfile

@@ -30,6 +30,7 @@ RUN pip freeze > /tmp/aether_fastapi_requirements_current.txt
 
 # Docker health check — verifies DB + Redis connectivity via the /health route.
 HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
-    CMD curl -f http://localhost/health || exit 1
+    # CMD curl -f http://localhost/health || exit 1
+    CMD curl -f http://localhost:5005/health || exit 1
 
 CMD ["gunicorn", "--conf", "/conf/gunicorn_fastapi_conf.py"]

README.md

@@ -1,67 +1,96 @@
-# Aether API v3.00.x (FastAPI)
 
-The **Aether API** is a high-performance, multi-tenant backend infrastructure built using the Python **FastAPI** framework. It serves as the central data and logic hub for the Aether Platform, supporting both legacy applications and modern V3/V4 standards.
+# Aether API v3.00.20 (FastAPI)
+
+The **Aether API** is a high-performance, multi-tenant backend for the Aether Platform, built on Python **FastAPI**. It powers both legacy and modern (V3/V4) applications, and is now fully containerized for robust, scalable deployment.
+
+---
+
 
 ---
 
 ## 🏗️ Architecture Overview
 
-The API is currently in a transitional state between legacy (V1/V2) patterns and the modern **V3 CRUD Architecture**.
+The API is in transition from legacy (V1/V2) to the modern **V3 CRUD Architecture**. All new development follows V3 standards.
 
-### **V3 CRUD (Modern)**
+
+### V3 CRUD (Modern)
 - **Path:** `/v3/crud/`
-- **Core Principles:**
-    - **`id_random` Primary:** All public communication uses URL-safe string IDs. Internal integer IDs are hidden.
-    - **Nested URL Structure:** Enforces parent-child relationships (e.g., `/v3/crud/journal/{id}/entry/`).
-    - **Nested Advanced Search:** Full support for POST-based search on nested objects.
-    - **Granular Dependencies:** Uses specialized FastAPI dependencies for Account Context, Pagination, Filtering, and Serialization.
-    - **Advanced Search:** POST-based search with recursive logic and standardized operators.
-    - **Schema Discovery:** Dynamic introspection of database and Pydantic models via `/v3/crud/{obj_type}/schema`.
+- **Principles:**
+    - **String IDs:** All public APIs use `id_random` (URL-safe string IDs); internal integer IDs are hidden.
+    - **Nested URLs:** Parent-child relationships enforced in URL structure.
+    - **Advanced Search:** POST-based, recursive, with standardized operators.
+    - **Schema Discovery:** Dynamic model/database introspection at `/v3/crud/{obj_type}/schema`.
+    - **Granular Dependencies:** Specialized FastAPI dependencies for account context, pagination, filtering, serialization.
 
-### **V3 Actions**
+
+### V3 Actions
 - **Path:** `/v3/action/`
-- Handles complex binary operations and atomic business logic separately from standard metadata CRUD.
-- **Key Features:**
-    - **Atomic Event Uploads:** Marriage of physical storage and complex event relations in one request.
-    - **Content-Addressable Downloads:** Direct file retrieval by SHA256 hash for high-performance local caching.
-    - **Intelligent ID Resolution:** Standard download endpoints now automatically resolve container IDs (e.g., event_file) to underlying binaries.
+- Handles complex/atomic business logic and binary operations outside standard CRUD.
+- **Features:**
+    - **Atomic Event Uploads:** File storage + event relations in one request.
+    - **Content-Addressable Downloads:** SHA256-based file retrieval for high-performance caching.
+    - **Intelligent ID Resolution:** Download endpoints auto-resolve container IDs.
 
-### **Legacy API (V1/V2)**
-- **Path:** `/`, `/api/`, `/crud/`, `/v2/crud/`
-- Maintained for backward compatibility but currently being systematically audited and deprecated.
-- **Deprecation System:** Accessing legacy routes triggers a `!!! DEPRECATED ROUTE ACCESSED` warning in logs.
+
+### Legacy API (V1/V2)
+- **Paths:** `/`, `/api/`, `/crud/`, `/v2/crud/`
+- Maintained for backward compatibility, but being systematically deprecated. Accessing legacy routes triggers a warning in logs.
 
 ---
 
+
 ## 🛠️ Core Technologies
-- **Framework:** FastAPI (v0.95.1)
-- **Database:** MariaDB (Remote Master) + SQLAlchemy (v1.4.52)
+- **Framework:** FastAPI (v0.95.1+)
+- **Database:** MariaDB (Docker, shared) + SQLAlchemy (v1.4.52)
 - **Caching/ID Resolution:** Redis
-- **Security:** JWT (JSON Web Tokens) + API Key Machine Authorization
-- **Logging:** Structured logging with module-level isolation and rotation.
+- **Security:** JWT (JSON Web Tokens), API Key Machine Auth
+- **Logging:** Structured, module-level, with rotation
 
 ---
 
-## 🚀 Getting Started
 
-### **Local Development**
-1.  **Environment:** Requires Python 3.9+.
-2.  **Setup:**
+## 🚀 Quick Start
+
+The Aether API is designed for containerized deployment as part of the unified Aether Docker environment. For full-stack orchestration, see the documentation in the `aether_container_env` project.
+
+### Prerequisites
+- Docker & Docker Compose (for containerized use)
+- Python 3.9+ (for local-only development)
+
+### Local Development (Optional)
+You can run the API locally for debugging:
 ```bash
 virtualenv environment
 source environment/bin/activate
 pip install -r requirements.txt
-    ```
-3.  **Run:**
-    ```bash
 uvicorn app.main:app --host 0.0.0.0 --port 5005 --reload
 ```
-- **Documentation:** [GUIDE__LOCAL_DEVELOPMENT.md](documentation/GUIDE__LOCAL_DEVELOPMENT.md)
+See [GUIDE__LOCAL_DEVELOPMENT.md](documentation/GUIDE__LOCAL_DEVELOPMENT.md) for details.
+
+### Docker Usage
+The API is run and managed via Docker Compose as part of the full Aether stack. Refer to the `aether_container_env` project for orchestration, environment setup, and advanced deployment instructions.
+
+### Service Endpoints (Default Ports)
+- **API Docs:** https://dev-api.oneskyit.com/docs
+- **Frontend:** http://localhost:8888
+- **phpMyAdmin:** http://localhost:8081 (if enabled)
+- **Logs (Dozzle):** http://localhost:8881
+
+
+---
+
+## 🗄️ Database & Backups
+
+All database operations are managed via Docker scripts in `aether_container_env/`:
+- **Backup:** `./backup_db.sh` (saves to `backups/`)
+- **Restore:** `./restore_db.sh [backup_file.gz]`
+- **Export:** `./export_db.sh` (conference-ready backup)
+- **Automated Import:** Drop file in `backups/import/` and run `./check_and_import.sh`
+
+See [GUIDE__DEPLOYMENT_MANUAL.md](documentation/GUIDE__DEPLOYMENT_MANUAL.md) for full deployment and backup/restore instructions.
+
+---
 
-### **Deployment**
-- The API is deployed via **Docker Compose** within the **Aether Docker Environment** (`aether_container_env`).
-- **Configuration (Docker)**: All settings (Database, SMTP, Ports) are managed via the master `.env` file in the `aether_container_env/` directory. No local `.env` file is required in this repository.
-- **Manual Deployment:** [GUIDE__DEPLOYMENT_MANUAL.md](documentation/GUIDE__DEPLOYMENT_MANUAL.md)
 
 ---
 
@@ -82,28 +111,66 @@ The API is currently in a transitional state between legacy (V1/V2) patterns and
 
 ---
 
+
 ## 🧪 Testing Suite
-The project maintains an exhaustive test suite under the `tests/` directory.
-- **Unit Tests:** `tests/unit/` (Mocked logic).
-- **Integration Tests:** `tests/integration/` (Local DB/Redis connectivity).
-- **E2E Tests:** `tests/e2e/` (Network-based API validation).
-- **Documentation:** [tests/README.md](tests/README.md)
+Tests are under `tests/`:
+- **Unit:** `tests/unit/` (mocked logic)
+- **Integration:** `tests/integration/` (DB/Redis connectivity)
+- **E2E:** `tests/e2e/` (API validation)
+- **Docs:** [tests/README.md](tests/README.md)
 
 ---
 
-## 🚧 Current Status & Work in Progress
 
-### **Active Workstreams**
-- **[Backend] API Deprecation:** Systematic pruning of orphaned routers and methods (ID: 111523094).
-- **[ID Vision]:** Phase 2 complete. String-ID standardization extended to Page, Post, Person, Journal, Contact, and User models.
-- **[V3 Migration]:** Implementation of atomic event actions and hash-based retrieval for high-performance Launcher caching complete.
+## 🚧 Status & Work in Progress
+
+### Active Workstreams
+- **API Deprecation:** Pruning orphaned routers/methods
+- **ID Vision:** String-ID standardization (Phase 2 complete)
+- **V3 Migration:** Atomic event actions, hash-based file retrieval
+
+### Known Issues
+- **Badge Rendering:** Corrupted numeric `id` fields in `event_badge_template` can cause template load failures
+- **Websockets:** Legacy modules need unification and stability improvements
+- **Intermittent Timeouts:** Some E2E tests occasionally reproduce 403s/timeouts on nested GET calls
+
+---
 
-### **Known Bugs / Issues**
-- **Badge Rendering:**Corrupted numeric `id` fields in `event_badge_template` table causing template load failures in Svelte 5 views.
-- **Websockets:** Legacy `websockets.py` and `websockets_redis.py` require unification and stability improvements.
-- **Intermittent Timeouts:** Some E2E tests occasionally reproduce 403s/Timeouts on nested GET calls (investigating).
 
 ---
 
 ## 📜 Release Snapshot
-Current Baseline: **`release/2026-01-28-v3_prod-snapshot`** (Stable v3.0.99).
+Current Baseline: **`release/2026-01-28-v3_prod-snapshot`** (Stable v3.0.99)
+
+---
+
+## 🔐 Security & Access
+- **SSH Required:** All git operations now require SSH (Bitbucket app passwords deprecated June 2026). See your Gitea or Bitbucket account for adding SSH keys.
+- **Never commit secrets:** `.env` and credentials are git-ignored.
+- **JWT Key:** Ensure `AE_API_JWT_KEY` is unique and high-entropy in production.
+- **.env precedence:** API uses `.env` credentials for core infra (SMTP/DB) over DB settings.
+
+---
+
+## 🧑‍💻 Management & Operations
+- **Restart API:** `docker compose restart ae_api`
+- **Restart Frontend:** `docker compose restart ae_app`
+- **Rebuild everything:** `docker compose up -d --build`
+- **Logs:** http://localhost:8881 (Dozzle)
+- **phpMyAdmin:** http://localhost:8081 (if enabled)
+
+---
+
+## 🏠 Directory Map (Key Mounts)
+- `conf/` — Nginx/Gunicorn config templates
+- `logs/` — Centralized logs
+- `srv/` — Data/source code mounts
+- `scripts/` — Automation scripts
+- `backups/` — MariaDB snapshots
+
+---
+
+## 📝 Notes
+- For multi-stack setups, ensure unique `AE_NETWORK_NAME` and `CONTAINER_` prefixes in `.env`.
+- All stacks must connect to `aether_shared_net` for shared DB/Redis.
+- See Docker env README and CHEATSHEET for advanced orchestration and troubleshooting.

app/config.py

@@ -1,6 +1,5 @@
 # Configuration for the Aether FastAPI application.
 # All settings are read directly from environment variables (injected by Docker via .env).
-# Previously this file was mounted from aether_container_env/conf/aether_api_config.py.
 from pydantic import BaseSettings, Field
 from typing import Any, Dict, List
 
@@ -28,6 +27,8 @@ class Settings(BaseSettings):
     # Connection tuning
     DB_CONNECT_TIMEOUT:  int = Field(20,   env='AE_DB_CONNECTION_TIMEOUT')
     DB_POOL_RECYCLE:     int = Field(1800, env='AE_DB_POOL_RECYCLE')
+    DB_POOL_SIZE:        int = Field(10,   env='AE_DB_POOL_SIZE')
+    DB_POOL_MAX_OVERFLOW: int = Field(20,  env='AE_DB_POOL_MAX_OVERFLOW')
 
     # --- Logging ---
     LOG_PATH_APP: str = Field('/logs/aether_api.log', env='AE_API_LOG_PATH')
@@ -76,6 +77,8 @@ class Settings(BaseSettings):
             'password':        self.DB_PASS,
             'connect_timeout':  self.DB_CONNECT_TIMEOUT,
             'pool_recycle':     self.DB_POOL_RECYCLE,
+            'pool_size':        self.DB_POOL_SIZE,
+            'max_overflow':     self.DB_POOL_MAX_OVERFLOW,
         }
 
     @property
@@ -103,7 +106,7 @@ class Settings(BaseSettings):
         }
 
     class Config:
-        case_sensitive = True
+        case_sensitive = False
 
 
 settings = Settings()

app/lib_api_crud_v3.py

@@ -8,6 +8,19 @@ from app.models.error_models import StandardError
 
 log = logging.getLogger(__name__)
 
+def apply_vision_id_fix(resp_data: dict, obj_type: str, by_alias: bool) -> dict:
+    """
+    V3 contract: {obj_type}_id in responses must be the random string, never the DB integer.
+    Applies to models not yet migrated to the Vision ID pattern (root_validator).
+    Safe to call on already-migrated models — no-op if the value is already a string.
+    """
+    _id_key = f'{obj_type}_id' if by_alias else 'id'
+    _rand_key = f'{obj_type}_id_random' if by_alias else 'id_random'
+    if isinstance(resp_data.get(_id_key), int) and resp_data.get(_rand_key):
+        resp_data[_id_key] = resp_data[_rand_key]
+    return resp_data
+
+
 def format_db_error(raw_error: str) -> StandardError:
     """
     Parses raw SQLAlchemy/MariaDB errors into structured StandardError objects.
@@ -43,6 +56,12 @@ def format_db_error(raw_error: str) -> StandardError:
         recoverable = True
     elif code in [1054, 1146]: # Unknown column / Table
         category = "database_schema"
+    elif code == 1364: # Field has no default value — model/schema mismatch
+        category = "database_schema"
+        field_match = re.search(r"Field '([^']+)' doesn't have a default value", message)
+        if field_match:
+            field_name = field_match.group(1)
+            message = f"Schema mismatch: column '{field_name}' is NOT NULL with no default but was not included in the insert. Check the model definition and database schema."
     else:
         category = "database"
 

app/lib_sql_core.py

@@ -43,9 +43,15 @@ def create_ae_engine(uri: str):
 
 engine = create_ae_engine(db_uri)
 
-# DEPRECATED: Global shared 'db' connection. Use engine.connect() in context managers instead.
-# Keeping for legacy compatibility but will phase out usage in crud lib.
+# DEPRECATED: Global shared 'db' connection. Still used by lib_schema_v3.py and lib_api_crud_v3.py.
+# TODO (P3 full fix): migrate those two call sites to engine.connect() context managers, then remove this.
+# Bare connect guarded so a Docker startup race (MariaDB not yet ready) doesn't crash the worker.
+# If this fails, db=None — callers that hit it before reconnect_db() runs will raise AttributeError.
+try:
     db = engine.connect()
+except Exception:
+    log.warning("DB SQL Core: Initial db connection failed at startup (MariaDB not ready?). Will retry via reconnect_db().")
+    db = None
 
 log.info('DB SQL Core: Initializing engine...')
 

app/lib_sql_crud.py

@@ -11,7 +11,7 @@ from sqlalchemy.exc import IntegrityError, OperationalError, ProgrammingError
 from app.log import log, logger_reset
 # CRITICAL: Import the core module to access current global state
 from app import lib_sql_core
-from app.lib_sql_core import sql_connect, set_last_sql_error
+from app.lib_sql_core import set_last_sql_error
 
 # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
 
@@ -63,11 +63,29 @@ def sql_insert(
                 return result_insert.lastrowid
             return False
     except IntegrityError as e:
+        # Data constraint violation (duplicate key, FK mismatch, NOT NULL) — do NOT retry;
+        # the same data would fail again. Return None so callers can distinguish from errors.
         if trans: trans.rollback()
         log.error('Integrity error (likely duplicate). Returning None')
         log.debug(e)
         set_last_sql_error(e)
         return None
+    except OperationalError:
+        # Transient connection failure. The broken connection rolls back on MariaDB's side,
+        # so retrying with a fresh connection is safe.
+        if trans: trans.rollback()
+        log.warning('Operational error in sql_insert. Retrying once with fresh connection...')
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                trans = conn.begin()
+                result_insert = conn.execute(sql_insert_stmt, data)
+                trans.commit()
+                if result_insert.rowcount == 1 and result_insert.lastrowid > 0:
+                    return result_insert.lastrowid
+                return False
+        except Exception as e:
+            set_last_sql_error(e)
+            return False
     except Exception as e:
         if trans: trans.rollback()
         log.error('Unknown exception in sql_insert. Returning False')
@@ -138,7 +156,6 @@ def sql_update(
     except OperationalError:
         if trans: trans.rollback()
         log.error('Operational error (gone away?). Retrying once...')
-        sql_connect()
         try:
             with lib_sql_core.engine.connect() as conn:
                 trans = conn.begin()
@@ -199,6 +216,19 @@ def sql_insert_or_update(
             res = conn.execute(stmt, data)
             trans.commit()
             return res.lastrowid if res.lastrowid > 0 else True
+    except OperationalError:
+        # ON DUPLICATE KEY UPDATE is idempotent — safe to retry.
+        if trans: trans.rollback()
+        log.warning('Operational error in sql_insert_or_update. Retrying once...')
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                trans = conn.begin()
+                res = conn.execute(stmt, data)
+                trans.commit()
+                return res.lastrowid if res.lastrowid > 0 else True
+        except Exception as e:
+            set_last_sql_error(e)
+            return False
     except Exception as e:
         if trans: trans.rollback()
         log.exception(e)
@@ -309,6 +339,21 @@ def sql_select(
                 return [] if as_list else None
 
             rows = result.all()
+    except OperationalError:
+        # Transient connection failure — reads are always safe to retry.
+        log.error('Operational error in sql_select. Retrying once with fresh connection...')
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                result = conn.execute(stmt, data)
+                if not result:
+                    return [] if as_list else None
+                if hasattr(result, 'returns_rows') and not result.returns_rows:
+                    return [] if as_list else None
+                rows = result.all()
+        except Exception as e:
+            log.error(f"SQL Fetch Error on retry: {e}")
+            set_last_sql_error(e)
+            return False
     except Exception as e:
         log.error(f"SQL Fetch Error: {e}")
         set_last_sql_error(e)
@@ -343,7 +388,6 @@ def run_sql_select(
             return conn.execute(sql, data)
     except (OperationalError, ProgrammingError) as e:
         log.error(f'DB Error: {e}. Retrying once...')
-        sql_connect()
         try:
             with lib_sql_core.engine.connect() as conn:
                 return conn.execute(sql, data)

app/lib_sql_search.py

@@ -199,7 +199,11 @@ def sql_search_qry_part(
                     if hasattr(item, 'field'):
                         clause, item_data = process_filter(item)
                         node_clauses.append(clause); data.update(item_data)
-                    else: node_clauses.append(f"({process_node(item, current_depth + 1)})")
+                    else:
+                        # Recurse into nested SearchQuery; only append if non-empty
+                        sub_clause = process_node(item, current_depth + 1)
+                        if sub_clause:
+                            node_clauses.append(f"({sub_clause})")
                 if node_clauses:
                     joiner = ' AND ' if 'and' in filter_attr else ' OR '
                     clauses.append(f"({joiner.join(node_clauses)})")
@@ -261,6 +265,18 @@ def sql_search_qry_part(
                     except Exception as e:
                         log.warning(f"Failed to resolve random ID for field {target_field}: {e}")
 
+        # site_domain: 'access_key' is a virtual field.
+        # site_access_key (site-level) takes priority; fall back to site_domain_access_key
+        # when site_access_key is not set (NULL or empty).
+        if target_field == 'access_key' and table_name and 'site_domain' in table_name:
+            sql_op = operator_map.get(f.op.lower())
+            if not sql_op: raise HTTPException(status_code=400, detail=f"Unsupported operator: {f.op}")
+            p1, p2 = get_param_name(), get_param_name()
+            return (
+                f"(site_access_key {sql_op} :{p1} OR "
+                f"((site_access_key IS NULL OR site_access_key = '') AND site_domain_access_key {sql_op} :{p2}))"
+            ), {p1: f.value, p2: f.value}
+
         if searchable_fields is not None and target_field not in searchable_fields:
             # Fallback check for original field just in case
             if f.field not in searchable_fields:

app/main.py

@@ -26,7 +26,7 @@ from app.db_sql import sql_select, reset_redis, reconnect_db
 from app.lib_config_v3 import bootstrap_db_config, validate_critical_config
 
 
-print('### **** *** ** * The Aether API v4 using FastAPI is loading... * ** *** **** ###')
+print('### **** *** ** * The Aether API v3.0 using FastAPI is loading... * ** *** **** ###')
 
 
 log = logging.getLogger(__name__)
@@ -42,7 +42,7 @@ async def lifespan(app: FastAPI):
     """
     # 1. Initialize Logging early but safely
     setup_logging(config.settings)
-    log.info('### **** *** ** * Aether API v4 using FastAPI - Startup Lifespan Initiated * ** *** **** ###')
+    log.info('### **** *** ** * Aether API v3.0 using FastAPI - Startup Lifespan Initiated * ** *** **** ###')
 
     # 2. Bootstrapping Configuration from DB with robust error handling
     log.info("Bootstrapping Configuration...")
@@ -82,21 +82,21 @@ async def lifespan(app: FastAPI):
     # 3. Final validation of critical infrastructure
     validate_critical_config(config.settings)
 
-    log.info('### **** *** ** * Aether API v4 using FastAPI - Startup Sequence Complete * ** *** **** ###')
+    log.info('### **** *** ** * Aether API v3.0 using FastAPI - Startup Sequence Complete * ** *** **** ###')
 
     yield
 
     # Shutdown logic
-    log.info('### **** *** ** * Aether API v4 using FastAPI - Shutdown Lifespan Initiated * ** *** **** ###')
+    log.info('### **** *** ** * Aether API v3.0 using FastAPI - Shutdown Lifespan Initiated * ** *** **** ###')
     log.info('The Aether FastAPI API is shutting down...')
 
 
-print('### **** *** ** * Aether API v4 using FastAPI - About to try FastAPI() while loading... * ** *** **** ###')
+print('### **** *** ** * Aether API v3.0 using FastAPI - About to try FastAPI() while loading... * ** *** **** ###')
 app = FastAPI(
     # debug = True,
     title = 'Aether API',
-    description = 'One Sky IT\'s Aether API v4 using FastAPI.',
-    version = '3.00.01',
+    description = 'One Sky IT\'s Aether API v3.0 using FastAPI.',
+    version = '3.00.10',
     operationsSorter = 'method',
     lifespan = lifespan,
     )

app/methods/e_novi_mailman_methods.py

@@ -1,50 +1,50 @@
 import json, requests
 from typing import Dict, List, Optional
-from app.db_sql import sql_select, sql_update, sql_insert
+from app.db_sql import sql_select
 from app.lib_general import log, logging, logger_reset
 
 # ---------------------------------------------------------------------------
-# Novi-Mailman Bridge
-# Synchronizes Novi AMS membership data with Mailman 3 mailing lists.
+# Novi-Mailman Bridge — IDAA
 #
-# Credential Storage (data_store table):
-#   code='novi_api_config'     → JSON: { "api_key": "...", "base_url": "https://..." }
-#   code='mailman_api_config'  → JSON: { "base_url": "http://...:8001", "username": "...", "password": "..." }
+# Credentials live in site.cfg_json for the IDAA site (id_random='58_gJESdlUh').
+# Novi keys already present:
+#   novi_api_root_url  — e.g. "https://www.idaa.org/api"
+#   novi_idaa_api_key  — Base64 API key (Basic auth)
 #
-# Sync Logic:
-#   - Active Novi members → subscribed in target Mailman list(s)
-#   - Lapsed/expired members → unsubscribed (or held, depending on policy)
-#   - Driven by webhooks (Novi membership events) + optional full-sync
+# Keys that must be added to cfg_json before Mailman or webhooks can work:
+#   mailman_base_url     — e.g. "http://lists.idaa.org:8001"
+#   mailman_username     — Mailman REST admin user (usually "restadmin")
+#   mailman_password     — Mailman REST admin password
+#   mailman_list_id      — Target list, e.g. "members@idaa.org"
+#   novi_webhook_secret  — Shared secret for HMAC-SHA256 webhook validation
 # ---------------------------------------------------------------------------
 
-# ── Credential Helpers ────────────────────────────────────────────────────
+IDAA_SITE_ID_RANDOM = '58_gJESdlUh'
+
+
+# ── Config Helper ─────────────────────────────────────────────────────────
 
 @logger_reset
-def _load_novi_config() -> Optional[Dict]:
-    """Load Novi AMS API credentials from data_store (code='novi_api_config')."""
-    rec = sql_select(table_name='data_store', field_name='code', field_value='novi_api_config')
-    if not rec:
-        log.error("Novi API config not found in data_store (code='novi_api_config').")
+def _load_idaa_cfg() -> Optional[Dict]:
+    """
+    Load IDAA site cfg_json.  Returns the parsed dict, or None on failure.
+    """
+    from app.methods.site_methods import load_site_obj
+    site = load_site_obj(site_id=IDAA_SITE_ID_RANDOM, model_as_dict=True)
+    if not site:
+        log.error("Could not load IDAA site record (id_random='%s').", IDAA_SITE_ID_RANDOM)
         return None
+    cfg = site.get('cfg_json')
+    if isinstance(cfg, str):
         try:
-        return json.loads(rec['text'])
+            cfg = json.loads(cfg)
         except Exception as e:
-        log.error(f"Failed to parse Novi config: {e}")
+            log.error("Failed to parse IDAA cfg_json: %s", e)
             return None
-
-
-@logger_reset
-def _load_mailman_config() -> Optional[Dict]:
-    """Load Mailman 3 REST API credentials from data_store (code='mailman_api_config')."""
-    rec = sql_select(table_name='data_store', field_name='code', field_value='mailman_api_config')
-    if not rec:
-        log.error("Mailman API config not found in data_store (code='mailman_api_config').")
-        return None
-    try:
-        return json.loads(rec['text'])
-    except Exception as e:
-        log.error(f"Failed to parse Mailman config: {e}")
+    if not isinstance(cfg, dict):
+        log.error("IDAA cfg_json is not a dict after parsing.")
         return None
+    return cfg
 
 
 # ── Novi AMS Methods ──────────────────────────────────────────────────────
@@ -52,26 +52,35 @@ def _load_mailman_config() -> Optional[Dict]:
 @logger_reset
 def test_novi_connection() -> Dict:
     """
-    Verify Novi AMS API credentials are valid.
-    Returns a dict with 'ok' bool and optional error message.
+    Verify Novi AMS API credentials from IDAA site cfg_json.
+    Uses the first group GUID in novi_idaa_group_guid_li as a lightweight auth probe.
+    Returns {'ok': True, 'member_count': N} on success.
     """
-    config = _load_novi_config()
-    if not config:
-        return {"ok": False, "error": "Credentials not configured."}
+    cfg = _load_idaa_cfg()
+    if not cfg:
+        return {"ok": False, "error": "Could not load IDAA site config."}
+
+    base_url = cfg.get('novi_api_root_url', '').rstrip('/')
+    api_key  = cfg.get('novi_idaa_api_key', '')
+
+    if not base_url or not api_key:
+        return {"ok": False, "error": "novi_api_root_url or novi_idaa_api_key missing from cfg_json."}
+
+    group_guid_li = cfg.get('novi_idaa_group_guid_li') or []
+    if not group_guid_li:
+        return {"ok": False, "error": "novi_idaa_group_guid_li missing from cfg_json."}
 
-    # Novi uses Basic auth with a Base64-encoded API key.
-    # Confirmed from IDAA Jitsi integration: Authorization: Basic {api_key}
-    base_url = config.get('base_url', '').rstrip('/')
-    api_key  = config.get('api_key', '')
     headers = {"Authorization": f"Basic {api_key}", "Accept": "application/json"}
-
     try:
-        resp = requests.get(f"{base_url}/api/v1/members", headers=headers, params={"$top": 1}, timeout=10)
+        # Use first group as a lightweight auth probe (pageSize=1)
+        guid = group_guid_li[0]
+        resp = requests.get(f"{base_url}/groups/{guid}/members",
+                            headers=headers, params={"pageSize": 1}, timeout=10)
         if resp.status_code == 200:
-            return {"ok": True}
+            return {"ok": True, "probe_group": guid}
         return {"ok": False, "error": f"HTTP {resp.status_code}: {resp.text[:200]}"}
     except Exception as e:
-        log.exception(f"Novi connection test failed: {e}")
+        log.exception("Novi connection test failed: %s", e)
         return {"ok": False, "error": str(e)}
 
 
@@ -80,46 +89,59 @@ def get_novi_members(status_filter: Optional[str] = None, page_size: int = 500,
     """
     Fetch member records from Novi AMS.
 
-    Args:
-        status_filter: Optional Novi membership status (e.g. 'Active', 'Lapsed').
-                       None returns all members.
-        page_size:     Records per page (Novi uses OData $top/$skip).
-        offset:        Pagination offset ($skip).
+    Novi has no flat member-list endpoint.  Members are fetched per group from
+    novi_idaa_group_guid_li, deduped by UniqueID, then each member's full record
+    (including Email) is fetched via GET /customers/{uuid}.
 
-    Returns:
-        List of member dicts, or None on failure.
-
-    TODO: Confirm OData filter field names against your Novi instance schema.
+    status_filter and pagination (page_size/offset) are not supported at the
+    Novi API level for this approach — all group members are returned.
     """
-    config = _load_novi_config()
-    if not config:
+    cfg = _load_idaa_cfg()
+    if not cfg:
         return None
 
-    base_url = config.get('base_url', '').rstrip('/')
-    api_key  = config.get('api_key', '')
+    base_url      = cfg.get('novi_api_root_url', '').rstrip('/')
+    api_key       = cfg.get('novi_idaa_api_key', '')
+    group_guid_li = cfg.get('novi_idaa_group_guid_li') or []
     headers       = {"Authorization": f"Basic {api_key}", "Accept": "application/json"}
-    params   = {"pageSize": page_size, "offset": offset}
 
-    if status_filter:
-        params["membershipStatus"] = status_filter
+    if not group_guid_li:
+        log.error("novi_idaa_group_guid_li missing from cfg_json.")
+        return None
 
-    # TODO: Confirm the bulk member list endpoint for this Novi instance.
-    # The IDAA Jitsi code uses /customers/{uuid} for individual lookups and
-    # /groups/{guid}/members for group membership. A bulk member list may be
-    # /members, /customers, or require a group-based approach.
+    # Step 1: collect unique member UUIDs across all configured groups
+    seen_uuids: set = set()
+    uuid_list: List[str] = []
+    for guid in group_guid_li:
         try:
-        resp = requests.get(f"{base_url}/members", headers=headers, params=params, timeout=30)
+            resp = requests.get(f"{base_url}/groups/{guid}/members",
+                                headers=headers, params={"pageSize": page_size}, timeout=30)
             if resp.status_code != 200:
-            log.error(f"Novi API error: {resp.status_code} - {resp.text[:200]}")
-            return None
-        data = resp.json()
-        # Novi may return array directly, or wrap in Results/Members key
-        if isinstance(data, list):
-            return data
-        return data.get('Results') or data.get('Members') or data.get('value') or []
+                log.error("Novi group %s fetch error: %s", guid, resp.status_code)
+                continue
+            for entry in resp.json():
+                uid = entry.get('UniqueID')
+                if uid and uid not in seen_uuids:
+                    seen_uuids.add(uid)
+                    uuid_list.append(uid)
         except Exception as e:
-        log.exception(f"Failed to fetch Novi members: {e}")
-        return None
+            log.exception("Failed to fetch Novi group %s: %s", guid, e)
+
+    log.info("Novi: %d unique members across %d group(s).", len(uuid_list), len(group_guid_li))
+
+    # Step 2: fetch full customer record (including Email) for each UUID
+    members: List[Dict] = []
+    for uid in uuid_list:
+        try:
+            resp = requests.get(f"{base_url}/customers/{uid}", headers=headers, timeout=10)
+            if resp.status_code == 200:
+                members.append(resp.json())
+            else:
+                log.warning("Novi customer %s fetch error: %s", uid, resp.status_code)
+        except Exception as e:
+            log.exception("Failed to fetch Novi customer %s: %s", uid, e)
+
+    return members
 
 
 # ── Mailman 3 Methods ─────────────────────────────────────────────────────
@@ -127,72 +149,101 @@ def get_novi_members(status_filter: Optional[str] = None, page_size: int = 500,
 @logger_reset
 def test_mailman_connection() -> Dict:
     """
-    Verify Mailman 3 REST API credentials are valid.
-    Returns a dict with 'ok' bool and optional error message.
+    Verify Mailman 3 REST API credentials from IDAA site cfg_json.
+    Returns {'ok': True, 'version': '...'} on success.
     """
-    config = _load_mailman_config()
-    if not config:
-        return {"ok": False, "error": "Credentials not configured."}
+    cfg = _load_idaa_cfg()
+    if not cfg:
+        return {"ok": False, "error": "Could not load IDAA site config."}
 
-    base_url = config.get('base_url', '').rstrip('/')
-    auth     = (config.get('username', 'restadmin'), config.get('password', ''))
+    base_url = cfg.get('mailman_base_url', '').rstrip('/')
+    username = cfg.get('mailman_username', 'restadmin')
+    password = cfg.get('mailman_password', '')
+
+    if not base_url or not password:
+        return {"ok": False, "error": "mailman_base_url or mailman_password missing from cfg_json."}
 
     try:
-        resp = requests.get(f"{base_url}/3.1/system/versions", auth=auth, timeout=10)
+        resp = requests.get(f"{base_url}/3.1/system/versions", auth=(username, password), timeout=10)
         if resp.status_code == 200:
             return {"ok": True, "version": resp.json().get('mailman_version')}
         return {"ok": False, "error": f"HTTP {resp.status_code}: {resp.text[:200]}"}
     except Exception as e:
-        log.exception(f"Mailman connection test failed: {e}")
+        log.exception("Mailman connection test failed: %s", e)
         return {"ok": False, "error": str(e)}
 
 
 @logger_reset
-def get_mailman_lists() -> Optional[List[Dict]]:
+def get_mailman_list_members(list_id: str, count: int = 100, page: int = 1) -> Optional[List[Dict]]:
     """
-    Return all mailing lists from this Mailman 3 instance.
+    Return members of a specific Mailman 3 list.
+
+    Args:
+        list_id: fqdn_listname e.g. 'mm3@idaa.org' or dot-notation 'mm3.idaa.org'
+        count:   page size (Mailman default 20, max typically 100)
+        page:    1-based page number
     """
-    config = _load_mailman_config()
-    if not config:
+    cfg = _load_idaa_cfg()
+    if not cfg:
         return None
 
-    base_url = config.get('base_url', '').rstrip('/')
-    auth     = (config.get('username', 'restadmin'), config.get('password', ''))
+    base_url    = cfg.get('mailman_base_url', '').rstrip('/')
+    auth        = (cfg.get('mailman_username', 'restadmin'), cfg.get('mailman_password', ''))
+    list_id_dot = list_id.replace('@', '.')
+
+    try:
+        resp = requests.get(
+            f"{base_url}/3.1/lists/{list_id_dot}/roster/member",
+            auth=auth,
+            params={"count": count, "page": page},
+            timeout=10,
+        )
+        if resp.status_code != 200:
+            log.error("Mailman member list fetch failed for %s: %s", list_id, resp.status_code)
+            return None
+        data = resp.json()
+        return data.get('entries', [])
+    except Exception as e:
+        log.exception("Failed to fetch members for list %s: %s", list_id, e)
+        return None
+
+
+@logger_reset
+def get_mailman_lists() -> Optional[List[Dict]]:
+    """Return all mailing lists from this Mailman 3 instance."""
+    cfg = _load_idaa_cfg()
+    if not cfg:
+        return None
+
+    base_url = cfg.get('mailman_base_url', '').rstrip('/')
+    auth     = (cfg.get('mailman_username', 'restadmin'), cfg.get('mailman_password', ''))
 
     try:
         resp = requests.get(f"{base_url}/3.1/lists", auth=auth, timeout=10)
         if resp.status_code != 200:
-            log.error(f"Mailman list fetch failed: {resp.status_code}")
+            log.error("Mailman list fetch failed: %s", resp.status_code)
             return None
         return resp.json().get('entries', [])
     except Exception as e:
-        log.exception(f"Failed to fetch Mailman lists: {e}")
+        log.exception("Failed to fetch Mailman lists: %s", e)
         return None
 
 
 @logger_reset
 def subscribe_member_to_list(list_id: str, email: str, display_name: str = '') -> bool:
     """
-    Subscribe an email address to a Mailman 3 list.
-    Uses pre-confirmed subscription (no confirmation email sent).
-
-    Args:
-        list_id:      Mailman list ID, e.g. 'members@yourdomain.org'
-        email:        Member email address
-        display_name: Optional display name
-
-    Returns:
-        True on success or already subscribed, False on error.
+    Subscribe an email address to a Mailman 3 list (pre-confirmed, no welcome email).
+    Returns True on success or already-subscribed, False on error.
     """
-    config = _load_mailman_config()
-    if not config:
+    cfg = _load_idaa_cfg()
+    if not cfg:
         return False
 
-    base_url = config.get('base_url', '').rstrip('/')
-    auth     = (config.get('username', 'restadmin'), config.get('password', ''))
+    base_url = cfg.get('mailman_base_url', '').rstrip('/')
+    auth     = (cfg.get('mailman_username', 'restadmin'), cfg.get('mailman_password', ''))
 
     payload = {
-        "list_id":            list_id.replace('@', '.'),  # Mailman uses dot notation for list IDs
+        "list_id":              list_id.replace('@', '.'),
         "subscriber":           email,
         "display_name":         display_name,
         "pre_verified":         True,
@@ -204,15 +255,15 @@ def subscribe_member_to_list(list_id: str, email: str, display_name: str = '') -
     try:
         resp = requests.post(f"{base_url}/3.1/members", auth=auth, json=payload, timeout=10)
         if resp.status_code in (200, 201):
-            log.info(f"Subscribed {email} to {list_id}")
+            log.info("Subscribed %s to %s", email, list_id)
             return True
         if resp.status_code == 409:
-            log.debug(f"{email} already subscribed to {list_id} — skipping.")
-            return True  # Already a member, treat as success
-        log.error(f"Subscribe failed for {email}: {resp.status_code} - {resp.text[:200]}")
+            log.debug("%s already subscribed to %s — skipping.", email, list_id)
+            return True
+        log.error("Subscribe failed for %s: %s - %s", email, resp.status_code, resp.text[:200])
         return False
     except Exception as e:
-        log.exception(f"Error subscribing {email} to {list_id}: {e}")
+        log.exception("Error subscribing %s to %s: %s", email, list_id, e)
         return False
 
 
@@ -220,165 +271,178 @@ def subscribe_member_to_list(list_id: str, email: str, display_name: str = '') -
 def unsubscribe_member_from_list(list_id: str, email: str) -> bool:
     """
     Unsubscribe an email address from a Mailman 3 list.
-
-    Returns:
-        True on success or not found (already unsubscribed), False on error.
+    Returns True on success or not-found, False on error.
     """
-    config = _load_mailman_config()
-    if not config:
+    cfg = _load_idaa_cfg()
+    if not cfg:
         return False
 
-    base_url = config.get('base_url', '').rstrip('/')
-    auth     = (config.get('username', 'restadmin'), config.get('password', ''))
+    base_url    = cfg.get('mailman_base_url', '').rstrip('/')
+    auth        = (cfg.get('mailman_username', 'restadmin'), cfg.get('mailman_password', ''))
     list_id_dot = list_id.replace('@', '.')
 
     try:
-        # Mailman member ID is {email}_{list_id} (base64 encoded in REST path)
         resp = requests.delete(
             f"{base_url}/3.1/lists/{list_id_dot}/member/{email}",
             auth=auth,
-            timeout=10
+            timeout=10,
         )
         if resp.status_code in (200, 204):
-            log.info(f"Unsubscribed {email} from {list_id}")
+            log.info("Unsubscribed %s from %s", email, list_id)
             return True
         if resp.status_code == 404:
-            log.debug(f"{email} not found in {list_id} — skipping.")
-            return True  # Not a member, treat as success
-        log.error(f"Unsubscribe failed for {email}: {resp.status_code} - {resp.text[:200]}")
+            log.debug("%s not found in %s — skipping.", email, list_id)
+            return True
+        log.error("Unsubscribe failed for %s: %s - %s", email, resp.status_code, resp.text[:200])
         return False
     except Exception as e:
-        log.exception(f"Error unsubscribing {email} from {list_id}: {e}")
+        log.exception("Error unsubscribing %s from %s: %s", email, list_id, e)
         return False
 
 
-# ── Sync Engine ───────────────────────────────────────────────────────────
+# ── Mirror Sync Engine ────────────────────────────────────────────────────
 
 @logger_reset
-def sync_single_member(email: str, display_name: str, list_id: str, is_active: bool) -> str:
+def mirror_novi_group_to_mailman_list(novi_group_guid: str, mailman_list_id: str) -> Optional[Dict]:
     """
-    Sync one member's subscription state for a given Mailman list.
+    Mirror a single Novi group to a Mailman 3 list.
 
-    Args:
-        email:        Member email
-        display_name: Member display name
-        list_id:      Target Mailman list ID
-        is_active:    True = subscribe, False = unsubscribe
-
-    Returns:
-        'subscribed', 'unsubscribed', or 'error'
+    - Fetches all members of `novi_group_guid` from Novi.
+    - For each member, fetches their customer record to get Email, name, and
+      membership flags.  Members with `Active=False` or `UnsubscribeFromEmails=True`
+      are excluded from the target set.
+    - Fetches current members of `mailman_list_id`.
+    - Subscribes addresses in Novi but not in Mailman.
+    - Unsubscribes addresses in Mailman but not in Novi (mirror / full reconcile).
+    - Returns a result dict with counts.
     """
-    if is_active:
-        ok = subscribe_member_to_list(list_id, email, display_name)
-        return 'subscribed' if ok else 'error'
-    else:
-        ok = unsubscribe_member_from_list(list_id, email)
-        return 'unsubscribed' if ok else 'error'
-
-
-@logger_reset
-def sync_novi_to_mailman(list_id: str, active_status: str = 'Active') -> Optional[Dict]:
-    """
-    Full sync: pull all Novi members and reconcile their Mailman subscription state.
-
-    - Novi members with `active_status` → subscribed to `list_id`
-    - All other statuses → unsubscribed from `list_id`
-
-    Args:
-        list_id:       Target Mailman list ID (e.g. 'members@yourdomain.org')
-        active_status: Novi membership status string that counts as active.
-
-    Returns:
-        Result dict with counts, or None on fatal error.
-
-    TODO: Adjust the Novi member field names (Email, FirstName, LastName,
-          MembershipStatus) to match your actual Novi instance schema.
-    """
-    log.info(f"Starting full Novi → Mailman sync for list: {list_id}")
-
-    members = get_novi_members()
-    if members is None:
-        log.error("Novi member fetch failed — aborting sync.")
+    cfg = _load_idaa_cfg()
+    if not cfg:
         return None
 
-    log.info(f"Fetched {len(members)} members from Novi.")
+    base_url = cfg.get('novi_api_root_url', '').rstrip('/')
+    api_key  = cfg.get('novi_idaa_api_key', '')
+    headers  = {"Authorization": f"Basic {api_key}", "Accept": "application/json"}
 
-    results = {"total": len(members), "subscribed": 0, "unsubscribed": 0, "error": 0, "skipped": 0}
+    log.info("Mirror sync: Novi group %s → Mailman list %s", novi_group_guid, mailman_list_id)
 
-    for member in members:
-        # Field names confirmed PascalCase from Novi API (verified via IDAA Jitsi integration).
-        # MembershipStatus field name still needs confirmation against the bulk member endpoint.
-        email  = (member.get('Email') or '').strip().replace(' ', '+')
-        fname  = member.get('FirstName') or ''
-        lname  = member.get('LastName')  or ''
-        status = member.get('MembershipStatus') or member.get('Status') or ''
+    # ── Step 1: Novi group → UUIDs ────────────────────────────────────────
+    try:
+        resp = requests.get(f"{base_url}/groups/{novi_group_guid}/members",
+                            headers=headers, params={"pageSize": 500}, timeout=30)
+        if resp.status_code != 200:
+            log.error("Novi group fetch failed (%s): %s", novi_group_guid, resp.status_code)
+            return None
+        uuid_list = [m['UniqueID'] for m in resp.json() if m.get('UniqueID')]
+    except Exception as e:
+        log.exception("Failed to fetch Novi group %s: %s", novi_group_guid, e)
+        return None
 
-        if not email:
-            results['skipped'] += 1
+    log.info("Novi group %s has %d member(s).", novi_group_guid, len(uuid_list))
+
+    # ── Step 2: UUID → customer record → email ────────────────────────────
+    # email.lower() → display_name
+    novi_members: Dict[str, str] = {}
+    skipped_inactive = 0
+    skipped_unsub    = 0
+    skipped_no_email = 0
+
+    for uid in uuid_list:
+        try:
+            r = requests.get(f"{base_url}/customers/{uid}", headers=headers, timeout=10)
+            if r.status_code != 200:
+                log.warning("Novi customer %s fetch failed: %s", uid, r.status_code)
                 continue
+            c = r.json()
+            if not c.get('Active', False):
+                skipped_inactive += 1
+                continue
+            if c.get('UnsubscribeFromEmails', False):
+                skipped_unsub += 1
+                continue
+            email = (c.get('Email') or '').strip()
+            if not email:
+                skipped_no_email += 1
+                continue
+            display = f"{c.get('FirstName', '')} {c.get('LastName', '')}".strip()
+            novi_members[email.lower()] = display
+        except Exception as e:
+            log.exception("Failed to fetch Novi customer %s: %s", uid, e)
 
-        display_name = f"{fname} {lname}".strip()
-        is_active    = (status == active_status)
+    log.info("Novi active/subscribed members with email: %d (skipped: inactive=%d unsub=%d no_email=%d)",
+             len(novi_members), skipped_inactive, skipped_unsub, skipped_no_email)
 
-        outcome = sync_single_member(email, display_name, list_id, is_active)
-        results[outcome] = results.get(outcome, 0) + 1
+    # ── Step 3: Current Mailman members ───────────────────────────────────
+    mailman_entries = get_mailman_list_members(mailman_list_id)
+    if mailman_entries is None:
+        log.error("Could not fetch current Mailman members for %s — aborting.", mailman_list_id)
+        return None
+    mailman_emails = {m['email'].lower() for m in mailman_entries}
 
-    log.info(f"Novi → Mailman sync complete: {results}")
+    # ── Step 4: Diff ──────────────────────────────────────────────────────
+    novi_email_set  = set(novi_members.keys())
+    to_subscribe    = novi_email_set - mailman_emails
+    to_unsubscribe  = mailman_emails - novi_email_set
+
+    log.info("Diff — to subscribe: %d, to unsubscribe: %d", len(to_subscribe), len(to_unsubscribe))
+
+    results = {
+        "novi_group_guid":    novi_group_guid,
+        "mailman_list_id":    mailman_list_id,
+        "novi_count":         len(novi_email_set),
+        "mailman_count_before": len(mailman_emails),
+        "subscribed":         0,
+        "unsubscribed":       0,
+        "errors":             0,
+        "skipped_inactive":   skipped_inactive,
+        "skipped_unsub":      skipped_unsub,
+        "skipped_no_email":   skipped_no_email,
+    }
+
+    # ── Step 5: Apply ─────────────────────────────────────────────────────
+    for email in to_subscribe:
+        ok = subscribe_member_to_list(mailman_list_id, email, novi_members[email])
+        if ok: results['subscribed'] += 1
+        else:  results['errors'] += 1
+
+    for email in to_unsubscribe:
+        ok = unsubscribe_member_from_list(mailman_list_id, email)
+        if ok: results['unsubscribed'] += 1
+        else:  results['errors'] += 1
+
+    log.info("Mirror sync complete: %s", results)
     return results
 
 
 @logger_reset
-def handle_novi_webhook(payload: Dict) -> Optional[Dict]:
+def mirror_all_configured_mappings() -> Optional[List[Dict]]:
     """
-    Process a Novi membership webhook event and update the appropriate Mailman list.
+    Run mirror_novi_group_to_mailman_list for every entry in
+    cfg_json['novi_mailman_sync'].
 
-    Expected payload shape (Novi webhook format — confirm against Novi docs):
-        {
-            "EventType": "MembershipActivated" | "MembershipLapsed" | "MembershipExpired" | ...,
-            "Member": {
-                "Email": "...",
-                "FirstName": "...",
-                "LastName": "...",
-                "MembershipStatus": "Active" | "Lapsed" | ...
-            }
-        }
-
-    TODO: Confirm Novi webhook payload format and EventType values.
-    TODO: Pull target list_id from data_store config or per-account settings.
+    Expected cfg_json shape:
+        "novi_mailman_sync": [
+            {"novi_group_guid": "...", "mailman_list_id": "members@idaa.org"},
+            ...
+        ]
     """
-    event_type = payload.get('EventType', '')
-    member     = payload.get('Member', {})
-    email      = (member.get('Email') or '').strip().replace(' ', '+')
-    fname      = member.get('FirstName', '')
-    lname      = member.get('LastName', '')
-    status     = member.get('MembershipStatus') or member.get('Status', '')
-
-    if not email:
-        log.warning(f"Novi webhook received with no email — skipping. Payload: {payload}")
+    cfg = _load_idaa_cfg()
+    if not cfg:
         return None
 
-    # Load target list_id from config
-    # TODO: Support per-account list routing (e.g. multiple orgs, each with their own list)
-    novi_config = _load_novi_config()
-    if not novi_config:
-        return None
-    list_id = novi_config.get('mailman_list_id', '')
-    if not list_id:
-        log.error("'mailman_list_id' not set in novi_api_config — cannot route webhook.")
-        return None
+    sync_map = cfg.get('novi_mailman_sync') or []
+    if not sync_map:
+        log.warning("novi_mailman_sync not configured in IDAA cfg_json.")
+        return []
 
-    ACTIVE_EVENTS = {'MembershipActivated', 'MembershipRenewed', 'MembershipCreated'}
-    INACTIVE_EVENTS = {'MembershipLapsed', 'MembershipExpired', 'MembershipTerminated', 'MembershipCancelled'}
+    results = []
+    for mapping in sync_map:
+        guid    = mapping.get('novi_group_guid', '').strip()
+        list_id = mapping.get('mailman_list_id', '').strip()
+        if not guid or not list_id:
+            log.warning("Skipping incomplete novi_mailman_sync entry: %s", mapping)
+            continue
+        result = mirror_novi_group_to_mailman_list(guid, list_id)
+        results.append(result or {"novi_group_guid": guid, "mailman_list_id": list_id, "error": "sync failed"})
 
-    if event_type in ACTIVE_EVENTS:
-        is_active = True
-    elif event_type in INACTIVE_EVENTS:
-        is_active = False
-    else:
-        log.info(f"Novi webhook: unhandled EventType '{event_type}' for {email} — ignoring.")
-        return {"action": "ignored", "reason": f"Unhandled EventType: {event_type}"}
-
-    display_name = f"{fname} {lname}".strip()
-    outcome = sync_single_member(email, display_name, list_id, is_active)
-    log.info(f"Novi webhook processed: {email} → {outcome} (EventType: {event_type})")
-    return {"action": outcome, "email": email, "event_type": event_type}
+    return results

app/methods/event_badge_methods.py

@@ -322,10 +322,9 @@ def create_update_event_badge_obj_v4(
     elif event_person_id := event_badge_obj.event_person_id: pass
 
     if event_badge_id:
-        if event_badge_dict_up_result := sql_update(data=event_badge_dict, table_name='event_badge', rm_id_random=True): pass
-        else:
-            log.warning(f'Event Badge not updated. Event Badge ID: {event_badge_id}')
-            log.debug(event_badge_dict_up_result)
+        event_badge_dict_up_result = sql_update(data=event_badge_dict, table_name='event_badge', record_id=event_badge_id, rm_id_random=True)
+        if event_badge_dict_up_result is False:
+            log.warning(f'Event Badge update failed (DB error). Event Badge ID: {event_badge_id}')
             return False
         log.debug(event_badge_dict_up_result)
     else:

app/methods/event_list_methods.py

@@ -1,122 +0,0 @@
-from __future__ import annotations
-import datetime
-
-from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, PrivateAttr, ValidationError, validator
-
-from app.db_sql import redis_lookup_id_random, sql_insert, sql_select, sql_update
-from app.lib_general import log, logging
-
-from app.methods.event_methods import load_event_obj
-
-
-# ### BEGIN ### API Event Methods ### load_event_obj_list() ###
-def load_event_obj_list(
-        account_id: int|str,
-        limit: int = 1000,
-        model_as_dict: bool = False,
-        enabled: str = 'enabled', # enabled, disabled, all
-        inc_contact_1: bool = False,
-        inc_contact_2: bool = False,
-        inc_contact_3: bool = False,
-        inc_event_abstract_list: bool = False,
-        inc_event_badge_list: bool = False,
-        inc_event_cfg: bool = False,
-        inc_event_device_list: bool = False,
-        inc_event_exhibit_list: bool = False,
-        inc_event_file_list: bool = False,
-        inc_event_location: bool = False, # For event_session child object
-        inc_event_location_list: bool = False,
-        inc_event_person_list: bool = False,
-        inc_event_presentation_list: bool = False,
-        inc_event_presenter_cat: bool = False, # For event_session child object
-        inc_event_presenter_list: bool = False,
-        inc_event_registration_cfg: bool = False,
-        inc_event_registration_list: bool = False,
-        inc_event_session_list: bool = False,
-        inc_event_track: bool = False, # For event_session child object
-        inc_event_track_list: bool = False,
-        inc_location_address: bool = False,
-        inc_poc_event_person: bool = False,
-        inc_person: bool = False,
-        inc_user: bool = False,
-        ) -> list|bool:
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    if account_id := redis_lookup_id_random(record_id_random=account_id, table_name='account'): pass
-    else: return False
-
-    data = {}
-    data['account_id'] = account_id
-
-    if enabled in ['enabled', 'disabled', 'all']:
-        if enabled == 'enabled':
-            data['enable'] = True
-            sql_enabled = f'AND `tbl`.enable = :enable'
-        elif enabled == 'disabled':
-            data['enable'] = False
-            sql_enabled = f'AND `tbl`.enable = :enable'
-        elif enabled == 'all':
-            sql_enabled = ''
-    # else: tbl_obj['account'] = None
-
-    if limit:
-        data['limit'] = limit
-        sql_limit = f'LIMIT :limit'
-    else:
-        sql_limit = ''
-
-    sql = f"""
-        SELECT `tbl`.id AS 'event_id', `tbl`.id_random AS 'event_id_random'
-        FROM `event` AS `tbl`
-        WHERE `tbl`.account_id = :account_id
-        {sql_enabled}
-        ORDER BY `tbl`.created_on DESC, `tbl`.updated_on DESC
-        {sql_limit};
-        """
-
-    if event_rec_li_result := sql_select(data=data, sql=sql, as_list=True):
-        log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-        log.debug(event_rec_li_result)
-        event_result_li = []
-        for event_rec in event_rec_li_result:
-            event_id = event_rec.get('event_id', None)
-            if event_result := load_event_obj(
-                event_id = event_id,
-                limit = limit,
-                model_as_dict = model_as_dict,
-                enabled = enabled,
-                # inc_location_address = inc_address,
-                # inc_contact_1 = inc_contact,
-                # inc_contact_2 = inc_contact,
-                # inc_contact_3 = inc_contact,
-                # inc_event_abstract_list = inc_event_abstract_list,
-                # inc_event_badge_list = inc_event_badge_list,
-                # inc_event_device_list = inc_event_device_list,
-                inc_event_exhibit_list = inc_event_exhibit_list,
-                inc_event_file_list = inc_event_file_list,
-                inc_event_location_list = inc_event_location_list,
-                inc_event_person_list = inc_event_person_list,
-                inc_event_presentation_list = inc_event_presentation_list,
-                inc_event_presenter_list = inc_event_presenter_list,
-                inc_event_registration_list = inc_event_registration_list,
-                inc_event_session_list = inc_event_session_list,
-                inc_event_track_list = inc_event_track_list,
-                # inc_person = inc_person,
-                # inc_user = inc_user,
-                ):
-                log.debug(event_result)
-                event_result_li.append(event_result)
-            else:
-                log.debug(event_result)
-                event_result_li.append(None)
-        log.debug(event_result_li)
-    else:
-        log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-        log.debug(event_rec_li_result)
-        event_result_li = []
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-
-    return event_result_li
-# ### END ### API Event Methods ### load_event_obj_list() ###

app/methods/event_person_methods.py

@@ -3,7 +3,7 @@ import datetime
 from typing import Dict, List, Optional, Set, Union
 from pydantic import BaseModel, EmailStr, Field, PrivateAttr, ValidationError, validator
 
-from app.db_sql import get_account_id_w_for_type_id, redis_lookup_id_random, sql_insert, sql_select, sql_update
+from app.db_sql import get_account_id_w_for_type_id, redis_lookup_id_random, sql_insert, sql_select, sql_update, get_id_random
 from app.lib_general import log, logging, logger_reset
 
 # from app.methods.event_abstract_methods import load_event_abstract_obj
@@ -355,7 +355,7 @@ def create_update_event_person_obj_v4(
         fail_any: bool = False, # Fail if any thing goes wrong for sub objects
         return_outline: bool = False,
         ) -> int|bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     # ### SECTION ### Secondary data validation
@@ -420,6 +420,18 @@ def create_update_event_person_obj_v4(
         if account_id:
             event_person_dict['account_id'] = account_id
         if event_id:
+            # The model expects random-string IDs (eg. id_random). If we have an
+            # integer internal ID, convert it to the random string form so the
+            # Pydantic root_validator preserves it. This ensures `event_id` is
+            # present when inserting a new `event_person` record.
+            if isinstance(event_id, int):
+                if idr := get_id_random(record_id=event_id, table_name='event'):
+                    event_person_dict['event_id_random'] = idr
+                else:
+                    # Fallback: set the integer (will likely be removed by the model),
+                    # but allow downstream logic to attempt insertion.
+                    event_person_dict['event_id'] = event_id
+            else:
                 event_person_dict['event_id'] = event_id
         try:
             event_person_obj = Event_Person_Base(**event_person_dict)
@@ -434,6 +446,15 @@ def create_update_event_person_obj_v4(
         if account_id:
             event_person_obj.account_id = account_id
         if event_id:
+            # If an integer internal ID was provided, convert to the random ID
+            # string form for the Pydantic object so it is preserved when
+            # serializing to the DB insert/update payload.
+            if isinstance(event_id, int):
+                if idr := get_id_random(record_id=event_id, table_name='event'):
+                    event_person_obj.event_id = idr
+                else:
+                    event_person_obj.event_id = event_id
+            else:
                 event_person_obj.event_id = event_id
     log.debug(event_person_obj)
 
@@ -453,11 +474,11 @@ def create_update_event_person_obj_v4(
             event_person_profile_id = event_person_obj.event_person_profile_id
 
     if event_person_id:
-        if event_person_dict_up_result := sql_update(data=event_person_dict, table_name='event_person', rm_id_random=True): pass
-        else:
-            log.warning(f'Event Person not updated. Event Person ID: {event_person_id}')
-            log.debug(event_person_dict_up_result)
+        event_person_dict_up_result = sql_update(data=event_person_dict, table_name='event_person', record_id=event_person_id, rm_id_random=True)
+        if event_person_dict_up_result is False:
+            log.warning(f'Event Person update failed (DB error). Event Person ID: {event_person_id}')
             return False
+        # None means 0 rows affected (record unchanged) — not an error, continue to sub-objects
         log.debug(event_person_dict_up_result)
     else:
         if event_person_dict_in_result := sql_insert(data=event_person_dict, table_name='event_person', rm_id_random=True, id_random_length=None): pass

app/methods/event_person_profile_methods.py

@@ -154,11 +154,12 @@ def create_update_event_person_profile_obj_v4(
         contact_id = event_person_profile_obj.contact_id
 
     if event_person_profile_id:
-        if event_person_profile_dict_up_result := sql_update(data=event_person_profile_dict, table_name='event_person_profile', rm_id_random=True): pass
-        else:
-            log.warning(f'Event Person Profile not updated. Event Person Profile ID: {event_person_profile_id}')
+        event_person_profile_dict_up_result = sql_update(data=event_person_profile_dict, table_name='event_person_profile', record_id=event_person_profile_id, rm_id_random=True)
+        if event_person_profile_dict_up_result is False:
+            log.warning(f'Event Person Profile update failed (DB error). Event Person Profile ID: {event_person_profile_id}')
             log.debug(event_person_profile_dict_up_result)
             return False
+        # None means 0 rows affected (record unchanged) — not an error
         log.debug(event_person_profile_dict_up_result)
     else:
         if event_person_profile_dict_in_result := sql_insert(data=event_person_profile_dict, table_name='event_person_profile', rm_id_random=True, id_random_length=8): pass

app/methods/event_presentation_methods.py

@@ -429,9 +429,9 @@ def create_update_event_presentation_obj_v4(
         event_presentation_dict = event_presentation_obj.dict(by_alias=False, exclude_defaults=False, exclude_unset=True, exclude={'event_presenter', 'event_presenter_list', 'created_on', 'updated_on'})
 
     if event_presentation_id:
-        if event_presentation_dict_up_result := sql_update(data=event_presentation_dict, table_name='event_presentation', rm_id_random=True): pass
-        else:
-            log.warning(f'Event Presentation not updated. Event Presentation ID: {event_presentation_id}')
+        event_presentation_dict_up_result = sql_update(data=event_presentation_dict, table_name='event_presentation', record_id=event_presentation_id, rm_id_random=True)
+        if event_presentation_dict_up_result is False:
+            log.warning(f'Event Presentation update failed (DB error). Event Presentation ID: {event_presentation_id}')
             log.debug(event_presentation_dict_up_result)
             return False
         log.debug(event_presentation_dict_up_result)

app/methods/event_presenter_methods.py

@@ -404,9 +404,9 @@ def create_update_event_presenter_obj_v4(
         event_presenter_dict = event_presenter_obj.dict(by_alias=False, exclude_defaults=False, exclude_unset=True, exclude={'created_on', 'updated_on'})
 
     if event_presenter_id:
-        if event_presenter_dict_up_result := sql_update(data=event_presenter_dict, table_name='event_presenter', rm_id_random=True): pass
-        else:
-            log.warning(f'Event Presenter not updated. Event Presenter ID: {event_presenter_id}')
+        event_presenter_dict_up_result = sql_update(data=event_presenter_dict, table_name='event_presenter', record_id=event_presenter_id, rm_id_random=True)
+        if event_presenter_dict_up_result is False:
+            log.warning(f'Event Presenter update failed (DB error). Event Presenter ID: {event_presenter_id}')
             log.debug(event_presenter_dict_up_result)
             return False
         log.debug(event_presenter_dict_up_result)

app/methods/idaa_novi_verify_methods.py

@@ -0,0 +1,154 @@
+import datetime
+import json
+import requests
+from typing import Dict, Optional
+
+from app.lib_general import log, logger_reset
+
+IDAA_SITE_ID_RANDOM = '58_gJESdlUh'
+_CACHE_TTL = datetime.timedelta(hours=4)
+
+
+# ── Config ────────────────────────────────────────────────────────────────
+
+@logger_reset
+def _load_idaa_cfg() -> Optional[Dict]:
+    """Load IDAA site cfg_json. Returns parsed dict or None on failure."""
+    from app.methods.site_methods import load_site_obj
+    site = load_site_obj(site_id=IDAA_SITE_ID_RANDOM, model_as_dict=True)
+    if not site:
+        log.error("Could not load IDAA site record (id_random='%s').", IDAA_SITE_ID_RANDOM)
+        return None
+    cfg = site.get('cfg_json')
+    if isinstance(cfg, str):
+        try:
+            cfg = json.loads(cfg)
+        except Exception as e:
+            log.error("Failed to parse IDAA cfg_json: %s", e)
+            return None
+    if not isinstance(cfg, dict):
+        log.error("IDAA cfg_json is not a dict after parsing.")
+        return None
+    return cfg
+
+
+def _cache_key(uuid: str) -> str:
+    return f'idaa:novi_member:{uuid}'
+
+
+# ── Public API ────────────────────────────────────────────────────────────
+
+@logger_reset
+def verify_novi_member(uuid: str) -> Dict:
+    """
+    Proxy GET /customers/{uuid} to Novi AMS and return normalized member data.
+
+    Returns a dict with one of:
+      {'status': 200, 'verified': True,  'full_name': '...', 'email': '...'}
+      {'status': 404, 'reason': '...'}
+      {'status': 429, 'reason': '...'}
+      {'status': 503, 'reason': '...'}
+
+    Redis cache key: idaa:novi_member:{uuid}, TTL 4 hours.
+    Only 200 (verified) results are cached — 404 is never cached.
+    """
+    from app.lib_redis_helpers import redis_client
+
+    cache_key = _cache_key(uuid)
+
+    # ── Cache hit ─────────────────────────────────────────────────────────
+    cached_raw = redis_client.get(cache_key)
+    if cached_raw:
+        try:
+            cached = json.loads(cached_raw)
+            log.info("Novi verify cache hit: %s", uuid)
+            return cached
+        except Exception:
+            pass  # corrupt cache entry — fall through to Novi
+
+    # ── Load credentials ──────────────────────────────────────────────────
+    cfg = _load_idaa_cfg()
+    if not cfg:
+        return {'status': 503, 'reason': 'IDAA site configuration unavailable.'}
+
+    base_url = cfg.get('novi_api_root_url', '').rstrip('/')
+    api_key  = cfg.get('novi_idaa_api_key', '')
+
+    if not base_url or not api_key:
+        log.error("novi_api_root_url or novi_idaa_api_key missing from IDAA cfg_json.")
+        return {'status': 503, 'reason': 'Novi credentials not configured.'}
+
+    headers = {'Authorization': f'Basic {api_key}', 'Accept': 'application/json'}
+
+    # ── Call Novi ─────────────────────────────────────────────────────────
+    try:
+        resp = requests.get(f'{base_url}/customers/{uuid}', headers=headers, timeout=10)
+    except requests.exceptions.ConnectionError as e:
+        log.error("Novi unreachable: %s", e)
+        return {'status': 503, 'reason': 'Novi API unreachable.'}
+    except requests.exceptions.Timeout:
+        log.error("Novi request timed out for UUID %s", uuid)
+        return {'status': 503, 'reason': 'Novi API timed out.'}
+    except Exception as e:
+        log.exception("Unexpected error calling Novi for UUID %s: %s", uuid, e)
+        return {'status': 503, 'reason': 'Unexpected error contacting Novi.'}
+
+    if resp.status_code == 429:
+        log.warning("Novi rate limit hit for UUID %s", uuid)
+        return {'status': 429, 'reason': 'Novi rate limit exceeded. Try again shortly.'}
+
+    if resp.status_code >= 500:
+        log.error("Novi server error %s for UUID %s", resp.status_code, uuid)
+        return {'status': 503, 'reason': f'Novi server error ({resp.status_code}).'}
+
+    if resp.status_code == 404:
+        log.info("Novi returned 404 for UUID %s", uuid)
+        return {'status': 404, 'reason': 'Member not found in Novi.'}
+
+    if resp.status_code != 200:
+        log.error("Unexpected Novi status %s for UUID %s: %s", resp.status_code, uuid, resp.text[:200])
+        return {'status': 503, 'reason': f'Unexpected Novi response ({resp.status_code}).'}
+
+    # ── Parse response ────────────────────────────────────────────────────
+    try:
+        data = resp.json()
+    except Exception:
+        log.error("Novi returned non-JSON for UUID %s", uuid)
+        return {'status': 503, 'reason': 'Novi returned an unparseable response.'}
+
+    if not isinstance(data, dict):
+        log.warning("Novi returned non-dict body for UUID %s", uuid)
+        return {'status': 404, 'reason': 'Member not found in Novi (empty response).'}
+
+    # Empty-member anti-pattern: Novi 200 with no identity data
+    email_raw = (data.get('Email') or '').strip()
+    if not email_raw:
+        log.info("Novi 200 with no Email for UUID %s — empty-member anti-pattern", uuid)
+        return {'status': 404, 'reason': 'Member not found in Novi (no identity data).'}
+
+    email = email_raw.replace(' ', '+')
+
+    # Build display name: "FirstName LastName[0]." — fall back to Name field
+    first = (data.get('FirstName') or '').strip()
+    last  = (data.get('LastName')  or '').strip()
+    if first and last:
+        full_name = f'{first} {last[0]}.'
+    elif first:
+        full_name = first
+    else:
+        full_name = (data.get('Name') or '').strip() or 'Member'
+
+    result = {
+        'status': 200,
+        'verified': True,
+        'full_name': full_name,
+        'email': email,
+    }
+
+    # ── Cache verified result ─────────────────────────────────────────────
+    try:
+        redis_client.setex(cache_key, _CACHE_TTL, json.dumps(result))
+    except Exception as e:
+        log.warning("Failed to cache Novi verify result for %s: %s", uuid, e)
+
+    return result

app/methods/site_domain_methods.py

@@ -147,6 +147,9 @@ def get_site_domain_rec_list(
 # ### BEGIN ### API Site Domain Methods ### lookup_site_domain_fqdn() ###
 def lookup_site_domain_fqdn(
         fqdn: str,
+    # Accept access_key as an argument for validation (str|None)
+    access_key: Optional[str] = None,
+    referrer: Optional[str] = None,
         enabled: str = 'enabled', # enabled, disabled, all
         limit: int = 100,
         offset: int = 0,
@@ -156,15 +159,37 @@ def lookup_site_domain_fqdn(
 
     data = {}
     data['fqdn'] = fqdn
+    # If access_key is provided, add it to the data dict for SQL parameterization
+    data['domain_access_key'] = access_key
+    if referrer:
+        data['required_referrer'] = referrer
 
     sql_enabled, data['enable'] = sql_enable_part(table_name='site_domain', enabled=enabled) # Reasonably safe return str and bool
     sql_limit = sql_limit_offset_part(limit=limit, offset=offset) # Reasonably safe return str
 
+    # Build access key / referrer SQL similar to router.lookup_fqdn behavior
+    if access_key and referrer:
+        sql_access_key_referrer = """
+            AND site_domain.domain_access_key = :domain_access_key
+            AND site_domain.required_referrer = :required_referrer
+            """
+    elif access_key:
+        sql_access_key_referrer = """
+            AND site_domain.domain_access_key = :domain_access_key
+            AND (site_domain.required_referrer IS NULL OR site_domain.required_referrer = '')
+            """
+    else:
+        sql_access_key_referrer = """
+            AND (site_domain.domain_access_key IS NULL OR site_domain.domain_access_key  = '')
+            AND (site_domain.required_referrer IS NULL OR site_domain.required_referrer  = '')
+            """
+
     sql = f"""
         SELECT `site_domain`.id AS 'site_domain_id', `site_domain`.id_random AS 'site_domain_id_random'
         FROM `v_site_domain` AS site_domain
         WHERE
             site_domain.fqdn = :fqdn
+            {sql_access_key_referrer}
             {sql_enabled}
         ORDER BY `site_domain`.fqdn ASC, `site_domain`.access_key ASC, `site_domain`.required_referrer ASC, `site_domain`.created_on DESC, `site_domain`.updated_on DESC
         {sql_limit};
@@ -176,4 +201,11 @@ def lookup_site_domain_fqdn(
         site_domain_rec_li = []
 
     return site_domain_rec_li
+
+# ---
+# To restore access_key validation:
+# 1. Accept access_key as a parameter to this function (and any API endpoint calling it).
+# 2. Add access_key to the SQL WHERE clause (see above) so only matching records are returned.
+# 3. If access_key is required, return empty or error if not matched.
+# 4. Update API docs and tests to reflect the new/required parameter.
 # ### END ### API Site Domain Methods ### get_site_domain_rec_list() ###

app/methods/user_methods.py

@@ -654,7 +654,7 @@ def email_user_auth_key_url(
     else: return False
     log.debug(account_cfg)
 
-    user_id_random = user_obj.id_random # NOTE: Not user_id_random because of alias
+    user_id_random = user_obj.id or user_obj.user_id # Vision ID: User_Out_Base uses 'id'/'user_id', not 'id_random'
 
     from_email = account_cfg.default_no_reply_email
     from_name = account_cfg.default_no_reply_name

app/models/archive_content_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union, ClassVar
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -18,13 +18,12 @@ class Archive_Content_Base(BaseModel):
     #     log.debug(test_var)
     #     return test_var
 
-    id_random: Optional[str] = Field(
-        # **base_fields['archive_content_id_random'],
-        alias = 'archive_content_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'archive_content_id'
-    )
+    # --- Vision IDs (primary public identifiers — always random strings) ---
+    id: Optional[str] = Field(None, **base_fields['archive_content_id_random'])
+    archive_content_id: Optional[str] = Field(None, **base_fields['archive_content_id_random'])
+    # Legacy alias kept for backward compatibility; populated by root_validator
+    id_random: Optional[str] = Field(None, alias='archive_content_id_random')
+
     account_id_random: Optional[str]
     account_id: Optional[int]
 
@@ -37,6 +36,9 @@ class Archive_Content_Base(BaseModel):
     lu_media_type_id: Optional[int]
     lu_media_type: Optional[str]
 
+    external_id: Optional[str]
+    code: Optional[str]
+
     name: Optional[str]
     description: Optional[str]
 
@@ -94,6 +96,7 @@ class Archive_Content_Base(BaseModel):
 
     # Fields that are part of the model (for reading) but should not be saved to the DB table
     fields_to_exclude_from_db: ClassVar[list] = [
+        'id', 'archive_content_id', 'id_random',
         'account_id', 'account_id_random', 'archive_id_random', 'hosted_file_id_random',
         'hosted_file_path', 'api_hosted_file_path_download', 'api_hosted_file_path_stream',
         'hosted_file_hash_sha256', 'hosted_file_content_type', 'hosted_file_size'
@@ -101,12 +104,21 @@ class Archive_Content_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('id', always=True)
-    def archive_content_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='archive_content')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer: Map DB random-string keys to clean Vision ID fields.
+        Collision prevention strips any integer that snuck into the string ID fields.
+        """
+        rid = values.get('id_random') or values.get('archive_content_id_random')
+        if rid and isinstance(rid, str):
+            values['id'] = rid
+            values['archive_content_id'] = rid
+        # Collision prevention: reject integer values in Vision string fields
+        for k in ['id', 'archive_content_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+        return values
 
     @validator('archive_id', always=True)
     def archive_id_lookup(cls, v, values, **kwargs):

app/models/event_badge_template_models.py

@@ -37,6 +37,7 @@ class Event_Badge_Template_Base(BaseModel):
     header_background: Optional[str]
 
     secondary_header_path: Optional[str] # Path to image file for back of badge and other sections
+    background_image_path: Optional[str]
 
     footer_path: Optional[str] # Path to image file
     footer_title: Optional[str]
@@ -73,7 +74,8 @@ class Event_Badge_Template_Base(BaseModel):
     script_src: Optional[str]
     passcode: Optional[str]
 
-    other_json: Optional[str]
+    other_json: Optional[Json]
+    cfg_json: Optional[Json]
 
     enable: Optional[bool]
     hide: Optional[bool]

app/models/event_models.py

@@ -171,6 +171,7 @@ class Event_Base(BaseModel):
 
     cfg_json: Optional[Union[Json, None]] # Store per event config options; Not currently used 2024-06-11
     data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields! Not currently used 2024-06-11
+    default_qry_str: Optional[str] # Default query string used for searching and filtering events. Updated using SQL triggers and a SQL function
 
     enable: Optional[bool] # Also in Event_Cfg_Base model
     enable_from: Optional[datetime.datetime] = None
@@ -396,6 +397,7 @@ class Event_Meeting_Flat_Base(BaseModel):
 
     cfg_json: Optional[Union[Json, None]] # Store per event config options; Not currently used 2024-06-11
     data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields! Not currently used 2024-06-11
+    default_qry_str: Optional[str] # Default query string used for searching and filtering events. Updated using SQL triggers and a SQL function
 
     enable: Optional[bool] # Also in Event_Cfg_Base model
     enable_from: Optional[datetime.datetime] = None

app/models/event_presenter_models.py

@@ -125,6 +125,7 @@ class Event_Presenter_Base(BaseModel):
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
+    default_qry_str: Optional[str] # Default query string used for searching and filtering presenters. Updated using SQL triggers and a SQL function
 
     # Including convenience data
     # This is only for convenience. Probably going to keep unless it causes a problem.
@@ -313,6 +314,7 @@ class Event_Presenter_Out_Base(BaseModel):
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
+    default_qry_str: Optional[str] # Default query string used for searching and filtering presenters. Updated using SQL triggers and a SQL function
 
     person_external_id: Optional[str]
     person_external_sys_id: Optional[str]

app/models/event_session_models.py

@@ -138,6 +138,9 @@ class Event_Session_Base(BaseModel):
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
+    default_qry_str: Optional[str] # Default query string used for searching and filtering sessions. Updated using SQL triggers and a SQL function
+    event_presentation_li_qry_str: Optional[str] # Concatenated query string of presentation data for this session (from v_event_session_w_file_count)
+    event_presenter_li_qry_str: Optional[str] # Concatenated query string of presenter data for this session (from v_event_session_w_file_count)
 
     # Including convenience data
     # This is only for convenience. Probably going to keep unless it causes a problem.

app/models/journal_entry_models.py

@@ -102,6 +102,7 @@ class Journal_Entry_Base(BaseModel):
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
+    default_qry_str: Optional[str] = None  # Default query string used for searching and filtering journal entries
 
     # Including other related objects
     # This is only for convenience. Probably going to keep unless it causes a problem.

app/models/post_models.py

@@ -32,7 +32,7 @@ class Post_Base(BaseModel):
     # type_id: Optional[int]
 
     # topic_id_random: Optional[str]
-    # topic_id: Optional[int]
+    topic_id: Optional[int]
 
     type: Optional[str]
 

app/models/site_domain_models.py

@@ -42,6 +42,23 @@ class Site_Domain_Base(BaseModel):
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
+    # Convenience fields from v_site_domain view (joined from account/site)
+    account_code: Optional[str] = None
+    account_name: Optional[str] = None
+    account_enable: Optional[bool] = None
+    account_enable_from: Optional[datetime.datetime] = None
+    account_enable_to: Optional[datetime.datetime] = None
+
+    site_enable_from: Optional[datetime.datetime] = None
+    site_enable_to: Optional[datetime.datetime] = None
+    site_domain_access_key: Optional[str] = None
+
+    logo_path: Optional[str] = None
+    style_href: Optional[str] = None
+    script_src: Optional[str] = None
+
+    google_tracking_id: Optional[str] = None
+
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
     @root_validator(pre=True)

app/models/user_models.py

@@ -1,6 +1,6 @@
 import datetime, hashlib, logging, os, pytz, redis, secrets
 
-from typing import Dict, List, Optional, Set, Union
+from typing import ClassVar, Dict, List, Optional, Set, Union
 from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import get_id_random, redis_lookup_id_random
@@ -169,6 +169,14 @@ class User_New_Base(BaseModel):
     # Including JSON data
     other_json: Optional[Json]
 
+    # Fields that are part of the model (for input) but must not be written to the DB table
+    fields_to_exclude_from_db: ClassVar[list] = [
+        'new_password',  # Virtual input field — the validator hashes it into 'password'; DB has no new_password column
+        'id', 'user_id',  # Vision ID strings — DB uses int 'id' (auto) and string 'id_random'
+        'account_id_random', 'contact_id_random', 'organization_id_random', 'person_id_random',
+        'account_name',
+    ]
+
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
     @root_validator(pre=True)
@@ -191,11 +199,21 @@ class User_New_Base(BaseModel):
         if p_rid := values.get('person_id_random'):
             values['person_id'] = p_rid
 
-        # 2. Prevent "Collision Population"
-        for k in ['id', 'user_id', 'account_id', 'contact_id', 'organization_id', 'person_id']:
+        # 2. Prevent "Collision Population" — only strip self-reference IDs.
+        # FK IDs (account_id, contact_id, etc.) are resolved to integers by sanitize_payload
+        # before model construction and must NOT be stripped, or they won't be written to the DB.
+        for k in ['id', 'user_id']:
             if k in values and not isinstance(values[k], str):
                 del values[k]
 
+        # 3. Pre-inject hashed password so it appears in __fields_set__.
+        # The @validator('password', always=True) below computes the same hash, but
+        # exclude_unset=True (used by the CRUD POST handler) only includes fields that
+        # were in the original input dict. By injecting 'password' here (pre=True),
+        # it is treated as part of the input and thus written to the DB.
+        if new_pw := values.get('new_password'):
+            values['password'] = secure_hash_string(string=new_pw)
+
         return values
 
     @validator('password', always=True)

app/object_definitions/cms.py

@@ -124,7 +124,7 @@ cms_obj_li = {
         'searchable_fields': [
             'id', 'account_id', 'site_id',
             'id_random', 'account_id_random', 'site_id_random',
-            'fqdn', 'access_key', 'site_access_key',
+            'fqdn', 'access_key', 'site_access_key', 'site_domain_access_key',
             'enable', 'created_on', 'updated_on'
         ],
     },

app/object_definitions/events_presentation.py

@@ -135,7 +135,8 @@ events_presentation_obj_li = {
             'poc_person_full_name',
             'public', 'public_hide', 'hide_event_launcher',
             'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on',
-            'default_qry_str', 'event_location_name'
+            'default_qry_str', 'event_location_name',
+            'event_presentation_li_qry_str', 'event_presenter_li_qry_str'
         ],
     },
     'event_track': {

app/object_definitions/events_registration.py

@@ -33,6 +33,12 @@ events_registration_obj_li = {
             'member_status', 'registration_type_code',
             'notes', 'created_on', 'updated_on', 'default_qry_str'
         ],
+        # Allow nested operations under both `event` and `event_person` parents.
+        # `event_badge` is directly linked to `event_person` (FK: event_person_id),
+        # but views expose it under `event` as well. Explicitly register both
+        # so nested CRUD routes like POST /v3/crud/event_person/{id}/event_badge/
+        # will be accepted by the generic nested router.
+        'parent_types': ['event', 'event_person'],
     },
     'event_badge_template': {
         'tbl': 'event_badge_template',

app/object_definitions/other.py

@@ -108,7 +108,7 @@ other_obj_li = {
         'searchable_fields': [
             'id', 'account_id', 'archive_id', 'hosted_file_id',
             'id_random', 'archive_content_id_random', 'account_id_random', 'archive_id_random',
-            'archive_content_type', 'lu_media_type', 'name', 'description',
+            'archive_content_type', 'lu_media_type', 'external_id', 'code', 'name', 'description',
             'filename', 'file_extension', 'original_location', 'original_url',
             'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on'
         ],

app/routers/aether_cfg.py

@@ -39,24 +39,24 @@ async def get_aether_cfg_obj(
         return mk_resp(data=None, status_code=404, response=commons.response)
 
 
-@router.get('/aether/flask/cfg/{aether_flask_cfg_id}', response_model=Resp_Body_Base)
-async def get_aether_flask_cfg_obj(
-        aether_flask_cfg_id: int,
-        # aether_flask_cfg_id: str = Path(min_length=11, max_length=22),
+# @router.get('/aether/flask/cfg/{aether_flask_cfg_id}', response_model=Resp_Body_Base)
+# async def get_aether_flask_cfg_obj(
+#         aether_flask_cfg_id: int,
+#         # aether_flask_cfg_id: str = Path(min_length=11, max_length=22),
 
-        # NOTE: The x_account_id header value is not required.
-        # commons: Common_Route_Params = Depends(common_route_params),
-        commons: Common_Route_Params_No_Account_ID = Depends(common_route_params_no_account_id),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+#         # NOTE: The x_account_id header value is not required.
+#         # commons: Common_Route_Params = Depends(common_route_params),
+#         commons: Common_Route_Params_No_Account_ID = Depends(common_route_params_no_account_id),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    if sql_select_result := sql_select(
-            table_name = 'cfg_flask',
-            record_id = aether_flask_cfg_id,
-            as_list = False,
-            max_count = 1,
-            ):
-        return mk_resp(data=sql_select_result, response=commons.response)
-    else:
-        return mk_resp(data=None, status_code=404, response=commons.response)
+#     if sql_select_result := sql_select(
+#             table_name = 'cfg_flask',
+#             record_id = aether_flask_cfg_id,
+#             as_list = False,
+#             max_count = 1,
+#             ):
+#         return mk_resp(data=sql_select_result, response=commons.response)
+#     else:
+#         return mk_resp(data=None, status_code=404, response=commons.response)

app/routers/api.py

@@ -4,15 +4,16 @@ from typing import Dict, List, Optional, Set, Union
 from sqlalchemy import text
 import json
 import time
-import secrets
+# import secrets
 import jwt as pyjwt # Avoid conflict with app.lib_jwt
 
-from app.db_connection import db
+# from app.db_connection import db
 from app.lib_general import sign_jwt, decode_jwt, log, logging
 from app.config import settings
 from app.db_sql import sql_insert, sql_update, sql_select, redis_lookup_id_random, get_id_random
 
-from app.routers.api_crud import delete_obj_template, get_obj_template, get_obj_li_template, patch_obj_template, post_obj_template
+# from app.routers.api_crud import delete_obj_template, get_obj_template, get_obj_li_template, patch_obj_template, post_obj_template
+from app.routers.dependencies_v3 import DeprecationParams
 from app.models.api_models import Api_Base
 from app.models.response_models import Resp_Body_Base, mk_resp
 
@@ -20,10 +21,21 @@ router = APIRouter()
 
 # --- Passcode Authentication ---
 
+ROLE_PRIORITY = ['super', 'manager', 'administrator', 'trusted', 'public', 'authenticated']
+
+ROLE_TTL = {
+    'super':         8  * 3600,   #  8 hours
+    'manager':       24 * 3600,   # 24 hours
+    'administrator': 48 * 3600,   # 48 hours
+    'trusted':       48 * 3600,   # 48 hours
+    'public':        24 * 3600,   # 24 hours
+    'authenticated': 12 * 3600,   # 12 hours
+}
+
 class PasscodeAuthRequest(BaseModel):
     """Request model for site-based passcode authentication."""
     site_id: str = Field(..., description="The random string ID of the site")
-    passcode: str = Field(..., description="The passcode to verify")
+    passcode: str = Field(..., min_length=5, description="The passcode to verify")
 
 @router.post('/authenticate_passcode', response_model=Resp_Body_Base)
 async def authenticate_passcode(
@@ -53,10 +65,11 @@ async def authenticate_passcode(
             except Exception as e:
                 log.error(f"Failed to parse access_code_kv_json for site {site_id}: {e}")
 
-        # 3. Verify Passcode and Resolve Role
+        # 3. Verify passcode in explicit priority order (highest privilege wins)
         matched_role = None
-        for role, code in access_codes.items():
-            if str(code) == str(passcode):
+        for role in ROLE_PRIORITY:
+            code = access_codes.get(role)
+            if code and str(code) == str(passcode):
                 matched_role = role
                 break
 
@@ -69,12 +82,15 @@ async def authenticate_passcode(
                 if account_id_int := record.get('account_id'):
                     account_id_random = get_id_random(record_id=account_id_int, table_name='account')
 
-            # 5. Mint JWT
+            # 5. Mint JWT with complete role flags and per-role TTL
             payload = {
                 'account_id':    account_id_random,
-                'administrator': (matched_role == 'administrator'),
-                'manager': (matched_role == 'manager'),
                 'super':         (matched_role == 'super'),
+                'manager':       (matched_role == 'manager'),
+                'administrator': (matched_role == 'administrator'),
+                'trusted':       (matched_role == 'trusted'),
+                'public':        (matched_role == 'public'),
+                'authenticated': (matched_role == 'authenticated'),
                 'json_str': json.dumps({
                     'auth_type': 'passcode',
                     'site_id':   site_id,
@@ -84,7 +100,7 @@ async def authenticate_passcode(
 
             token = sign_jwt(
                 secret_key=settings.JWT_KEY,
-                ttl=3600 * 24, # 24 hour session
+                ttl=ROLE_TTL[matched_role],
                 **payload
             )
 
@@ -98,7 +114,9 @@ async def authenticate_passcode(
 
 # --- JWT Request ---
 
-@router.get('/request_jwt', response_model=Resp_Body_Base)
+# DEPRECATED — no V3 replacement needed; passcode→JWT is the V3 auth pattern (/api/authenticate_passcode).
+# No frontend references found. Safe to remove after confirming no live traffic. TODO: remove.
+@router.get('/request_jwt', response_model=Resp_Body_Base, dependencies=[Depends(DeprecationParams)])
 async def request_jwt(
         x_aether_signing_key: Optional[str] = Header(None, min_length=22, max_length=22),
         x_aether_api_key: Optional[str] = Header(None, min_length=22, max_length=22),
@@ -151,7 +169,8 @@ async def request_jwt(
     token = sign_jwt(secret_key=signing_key, public_key=x_aether_api_key, ttl=max_ttl, max_renew=max_renew, **payload)
     return mk_resp(data={ 'jwt': token })
 
-@router.get('/temp_token', response_model=Resp_Body_Base)
+# DEPRECATED — no active use identified. TODO: remove after confirming no live traffic.
+@router.get('/temp_token', response_model=Resp_Body_Base, dependencies=[Depends(DeprecationParams)])
 async def get_api_temp_token(
         x_aether_api_key: Optional[str] = Header(None),
         response: Response = Response,
@@ -167,6 +186,10 @@ async def get_api_temp_token(
 
 # --- Jitsi Token ---
 
+# NOTE: Actively used by IDAA for video conferences on self-hosted Jitsi (jitsi.dgrzone.com).
+# JWT_APP_ID and JWT_APP_SECRET must match the values in the Jitsi server .env file.
+# TODO: Rotate JWT_APP_SECRET — update it here AND in /mnt/nfs_dgr_storage/env/dgr_zone_jitsi/.env (JWT_APP_SECRET) then restart prosody + jicofo.
+
 JWT_APP_ID = "my_jitsi_app_id"
 JWT_APP_SECRET = "my_jitsi_app_secret-9876543210"
 JITSI_DOMAIN = "jitsi.dgrzone.com"
@@ -184,14 +207,12 @@ class JitsiTokenRequest(BaseModel):
 @router.post("/jitsi_token")
 async def create_jitsi_jwt(request_data: JitsiTokenRequest = Body(...)):
     log.setLevel(logging.INFO)
-    if not request_data.is_moderator:
-        raise HTTPException(status_code=403, detail="JWT generation is only permitted for moderators.")
 
     try:
         payload = {
             "aud": JWT_APP_ID, "iss": JWT_APP_ID, "sub": JITSI_DOMAIN,
             "room": request_data.room,
-            "exp": int(time.time()) + 3600,
+            "exp": int(time.time()) + 7200, # 2 hour expiry
             "config": request_data.config or {},
             "context": {
                 "user": {
@@ -211,40 +232,43 @@ async def create_jitsi_jwt(request_data: JitsiTokenRequest = Body(...)):
         raise HTTPException(status_code=500, detail=f"Failed to create JWT: {str(e)}")
 
 # --- Api_Base CRUD ---
+# LEGACY (disabled) - superseded by V3 CRUD: /v3/crud/api/
 
-@router.post('', response_model=Resp_Body_Base)
-async def post_api_obj(obj: Api_Base, x_account_id: str = Header(...)):
-    return post_obj_template(obj_type='api', data=obj.dict(by_alias=False, exclude_unset=True), return_obj=True)
+# @router.post('', response_model=Resp_Body_Base)
+# async def post_api_obj(obj: Api_Base, x_account_id: str = Header(...)):
+#     return post_obj_template(obj_type='api', data=obj.dict(by_alias=False, exclude_unset=True), return_obj=True)
 
-@router.patch('/{obj_id}', response_model=Resp_Body_Base)
-async def patch_api_obj(obj_id: str, obj: Api_Base, x_account_id: str = Header(...)):
-    data = obj.dict(by_alias=False, exclude_unset=True)
-    data['id'] = redis_lookup_id_random(record_id_random=obj_id, table_name='api')
-    return patch_obj_template(obj_type='api', data=data, obj_id=obj_id, return_obj=True)
+# @router.patch('/{obj_id}', response_model=Resp_Body_Base)
+# async def patch_api_obj(obj_id: str, obj: Api_Base, x_account_id: str = Header(...)):
+#     data = obj.dict(by_alias=False, exclude_unset=True)
+#     data['id'] = redis_lookup_id_random(record_id_random=obj_id, table_name='api')
+#     return patch_obj_template(obj_type='api', data=data, obj_id=obj_id, return_obj=True)
 
-@router.get('/list', response_model=Resp_Body_Base)
-async def get_api_obj_li(for_obj_type: Optional[str] = Query(None), for_obj_id: Optional[str] = Query(None), x_account_id: str = Header(...)):
-    return get_obj_li_template(obj_type='api', for_obj_type=for_obj_type, for_obj_id=for_obj_id)
+# @router.get('/list', response_model=Resp_Body_Base)
+# async def get_api_obj_li(for_obj_type: Optional[str] = Query(None), for_obj_id: Optional[str] = Query(None), x_account_id: str = Header(...)):
+#     return get_obj_li_template(obj_type='api', for_obj_type=for_obj_type, for_obj_id=for_obj_id)
 
-@router.get('/{obj_id}', response_model=Resp_Body_Base)
-async def get_api_obj(obj_id: str, x_account_id: str = Header(...)):
-    return get_obj_template(obj_type='api', obj_id=obj_id)
+# @router.get('/{obj_id}', response_model=Resp_Body_Base)
+# async def get_api_obj(obj_id: str, x_account_id: str = Header(...)):
+#     return get_obj_template(obj_type='api', obj_id=obj_id)
 
-@router.delete('/{obj_id}', response_model=Resp_Body_Base)
-async def delete_api_obj(obj_id: str, x_account_id: str = Header(...)):
-    return delete_obj_template(obj_type='api', obj_id=obj_id)
+# @router.delete('/{obj_id}', response_model=Resp_Body_Base)
+# async def delete_api_obj(obj_id: str, x_account_id: str = Header(...)):
+#     return delete_obj_template(obj_type='api', obj_id=obj_id)
 
-@router.get('/get_id/{object_type}/{object_id_random}', response_model=Resp_Body_Base)
-async def get_api_object_id(object_type: str, object_id_random: str):
-    if object_id := redis_lookup_id_random(record_id_random=object_id_random, table_name=object_type):
-        return mk_resp(data={ 'object_id': object_id})
-    return mk_resp(data=None, status_code=404)
+# LEGACY (disabled) - exposes internal integer IDs, breaks id_random abstraction
+# @router.get('/get_id/{object_type}/{object_id_random}', response_model=Resp_Body_Base)
+# async def get_api_object_id(object_type: str, object_id_random: str):
+#     if object_id := redis_lookup_id_random(record_id_random=object_id_random, table_name=object_type):
+#         return mk_resp(data={ 'object_id': object_id})
+#     return mk_resp(data=None, status_code=404)
 
-@router.get('/sql_test', tags=['Testing'])
-async def sql_test(response: Response = Response):
-    sql = text("SELECT NOW() as current_time, VERSION() as version")
-    try:
-        result = db.execute(sql).fetchone()
-        return mk_resp(data={"current_time": str(result[0]), "version": result[1]})
-    except Exception as e:
-        return mk_resp(data=False, status_code=500, details=str(e), response=response)
+# LEGACY (disabled) - testing/debug endpoint
+# @router.get('/sql_test', tags=['Testing'])
+# async def sql_test(response: Response = Response):
+#     sql = text("SELECT NOW() as current_time, VERSION() as version")
+#     try:
+#         result = db.execute(sql).fetchone()
+#         return mk_resp(data={"current_time": str(result[0]), "version": result[1]})
+#     except Exception as e:
+#         return mk_resp(data=False, status_code=500, details=str(e), response=response)

app/routers/api_crud_v3.py

@@ -15,7 +15,8 @@ from app.lib_general_v3 import (
 )
 from app.lib_api_crud_v3 import (
     check_account_access, apply_forced_account_filter, filter_order_by,
-    get_supported_filters, safe_json_loads, sanitize_payload, format_db_error
+    get_supported_filters, safe_json_loads, sanitize_payload, format_db_error,
+    apply_vision_id_fix
 )
 from app.lib_schema_v3 import get_object_schema_info
 from app.db_sql import get_last_sql_error
@@ -157,6 +158,7 @@ async def get_obj(
             sql_result['inc_hosted_file'] = True
 
         resp_data = base_name(**sql_result).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none)
+        apply_vision_id_fix(resp_data, obj_name, serialization.by_alias)
         return mk_resp(data=resp_data, response=response)
     else:
         return mk_resp(data=False, status_code=404, response=response, status_message=f"Object with ID '{obj_id}' not found in database.")
@@ -283,7 +285,7 @@ async def get_obj_li(
         return mk_resp(data=False, status_code=status_code, response=response, status_message="Listing failed due to database error.", details=db_err.dict())
 
     if sql_result:
-        resp_data_li = [base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none) for record in sql_result]
+        resp_data_li = [apply_vision_id_fix(base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none), obj_name, serialization.by_alias) for record in sql_result]
         return mk_resp(data=resp_data_li, response=response)
     else:
         return mk_resp(data=[], status_code=200, response=response)
@@ -341,6 +343,31 @@ async def search_obj_li(
     status_filter = get_supported_filters(base_name, status_filter)
     searchable_fields = obj_cfg.get('searchable_fields')
 
+    # site_domain access-key enforcement:
+    # - site_access_key (site-level) takes priority; site_domain_access_key used as fallback.
+    # - A domain is public only if site_domain_access_key is NULL/empty (and site_access_key is also unset).
+    # - Falsy access_key values (empty string, None) are stripped — treated as "no key".
+    # - When a key IS provided, lib_sql_search handles the SQL expansion (see process_filter).
+    if obj_name == 'site_domain':
+        # Sanity check: drop access_key filters with falsy values
+        if search_query.and_filters:
+            search_query.and_filters = [
+                f for f in search_query.and_filters
+                if not (isinstance(f, SearchFilter) and f.field == 'access_key' and not f.value)
+            ]
+        key_fields = {'access_key', 'site_access_key', 'site_domain_access_key'}
+        has_key_filter = any(
+            isinstance(f, SearchFilter) and f.field in key_fields
+            for f in (search_query.and_filters or [])
+        )
+        if not has_key_filter:
+            if search_query.and_filters is None:
+                search_query.and_filters = []
+            for col in ('site_access_key', 'site_domain_access_key'):
+                search_query.and_filters.append(SearchQuery.parse_obj({
+                    'or': [{'field': col, 'op': 'is_null'}, {'field': col, 'op': 'eq', 'value': ''}]
+                }))
+
     if for_obj_type == 'account' and for_obj_id:
         if not account.super and for_obj_id != account.account_id_random:
             return mk_resp(data=False, status_code=403, response=response, status_message="Access denied to requested account.")
@@ -393,7 +420,7 @@ async def search_obj_li(
         return mk_resp(data=False, status_code=status_code, response=response, status_message="Search failed due to database error.", details=db_err.dict())
 
     if sql_result:
-        resp_data_li = [base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none) for record in sql_result]
+        resp_data_li = [apply_vision_id_fix(base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none), obj_name, serialization.by_alias) for record in sql_result]
         return mk_resp(data=resp_data_li, response=response)
     else:
         return mk_resp(data=[], status_code=200, response=response)
@@ -438,14 +465,9 @@ async def post_obj(
         return mk_resp(data=False, status_code=500, response=response, status_message=f"Configuration error.")
 
     if not account.super and account.auth_method != 'bypass' and account.account_id:
-        if 'account_id' in input_model.__fields__:
-            obj_data['account_id'] = account.account_id
-        elif obj_name == 'account':
+        if obj_name == 'account':
             return mk_resp(data=False, status_code=403, response=response, status_message="Account creation is restricted.")
 
-    # Sanitize payload (ID resolution, virtual fields, and optionally extra fields)
-    sanitize_payload(obj_data, input_model, ignore_extra=x_ae_ignore_extra_fields)
-
     try:
         validated_obj = input_model(**obj_data)
     except ValidationError as e:
@@ -457,6 +479,22 @@ async def post_obj(
 
     data_to_insert = validated_obj.dict(exclude_unset=True)
 
+    # Sanitize payload AFTER model validation so that:
+    # 1. The model receives raw Vision ID strings (passes field-length constraints).
+    # 2. ID resolution (string → integer) happens on the serialized dict that goes to the DB,
+    #    avoiding conflicts with root_validator collision-prevention logic.
+    sanitize_payload(data_to_insert, input_model, ignore_extra=x_ae_ignore_extra_fields)
+
+    # Enforce account ownership AFTER sanitize_payload so the integer account_id goes straight
+    # to the DB without conflicting with Vision ID string constraints in the model.
+    # Guard: skip if the model explicitly excludes account_id from DB writes (e.g. event_badge,
+    # event_device — the column does not exist in those tables).
+    if not account.super and account.auth_method != 'bypass' and account.account_id:
+        if 'account_id' in input_model.__fields__:
+            excluded = getattr(input_model, 'fields_to_exclude_from_db', [])
+            if 'account_id' not in excluded:
+                data_to_insert['account_id'] = account.account_id
+
     if sql_insert_result := sql_insert(data=data_to_insert, table_name=table_name_insert):
         new_obj_id = sql_insert_result
         new_obj_id_random = get_id_random(record_id=new_obj_id, table_name=obj_name)
@@ -464,8 +502,9 @@ async def post_obj(
         if return_obj:
             if sql_select_result := sql_select(table_name=table_name_select, record_id=new_obj_id):
                 resp_data = output_model(**sql_select_result).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset)
+                apply_vision_id_fix(resp_data, obj_name, serialization.by_alias)
                 return mk_resp(data=resp_data, response=response)
-        return mk_resp(data={"obj_id": new_obj_id, "obj_id_random": new_obj_id_random}, response=response)
+        return mk_resp(data={"obj_id": new_obj_id_random, "obj_id_random": new_obj_id_random}, response=response)
     else:
         # Standardized rich error bubbling
         db_err = format_db_error(get_last_sql_error())
@@ -527,6 +566,7 @@ async def patch_obj(
         if return_obj:
             if sql_select_result := sql_select(table_name=table_name_select, record_id=record_id):
                 resp_data = output_model(**sql_select_result).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset)
+                apply_vision_id_fix(resp_data, obj_name, serialization.by_alias)
                 return mk_resp(data=resp_data, response=response)
         return mk_resp(data=True, response=response, status_message="Object updated successfully.")
     else:

app/routers/api_crud_v3_nested.py

@@ -13,7 +13,8 @@ from app.lib_general_v3 import (
 )
 from app.lib_api_crud_v3 import (
     check_account_access, apply_forced_account_filter, filter_order_by,
-    get_supported_filters, safe_json_loads, sanitize_payload, format_db_error
+    get_supported_filters, safe_json_loads, sanitize_payload, format_db_error,
+    apply_vision_id_fix
 )
 from app.db_sql import get_last_sql_error
 from app.models.response_models import *
@@ -84,6 +85,9 @@ async def get_child_obj_li(
     table_name = obj_cfg.get(f'tbl_{view}', obj_cfg.get('tbl_default', obj_cfg.get('tbl')))
     base_name = obj_cfg.get(f'mdl_{view}', obj_cfg.get('mdl_default', obj_cfg.get('mdl')))
 
+    # Log parent/child resolution details (use INFO so logs appear in production)
+    log.info("nested.list start parent=%s parent_table=%s parent_id_random=%s child=%s table=%s allowed_parents=%s", parent_obj_type, parent_table, parent_obj_id, obj_name, table_name, obj_cfg.get('parent_types'))
+
     if not table_name or not base_name:
         return mk_resp(data=False, status_code=500, response=response, status_message=f"Configuration error.")
 
@@ -91,15 +95,26 @@ async def get_child_obj_li(
     status_filter = get_supported_filters(base_name, status_filter)
 
     resolved_parent_id = redis_lookup_id_random(record_id_random=parent_obj_id, table_name=parent_table)
+    log.info("nested.list resolved_parent_id=%s (random=%s) for parent_table=%s", resolved_parent_id, parent_obj_id, parent_table)
     if not resolved_parent_id:
+        log.info("nested.list parent resolution failed for random id=%s table=%s", parent_obj_id, parent_table)
         return mk_resp(data=False, status_code=404, response=response, status_message=f"Parent not found.")
 
+    # Enforce allowed parent types when configured on the child object
+    allowed_parents = obj_cfg.get('parent_types')
+    if allowed_parents and parent_obj_type not in allowed_parents:
+        log.info("nested.list invalid parent type: parent=%s allowed=%s", parent_obj_type, allowed_parents)
+        return mk_resp(data=False, status_code=400, response=response, status_message=f"Invalid parent type for this child.")
+
     parent_cfg = obj_type_kv_li[parent_obj_type]
     parent_table_select = parent_cfg.get('tbl_default', parent_cfg.get('tbl'))
     if parent_sql_res := sql_select(table_name=parent_table_select, record_id=resolved_parent_id):
+        log.info("nested.list parent_sql_res found for id=%s table=%s", resolved_parent_id, parent_table_select)
         if not check_account_access(parent_sql_res, account, parent_obj_type):
+             log.info("nested.list access denied to parent id=%s for account=%s", resolved_parent_id, getattr(account, 'account_id', None))
              return mk_resp(data=False, status_code=403, response=response, status_message="Access denied to parent.")
     else:
+        log.info("nested.list parent sql_select returned no row for id=%s table=%s", resolved_parent_id, parent_table_select)
         return mk_resp(data=False, status_code=404, response=response, status_message="Parent not found.")
 
     and_qry_dict_obj = apply_forced_account_filter(and_qry_dict_obj, account, base_name, obj_name, table_name=table_name)
@@ -132,7 +147,7 @@ async def get_child_obj_li(
         return mk_resp(data=False, status_code=status_code, response=response, status_message="Listing failed due to database error.", details=db_err.dict())
 
     if sql_result:
-        resp_data_li = [base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none) for record in sql_result]
+        resp_data_li = [apply_vision_id_fix(base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none), obj_name, serialization.by_alias) for record in sql_result]
         return mk_resp(data=resp_data_li, response=response)
     else:
         return mk_resp(data=[], status_code=200, response=response)
@@ -181,15 +196,26 @@ async def search_child_obj_li(
     searchable_fields = obj_cfg.get('searchable_fields')
 
     resolved_parent_id = redis_lookup_id_random(record_id_random=parent_obj_id, table_name=parent_table)
+    log.info("nested.search resolved_parent_id=%s (random=%s) for parent_table=%s", resolved_parent_id, parent_obj_id, parent_table)
     if not resolved_parent_id:
+        log.info("nested.search parent resolution failed for random id=%s table=%s", parent_obj_id, parent_table)
         return mk_resp(data=False, status_code=404, response=response, status_message="Parent not found.")
 
+    # Enforce allowed parent types when configured on the child object
+    allowed_parents = obj_cfg.get('parent_types')
+    if allowed_parents and parent_obj_type not in allowed_parents:
+        log.info("nested.search invalid parent type: parent=%s allowed=%s", parent_obj_type, allowed_parents)
+        return mk_resp(data=False, status_code=400, response=response, status_message="Invalid parent type for this child.")
+
     parent_cfg = obj_type_kv_li[parent_obj_type]
     parent_table_select = parent_cfg.get('tbl_default', parent_cfg.get('tbl'))
     if parent_sql_res := sql_select(table_name=parent_table_select, record_id=resolved_parent_id):
+        log.info("nested.search parent_sql_res found for id=%s table=%s", resolved_parent_id, parent_table_select)
         if not check_account_access(parent_sql_res, account, parent_obj_type):
+             log.info("nested.search access denied to parent id=%s for account=%s", resolved_parent_id, getattr(account, 'account_id', None))
              return mk_resp(data=False, status_code=403, response=response, status_message="Access denied to parent.")
     else:
+        log.info("nested.search parent sql_select returned no row for id=%s table=%s", resolved_parent_id, parent_table_select)
         return mk_resp(data=False, status_code=404, response=response, status_message="Parent not found.")
 
     # Enforce account isolation on the search query
@@ -218,7 +244,7 @@ async def search_child_obj_li(
         return mk_resp(data=False, status_code=status_code, response=response, status_message="Search failed due to database error.", details=db_err.dict())
 
     if sql_result:
-        resp_data_li = [base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none) for record in sql_result]
+        resp_data_li = [apply_vision_id_fix(base_name(**record).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset, exclude_defaults=serialization.exclude_defaults, exclude_none=serialization.exclude_none), obj_name, serialization.by_alias) for record in sql_result]
         return mk_resp(data=resp_data_li, response=response)
     else:
         return mk_resp(data=[], status_code=200, response=response)
@@ -256,18 +282,29 @@ async def post_child_obj(
     # ID Vision: Resolve physical table names from registry to support aliases
     parent_table = obj_type_kv_li[parent_obj_type].get('tbl')
     resolved_parent_id = redis_lookup_id_random(record_id_random=parent_obj_id, table_name=parent_table)
+    log.info("nested.post parent=%s parent_table=%s parent_id_random=%s", parent_obj_type, parent_table, parent_obj_id)
+    log.info("nested.post resolved_parent_id=%s for random=%s table=%s", resolved_parent_id, parent_obj_id, parent_table)
     if not resolved_parent_id:
+        log.info("nested.post parent resolution failed for random id=%s table=%s", parent_obj_id, parent_table)
         return mk_resp(data=False, status_code=404, response=response, status_message="Parent not found.")
 
     parent_cfg = obj_type_kv_li[parent_obj_type]
     parent_table_select = parent_cfg.get('tbl_default', parent_cfg.get('tbl'))
     if parent_sql_res := sql_select(table_name=parent_table_select, record_id=resolved_parent_id):
+        log.info("nested.post parent_sql_res found for id=%s table=%s", resolved_parent_id, parent_table_select)
         if not check_account_access(parent_sql_res, account, parent_obj_type):
+             log.info("nested.post access denied to parent id=%s for account=%s", resolved_parent_id, getattr(account, 'account_id', None))
              return mk_resp(data=False, status_code=403, response=response, status_message="Access denied to parent.")
     else:
+        log.info("nested.post parent sql_select returned no row for id=%s table=%s", resolved_parent_id, parent_table_select)
         return mk_resp(data=False, status_code=404, response=response, status_message="Parent not found.")
 
     obj_cfg = obj_type_kv_li[child_obj_type]
+    # Enforce allowed parent types when configured on the child object
+    allowed_parents = obj_cfg.get('parent_types')
+    if allowed_parents and parent_obj_type not in allowed_parents:
+        log.info("nested.post invalid parent type: parent=%s allowed=%s", parent_obj_type, allowed_parents)
+        return mk_resp(data=False, status_code=400, response=response, status_message="Invalid parent type for this child.")
     table_name_insert = obj_cfg.get('tbl_update', obj_cfg.get('tbl'))
     table_name_select = obj_cfg.get('tbl_default', obj_cfg.get('tbl'))
     input_model = obj_cfg.get('mdl_in', obj_cfg.get('mdl'))
@@ -276,15 +313,6 @@ async def post_child_obj(
     if not table_name_insert or not input_model or not table_name_select or not output_model:
         return mk_resp(data=False, status_code=500, response=response, status_message=f"Configuration error.")
 
-    if not account.super and account.auth_method != 'bypass' and account.account_id:
-        if 'account_id' in input_model.__fields__:
-            obj_data['account_id'] = account.account_id
-
-    obj_data[f'{parent_obj_type}_id'] = resolved_parent_id
-
-    # Sanitize payload (ID resolution, virtual fields, and optionally extra fields)
-    sanitize_payload(obj_data, input_model, ignore_extra=x_ae_ignore_extra_fields)
-
     try:
         validated_obj = input_model(**obj_data)
     except ValidationError as e:
@@ -295,6 +323,20 @@ async def post_child_obj(
 
     data_to_insert = validated_obj.dict(exclude_unset=True)
 
+    # Sanitize AFTER serialization so that:
+    # 1. The model receives raw Vision ID strings (passes field-length constraints).
+    # 2. ID resolution (string → integer) happens on the dict going to the DB,
+    #    avoiding the root_validator's integer-stripping anti-leakage guard.
+    # (Matches the flat V3 POST pattern in api_crud_v3.py.)
+    sanitize_payload(data_to_insert, input_model, ignore_extra=x_ae_ignore_extra_fields)
+
+    # Re-inject parent FK last — overrides anything sanitize_payload or the model may have
+    # set — ensuring the child is always linked to the correct parent.
+    # Note: account_id is intentionally NOT injected here. Child objects in the nested
+    # endpoint inherit account context from their parent via the FK relationship; they do
+    # not carry their own account_id column (e.g. event_badge, journal_entry).
+    data_to_insert[f'{parent_obj_type}_id'] = resolved_parent_id
+
     if sql_insert_result := sql_insert(data=data_to_insert, table_name=table_name_insert):
         new_obj_id = sql_insert_result
         new_obj_id_random = get_id_random(record_id=new_obj_id, table_name=child_obj_type)
@@ -302,8 +344,9 @@ async def post_child_obj(
         if return_obj:
             if sql_select_result := sql_select(table_name=table_name_select, record_id=new_obj_id):
                 resp_data = output_model(**sql_select_result).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset)
+                apply_vision_id_fix(resp_data, child_obj_type, serialization.by_alias)
                 return mk_resp(data=resp_data, response=response)
-        return mk_resp(data={"obj_id": new_obj_id, "obj_id_random": new_obj_id_random}, response=response)
+        return mk_resp(data={"obj_id": new_obj_id_random, "obj_id_random": new_obj_id_random}, response=response)
     else:
         # Standardized rich error bubbling
         db_err = format_db_error(get_last_sql_error())
@@ -345,6 +388,10 @@ async def get_child_obj(
         return mk_resp(data=False, status_code=404, response=response, status_message="Object(s) not found.")
 
     obj_cfg = obj_type_kv_li[child_obj_type]
+    # Enforce allowed parent types when configured on the child object
+    allowed_parents = obj_cfg.get('parent_types')
+    if allowed_parents and parent_obj_type not in allowed_parents:
+        return mk_resp(data=False, status_code=400, response=response, status_message="Invalid parent type for this child.")
     table_name = obj_cfg.get(f'tbl_{view}', obj_cfg.get('tbl_default', obj_cfg.get('tbl')))
     base_name = obj_cfg.get(f'mdl_{view}', obj_cfg.get('mdl_default', obj_cfg.get('mdl')))
 
@@ -353,6 +400,7 @@ async def get_child_obj(
             return mk_resp(data=False, status_code=404, response=response, status_message="Child not found under parent.")
 
         resp_data = base_name(**sql_result).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset)
+        apply_vision_id_fix(resp_data, child_obj_type, serialization.by_alias)
         return mk_resp(data=resp_data, response=response)
     return mk_resp(data=False, status_code=404, response=response, status_message="Child not found.")
 
@@ -414,6 +462,7 @@ async def patch_child_obj(
         if return_obj:
             if updated_child := sql_select(table_name=table_name_select, record_id=resolved_child_id):
                 resp_data = output_model(**updated_child).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset)
+                apply_vision_id_fix(resp_data, child_obj_type, serialization.by_alias)
                 return mk_resp(data=resp_data, response=response)
         return mk_resp(data=True, response=response, status_message="Updated successfully.")
     else:
@@ -421,116 +470,6 @@ async def patch_child_obj(
         return mk_resp(data=False, status_code=400, response=response, status_message="Update failed.", details=db_err.dict())
 
 
-@router.get('/{parent_obj_type}/{parent_obj_id}/{child_obj_type}/{child_obj_id}', response_model=Resp_Body_Base)
-async def get_child_obj(
-        response: Response,
-        parent_obj_type: str = Path(min_length=2, max_length=50),
-        parent_obj_id: str = Path(min_length=11, max_length=22),
-        child_obj_type: str = Path(min_length=2, max_length=50),
-        child_obj_id: str = Path(min_length=11, max_length=22),
-        view: str = Query('default'),
-        account: AccountContext = Depends(get_account_context),
-        serialization: SerializationParams = Depends(),
-        delay: DelayParams = Depends(),
-        ):
-    """
-    Retrieve Child Object.
-    
-    Verifies that the child belongs to the specified parent.
-    """
-    from app.db_sql import redis_lookup_id_random, sql_select
-
-    if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
-
-    # ID Vision: Resolve physical table names from registry to support aliases
-    if parent_obj_type not in obj_type_kv_li or child_obj_type not in obj_type_kv_li:
-        return mk_resp(data=False, status_code=400, response=response, status_message="Invalid object type(s).")
-
-    parent_table = obj_type_kv_li[parent_obj_type].get('tbl')
-    child_table = obj_type_kv_li[child_obj_type].get('tbl')
-
-    resolved_parent_id = redis_lookup_id_random(record_id_random=parent_obj_id, table_name=parent_table)
-    resolved_child_id = redis_lookup_id_random(record_id_random=child_obj_id, table_name=child_table)
-
-    if not resolved_parent_id or not resolved_child_id:
-        return mk_resp(data=False, status_code=404, response=response, status_message="Object(s) not found.")
-
-    obj_cfg = obj_type_kv_li[child_obj_type]
-    table_name = obj_cfg.get(f'tbl_{view}', obj_cfg.get('tbl_default', obj_cfg.get('tbl')))
-    base_name = obj_cfg.get(f'mdl_{view}', obj_cfg.get('mdl_default', obj_cfg.get('mdl')))
-
-    if sql_result := sql_select(table_name=table_name, record_id=resolved_child_id):
-        if sql_result.get(f'{parent_obj_type}_id') != resolved_parent_id:
-            return mk_resp(data=False, status_code=404, response=response, status_message="Child not found under parent.")
-
-        resp_data = base_name(**sql_result).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset)
-        return mk_resp(data=resp_data, response=response)
-    return mk_resp(data=False, status_code=404, response=response, status_message="Child not found.")
-
-
-@router.patch('/{parent_obj_type}/{parent_obj_id}/{child_obj_type}/{child_obj_id}', response_model=Resp_Body_Base)
-async def patch_child_obj(
-        request: Request,
-        response: Response,
-        parent_obj_type: str = Path(min_length=2, max_length=50),
-        parent_obj_id: str = Path(min_length=11, max_length=22),
-        child_obj_type: str = Path(min_length=2, max_length=50),
-        child_obj_id: str = Path(min_length=11, max_length=22),
-        return_obj: Optional[bool] = True,
-        x_ae_ignore_extra_fields: Optional[bool] = Header(False),
-        account: AccountContext = Depends(get_account_context),
-        serialization: SerializationParams = Depends(),
-        delay: DelayParams = Depends(),
-        ):
-    """
-    Update Child Object.
-    
-    Verifies that the child belongs to the specified parent before updating.
-    """
-    from app.db_sql import redis_lookup_id_random, sql_select, sql_update
-
-    if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
-
-    obj_data = await request.json()
-
-    # ID Vision: Resolve physical table names from registry to support aliases
-    if parent_obj_type not in obj_type_kv_li or child_obj_type not in obj_type_kv_li:
-        return mk_resp(data=False, status_code=400, response=response, status_message="Invalid object type(s).")
-
-    parent_table = obj_type_kv_li[parent_obj_type].get('tbl')
-    child_table = obj_type_kv_li[child_obj_type].get('tbl')
-
-    resolved_parent_id = redis_lookup_id_random(record_id_random=parent_obj_id, table_name=parent_table)
-    resolved_child_id = redis_lookup_id_random(record_id_random=child_obj_id, table_name=child_table)
-
-    if not resolved_parent_id or not resolved_child_id:
-        return mk_resp(data=False, status_code=404, response=response, status_message="Object(s) not found.")
-
-    obj_cfg = obj_type_kv_li[child_obj_type]
-    table_name_update = obj_cfg.get('tbl_update', obj_cfg.get('tbl'))
-    table_name_select = obj_cfg.get('tbl_default', obj_cfg.get('tbl'))
-    input_model = obj_cfg.get('mdl_in', obj_cfg.get('mdl'))
-    output_model = obj_cfg.get('mdl_out', obj_cfg.get('mdl_default', obj_cfg.get('mdl')))
-
-    if existing_child := sql_select(table_name=table_name_select, record_id=resolved_child_id):
-        if existing_child.get(f'{parent_obj_type}_id') != resolved_parent_id:
-            return mk_resp(data=False, status_code=404, response=response, status_message="Child not found under parent.")
-    else:
-        return mk_resp(data=False, status_code=404, response=response, status_message="Child not found.")
-
-    # Sanitize payload (ID resolution, virtual fields, and optionally extra fields)
-    sanitize_payload(obj_data, input_model, ignore_extra=x_ae_ignore_extra_fields)
-
-    if sql_update(data=obj_data, table_name=table_name_update, record_id=resolved_child_id):
-        if return_obj:
-            if updated_child := sql_select(table_name=table_name_select, record_id=resolved_child_id):
-                resp_data = output_model(**updated_child).dict(by_alias=serialization.by_alias, exclude_unset=serialization.exclude_unset)
-                return mk_resp(data=resp_data, response=response)
-        return mk_resp(data=True, response=response, status_message="Updated successfully.")
-    else:
-        db_err = format_db_error(get_last_sql_error())
-        return mk_resp(data=False, status_code=400, response=response, status_message="Update failed.", details=db_err.dict())
-
 
 @router.delete('/{parent_obj_type}/{parent_obj_id}/{child_obj_type}/{child_obj_id}', response_model=Resp_Body_Base)
 async def delete_child_obj(

app/routers/api_v3_actions_e_novi_mailman.py

@@ -1,17 +1,20 @@
-from fastapi import APIRouter, Body, Depends, Query
-from typing import Optional
 import asyncio
 
-from app.lib_general import log
+from fastapi import APIRouter, Depends, Query
+from typing import Optional
+
 from app.lib_general_v3 import AccountContext, get_account_context, DelayParams
 from app.models.response_models import Resp_Body_Base, mk_resp
 from app.methods.e_novi_mailman_methods import (
     test_novi_connection,
     test_mailman_connection,
     get_mailman_lists,
+    get_mailman_list_members,
+    subscribe_member_to_list,
+    unsubscribe_member_from_list,
     get_novi_members,
-    sync_novi_to_mailman,
-    handle_novi_webhook,
+    mirror_novi_group_to_mailman_list,
+    mirror_all_configured_mappings,
 )
 
 router = APIRouter()
@@ -24,7 +27,7 @@ async def test_novi(
         account: AccountContext = Depends(get_account_context),
         delay: DelayParams = Depends(),
         ):
-    """Verify Novi AMS API credentials stored in data_store."""
+    """Verify Novi AMS API credentials from IDAA site cfg_json."""
     if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
     result = test_novi_connection()
     if result.get('ok'):
@@ -37,7 +40,7 @@ async def test_mailman(
         account: AccountContext = Depends(get_account_context),
         delay: DelayParams = Depends(),
         ):
-    """Verify Mailman 3 REST API credentials stored in data_store."""
+    """Verify Mailman 3 REST API credentials from IDAA site cfg_json."""
     if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
     result = test_mailman_connection()
     if result.get('ok'):
@@ -47,6 +50,53 @@ async def test_mailman(
 
 # ── Inspection / Preview ──────────────────────────────────────────────────
 
+@router.get('/mailman/lists/{list_id}/members', response_model=Resp_Body_Base)
+async def list_mailman_list_members(
+        list_id: str,
+        count: int = Query(100, ge=1, le=500),
+        page: int = Query(1, ge=1),
+        account: AccountContext = Depends(get_account_context),
+        delay: DelayParams = Depends(),
+        ):
+    """Return members of a specific Mailman 3 list."""
+    if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
+    data = get_mailman_list_members(list_id=list_id, count=count, page=page)
+    if data is not None:
+        return mk_resp(data={"count": len(data), "members": data})
+    return mk_resp(data=False, status_code=500, status_message=f"Failed to fetch members for list '{list_id}'.")
+
+
+@router.post('/mailman/lists/{list_id}/subscribe', response_model=Resp_Body_Base)
+async def mailman_subscribe(
+        list_id: str,
+        email: str = Query(..., description="Email address to subscribe"),
+        display_name: str = Query('', description="Optional display name"),
+        account: AccountContext = Depends(get_account_context),
+        delay: DelayParams = Depends(),
+        ):
+    """Subscribe an email address to a Mailman 3 list."""
+    if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
+    ok = subscribe_member_to_list(list_id=list_id, email=email, display_name=display_name)
+    if ok:
+        return mk_resp(data={"list_id": list_id, "email": email}, status_message="Subscribed successfully.")
+    return mk_resp(data=False, status_code=500, status_message=f"Failed to subscribe {email} to {list_id}.")
+
+
+@router.delete('/mailman/lists/{list_id}/subscribe', response_model=Resp_Body_Base)
+async def mailman_unsubscribe(
+        list_id: str,
+        email: str = Query(..., description="Email address to unsubscribe"),
+        account: AccountContext = Depends(get_account_context),
+        delay: DelayParams = Depends(),
+        ):
+    """Unsubscribe an email address from a Mailman 3 list."""
+    if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
+    ok = unsubscribe_member_from_list(list_id=list_id, email=email)
+    if ok:
+        return mk_resp(data={"list_id": list_id, "email": email}, status_message="Unsubscribed successfully.")
+    return mk_resp(data=False, status_code=500, status_message=f"Failed to unsubscribe {email} from {list_id}.")
+
+
 @router.get('/mailman/lists', response_model=Resp_Body_Base)
 async def list_mailman_lists(
         account: AccountContext = Depends(get_account_context),
@@ -68,7 +118,7 @@ async def list_novi_members(
         account: AccountContext = Depends(get_account_context),
         delay: DelayParams = Depends(),
         ):
-    """Fetch a page of Novi AMS members. Useful for inspecting data before a full sync."""
+    """Fetch a page of Novi AMS members."""
     if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
     data = get_novi_members(status_filter=status_filter, page_size=page_size, offset=offset)
     if data is not None:
@@ -79,38 +129,36 @@ async def list_novi_members(
 # ── Sync ──────────────────────────────────────────────────────────────────
 
 @router.post('/sync', response_model=Resp_Body_Base)
-async def sync_full(
-        list_id: str = Query(..., description="Target Mailman list ID, e.g. 'members@yourdomain.org'"),
-        active_status: str = Query('Active', description="Novi membership status that maps to 'subscribed'"),
+async def sync_all_mappings(
         account: AccountContext = Depends(get_account_context),
         delay: DelayParams = Depends(),
         ):
     """
-    Full sync: pull all Novi members and reconcile Mailman subscription state.
-    Active members are subscribed; all others are unsubscribed.
-    This is the manual / scheduled trigger — the webhook handles real-time updates.
+    Run all Novi → Mailman mirror syncs configured in novi_mailman_sync (IDAA cfg_json).
+    This is the cron target — call on a schedule to keep all lists in sync.
     """
     if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
-    result = sync_novi_to_mailman(list_id=list_id, active_status=active_status)
-    if result:
-        return mk_resp(data=result, status_message="Novi → Mailman sync complete.")
-    return mk_resp(data=False, status_code=500, status_message="Sync failed.")
+    results = mirror_all_configured_mappings()
+    if results is not None:
+        return mk_resp(data=results, status_message=f"Mirror sync complete. {len(results)} mapping(s) processed.")
+    return mk_resp(data=False, status_code=500, status_message="Mirror sync failed.")
 
 
-# ── Webhook ───────────────────────────────────────────────────────────────
-
-@router.post('/webhook/novi', response_model=Resp_Body_Base, include_in_schema=True)
-async def novi_membership_webhook(
-        payload: dict = Body(...),
+@router.post('/sync/group/{novi_group_guid}', response_model=Resp_Body_Base)
+async def sync_single_group(
+        novi_group_guid: str,
+        mailman_list_id: str = Query(..., description="Target Mailman list, e.g. 'mm3@idaa.org'"),
+        account: AccountContext = Depends(get_account_context),
+        delay: DelayParams = Depends(),
         ):
     """
-    Receives Novi AMS membership webhook events and immediately updates
-    the corresponding Mailman subscription — no auth required (Novi pushes to this endpoint).
-
-    TODO: Add HMAC signature verification once Novi webhook secret is configured.
+    Mirror a single Novi group to a specific Mailman list.
+    Useful for testing or forcing a refresh of one mapping.
     """
-    log.info(f"Novi webhook received: EventType={payload.get('EventType')} Email={payload.get('Member', {}).get('Email')}")
-    result = handle_novi_webhook(payload)
+    if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
+    result = mirror_novi_group_to_mailman_list(novi_group_guid, mailman_list_id)
     if result:
-        return mk_resp(data=result)
-    return mk_resp(data=False, status_code=400, status_message="Webhook payload could not be processed.")
+        return mk_resp(data=result, status_message="Mirror sync complete.")
+    return mk_resp(data=False, status_code=500, status_message="Mirror sync failed.")
+
+

app/routers/api_v3_actions_email.py

@@ -0,0 +1,78 @@
+"""
+Aether API V3 - Email Action Router
+-------------------------------------
+Handles transactional email sending.
+
+Routes:
+  POST /send — send a transactional email
+
+Replaces: POST /util/email/send (legacy — see util_email.py)
+"""
+
+import logging
+from typing import Optional
+
+from fastapi import APIRouter, Depends, Query, Response
+from pydantic import BaseModel, Field
+
+from app.lib_email import send_email
+from app.lib_general_v3 import AccountContext, get_account_context
+from app.models.response_models import Resp_Body_Base, mk_resp
+
+log = logging.getLogger(__name__)
+
+router = APIRouter()
+
+
+class EmailSendRequest(BaseModel):
+    from_email: str = Field(..., description="Sender email address")
+    from_name: Optional[str] = None
+    to_email: str = Field(..., description="Recipient email address")
+    to_name: Optional[str] = None
+    cc_email: Optional[str] = None
+    cc_name: Optional[str] = None
+    bcc_email: Optional[str] = None
+    bcc_name: Optional[str] = None
+    subject: str = Field(..., description="Email subject line")
+    body_html: str = Field(..., description="HTML email body")
+    body_text: Optional[str] = None
+
+
+@router.post('/send', response_model=Resp_Body_Base)
+async def action_email_send(
+        req: EmailSendRequest,
+        test: bool = Query(False, description="Simulate send without delivering"),
+        account_ctx: AccountContext = Depends(get_account_context),
+        response: Response = Response,
+        ):
+    log.setLevel(logging.INFO)
+
+    success = send_email(
+        from_email=req.from_email,
+        from_name=req.from_name,
+        to_email=req.to_email,
+        to_name=req.to_name,
+        cc_email=req.cc_email or '',
+        cc_name=req.cc_name or '',
+        bcc_email=req.bcc_email or '',
+        bcc_name=req.bcc_name or '',
+        subject=req.subject,
+        body_text=req.body_text,
+        body_html=req.body_html,
+        test=test,
+    )
+
+    if success:
+        status_code = 200
+        status_message = f'Email sent to <{req.to_email}>.'
+    else:
+        status_code = 400
+        status_message = f'Email failed to send to <{req.to_email}>.'
+
+    log.info(status_message)
+    resp_data = {
+        'from_email': req.from_email,
+        'to_email': req.to_email,
+        'subject': req.subject[:40],
+    }
+    return mk_resp(data=resp_data, status_code=status_code, response=response, status_message=status_message)

app/routers/api_v3_actions_event_exhibit.py

@@ -0,0 +1,232 @@
+import datetime
+import logging
+import os
+import pathlib
+import re
+from typing import Optional
+
+import pandas
+from fastapi import APIRouter, Depends, HTTPException, Path, Query, status
+from fastapi.responses import FileResponse
+
+log = logging.getLogger(__name__)
+
+from app.config import settings
+from app.db_sql import redis_lookup_id_random, sql_select
+from app.lib_api_crud_v3 import check_account_access
+from app.lib_export import create_export_file, return_full_tmp_path
+from app.lib_general_v3 import AccountContext, get_account_context
+from app.methods.event_exhibit_tracking_methods import (
+    get_event_exhibit_tracking_rec_list,
+    load_event_exhibit_tracking_obj,
+)
+from app.models.response_models import mk_resp
+
+"""
+Aether API V3 - Event Exhibit Action Router
+---------------------------------------------
+Handles specialized actions for the Event Exhibit module, such as
+exporting tracking (lead) data for exhibitors.
+"""
+
+router = APIRouter()
+
+# --- Helpers ---
+
+_HTML_TAG_RE = re.compile(r'<[^>]+>')
+
+
+def _strip_html(text: Optional[str]) -> Optional[str]:
+    if not text:
+        return text
+    return _HTML_TAG_RE.sub('', text)
+
+
+def _flatten_responses(responses_json: Optional[dict]) -> dict:
+    """
+    Flatten responses_json into key→value pairs for CSV/Excel export.
+    New format:  { question_code: { response: <value> } }  → value = inner['response']
+    Legacy format: { label: <scalar> }                     → value = scalar
+    """
+    if not responses_json:
+        return {}
+    flat = {}
+    for key, value in responses_json.items():
+        if isinstance(value, dict):
+            flat[key] = value.get('response')
+        else:
+            flat[key] = value
+    return flat
+
+
+# --- Routes ---
+
+@router.get('/{exhibit_id}/tracking_export')
+async def export_exhibit_tracking(
+        exhibit_id: str = Path(..., min_length=11, max_length=22),
+        file_type: str = Query('CSV', regex=r'^(CSV|XLSX)$'),
+        return_file: bool = Query(True),
+        account: AccountContext = Depends(get_account_context),
+        ):
+    """
+    V3 Action: Export all tracking (lead capture) records for an exhibit.
+
+    Auth: Requires `leads_api_access == True` on the exhibit OR manager-level account access.
+    Returns a CSV or XLSX file attachment.
+    """
+    # 1. Resolve random ID → internal integer
+    exhibit_int_id = redis_lookup_id_random(record_id_random=exhibit_id, table_name='event_exhibit')
+    if not exhibit_int_id:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail='Exhibit not found.')
+
+    # 2. Load exhibit record for ownership + permission checks
+    exhibit_rec = sql_select(table_name='v_event_exhibit', record_id=exhibit_int_id)
+    if not exhibit_rec:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail='Exhibit not found.')
+
+    # 3. Multi-tenant ownership check
+    if not check_account_access(exhibit_rec, account, 'event_exhibit'):
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail='Access denied: this exhibit belongs to a different account.',
+        )
+
+    # 4. Permission: leads_api_access flag OR manager-level access
+    if not exhibit_rec.get('leads_api_access') and not account.manager:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail='Access denied: leads API access is not enabled for this exhibit.',
+        )
+
+    # 5. Fetch all tracking records — no hidden/enabled filter, full export
+    tracking_rec_list = get_event_exhibit_tracking_rec_list(
+        event_exhibit_id=exhibit_int_id,
+        hidden='all',
+        enabled='all',
+        limit=1500,
+    )
+    if tracking_rec_list is False:
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail='Failed to retrieve tracking records.',
+        )
+
+    # 6. Build export rows
+    data_rows = []
+    response_keys: list = []  # ordered unique custom-question column names
+
+    for rec in (tracking_rec_list or []):
+        tracking_obj = load_event_exhibit_tracking_obj(
+            event_exhibit_tracking_id=rec.get('event_exhibit_tracking_id'),
+        )
+        if not tracking_obj:
+            continue
+
+        # Apply exhibitor-side field overrides (override values replace badge defaults)
+        if tracking_obj.event_badge_full_name_override:
+            tracking_obj.event_badge_full_name = tracking_obj.event_badge_full_name_override
+        if tracking_obj.event_badge_professional_title_override:
+            tracking_obj.event_badge_professional_title = tracking_obj.event_badge_professional_title_override
+        if tracking_obj.event_badge_affiliations_override:
+            tracking_obj.event_badge_affiliations = tracking_obj.event_badge_affiliations_override
+        if tracking_obj.event_badge_email_override:
+            tracking_obj.event_badge_email = tracking_obj.event_badge_email_override
+        if tracking_obj.event_badge_location_override:
+            tracking_obj.event_badge_location = tracking_obj.event_badge_location_override
+
+        # Flatten custom Q&A responses and collect column keys (order-preserving dedup)
+        responses = _flatten_responses(tracking_obj.responses_json)
+        for key in responses:
+            if key not in response_keys:
+                response_keys.append(key)
+
+        row = {
+            'event_exhibit_tracking_id': tracking_obj.event_exhibit_tracking_id,
+            'created_on': tracking_obj.created_on,
+            'updated_on': tracking_obj.updated_on,
+            'event_exhibit_name': tracking_obj.event_exhibit_name,
+            'event_badge_full_name': tracking_obj.event_badge_full_name,
+            'event_badge_email': tracking_obj.event_badge_email,
+            'event_badge_professional_title': tracking_obj.event_badge_professional_title,
+            'event_badge_affiliations': tracking_obj.event_badge_affiliations,
+            'event_badge_location': tracking_obj.event_badge_location,
+            'event_badge_country': tracking_obj.event_badge_country,
+            'external_person_id': tracking_obj.external_person_id,
+            'exhibitor_notes': _strip_html(tracking_obj.exhibitor_notes),
+            'priority': tracking_obj.priority,
+            'enable': tracking_obj.enable,
+            'hide': tracking_obj.hide,
+            **responses,
+        }
+        data_rows.append(row)
+
+    # 7. Determine file format
+    export_type = 'Excel' if file_type == 'XLSX' else 'CSV'
+    content_type = (
+        'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
+        if file_type == 'XLSX' else 'text/csv'
+    )
+    datetime_str = datetime.datetime.utcnow().strftime('%Y-%m-%d_%H%M')
+    filename = f'leads_export_{datetime_str}'
+    ext = '.xlsx' if export_type == 'Excel' else '.csv'
+    filename_w_ext = filename + ext
+
+    fixed_columns = [
+        'event_exhibit_tracking_id',
+        'created_on',
+        'updated_on',
+        'event_exhibit_name',
+        'event_badge_full_name',
+        'event_badge_email',
+        'event_badge_professional_title',
+        'event_badge_affiliations',
+        'event_badge_location',
+        'event_badge_country',
+        'external_person_id',
+        'exhibitor_notes',
+        'priority',
+        'enable',
+        'hide',
+    ]
+    column_name_li = fixed_columns + response_keys
+
+    # 8. Handle empty result — write headers-only file
+    if not data_rows:
+        hosted_tmp_path = settings.FILES_PATH['hosted_tmp_root']
+        subdir = os.path.join(hosted_tmp_path, 'event_exhibit')
+        pathlib.Path(subdir).mkdir(parents=True, exist_ok=True)
+        full_path = os.path.join(subdir, filename_w_ext)
+        df = pandas.DataFrame(columns=fixed_columns)
+        if export_type == 'CSV':
+            df.to_csv(full_path, index=False)
+        else:
+            df.to_excel(full_path, index=False)
+        if return_file:
+            return FileResponse(path=full_path, filename=filename_w_ext, media_type=content_type)
+        return mk_resp(data=[], tmp_file_path=filename_w_ext)
+
+    # 9. Generate the export file
+    tmp_file_path = create_export_file(
+        data_dict_list=data_rows,
+        column_name_li=column_name_li,
+        subdir_path='event_exhibit',
+        filename=filename,
+        rm_id=False,
+        export_type=export_type,
+    )
+    if not tmp_file_path:
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail='Failed to generate export file.',
+        )
+
+    if return_file:
+        full_path = return_full_tmp_path(full_tmp_path=tmp_file_path)
+        if not full_path:
+            raise HTTPException(
+                status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                detail='Export file not found after creation.',
+            )
+        return FileResponse(path=full_path, filename=filename_w_ext, media_type=content_type)
+
+    return mk_resp(data=data_rows, tmp_file_path=tmp_file_path)

app/routers/api_v3_actions_hosted_file.py

@@ -15,7 +15,8 @@ from app.config import settings
 from app.db_sql import redis_lookup_id_random, sql_select, sql_update, sql_delete, get_id_random
 from app.methods.hosted_file_methods import (
     create_hosted_file_obj, load_hosted_file_obj, save_file,
-    create_hosted_file_link, delete_hosted_file_link, get_hosted_file_link_rec_list
+    create_hosted_file_link, delete_hosted_file_link, get_hosted_file_link_rec_list,
+    lookup_file_hash, check_for_hosted_file_hash_file
 )
 from app.methods.lib_media import convert_file_method
 from app.methods.lib_media import clip_video_method
@@ -354,6 +355,38 @@ async def download_file_by_hash_action(
     return FileResponse(full_file_path, filename=target_filename, media_type=media_type)
 
 
+@router.get('/hash/{hosted_file_hash}', response_model=Resp_Body_Base)
+async def check_hosted_file_obj_w_hash_action(
+        response: Response,
+        hosted_file_hash: str = Path(min_length=64, max_length=64),
+        check_for_local: Optional[bool] = Query(True),
+        account: AccountContext = Depends(get_account_context_optional),
+        delay: DelayParams = Depends(),
+        ):
+    """
+    Look up a hosted_file record by its hash (Deduplication Check).
+    Optionally verifies physical file existence on disk.
+    """
+    if delay.sleep_time_s > 0: await asyncio.sleep(delay.sleep_time_s)
+
+    if hfid := lookup_file_hash(file_hash=hosted_file_hash):
+        obj_model = load_hosted_file_obj(hosted_file_id=hfid, model_as_dict=False)
+        if not obj_model:
+             return mk_resp(data=False, status_code=404, response=response, status_message="Record found but data could not be loaded.")
+
+        if check_for_local:
+            # We use the model directly to access subdirectory_path even if it's excluded from dicts
+            sub_dir = getattr(obj_model, 'subdirectory_path', '') or ''
+            if check := check_for_hosted_file_hash_file(file_hash=hosted_file_hash, sub_dir=sub_dir):
+                obj_model.hosted_file_found_check = True
+                obj_model.hosted_file_size_check = check['file_size']
+
+        # mk_resp will handle model->dict conversion with proper ID Vision mapping
+        return mk_resp(data=obj_model)
+
+    return mk_resp(data=False, status_code=404, response=response, status_message="No record found for this hash.")
+
+
 @router.delete('/{hosted_file_id}', response_model=Resp_Body_Base)
 async def delete_file_action(
         hosted_file_id: str = Path(min_length=11, max_length=22),

app/routers/api_v3_actions_idaa.py

@@ -0,0 +1,41 @@
+import asyncio
+
+from fastapi import APIRouter, Depends
+
+from app.lib_general_v3 import AccountContext, get_account_context, DelayParams
+from app.models.response_models import Resp_Body_Base, mk_resp
+from app.methods.idaa_novi_verify_methods import verify_novi_member
+
+router = APIRouter()
+
+
+@router.get('/novi_member/{uuid}', response_model=Resp_Body_Base)
+async def get_novi_member_verification(
+        uuid: str,
+        account: AccountContext = Depends(get_account_context),
+        delay: DelayParams = Depends(),
+        ):
+    """
+    Proxy Novi AMS member lookup server-to-server.
+    Returns verified member identity or an appropriate error code.
+    """
+    if delay.sleep_time_s > 0:
+        await asyncio.sleep(delay.sleep_time_s)
+
+    result = verify_novi_member(uuid)
+    status = result.get('status', 503)
+
+    if status == 200:
+        return mk_resp(data={
+            'verified': result['verified'],
+            'full_name': result['full_name'],
+            'email': result['email'],
+        })
+
+    if status == 404:
+        return mk_resp(data=False, status_code=404, status_message=result.get('reason', 'Member not found.'))
+
+    if status == 429:
+        return mk_resp(data=False, status_code=429, status_message=result.get('reason', 'Novi rate limit exceeded.'))
+
+    return mk_resp(data=False, status_code=503, status_message=result.get('reason', 'Novi API unavailable.'))

app/routers/api_v3_actions_user.py

@@ -0,0 +1,299 @@
+"""
+Aether API V3 - User Action Router
+------------------------------------
+Handles secure, stateful user account operations that are not standard CRUD.
+
+Routes:
+  POST  /authenticate              — username+password or user_id+auth_key (body, not query params)
+  POST  /verify_password           — verify a user's current password without changing it
+  POST  /{user_id}/change_password — change password (with optional current-password verification)
+  GET   /{user_id}/new_auth_key    — generate a new one-time login auth key
+  GET   /{user_id}/email_auth_key_url — email a one-time login link to the user
+
+Security improvements over legacy /user/* routes:
+  - Credentials are in the POST body, never in query params (no URL logging exposure).
+  - Uses V3 AccountContext (x-aether-api-key mandatory).
+  - HTTPException for all error paths (native FastAPI status codes).
+"""
+
+from fastapi import APIRouter, Body, Depends, HTTPException, Path, Query, status
+import datetime
+import logging
+from typing import Optional
+from pydantic import BaseModel, Field
+
+from app.db_sql import redis_lookup_id_random, sql_select, sql_update
+from app.lib_general import secure_hash_string, verify_secure_hash_string
+from app.lib_general_v3 import AccountContext, get_account_context
+from app.methods.user_methods import email_user_auth_key_url, load_user_obj
+from app.models.common_field_schema import default_num_bytes
+from app.models.response_models import Resp_Body_Base, mk_resp
+
+log = logging.getLogger(__name__)
+
+router = APIRouter()
+
+
+# --- Request Body Models ---
+
+class ChangePasswordRequest(BaseModel):
+    new_password: str = Field(..., min_length=10, max_length=100)
+    current_password: Optional[str] = Field(None, description="If provided, verified before applying the change.")
+
+
+class AuthenticateRequest(BaseModel):
+    """Provide either username+password or user_id+auth_key."""
+    username: Optional[str] = Field(None, min_length=3, max_length=50)
+    password: Optional[str] = Field(None, min_length=8, max_length=100)
+    user_id: Optional[str] = Field(None, min_length=11, max_length=22, description="Vision ID (id_random) of the user.")
+    auth_key: Optional[str] = Field(None, min_length=11, max_length=22)
+    valid_email: Optional[bool] = Field(None, description="If True, marks email_verified=True on successful auth.")
+
+
+class VerifyPasswordRequest(BaseModel):
+    """Provide user_id (Vision ID) or username, plus the password to verify."""
+    current_password: str = Field(..., min_length=1, max_length=100)
+    user_id: Optional[str] = Field(None, min_length=11, max_length=22)
+    username: Optional[str] = Field(None, min_length=2, max_length=50)
+
+
+# --- Internal Helper ---
+
+def _check_user_enabled(rec: dict) -> Optional[str]:
+    """
+    Returns an error message string if the user account is not currently active, None if OK.
+    Checks: enable flag, enable_from, enable_to (all treated as UTC).
+    """
+    if not rec.get('enable'):
+        return 'This user account is not enabled.'
+    now = datetime.datetime.now(datetime.timezone.utc)
+    if enable_from := rec.get('enable_from'):
+        ef = enable_from.replace(tzinfo=datetime.timezone.utc)
+        if ef > now:
+            return f'This user account is not yet enabled (active from: {ef}).'
+    if enable_to := rec.get('enable_to'):
+        et = enable_to.replace(tzinfo=datetime.timezone.utc)
+        if et < now:
+            return f'This user account has expired (expired: {et}).'
+    return None
+
+
+# --- Routes ---
+
+@router.post('/authenticate', response_model=Resp_Body_Base)
+async def action_authenticate(
+    body: AuthenticateRequest = Body(...),
+    inc_user_role_list: bool = Query(False),
+    account: AccountContext = Depends(get_account_context),
+):
+    """
+    Authenticate a user by username+password or user_id+auth_key.
+
+    - Credentials are in the POST body (not query params) — safe from URL logging.
+    - Auth key is one-time-use: cleared on successful authentication.
+    - On success: stamps logged_in_on, returns the full user object.
+    - Provide x-account-id to scope username lookups to the correct account.
+    """
+    account_id = account.account_id
+
+    if body.username and body.password:
+        sql = """
+            SELECT id AS user_id, id_random AS user_id_random, password,
+                   enable, enable_from, enable_to
+            FROM `user`
+            WHERE account_id = :account_id AND username = :username
+            LIMIT 1
+        """
+        rec = sql_select(sql=sql, data={'account_id': account_id, 'username': body.username})
+        if not rec:
+            raise HTTPException(status_code=status.HTTP_404_NOT_FOUND,
+                                detail='User not found for this account and username.')
+        if not rec.get('password'):
+            raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST,
+                                detail='No password is set for this user.')
+        if not verify_secure_hash_string(string=body.password, string_hash=rec['password']):
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN,
+                                detail='Password did not match.')
+        if err := _check_user_enabled(rec):
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=err)
+
+        db_user_id = rec['user_id']
+        update_data = {'id': db_user_id, 'logged_in_on': datetime.datetime.utcnow()}
+        if body.valid_email:
+            update_data['email_verified'] = True
+        sql_update(table_name='user', data=update_data)
+
+    elif body.user_id and body.auth_key:
+        sql = """
+            SELECT id AS user_id, id_random AS user_id_random, password,
+                   enable, enable_from, enable_to
+            FROM `user`
+            WHERE id_random = :user_id_random
+              AND auth_key = :auth_key
+              AND allow_auth_key = 1
+            LIMIT 1
+        """
+        rec = sql_select(sql=sql, data={'user_id_random': body.user_id, 'auth_key': body.auth_key})
+        if not rec:
+            raise HTTPException(status_code=status.HTTP_404_NOT_FOUND,
+                                detail='User + auth key combination not found.')
+        if err := _check_user_enabled(rec):
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail=err)
+
+        db_user_id = rec['user_id']
+        # Auth key is one-time-use — clear it immediately.
+        update_data = {'id': db_user_id, 'auth_key': None, 'logged_in_on': datetime.datetime.utcnow()}
+        if body.valid_email:
+            update_data['email_verified'] = True
+        sql_update(table_name='user', data=update_data)
+
+    else:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST,
+                            detail='Provide either username+password or user_id+auth_key.')
+
+    user_obj = load_user_obj(user_id=db_user_id, inc_user_role_list=inc_user_role_list)
+    if not user_obj:
+        raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                            detail='Authentication succeeded but user record could not be loaded.')
+
+    return mk_resp(data=user_obj.dict(by_alias=True), status_message='Authentication successful.')
+
+
+@router.post('/verify_password', response_model=Resp_Body_Base)
+async def action_verify_password(
+    body: VerifyPasswordRequest = Body(...),
+    account: AccountContext = Depends(get_account_context),
+):
+    """
+    Verify a user's current password without changing it.
+
+    Provide user_id (Vision ID) or username + current_password.
+    Returns data=True on match, 403 on mismatch.
+    """
+    account_id = account.account_id
+
+    if body.user_id:
+        sql = """
+            SELECT id AS user_id, username, password
+            FROM `user`
+            WHERE id_random = :user_id_random
+            LIMIT 1
+        """
+        rec = sql_select(sql=sql, data={'user_id_random': body.user_id})
+    elif body.username:
+        sql = """
+            SELECT id AS user_id, username, password
+            FROM `user`
+            WHERE account_id = :account_id AND username = :username
+            LIMIT 1
+        """
+        rec = sql_select(sql=sql, data={'account_id': account_id, 'username': body.username})
+    else:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST,
+                            detail='Provide user_id or username.')
+
+    if not rec:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail='User not found.')
+    if not rec.get('password'):
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST,
+                            detail='No password is set for this user.')
+    if not verify_secure_hash_string(string=body.current_password, string_hash=rec['password']):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Password did not match.')
+
+    return mk_resp(data=True, status_message='Password verified.')
+
+
+@router.post('/{user_id}/change_password', response_model=Resp_Body_Base)
+async def action_change_password(
+    user_id: str = Path(min_length=11, max_length=22),
+    body: ChangePasswordRequest = Body(...),
+    account: AccountContext = Depends(get_account_context),
+):
+    """
+    Change a user's password.
+
+    - new_password is required (min 10 chars).
+    - If current_password is provided, it is verified before the change is applied.
+    - Stamps password_set_on on success.
+    """
+    db_user_id = redis_lookup_id_random(record_id_random=user_id, table_name='user')
+    if not db_user_id:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail='User not found.')
+
+    if body.current_password:
+        sql = "SELECT password FROM `user` WHERE id = :uid LIMIT 1"
+        rec = sql_select(sql=sql, data={'uid': db_user_id})
+        if not rec or not rec.get('password'):
+            raise HTTPException(status_code=status.HTTP_404_NOT_FOUND,
+                                detail='User not found or password not set.')
+        if not verify_secure_hash_string(string=body.current_password, string_hash=rec['password']):
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN,
+                                detail='Current password is incorrect.')
+
+    update_data = {
+        'id': db_user_id,
+        'password': secure_hash_string(string=body.new_password),
+        'password_set_on': datetime.datetime.utcnow(),
+    }
+    if not sql_update(table_name='user', data=update_data):
+        raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                            detail='Password update failed.')
+
+    return mk_resp(data=True, status_message='Password changed successfully.')
+
+
+@router.get('/{user_id}/new_auth_key', response_model=Resp_Body_Base)
+async def action_new_auth_key(
+    user_id: str = Path(min_length=11, max_length=22),
+    account: AccountContext = Depends(get_account_context),
+):
+    """
+    Generate a new one-time-use auth key for the user.
+
+    The key is written to the DB and returned in the response body.
+    The user record must have allow_auth_key=1 for the key to be usable
+    with the /authenticate endpoint.
+    """
+    import secrets
+    db_user_id = redis_lookup_id_random(record_id_random=user_id, table_name='user')
+    if not db_user_id:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail='User not found.')
+
+    new_key = secrets.token_urlsafe(default_num_bytes)
+    update_data = {'id': db_user_id, 'auth_key': new_key}
+    if not sql_update(table_name='user', data=update_data):
+        raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                            detail='Failed to write auth key.')
+
+    return mk_resp(data={'auth_key': new_key}, status_message='New auth key generated.')
+
+
+@router.get('/{user_id}/email_auth_key_url', response_model=Resp_Body_Base)
+async def action_email_auth_key_url(
+    user_id: str = Path(min_length=11, max_length=22),
+    root_url: str = Query(..., min_length=10, max_length=200),
+    key_param_name: str = Query('auth_key', min_length=2, max_length=30),
+    account: AccountContext = Depends(get_account_context),
+):
+    """
+    Generate a new auth key and email a one-time login URL to the user.
+
+    root_url is the base URL the login link will be built from.
+    key_param_name controls the query param name used for the auth key in the link (default: auth_key).
+    Returns data=True on success (email sent), 500 if delivery failed.
+    """
+    db_user_id = redis_lookup_id_random(record_id_random=user_id, table_name='user')
+    if not db_user_id:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail='User not found.')
+
+    result = email_user_auth_key_url(
+        account_id=account.account_id,
+        user_id=db_user_id,
+        root_url=root_url,
+        key_param_name=key_param_name,
+    )
+    if not result:
+        raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                            detail='Auth key email could not be sent. Check account email config and user enable status.')
+
+    return mk_resp(data=True, status_message='Auth key email sent.')

app/routers/archive/event_reports.py

@@ -1,112 +0,0 @@
-import datetime
-from fastapi import APIRouter, Body, Depends, Header, HTTPException, Path, Query, Response, status
-from pydantic import BaseModel, EmailStr, Field
-from typing import Dict, List, Optional, Set, Union
-
-from app.lib_general import log, logging, common_route_params, Common_Route_Params
-from app.config import settings
-from app.db_sql import sql_insert, sql_update, sql_insert_or_update, sql_select, sql_delete, get_id_random, redis_lookup_id_random
-
-from app.routers.api_crud import delete_obj_template, get_obj_template, get_obj_li_template, patch_obj_template, post_obj_template
-
-from app.methods.event_presenter_methods import get_event_presenter_url_list
-
-from app.models.response_models import Resp_Body_Base, mk_resp
-
-
-router = APIRouter()
-
-
-# ### BEGIN ### API Event Reports ### event_id_rpt_presenter_links() ###
-# Updated 2022-04-12
-@router.get('/event/{event_id}/rpt_presenter_links', response_model=Resp_Body_Base)
-async def event_id_rpt_presenter_links(
-        event_id: str = Path(min_length=11, max_length=22),
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    if event_id := redis_lookup_id_random(record_id_random=event_id, table_name='event'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response)
-
-
-    if order_line_rec_list_result := get_order_line_rec_list(
-            for_obj_type = obj_type,
-            for_obj_id = obj_id,
-            from_datetime = from_datetime,
-            to_datetime = to_datetime,
-            product_for_type = prod_type,
-            status = status,
-            full_detail = full_detail,
-            # enabled = enabled,
-            limit = limit,
-            ):
-        order_line_result_list = []
-        data_dict_list_for_export = []
-        for order_line_rec in order_line_rec_list_result:
-            if not full_detail:
-                if load_order_line_result := load_order_obj_line(
-                        order_line_id = order_line_rec.get('order_line_id', None),
-                        by_alias = by_alias,
-                        exclude_unset = exclude_unset,
-                        # model_as_dict = model_as_dict,
-                        ):
-                    order_line_result_list.append(load_order_line_result)
-                else:
-                    order_line_result_list.append(None)
-            else: # Uses a different view: v_order_line_full_detail
-                if load_order_line_result := load_order_obj_line_full_detail(
-                        order_line_rec = order_line_rec,
-                        by_alias = by_alias,
-                        exclude_unset = exclude_unset,
-                        model_as_dict = False,
-                        ):
-                    if create_export:
-                        data_dict = load_order_line_result.dict(by_alias=by_alias, exclude_unset=exclude_unset)
-                        data_dict_list_for_export.append(data_dict)
-                    order_line_result_list.append(load_order_line_result)
-                else:
-                    order_line_result_list.append(None)
-        response_data = order_line_result_list
-    elif isinstance(order_line_rec_list_result, list) or order_line_rec_list_result is None: # Empty list or None
-        log.info('No results')
-        return mk_resp(data=None, status_code=404, response=response) # Not Found
-    else:
-        log.warning('Likely bad request')
-        return mk_resp(data=False, status_code=400, response=response) # Bad Request
-
-    if create_export:
-        # column_name_li = ['order_id_random', 'order_line_id_random', '', 'product_name', 'quantity', 'amount', 'dollar_amount', 'person_email']
-
-        # column_name_li = ['order_line_id_random', 'order_id_random', 'product_id_random', 'product_type', 'product_name', 'product_unit_price', 'product_recurring', 'curr_product_id_random', 'curr_product_type', 'curr_product_type_name', 'curr_product_name', 'name', 'quantity', 'amount', 'dollar_amount', 'recurring', 'message', 'person_id_random', 'person_given_name', 'person_family_name', 'person_full_name', 'person_full_name_override', 'person_contact_email', 'person_contact_cc_email', 'person_contact_phone_mobile', 'person_contact_phone_home', 'person_contact_phone_office', 'person_contact_phone_land', 'person_contact_phone_fax', 'person_contact_phone_other', 'person_contact_address_name', 'person_contact_address_organization_name', 'person_contact_address_line_1', 'person_contact_address_line_2', 'person_contact_address_line_3', 'person_contact_address_city', 'person_contact_address_country_subdivision_code', 'person_contact_address_state_province', 'person_contact_address_postal_code', 'person_contact_address_country_alpha_2_code', 'person_contact_address_country_name', 'person_contact_address_country', 'order_status', 'order_created_on', 'order_updated_on', 'created_on', 'updated_on']
-
-        column_name_li = [
-            'event_presenter_id_random',
-            'event_id_random',
-            'events_session_id_random',
-            'events_presentation_id_random',
-            'event_presenter_given_name',
-            'event_presenter_family_name',
-            'event_presenter_email',
-            'event_presenter_created_on', 'event_presenter_updated_on'
-            ]
-
-
-        # column_name_li = []
-        datetime_format='%Y-%m-%d_%H%M'
-
-        # current_datetime = datetime.datetime.now() # Servers timezone (Eastern)
-        current_datetime_utc = datetime.datetime.utcnow()
-        current_datetime_utc = current_datetime_utc.strftime(datetime_format)
-        filename = f'order_line_list_{current_datetime_utc}'
-        if result := create_export_file(data_dict_list=data_dict_list_for_export, column_name_li=column_name_li, subdir_path='order_line', filename=filename, export_type='Excel'):
-            tmp_file_path = result
-        else:
-            log.error('Something went wrong while creating or saving the export file')
-            tmp_file_path = result
-    else: tmp_file_path = None
-
-    return mk_resp(data=response_data, tmp_file_path=tmp_file_path, response=response)
-# ### END ### API Event Reports ### get_obj_id_order_line_list() ###

app/routers/archive/order_v3.py

@@ -1,480 +0,0 @@
-import datetime
-from fastapi import APIRouter, Body, Depends, Header, HTTPException, Path, Query, Response, status
-from pydantic import BaseModel, EmailStr, Field
-from typing import Dict, List, Optional, Set, Union
-
-from app.lib_general import *
-from app.config import settings
-from app.db_sql import *
-
-from app.routers.api_crud import delete_obj_template, get_obj_template, get_obj_li_template, patch_obj_template, post_obj_template
-
-from app.methods.order_methods import create_order_obj, update_order_obj, get_order_rec_list, load_order_obj, save_order_obj
-from app.methods.order_line_methods import create_order_obj_line, update_order_obj_line, load_order_obj_line
-
-from app.models.response_models import Resp_Body_Base, mk_resp
-from app.models.order_models_v3 import Order_Base
-from app.models.order_line_models_v3 import Order_Line_Base
-
-
-router = APIRouter()
-
-
-# ### BEGIN ### API Order Routers ### post_order_obj() ###
-# Updated 2022-01-18
-@router.post('/v3/order', response_model=Resp_Body_Base)
-@router.post('/v3/person/{person_id}/order', response_model=Resp_Body_Base)
-async def post_order_obj(
-        order_obj: Order_Base,
-        person_id: str = Path(min_length=11, max_length=22),
-
-        inc_address: bool = False,
-        inc_contact: bool = False,
-        inc_order_line_list: bool = True,
-        inc_person: bool = False,
-        return_obj: bool = True,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    if person_id := redis_lookup_id_random(record_id_random=person_id, table_name='person'): pass
-    # elif person_id is None: pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The person ID was invalid or not found.')
-
-    # ### SECTION ### Process data
-    if order_id := create_order_obj(
-            account_id = commons.x_account_id,
-            person_id = person_id,
-            order_dict_obj = order_obj,
-            ): pass
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='Something failed while processing the data.') # Bad Request
-
-    # ### SECTION ### Return successful results
-    if return_obj:
-        if load_order_obj_result := load_order_obj(
-                order_id = order_id,
-                inc_address = inc_address,
-                inc_contact = inc_contact,
-                inc_order_line_list = inc_order_line_list,
-                inc_person = inc_person,
-                ).dict(by_alias=commons.by_alias, exclude_unset=commons.exclude_unset):
-            log.info('Loading successful. Returning result')
-            log.debug(load_order_obj_result)
-            return mk_resp(data=load_order_obj_result, response=commons.response)
-        elif isinstance(load_order_obj_result, list) or load_order_obj_result is None: # Empty list or None
-            return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-        else:
-            return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
-    else:
-        order_id_random = get_id_random(record_id=order_id, table_name='order')
-        data = {}
-        data['order_id'] = order_id
-        data['order_id_random'] = order_id_random
-        return mk_resp(data=data, response=commons.response)
-# ### END ### API Order Routers ### post_order_obj() ###
-
-
-# ### BEGIN ### API Order Routers ### patch_order_obj() ###
-# Updated 2022-01-18
-@router.patch('/v3/order/{order_id}', response_model=Resp_Body_Base)
-# @router.patch('/v3/person/{person_id}/order/{order_id}', response_model=Resp_Body_Base)
-async def patch_order_obj(
-        order_obj: Order_Base,
-        order_id: str = Path(min_length=11, max_length=22),
-        # person_id: str = Query(None, min_length=11, max_length=22),
-
-        inc_address: bool = False,
-        inc_contact: bool = False,
-        inc_order_line_list: bool = True,
-        inc_person: bool = False,
-        return_obj: Optional[bool] = True,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    order_id_random = order_id # This is used later for the response data
-    # person_id_random = person_id # This is used later for the response data
-
-    if order_id := redis_lookup_id_random(record_id_random=order_id, table_name='order'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order ID was invalid or not found.')
-
-    # if person_id := redis_lookup_id_random(record_id_random=person_id, table_name='person'): pass
-    # elif person_id is None: pass
-    # else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The person ID was invalid or not found.')
-
-    # ### SECTION ### Process data
-    if update_order_obj_result := update_order_obj(
-            order_id = order_id,
-            order_dict_obj = order_obj,
-            # person_id = person_id,
-            ): pass
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='Something failed while processing the data.') # Bad Request
-
-    # ### SECTION ### Return successful results
-    if return_obj:
-        if load_order_obj_result := load_order_obj(
-                order_id = order_id,
-                inc_address = inc_address,
-                inc_contact = inc_contact,
-                inc_order_line_list = inc_order_line_list,
-                inc_person = inc_person,
-                ).dict(by_alias=commons.by_alias, exclude_unset=commons.exclude_unset):
-            log.info('Loading successful. Returning result')
-            log.debug(load_order_obj_result)
-            return mk_resp(data=load_order_obj_result, response=commons.response)
-        elif isinstance(load_order_obj_result, list) or load_order_obj_result is None: # Empty list or None
-            return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-        else:
-            return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
-    else:
-        data = {}
-        data['order_id'] = order_id
-        data['order_id_random'] = order_id_random
-        return mk_resp(data=data, response=commons.response)
-# ### END ### API Order Routers ### patch_order_obj() ###
-
-
-# ### BEGIN ### API Order Routers ### patch_order_obj_add_line() ###
-# Updated 2022-01-18
-@router.patch('/v3/order/{order_id}/line/add', response_model=Resp_Body_Base)
-async def patch_order_obj_add_line(
-        order_line_obj: Order_Line_Base,
-        order_id: str = Path(min_length=11, max_length=22),
-
-        # inc_order: bool = False,
-        inc_order_line_list: bool = True,
-        return_obj: Optional[bool] = True,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    order_id_random = order_id # This is used later for the response data
-
-    if order_id := redis_lookup_id_random(record_id_random=order_id, table_name='order'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order ID was invalid or not found.')
-
-    # ### SECTION ### Process data
-    if order_line_id := add_order_obj_line(
-            order_id = order_id,
-            order_line_dict_obj = order_line_obj,
-            ): pass
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='Something failed while processing the data.') # Bad Request
-
-    # ### SECTION ### Return successful results
-    if return_obj:
-        if load_order_obj_result := load_order_obj(
-                order_id = order_id,
-                # inc_address = inc_address,
-                # inc_contact = inc_contact,
-                inc_order_line_list = inc_order_line_list,
-                # inc_person = inc_person,
-                ).dict(by_alias=commons.by_alias, exclude_unset=commons.exclude_unset):
-            log.info('Loading successful. Returning result')
-            log.debug(load_order_obj_result)
-            return mk_resp(data=load_order_obj_result, response=commons.response)
-        elif isinstance(load_order_obj_result, list) or load_order_obj_result is None: # Empty list or None
-            return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-        else:
-            return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
-    else:
-        order_line_id = order_line_add_result
-        order_line_id_random = get_id_random(record_id=order_line_id, table_name='order_line')
-        data = {}
-        data['order_id'] = order_id
-        data['order_id_random'] = order_id_random
-        data['order_line_id'] = order_line_id
-        data['order_line_id_random'] = order_line_id_random
-        return mk_resp(data=data, response=commons.response)
-# ### END ### API Order Routers ### patch_order_obj_add_line() ###
-
-
-# ### BEGIN ### API Order Routers ### patch_order_obj_update_line() ###
-# Updated 2022-01-18
-@router.patch('/v3/order/{order_id}/line/{order_line_id}/update', response_model=Resp_Body_Base)
-async def patch_order_obj_update_line(
-        order_obj: Order_Line_Base,
-        order_id: str = Path(min_length=11, max_length=22),
-        order_line_id: str = Path(min_length=11, max_length=22),
-
-        # inc_order: bool = False,
-        inc_order_line_list: bool = True,
-        return_obj: Optional[bool] = True,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    order_id_random = order_id # This is used later for the response data
-    order_line_id_random = order_line_id # This is used later for the response data
-
-    if order_id := redis_lookup_id_random(record_id_random=order_id, table_name='order'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order ID was invalid or not found.')
-
-    if order_line_id := redis_lookup_id_random(record_id_random=order_line_id, table_name='order_line'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order line ID was invalid or not found.')
-
-    # ### SECTION ### Process data
-    if update_order_obj_line_result := update_order_obj_line(
-            order_line_id = order_line_id,
-            order_line_dict_obj = order_line_obj,
-            ): pass
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='Something failed while processing the data.') # Bad Request
-
-    # ### SECTION ### Return successful results
-    if return_obj:
-        if load_order_obj_result := load_order_obj(
-                order_id = order_id,
-                # inc_address = inc_address,
-                # inc_contact = inc_contact,
-                inc_order_line_list = inc_order_line_list,
-                # inc_person = inc_person,
-                ).dict(by_alias=commons.by_alias, exclude_unset=commons.exclude_unset):
-            log.info('Loading successful. Returning result')
-            log.debug(order_dict)
-            return mk_resp(data=order_dict, response=commons.response)
-        elif isinstance(load_order_obj_result, list) or load_order_obj_result is None: # Empty list or None
-            return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-        else:
-            return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
-    else:
-        data = {}
-        data['order_id'] = order_id
-        data['order_id_random'] = order_id_random
-        data['order_line_id'] = order_line_id
-        data['order_line_id_random'] = order_line_id_random
-        return mk_resp(data=data, response=commons.response)
-# ### END ### API Order Routers ### patch_order_obj_update_line() ###
-
-
-# ### BEGIN ### API Order Routers ### patch_order_obj_remove_line() ###
-# Updated 2022-01-18
-@router.patch('/v3/order/{order_id}/line/{order_line_id}/remove', response_model=Resp_Body_Base)
-async def patch_order_obj_remove_line(
-        order_obj: Order_Line_Base,
-        order_id: str = Path(min_length=11, max_length=22),
-        order_line_id: str = Path(min_length=11, max_length=22),
-
-        # inc_order: bool = False,
-        inc_order_line_list: bool = True,
-        return_obj: Optional[bool] = True,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    order_id_random = order_id # This is used later for the response data
-    order_line_id_random = order_line_id # This is used later for the response data
-
-    if order_id := redis_lookup_id_random(record_id_random=order_id, table_name='order'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order ID was invalid or not found.')
-
-    if order_line_id := redis_lookup_id_random(record_id_random=order_line_id, table_name='order_line'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order line ID was invalid or not found.')
-
-    # ### SECTION ### Process data
-    if remove_order_obj_line_result := remove_order_obj_line(
-            order_line_id = order_line_id,
-            ): pass
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='Something failed while processing the data.') # Bad Request
-
-    # ### SECTION ### Return successful results
-    if return_obj:
-        if load_order_obj_result := load_order_obj(
-                order_id = order_id,
-                # inc_address = inc_address,
-                # inc_contact = inc_contact,
-                inc_order_line_list = inc_order_line_list,
-                # inc_person = inc_person,
-                ).dict(by_alias=commons.by_alias, exclude_unset=commons.exclude_unset):
-            log.info('Loading successful. Returning result')
-            log.debug(order_dict)
-            return mk_resp(data=order_dict, response=commons.response)
-        elif isinstance(load_order_obj_result, list) or load_order_obj_result is None: # Empty list or None
-            return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-        else:
-            return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
-    else:
-        data = {}
-        data['order_id'] = order_id
-        data['order_id_random'] = order_id_random
-        data['order_line_id'] = order_line_id
-        data['order_line_id_random'] = order_line_id_random
-        return mk_resp(data=data, response=commons.response)
-# ### END ### API Order Routers ### patch_order_obj_remove_line() ###
-
-
-# ### BEGIN ### API Order Routers ### get_order_obj_li() ###
-# Updated 2022-01-18
-@router.get('/v3/{for_obj_type}/{for_obj_id}/order/list', response_model=Resp_Body_Base)
-async def get_order_obj_li(
-        for_obj_type: str = Path(min_length=2, max_length=50),
-        for_obj_id: str = Path(min_length=11, max_length=22),
-        order_status: str = 'complete',
-        order_checkout_status: str = 'complete',
-        from_datetime: datetime.datetime = None,
-        to_datetime: datetime.datetime = None,
-
-        inc_address: bool = False,
-        inc_contact: bool = False,
-        inc_order_cfg: bool = False,
-        inc_order_line_list: bool = False,
-        inc_person: bool = False,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    if obj_type in ['account', 'person']: pass
-    else: return mk_resp(data=False, status_code=400, response=response, status_message='The object type passed was invalid or not found. Expecting "account" or "person".') # Bad Request
-
-    if obj_type_id := redis_lookup_id_random(record_id_random=for_obj_id, table_name=for_obj_type): pass
-    else: return mk_resp(data=False, status_code=404, response=commons.response) # Not Found
-
-    if get_order_rec_list_result := get_order_rec_list(
-            for_obj_type = for_obj_type,
-            for_obj_id = for_obj_id,
-            from_datetime = from_datetime,
-            to_datetime = to_datetime,
-            status = order_status,
-            # checkout_status = order_checkout_status,
-            enabled = commons.enabled,
-            limit = commons.limit,
-            offset = commons.offset,
-            ):
-        order_obj_list = []
-        for order_rec in get_order_rec_list_result:
-            if load_order_obj_result := load_order_obj(
-                    order_id = order_rec.get('order_id'),
-                    inc_address = inc_address,
-                    inc_contact = inc_contact,
-                    inc_order_cfg = inc_order_cfg,
-                    inc_order_line_list = inc_order_line_list,
-                    inc_person = inc_person,
-                    enabled = commons.enabled,
-                    limit = commons.limit,
-                    by_alias = commons.by_alias,
-                    exclude_unset = commons.exclude_unset,
-                    # model_as_dict = model_as_dict,
-                    ):
-                log.debug(load_order_obj_result)
-                order_obj_list.append(load_order_obj_result)
-            else:
-                order_obj_list.append(None)
-        log.info('Loading successful. Returning result')
-        log.debug(order_obj_list)
-        return mk_resp(data=order_obj_list, response=commons.response)
-    elif isinstance(get_order_rec_list_result, list) or get_order_rec_list_result is None: # Empty list or None
-        return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
-# ### END ### API Order Routers ### get_order_obj_li() ###
-
-
-# ### BEGIN ### API Order Routes ### get_order_obj() ###
-# NOTE 2021-08-09: Use with rework of order_cart
-# Updated 2022-12-18
-@router.get('/v3/order/{order_id}', response_model=Resp_Body_Base)
-async def get_order_obj(
-        order_id: str = Path(min_length=11, max_length=22),
-
-        inc_address: bool = False,
-        inc_contact: bool = False,
-        inc_order_cfg: bool = False,
-        inc_order_line_list: bool = False,
-        inc_person: bool = False,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    if order_id := redis_lookup_id_random(record_id_random=order_id, table_name='order'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order ID was invalid or not found.')
-
-    if load_order_obj_result := load_order_obj(
-            order_id = order_id,
-            inc_address = inc_address,
-            inc_contact = inc_contact,
-            inc_order_cfg = inc_order_cfg,
-            inc_order_line_list = inc_order_line_list,
-            inc_person = inc_person,
-            limit = commons.limit,
-            enabled = commons.enabled,
-            by_alias = commons.by_alias,
-            exclude_unset = commons.exclude_unset,
-            # model_as_dict = model_as_dict,
-            ):
-        log.debug(load_order_obj_result)
-        order_dict = load_order_obj_result.dict(by_alias=commons.by_alias, exclude_unset=False) # NOTE NOTE NOTE NOTE exclude_unset is forced to False for now. Will return more fields than is ideal. Need to create another Order_Line_Base. Probably Order_Line_OUT_Base
-        log.info('Loading successful. Returning result')
-        return mk_resp(data=order_dict, response=commons.response)
-    elif isinstance(load_order_obj_result, list) or load_order_obj_result is None: # Empty list or None
-        return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
-# ### END ### API Order Routes ### get_order_obj() ###
-
-
-# ### BEGIN ### API Order ### get_person_id_order_cart() ###
-# NOTE 2021-08-09: Use with rework of order_cart. The most recent (hopefully only one) "open" order for a person.
-# Updated 2022-12-18
-@router.get('/v3/person/{person_id}/order/cart', response_model=Resp_Body_Base)
-async def get_person_id_order_cart(
-        person_id: str = Path(min_length=11, max_length=22),
-        enabled: str = 'enabled',
-        inc_order_line_list: bool = False,
-        inc_order_cfg: bool = False,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    if person_id := redis_lookup_id_random(record_id_random=person_id, table_name='person'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The person ID was invalid or not found.')
-
-    # Query to get the one "open" order status for a person ID
-
-    return False
-# ### END ### API Order ### get_person_id_order_cart() ###
-
-
-# ### BEGIN ### API Order Routers ### delete_order_obj() ###
-# Updated 2022-01-18
-@router.delete('/v3/order/{order_id}', response_model=Resp_Body_Base)
-async def delete_order_obj(
-        order_id: str = Path(min_length=11, max_length=22),
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    if order_id := redis_lookup_id_random(record_id_random=order_id, table_name='order'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The order ID was invalid or not found.')
-
-    obj_type = 'order'
-    result = delete_obj_template(
-        obj_type = obj_type,
-        obj_id = obj_id,
-        )
-    return result
-# ### END ### API Order Routers ### delete_order_obj() ###

app/routers/data_store.py

@@ -24,119 +24,125 @@ router = APIRouter()
 
 
 # ### BEGIN ### API Data Store Routers ### post_data_store_obj() ###
+# LEGACY (disabled) - superseded by V3 CRUD: POST /v3/crud/data_store/
 # Updated 2026-01-28
-@router.post('/data_store', response_model=Resp_Body_Base)
-async def post_data_store_obj(
-        data_store_obj: Data_Store_Base,
-
-        return_obj: bool = True,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    # None
-
-    # ### SECTION ### Process data
-    if data_store_id := create_update_data_store_obj(
-            data_store_dict_obj = data_store_obj,
-            ): pass
-    else:
-        log.warning('Likely bad request')
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='Not created. Something failed while processing the data. Check the field names and data types.') # Bad Request
-
-    # ### SECTION ### Return successful results
-    if return_obj:
-        data_store_obj = load_data_store_obj(
-            data_store_id = data_store_id,
-            )
-        data = data_store_obj
-    else:
-        data_store_id_random = get_id_random(record_id=data_store_id, table_name='data_store')
-        data = {}
-        data['data_store_id'] = data_store_id
-        data['data_store_id_random'] = data_store_id_random
-    return mk_resp(data=data, response=commons.response)
+# @router.post('/data_store', response_model=Resp_Body_Base)
+# async def post_data_store_obj(
+#         data_store_obj: Data_Store_Base,
+#
+#         return_obj: bool = True,
+#
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
+#
+#     # ### SECTION ### Secondary data validation
+#     # None
+#
+#     # ### SECTION ### Process data
+#     if data_store_id := create_update_data_store_obj(
+#             data_store_dict_obj = data_store_obj,
+#             ): pass
+#     else:
+#         log.warning('Likely bad request')
+#         return mk_resp(data=False, status_code=400, response=commons.response, status_message='Not created. Something failed while processing the data. Check the field names and data types.') # Bad Request
+#
+#     # ### SECTION ### Return successful results
+#     if return_obj:
+#         data_store_obj = load_data_store_obj(
+#             data_store_id = data_store_id,
+#             )
+#         data = data_store_obj
+#     else:
+#         data_store_id_random = get_id_random(record_id=data_store_id, table_name='data_store')
+#         data = {}
+#         data['data_store_id'] = data_store_id
+#         data['data_store_id_random'] = data_store_id_random
+#     return mk_resp(data=data, response=commons.response)
 # ### END ### API Data Store Routers ### post_data_store_obj() ###
 
 
 # ### BEGIN ### API Data Store Routers ### patch_data_store_obj() ###
+# LEGACY (disabled) - superseded by V3 CRUD: PATCH /v3/crud/data_store/{id}
 # Updated 2022-03-11
-@router.patch('/data_store/{data_store_id}', response_model=Resp_Body_Base)
-async def patch_data_store_obj(
-        data_store_obj: Data_Store_Base,
-        data_store_id: str = Path(min_length=11, max_length=22),
-
-        return_obj: Optional[bool] = True,
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    data_store_id_random = data_store_id # This is used later for the response data
-    if data_store_id := redis_lookup_id_random(record_id_random=data_store_id, table_name='data_store'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The Data Store ID was invalid or not found.')
-
-    # ### SECTION ### Process data
-    if data_store_up_result := create_update_data_store_obj(
-            data_store_dict_obj = data_store_obj,
-            data_store_id = data_store_id,
-            ): pass
-    else:
-        log.warning('Likely bad request')
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='Not updated. Something failed while processing the data. Check the field names and data types.') # Bad Request
-
-    # ### SECTION ### Return successful results
-    if return_obj:
-        data_store_obj = load_data_store_obj(
-            data_store_id = data_store_id,
-            )
-        data = data_store_obj
-    else:
-        data = {}
-        data['data_store_id'] = data_store_id
-        data['data_store_id_random'] = data_store_id_random
-    return mk_resp(data=data, response=commons.response)
+# @router.patch('/data_store/{data_store_id}', response_model=Resp_Body_Base)
+# async def patch_data_store_obj(
+#         data_store_obj: Data_Store_Base,
+#         data_store_id: str = Path(min_length=11, max_length=22),
+#
+#         return_obj: Optional[bool] = True,
+#
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
+#
+#     # ### SECTION ### Secondary data validation
+#     data_store_id_random = data_store_id # This is used later for the response data
+#     if data_store_id := redis_lookup_id_random(record_id_random=data_store_id, table_name='data_store'): pass
+#     else: return mk_resp(data=None, status_code=404, response=commons.response, status_message='The Data Store ID was invalid or not found.')
+#
+#     # ### SECTION ### Process data
+#     if data_store_up_result := create_update_data_store_obj(
+#             data_store_dict_obj = data_store_obj,
+#             data_store_id = data_store_id,
+#             ): pass
+#     else:
+#         log.warning('Likely bad request')
+#         return mk_resp(data=False, status_code=400, response=commons.response, status_message='Not updated. Something failed while processing the data. Check the field names and data types.') # Bad Request
+#
+#     # ### SECTION ### Return successful results
+#     if return_obj:
+#         data_store_obj = load_data_store_obj(
+#             data_store_id = data_store_id,
+#             )
+#         data = data_store_obj
+#     else:
+#         data = {}
+#         data['data_store_id'] = data_store_id
+#         data['data_store_id_random'] = data_store_id_random
+#     return mk_resp(data=data, response=commons.response)
 # ### END ### API Data Store Routers ### patch_data_store_obj() ###
 
 
 # ### BEGIN ### API Data Store ### get_data_store_obj() ###
+# LEGACY (disabled) - superseded by V3 CRUD: GET /v3/crud/data_store/{id}
 # Updated 2026-01-28
-@router.get('/data_store/{data_store_id}', response_model=Resp_Body_Base)
-async def get_data_store_obj(
-        data_store_id: str = Path(min_length=11, max_length=22),
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    # ### SECTION ### Secondary data validation
-    if data_store_id := redis_lookup_id_random(record_id_random=data_store_id, table_name='data_store'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response)
-
-    if data_store_rec_result := load_data_store_obj(
-            data_store_id = data_store_id,
-            limit = commons.limit,
-            enabled = commons.enabled,
-            ):
-        log.info('Loading successful. Returning result')
-        return mk_resp(data=data_store_rec_result, response=commons.response)
-    elif isinstance(data_store_rec_result, list) or data_store_rec_result is None: # Empty list or None
-        log.info('No results')
-        return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-    else:
-        log.warning('Likely bad request')
-        return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
+# @router.get('/data_store/{data_store_id}', response_model=Resp_Body_Base)
+# async def get_data_store_obj(
+#         data_store_id: str = Path(min_length=11, max_length=22),
+#
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
+#
+#     # ### SECTION ### Secondary data validation
+#     if data_store_id := redis_lookup_id_random(record_id_random=data_store_id, table_name='data_store'): pass
+#     else: return mk_resp(data=None, status_code=404, response=commons.response)
+#
+#     if data_store_rec_result := load_data_store_obj(
+#             data_store_id = data_store_id,
+#             limit = commons.limit,
+#             enabled = commons.enabled,
+#             ):
+#         log.info('Loading successful. Returning result')
+#         return mk_resp(data=data_store_rec_result, response=commons.response)
+#     elif isinstance(data_store_rec_result, list) or data_store_rec_result is None: # Empty list or None
+#         log.info('No results')
+#         return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
+#     else:
+#         log.warning('Likely bad request')
+#         return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
 # ### END ### API Data Store ### get_data_store_obj() ###
 
 
 # ### BEGIN ### API Data Store ### get_v3_data_store_obj_w_code() ###
 # NEW V3 Endpoint for Code Lookup
+# TODO: Migrate to a dedicated api_v3_actions_data_store.py router and rename path to
+#       /v3/action/data_store/code/{data_store_code} to match the V3 action naming convention.
+#       Requires a coordinated frontend update before the path rename can happen.
 # Updated 2026-01-28
 @router.get('/v3/data_store/code/{data_store_code}', response_model=Resp_Body_Base, tags=['Data Store V3'])
 async def get_v3_data_store_obj_w_code(
@@ -177,57 +183,60 @@ async def get_v3_data_store_obj_w_code(
 
 
 # ### BEGIN ### API Data Store ### get_data_store_obj_w_code() ###
-# NOTE: Adding some explanation because this is not quickly obvious how it fully works.
-# The look up order starts with a required data_store_code. Then the first result that matches the most specific method. The for_type and for_id fields are not required. I think it makes the most sense to be a part of the URL path, not the GET params. Either should work with no problem though.
+# LEGACY (disabled) - legacy code-based lookup; use GET /v3/data_store/code/{code} instead.
+# NOTE: The look up order starts with a required data_store_code. Then the first result that matches the most specific method. The for_type and for_id fields are not required. I think it makes the most sense to be a part of the URL path, not the GET params. Either should work with no problem though.
 # Lookup using: for_type and for_id > account_id > data_store_code
 # This is a nice way to have global default data along with account and object specific data.
 # Updated 2023-05-22
 
-
-@router.get('/data_store/code/{data_store_code}/{for_type}/{for_id}', response_model=Resp_Body_Base)
-async def get_data_store_obj_w_code_path(
-        data_store_code: str = Path(min_length=3, max_length=50),
-        for_type: Optional[str] = Path(min_length=1, max_length=25),
-        for_id: Optional[str] = Path(min_length=11, max_length=22),
-        limit: int = Query(1, ge=1),
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-    log.info('Using path parameters')
-    # ### SECTION ### Call generic function to get the data_store object
-    return await handle_get_data_store_obj_w_code(
-        data_store_code = data_store_code,
-        for_type = for_type,
-        for_id = for_id,
-        commons = commons,
-        limit = limit,
-        )
+# @router.get('/data_store/code/{data_store_code}/{for_type}/{for_id}', response_model=Resp_Body_Base)
+# async def get_data_store_obj_w_code_path(
+#         data_store_code: str = Path(min_length=3, max_length=50),
+#         for_type: Optional[str] = Path(min_length=1, max_length=25),
+#         for_id: Optional[str] = Path(min_length=11, max_length=22),
+#         limit: int = Query(1, ge=1),
+#
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
+#     log.info('Using path parameters')
+#     # ### SECTION ### Call generic function to get the data_store object
+#     return await handle_get_data_store_obj_w_code(
+#         data_store_code = data_store_code,
+#         for_type = for_type,
+#         for_id = for_id,
+#         commons = commons,
+#         limit = limit,
+#         )
 
 
-@router.get('/data_store/code/{data_store_code}', response_model=Resp_Body_Base)
-async def get_data_store_obj_w_code_query(
-        data_store_code: str = Path(min_length=3, max_length=50),
-        for_type: Optional[str] = Query(None, min_length=1, max_length=25),
-        for_id: Optional[str] = Query(None, min_length=11, max_length=22),
-        limit: int = Query(1, ge=1),
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-    log.info('Using query parameters')
-    # ### SECTION ### Call generic function to get the data_store object
-    return await handle_get_data_store_obj_w_code(
-        data_store_code = data_store_code,
-        for_type = for_type,
-        for_id = for_id,
-        commons = commons,
-        limit = limit,
-        )
+# @router.get('/data_store/code/{data_store_code}', response_model=Resp_Body_Base)
+# async def get_data_store_obj_w_code_query(
+#         data_store_code: str = Path(min_length=3, max_length=50),
+#         for_type: Optional[str] = Query(None, min_length=1, max_length=25),
+#         for_id: Optional[str] = Query(None, min_length=11, max_length=22),
+#         limit: int = Query(1, ge=1),
+#
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
+#     log.info('Using query parameters')
+#     # ### SECTION ### Call generic function to get the data_store object
+#     return await handle_get_data_store_obj_w_code(
+#         data_store_code = data_store_code,
+#         for_type = for_type,
+#         for_id = for_id,
+#         commons = commons,
+#         limit = limit,
+#         )
+# ### END ### API Data Store ### get_data_store_obj_w_code() ###
 
 
+# TODO: Migrate to a dedicated api_v3_actions_data_store.py router and rename path to
+#       /v3/action/data_store/code/{data_store_code}/search to match the V3 action naming convention.
+#       Requires a coordinated frontend update before the path rename can happen.
 @router.post('/v3/data_store/code/{data_store_code}/search', response_model=Resp_Body_Base, tags=['Data Store V3'])
 async def search_v3_data_store_obj_w_code(
         data_store_code: str,
@@ -359,43 +368,44 @@ async def handle_get_data_store_obj_w_code(
 
 
 # ### BEGIN ### API Data Store ### get_account_obj_data_store_list() ###
+# LEGACY (disabled) - superseded by V3 CRUD search: POST /v3/crud/data_store/search
 # Updated 2022-03-11
-@router.get('/account/{account_id}/data_store/list', response_model=Resp_Body_Base)
-async def get_account_obj_data_store_list(
-        account_id: str = Path(min_length=11, max_length=22),
-
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    if account_id := redis_lookup_id_random(record_id_random=account_id, table_name='account'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response)
-
-    # Updated 2022-03-11
-    if data_store_rec_list_result := get_data_store_rec_list(
-            account_id = account_id,
-            for_type = 'account',
-            for_id = account_id,
-            enabled = commons.enabled,
-            limit = commons.limit,
-            offset = commons.offset,
-            ):
-        data_store_result_list = []
-        for data_store_rec in data_store_rec_list_result:
-            if load_data_store_result := load_data_store_obj(
-                    data_store_id = data_store_rec.get('data_store_id', None),
-                    enabled = commons.enabled,
-                    ):
-                data_store_result_list.append(load_data_store_result)
-            else:
-                data_store_result_list.append(None)
-        response_data = data_store_result_list
-        return mk_resp(data=response_data, response=commons.response)
-    elif isinstance(data_store_rec_list_result, list) or data_store_rec_list_result is None: # Empty list or None
-        log.info('No results')
-        return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
-    else:
-        log.warning('Likely bad request')
-        return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
+# @router.get('/account/{account_id}/data_store/list', response_model=Resp_Body_Base)
+# async def get_account_obj_data_store_list(
+#         account_id: str = Path(min_length=11, max_length=22),
+#
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
+#
+#     if account_id := redis_lookup_id_random(record_id_random=account_id, table_name='account'): pass
+#     else: return mk_resp(data=None, status_code=404, response=commons.response)
+#
+#     # Updated 2022-03-11
+#     if data_store_rec_list_result := get_data_store_rec_list(
+#             account_id = account_id,
+#             for_type = 'account',
+#             for_id = account_id,
+#             enabled = commons.enabled,
+#             limit = commons.limit,
+#             offset = commons.offset,
+#             ):
+#         data_store_result_list = []
+#         for data_store_rec in data_store_rec_list_result:
+#             if load_data_store_result := load_data_store_obj(
+#                     data_store_id = data_store_rec.get('data_store_id', None),
+#                     enabled = commons.enabled,
+#                     ):
+#                 data_store_result_list.append(load_data_store_result)
+#             else:
+#                 data_store_result_list.append(None)
+#         response_data = data_store_result_list
+#         return mk_resp(data=response_data, response=commons.response)
+#     elif isinstance(data_store_rec_list_result, list) or data_store_rec_list_result is None: # Empty list or None
+#         log.info('No results')
+#         return mk_resp(data=None, status_code=404, response=commons.response) # Not Found
+#     else:
+#         log.warning('Likely bad request')
+#         return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
 # ### END ### API Data Store ### get_account_obj_data_store_list() ###

app/routers/event_badge_importing.py

@@ -16,9 +16,9 @@ from app.methods.event_person_methods import create_event_person_obj, create_upd
 # from app.methods.event_presenter_methods import create_update_event_presenter_obj_v4, get_event_presenter_rec_list, load_event_presenter_obj
 from app.methods.hosted_file_methods import load_hosted_file_obj, save_file
 
-from app.models.event_models import Event_Base
+#from app.models.event_models import Event_Base
 # from app.models.event_location_models import Event_Location_Base
-from app.models.event_person_models import Event_Person_Base
+#from app.models.event_person_models import Event_Person_Base
 # from app.models.event_presentation_models import Event_Presentation_Base
 # from app.models.event_presenter_models import Event_Presenter_Base
 # from app.models.event_session_models import Event_Session_Base
@@ -432,6 +432,11 @@ async def event_id_badge_import(
                 event_badge_id = event_person_result.get('event_badge_id')
                 event_person_profile_id = event_person_result.get('event_person_profile_id')
                 log.info(f'Found Event Person. Updating existing... Event Person ID: {event_person_id}')
+                # Don't touch enable on update — a manually disabled record is effectively
+                # blacklisted and should survive repeated re-imports of the same file.
+                event_person_data.pop('enable', None)
+                event_person_data.get('event_badge', {}).pop('enable', None)
+                event_person_data.get('event_person_profile', {}).pop('enable', None)
                 if create_event_person_obj_result := create_update_event_person_obj_v4(
                         event_person_dict_obj = event_person_data,
                         event_person_id = event_person_id,
@@ -473,3 +478,578 @@ async def event_id_badge_import(
         return mk_resp(data=event_badge_person_li, status_message=f'Importing badges from file. Found {len(person_li)} badges.', response=commons.response)
     else:
         return mk_resp(data=event_badge_person_summary_li, status_message=f'Checked for badges from file. Found {len(event_badge_person_li)} badges.', response=commons.response)
+
+
+# ### BEGIN ### Zoom Events CSV Badge Import ### event_id_badge_import_zoom_csv() ###
+# Accepts a Zoom Events registrant CSV export and upserts event_person records.
+# Zoom CSV format: fixed columns (First name, Last name, Registrant email, Ticket name,
+# Unique identifier, etc.) plus per-ticket-type custom fields using the pattern
+# "FieldLabel_*_TicketTypeName". Delimiter is auto-detected (Zoom exports vary).
+# Updated 2026-04-06
+
+# Notes specific to Axonius 2026
+
+# SELECT id, badge_type, badge_type_code
+# FROM event_badge
+# WHERE badge_type = 'In-Person Attendee';
+
+# UPDATE event_badge
+# SET badge_type_code = 'attendee'
+# WHERE badge_type = 'In-Person Attendee';
+
+# SELECT id, badge_type, badge_type_code
+# FROM event_badge
+# WHERE badge_type = 'Adapt26 Sponsor';
+
+# UPDATE event_badge
+# SET badge_type_code = 'sponsor'
+# WHERE badge_type = 'Adapt26 Sponsor';
+
+
+def _split_full_name(full_name: str) -> tuple:
+    """Split 'First Last' on last space into (given_name, family_name)."""
+    parts = full_name.strip().rsplit(' ', 1)
+    if len(parts) == 2:
+        return parts[0], parts[1]
+    return full_name.strip(), ''
+
+
+def _zoom_ticket_field(record: dict, field_prefix: str, ticket_name: str) -> str:
+    """
+    Extracts a per-ticket-type field value from a Zoom CSV row.
+    Tries the exact ticket match first, then falls back to the first non-empty value
+    across all variants of that field prefix.
+    """
+    exact_key = f'{field_prefix}_*_{ticket_name}'
+    if val := str(record.get(exact_key, '')).strip():
+        return val
+    for key, val in record.items():
+        if key.startswith(f'{field_prefix}_*_') and str(val).strip():
+            return str(val).strip()
+    return ''
+
+
+@router.post('/event/{event_id}/badge/import/zoom_csv', response_model=Resp_Body_Base)
+async def event_id_badge_import_zoom_csv(
+        event_id: str = Path(min_length=11, max_length=22),
+        file: UploadFile = File(...),
+
+        begin_at: int = 0,
+        end_at: int = 20000,
+
+        return_detail: bool = False,
+
+        commons: Common_Route_Params = Depends(common_route_params),
+        ):
+    """
+    Import event badges from a Zoom Events registrant CSV export.
+
+    Zoom exports fixed columns (First name, Last name, Registrant email, Ticket name,
+    Unique identifier) plus per-ticket-type custom fields in the format
+    "FieldLabel_*_TicketTypeName". The 'Unique identifier' column is used as the
+    external_registration_id. Delimiter is auto-detected.
+    """
+    log.setLevel(logging.INFO)
+
+    account_id = commons.x_account_id
+
+    event_id_random = event_id
+    if event_id := redis_lookup_id_random(record_id_random=event_id, table_name='event'): pass
+    else: return mk_resp(data=None, status_code=404, response=commons.response)
+
+    link_to_type = 'event'
+    link_to_id = event_id
+
+    file_info = await save_file(
+        file=file,
+        account_id=account_id,
+        link_to_type=link_to_type,
+        link_to_id=link_to_id,
+        )
+    if file_info['saved']:
+        log.info('File saved')
+    else:
+        log.error('Something may have gone wrong while saving the uploaded file?')
+        return mk_resp(data=None, status_code=500, response=commons.response)
+
+    hosted_files_path = settings.FILES_PATH['hosted_files_root']
+    subdirectory_dest = os.path.join(hosted_files_path, file_info.get('subdirectory_path'))
+    hash_filename = file_info.get('hash_sha256') + '.file'
+    full_file_path = pathlib.Path(os.path.join(subdirectory_dest, hash_filename))
+
+    if not full_file_path.exists():
+        log.warning(f'Not found at full file path: {full_file_path}')
+        return mk_resp(data=None, status_code=500, response=commons.response)
+
+    # Zoom CSV layout: row 1 = "Report generated" metadata, row 2 = blank, row 3 = headers
+    # Delimiter is auto-detected (Zoom exports vary between comma and tab)
+    df = pandas.read_csv(
+        full_file_path,
+        sep=None,
+        engine='python',
+        skiprows=2,
+        na_filter=False,
+        dtype=str,
+        )
+
+    df_dict = df.to_dict(orient='records')
+    log.info(f'Zoom CSV total record count: {len(df_dict)}')
+
+    loop_count = 0
+    event_badge_person_li = []
+    event_badge_person_summary_li = []
+
+    log.setLevel(logging.DEBUG)
+    for record in df_dict:
+        log.info(f'Loop Count: {loop_count}')
+        loop_count += 1
+        if loop_count <= begin_at: continue
+        if loop_count > end_at: break
+
+        # Force use of Registrant email as the external_id for Zoom CSV imports.
+        # Many Zoom exports (for this group) have a useless "Unique identifier"
+        # column that contains "N/A" for every row — rely on email instead.
+        email = str(record.get('Registrant email', '')).strip()
+        if not email:
+            log.warning('Row missing registrant email — skipping.')
+            continue
+        external_id = email
+
+        # Sanitize the Unique identifier value and only use it as the
+        # external_registration_id if it appears meaningful. Treat common
+        # placeholders like 'N/A'/'NA'/'UNKNOWN' as missing.
+        unique_id_raw = str(record.get('Unique identifier', '')).strip()
+        if unique_id_raw and unique_id_raw.upper() not in ('N/A', 'NA', 'UNKNOWN'):
+            external_registration_id = unique_id_raw
+        else:
+            external_registration_id = None
+
+        ticket_name = str(record.get('Ticket name', '')).strip()
+        given_name = str(record.get('First name', '')).strip()
+        family_name = str(record.get('Last name', '')).strip()
+        display_name = str(record.get('Display name', '')).strip()
+
+        # Per-ticket-type custom fields
+        organization = _zoom_ticket_field(record, 'Organization', ticket_name)
+        professional_title = _zoom_ticket_field(record, 'Job title', ticket_name)
+        phone = (_zoom_ticket_field(record, 'Phone', ticket_name)
+                 or _zoom_ticket_field(record, 'Phone number', ticket_name))
+        address_line_1 = (_zoom_ticket_field(record, 'Address line 1', ticket_name)
+                          or _zoom_ticket_field(record, 'Address', ticket_name))
+        address_line_2 = _zoom_ticket_field(record, 'Address line 2', ticket_name)
+        address_line_3 = _zoom_ticket_field(record, 'Address line 3', ticket_name)
+        city = _zoom_ticket_field(record, 'City', ticket_name)
+        state_province = _zoom_ticket_field(record, 'State/Province', ticket_name)
+        state_province_abb = _zoom_ticket_field(record, 'State/Province Abb', ticket_name)
+        postal_code = (_zoom_ticket_field(record, 'Postal code', ticket_name)
+                       or _zoom_ticket_field(record, 'Zip code', ticket_name)
+                       or _zoom_ticket_field(record, 'Zip/Postal Code', ticket_name))
+        country = _zoom_ticket_field(record, 'Country/Region', ticket_name)
+        country_alpha_2_code = _zoom_ticket_field(record, 'Country Alpha 2 Code', ticket_name)
+        country_subdivision_code = _zoom_ticket_field(record, 'Country Subdivision Code', ticket_name)
+        # location, full_address, location_long, location_short are computed by DB triggers
+
+        event_person_summary = {
+            'event_id': event_id,
+            'event_id_random': event_id_random,
+            'external_id': external_id,
+            'given_name': given_name,
+            'family_name': family_name,
+            'email': email,
+        }
+
+        # TEMPORARY: Axonius-specific mapping for certain ticket / badge labels
+        # to internal `badge_type_code` values. Remove after the event (~2 weeks).
+        normalized_ticket = ticket_name.strip().lower()
+        badge_type_code = None
+        if 'sponsor' in normalized_ticket:
+            badge_type_code = 'sponsor'
+        elif 'attend' in normalized_ticket or 'attendee' in normalized_ticket:
+            badge_type_code = 'attendee'
+        if badge_type_code:
+            log.info(f"Axonius mapping applied: '{ticket_name}' -> '{badge_type_code}'")
+
+        # Parse marketing consent column (if present) and map to badge fields.
+        # Expected values: "Opt-in" => agree_to_tc=True, allow_tracking=True
+        #                  "Opt-out" => agree_to_tc=False, allow_tracking=False
+        #                  "N/A" => None/NULL
+        marketing_raw = None
+        for _k in ('Agree to receive marketing communication?', 'Agree to receive marketing communication', 'Agree to TC', 'agree_to_tc'):
+            if _k in record and str(record.get(_k)).strip() != '':
+                marketing_raw = str(record.get(_k)).strip()
+                break
+
+        agree_to_tc_val = None
+        allow_tracking_val = None
+        if marketing_raw is not None:
+            m = marketing_raw.strip()
+            m_low = m.lower()
+            if m_low in ('n/a', 'na'):
+                agree_to_tc_val = None
+                allow_tracking_val = None
+            elif m_low in ('opt-in', 'optin', 'opt in'):
+                agree_to_tc_val = True
+                allow_tracking_val = True
+            elif m_low in ('opt-out', 'optout', 'opt out'):
+                agree_to_tc_val = False
+                allow_tracking_val = False
+            else:
+                if m_low in ('yes', 'y', 'true', '1'):
+                    agree_to_tc_val = True
+                    allow_tracking_val = True
+                elif m_low in ('no', 'n', 'false', '0'):
+                    agree_to_tc_val = False
+                    allow_tracking_val = False
+                else:
+                    agree_to_tc_val = None
+                    allow_tracking_val = None
+
+        # Need to deal with this special field/column for Axonius
+        # "Agree to receive marketing communication?"
+
+        event_person_data = {
+            'account_id': account_id,
+            'event_id': event_id,
+            'enable': True,
+            'external_id': external_id,
+            'external_registration_id': external_registration_id,
+            'event_person_profile': {
+                'event_id': event_id,
+                'enable': True,
+                'given_name': given_name,
+                'family_name': family_name,
+                'full_name': display_name or f'{given_name} {family_name}'.strip(),
+                'email': email,
+                'phone': phone,
+                'address_line_1': address_line_1,
+                'address_line_2': address_line_2,
+                'address_line_3': address_line_3,
+                'city': city,
+                'state_province': state_province,
+                'state_province_abb': state_province_abb,
+                'postal_code': postal_code,
+                'country': country,
+                'country_alpha_2_code': country_alpha_2_code,
+                'country_subdivision_code': country_subdivision_code,
+                'professional_title': professional_title,
+                'affiliations': organization,
+            },
+            'event_badge': {
+                # 'event_id': event_id,
+                'enable': True,
+                'external_id': external_id,
+                'external_registration_id': external_registration_id,
+                'given_name': given_name,
+                'family_name': family_name,
+                'full_name': display_name or f'{given_name} {family_name}'.strip(),
+                'email': email,
+                'phone': phone,
+                'address_line_1': address_line_1,
+                'address_line_2': address_line_2,
+                'address_line_3': address_line_3,
+                'city': city,
+                'state_province': state_province,
+                'state_province_abb': state_province_abb,
+                'postal_code': postal_code,
+                'country': country,
+                'country_alpha_2_code': country_alpha_2_code,
+                'country_subdivision_code': country_subdivision_code,
+                'professional_title': professional_title,
+                'affiliations': organization,
+                # TEMPORARY: Axonius export does not include a badge template id.
+                # Default to the Axonius group's badge template `RKYp2HcQm9o (21)`.
+                # This is a temporary hardcode — remove or replace when mapping is provided.
+                'event_badge_template_id': 21,
+                'event_badge_template_id_random': 'RKYp2HcQm9o',
+                'badge_type': ticket_name,
+                'badge_type_code': badge_type_code,
+                'agree_to_tc': agree_to_tc_val,
+                'allow_tracking': allow_tracking_val,
+            },
+        }
+
+        # Look up existing event_person by event_id + external_id (should be 0 or 1).
+        sql_select_event_person = """
+            SELECT id AS event_person_id, id_random AS event_person_id_random,
+                   external_id AS event_person_external_id,
+                   event_badge_id AS event_badge_id,
+                   event_person_profile_id AS event_person_profile_id
+            FROM `event_person`
+            WHERE event_person.event_id = :event_id
+              AND event_person.external_id = :external_id
+            /*LIMIT 2*/;
+            """
+
+        event_person_result = sql_select(sql=sql_select_event_person, data=event_person_summary)
+        if event_person_result:
+            # If multiple rows are returned that's an integrity problem — log it and
+            # use the first row for the update to avoid creating duplicates.
+            if isinstance(event_person_result, list):
+                log.error(f'Found more than one Event Person with external_id={external_id}. Count: {len(event_person_result)}')
+                event_person_result = event_person_result[0]
+
+            event_person_id = event_person_result.get('event_person_id')
+            event_badge_id = event_person_result.get('event_badge_id')
+            event_person_profile_id = event_person_result.get('event_person_profile_id')
+            log.info(f'Found Event Person. Updating existing... Event Person ID: {event_person_id}')
+
+            # Don't touch enable on update — a manually disabled record is effectively
+            # blacklisted and should survive repeated re-imports of the same file.
+            event_person_data.pop('enable', None)
+            event_person_data.get('event_badge', {}).pop('enable', None)
+            event_person_data.get('event_person_profile', {}).pop('enable', None)
+            updated_id = create_update_event_person_obj_v4(
+                event_person_dict_obj=event_person_data,
+                event_person_id=event_person_id,
+                account_id=account_id,
+                event_id=event_id,
+                event_badge_id=event_badge_id,
+                event_person_profile_id=event_person_profile_id,
+            )
+            if updated_id:
+                log.warning(f'Event Person updated. ID: {updated_id}')
+            else:
+                log.warning(f'Event Person not updated. ID: {event_person_id}')
+        else:
+            log.info('No Event Person found. Creating new...')
+            result_id = create_update_event_person_obj_v4(
+                event_person_dict_obj=event_person_data,
+                account_id=account_id,
+                event_id=event_id,
+            )
+            if result_id:
+                log.warning(f'Event Person created. ID: {result_id}')
+            else:
+                log.warning('Event Person not created.')
+
+        # Record the processed input for response summary after DB ops.
+        event_badge_person_li.append(event_person_data)
+        event_badge_person_summary_li.append(event_person_summary)
+
+    if return_detail:
+        return mk_resp(data=event_badge_person_li, status_message=f'Zoom CSV import complete. Processed {len(event_badge_person_li)} records.', response=commons.response)
+    else:
+        return mk_resp(data=event_badge_person_summary_li, status_message=f'Zoom CSV import complete. Processed {len(event_badge_person_summary_li)} records.', response=commons.response)
+
+
+# ### BEGIN ### Splash (Cvent) XLSX Badge Import ### event_id_badge_import_splash_xlsx() ###
+# Accepts a Splash (Cvent) registrant XLSX export and inserts/updates event_person records.
+# Splash exports fixed columns: Full Name, Email, Time of RSVP, Status, plus custom
+# fields prefixed with "Custom: ". Email is used as external_id. Full Name is split
+# on the last space into given_name/family_name and also stored directly as full_name.
+# Updated 2026-06-02
+
+@router.post('/event/{event_id}/badge/import/splash_xlsx', response_model=Resp_Body_Base)
+async def event_id_badge_import_splash_xlsx(
+        event_id: str = Path(min_length=11, max_length=22),
+        file: UploadFile = File(...),
+
+        begin_at: int = 0,
+        end_at: int = 20000,
+
+        import_status_filter: str = 'Attending',  # set to '' to import all statuses
+
+        return_detail: bool = False,
+
+        commons: Common_Route_Params = Depends(common_route_params),
+        ):
+    """
+    Import event badges from a Splash (Cvent) registrant XLSX export.
+
+    Splash exports fixed columns (Full Name, Email, Time of RSVP, Status) plus
+    custom fields prefixed with "Custom: ". Email is used as external_id.
+    Full Name is split on the last space into given_name/family_name and also
+    stored directly as full_name. Pass import_status_filter='' to import all
+    statuses (default is 'Attending').
+    """
+    log.setLevel(logging.INFO)
+
+    account_id = commons.x_account_id
+
+    event_id_random = event_id
+    if event_id := redis_lookup_id_random(record_id_random=event_id, table_name='event'): pass
+    else: return mk_resp(data=None, status_code=404, response=commons.response)
+
+    link_to_type = 'event'
+    link_to_id = event_id
+
+    file_info = await save_file(
+        file=file,
+        account_id=account_id,
+        link_to_type=link_to_type,
+        link_to_id=link_to_id,
+    )
+    if file_info['saved']:
+        log.info('File saved')
+    else:
+        log.error('Something may have gone wrong while saving the uploaded file?')
+        return mk_resp(data=None, status_code=500, response=commons.response)
+
+    hosted_files_path = settings.FILES_PATH['hosted_files_root']
+    subdirectory_dest = os.path.join(hosted_files_path, file_info.get('subdirectory_path'))
+    hash_filename = file_info.get('hash_sha256') + '.file'
+    full_file_path = pathlib.Path(os.path.join(subdirectory_dest, hash_filename))
+
+    if not full_file_path.exists():
+        log.warning(f'Not found at full file path: {full_file_path}')
+        return mk_resp(data=None, status_code=500, response=commons.response)
+
+    df = pandas.read_excel(full_file_path, dtype=str, na_filter=False)
+
+    df_dict = df.to_dict(orient='records')
+    log.info(f'Splash XLSX total record count: {len(df_dict)}')
+
+    loop_count = 0
+    skipped_count = 0
+    event_badge_person_li = []
+    event_badge_person_summary_li = []
+
+    log.setLevel(logging.DEBUG)
+    for record in df_dict:
+        log.info(f'Loop Count: {loop_count}')
+        loop_count += 1
+        if loop_count <= begin_at: continue
+        if loop_count > end_at: break
+
+        # Status filter — skip rows that don't match when a filter is set.
+        if import_status_filter:
+            status = str(record.get('Status', '')).strip()
+            if status != import_status_filter:
+                log.info(f'Skipping row with status "{status}" (filter: "{import_status_filter}")')
+                skipped_count += 1
+                continue
+
+        email = str(record.get('Email', '')).strip()
+        if not email:
+            log.warning('Row missing Email — skipping.')
+            skipped_count += 1
+            continue
+        external_id = email
+
+        full_name = str(record.get('Full Name', '')).strip()
+        given_name, family_name = _split_full_name(full_name)
+
+        professional_title = str(record.get('Custom: Job Title', '')).strip()
+        organization = str(record.get('Custom: Company Name', '')).strip()
+        country = str(record.get('Custom: Country', '')).strip()
+        state_province = str(record.get('Custom: State', '')).strip()
+        dietary_restrictions = str(record.get('Custom: Please note any dietary restrictions or preferences.', '')).strip()
+
+        # "Custom: Opt-In" → agree_to_tc / allow_tracking
+        opt_in_raw = str(record.get('Custom: Opt-In', '')).strip().lower()
+        if opt_in_raw in ('yes', 'y', 'true', '1', 'opt-in', 'opt_in'):
+            agree_to_tc_val = True
+            allow_tracking_val = True
+        elif opt_in_raw in ('no', 'n', 'false', '0', 'opt-out', 'opt_out'):
+            agree_to_tc_val = False
+            allow_tracking_val = False
+        else:
+            agree_to_tc_val = None
+            allow_tracking_val = None
+
+        event_person_summary = {
+            'event_id': event_id,
+            'event_id_random': event_id_random,
+            'external_id': external_id,
+            'given_name': given_name,
+            'family_name': family_name,
+            'email': email,
+        }
+
+        event_person_data = {
+            'account_id': account_id,
+            'event_id': event_id,
+            'enable': True,
+            'external_id': external_id,
+            'event_person_profile': {
+                'event_id': event_id,
+                'enable': True,
+                'given_name': given_name,
+                'family_name': family_name,
+                'full_name': full_name,
+                'email': email,
+                'professional_title': professional_title,
+                'affiliations': organization,
+                'country': country,
+                'state_province': state_province,
+            },
+            'event_badge': {
+                'enable': True,
+                'external_id': external_id,
+                'given_name': given_name,
+                'family_name': family_name,
+                'full_name': full_name,
+                'email': email,
+                'professional_title': professional_title,
+                'affiliations': organization,
+                'country': country,
+                'state_province': state_province,
+                'other_1': dietary_restrictions,
+                # TEMPORARY: Axonius DC event badge template mu_7SRuJYum (23).
+                'event_badge_template_id': 23,
+                'event_badge_template_id_random': 'mu_7SRuJYum',
+                'badge_type_code': 'attendee',
+                'agree_to_tc': agree_to_tc_val,
+                'allow_tracking': allow_tracking_val,
+            },
+        }
+
+        sql_select_event_person = """
+            SELECT id AS event_person_id, id_random AS event_person_id_random,
+                   external_id AS event_person_external_id,
+                   event_badge_id AS event_badge_id,
+                   event_person_profile_id AS event_person_profile_id
+            FROM `event_person`
+            WHERE event_person.event_id = :event_id
+              AND event_person.external_id = :external_id
+            /*LIMIT 2*/;
+            """
+
+        event_person_result = sql_select(sql=sql_select_event_person, data=event_person_summary)
+        if event_person_result:
+            if isinstance(event_person_result, list):
+                log.error(f'Found more than one Event Person with external_id={external_id}. Count: {len(event_person_result)}')
+                event_person_result = event_person_result[0]
+
+            event_person_id = event_person_result.get('event_person_id')
+            event_badge_id = event_person_result.get('event_badge_id')
+            event_person_profile_id = event_person_result.get('event_person_profile_id')
+            log.info(f'Found Event Person. Updating existing... Event Person ID: {event_person_id}')
+
+            # Don't touch enable on update — a manually disabled record is effectively
+            # blacklisted and should survive repeated re-imports of the same file.
+            event_person_data.pop('enable', None)
+            event_person_data.get('event_badge', {}).pop('enable', None)
+            event_person_data.get('event_person_profile', {}).pop('enable', None)
+            updated_id = create_update_event_person_obj_v4(
+                event_person_dict_obj=event_person_data,
+                event_person_id=event_person_id,
+                account_id=account_id,
+                event_id=event_id,
+                event_badge_id=event_badge_id,
+                event_person_profile_id=event_person_profile_id,
+            )
+            if updated_id:
+                log.warning(f'Event Person updated. ID: {updated_id}')
+            else:
+                log.warning(f'Event Person not updated. ID: {event_person_id}')
+        else:
+            log.info('No Event Person found. Creating new...')
+            result_id = create_update_event_person_obj_v4(
+                event_person_dict_obj=event_person_data,
+                account_id=account_id,
+                event_id=event_id,
+            )
+            if result_id:
+                log.warning(f'Event Person created. ID: {result_id}')
+            else:
+                log.warning('Event Person not created.')
+
+        event_badge_person_li.append(event_person_data)
+        event_badge_person_summary_li.append(event_person_summary)
+
+    processed = len(event_badge_person_li)
+    if return_detail:
+        return mk_resp(data=event_badge_person_li, status_message=f'Splash XLSX import complete. Processed {processed} records, skipped {skipped_count}.', response=commons.response)
+    else:
+        return mk_resp(data=event_badge_person_summary_li, status_message=f'Splash XLSX import complete. Processed {processed} records, skipped {skipped_count}.', response=commons.response)

app/routers/event_importing.py

@@ -28,6 +28,21 @@ from app.models.response_models import Resp_Body_Base, mk_resp
 router = APIRouter()
 
 
+def _clean_datetime(value) -> str | None:
+    """Normalize datetime strings from CSV imports (handles \xa0 from Excel, 12-hour format)."""
+    if not value:
+        return None
+    cleaned = str(value).replace('\xa0', ' ').strip()
+    if not cleaned:
+        return None
+    for fmt in ('%m/%d/%Y %I:%M %p', '%m/%d/%Y %H:%M', '%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M'):
+        try:
+            return datetime.datetime.strptime(cleaned, fmt).strftime('%Y-%m-%d %H:%M:%S')
+        except ValueError:
+            continue
+    return cleaned
+
+
 # No longer needed? 2024-08-15
 # Based on the program import template the clients are given.
 # Ideally the import file should only contain records with new External IDs. Old records will be checked and only updated if needed.
@@ -332,7 +347,10 @@ router = APIRouter()
 # ### BEGIN ### Event Importing ### event_importing_program_data() ###
 # Based on the program import template the clients are given.
 # Create and update locations, sessions, presentations, and presenters as needed.
-# Updated 2024-03-25
+# Careful with how date and time fields are combined
+# This should work: =TEXT(G2,"M/D/YYYY")&" "&TEXT(H2,"H:MM AM/PM")
+# Simply adding the fields (=D264+E264) sort of works. This produces non breaking spaces but clean up on import.
+# Updated 2026-05-15
 @router.post('/event/{event_id}/importing/program_data', response_model=Resp_Body_Base)
 async def event_importing_program_data(
         event_id: str = Path(min_length=11, max_length=22),
@@ -656,13 +674,8 @@ async def event_importing_program_data(
         if record.get('session_description'):
             event_session_data['description'] = record.get('session_description', '').strip()
 
-        event_session_data['start_datetime'] = record.get('session_start_datetime', '').strip()
-        # event_session_start_datetime = record.get('event_session_start_date', '') + ' ' + record.get('event_session_start_time', '')
-        # event_session_data['start_datetime'] = event_session_start_datetime
-
-        event_session_data['end_datetime'] = record.get('session_end_datetime', '').strip()
-        # event_session_end_datetime = record.get('event_session_end_date', '') + ' ' + record.get('event_session_end_time', '')
-        # event_session_data['end_datetime'] = event_session_end_datetime
+        event_session_data['start_datetime'] = _clean_datetime(record.get('session_start_datetime'))
+        event_session_data['end_datetime'] = _clean_datetime(record.get('session_end_datetime'))
 
         event_session_data['sort'] = record.get('session_sort')
 
@@ -736,19 +749,11 @@ async def event_importing_program_data(
         if record.get('presentation_description'):
             event_presentation_data['description'] = record.get('presentation_description', '').strip()
 
-        if record.get('presentation_start_datetime'):
-            event_presentation_data['start_datetime'] = record.get('presentation_start_datetime', '').strip()
+        event_presentation_data['start_datetime'] = _clean_datetime(record.get('presentation_start_datetime'))
         data['presentation_start_datetime'] = event_presentation_data['start_datetime']
-        else:
-            event_presentation_data['start_datetime'] = None
-            data['presentation_start_datetime'] = None
 
-        if record.get('presentation_end_datetime'):
-            event_presentation_data['end_datetime'] = record.get('presentation_end_datetime', '').strip()
+        event_presentation_data['end_datetime'] = _clean_datetime(record.get('presentation_end_datetime'))
         data['presentation_end_datetime'] = event_presentation_data['end_datetime']
-        else:
-            event_presentation_data['end_datetime'] = None
-            data['presentation_end_datetime'] = None
 
         if record.get('presentation_abstract_code'):
             event_presentation_data['abstract_code'] = record.get('presentation_abstract_code', '').strip()

app/routers/registry.py

@@ -1,16 +1,14 @@
 from fastapi import FastAPI, Depends
 from app.routers.dependencies_v3 import DeprecationParams
 from app.routers import (
-    ae_obj, aether_cfg, api_crud, api_crud_v2, api_crud_v3, api, health, importing, sql,
-    account, contact, data_store,
-    event, event_badge, event_badge_importing, event_badge_template,
-    event_device, event_exhibit, event_exhibit_tracking, event_file, event_importing,
-    event_location, event_person,
-    event_presentation, event_presenter, event_session,
-    flask_cfg, hosted_file, api_v3_actions_hosted_file, api_v3_actions_event_file, api_v3_actions_e_zoom, api_v3_actions_e_novi_mailman, lookup, lookup_v3,
-    organization, page, person,
-    person_user, qr, site, site_domain, user,
-    util_email, websockets, websockets_redis, websockets_v3, e_confex, e_cvent, e_impexium, e_stripe
+    ae_obj, aether_cfg, api_crud_v3, api, health, importing,
+    data_store,
+    event_badge_importing,
+    event_importing,
+    api_v3_actions_email,
+    api_v3_actions_hosted_file, api_v3_actions_event_file, api_v3_actions_event_exhibit, api_v3_actions_e_zoom, api_v3_actions_e_novi_mailman, api_v3_actions_idaa, api_v3_actions_user, lookup_v3,
+    user,
+    util_email, websockets_v3, e_confex, e_cvent, e_impexium, e_stripe
 )
 
 def setup_routers(app: FastAPI):
@@ -21,13 +19,13 @@ def setup_routers(app: FastAPI):
     app.include_router(ae_obj.router, prefix='/ae_obj', tags=['AE Object'])
     app.include_router(aether_cfg.router, tags=['Aether Config'])
     # app.include_router(api_crud.router, prefix='/crud', tags=['CRUD v1.2 (Legacy)'], dependencies=[Depends(DeprecationParams)])
-    app.include_router(api_crud_v2.router, prefix='/v2/crud', tags=['CRUD v2.5'], dependencies=[Depends(DeprecationParams)])
+    # app.include_router(api_crud_v2.router, prefix='/v2/crud', tags=['CRUD v2.5'], dependencies=[Depends(DeprecationParams)])
     app.include_router(api_crud_v3.router, prefix='/v3/crud', tags=['CRUD v3'])
 
     app.include_router(api.router, prefix='/api', tags=['API'])
     # app.include_router(flask_cfg.router, prefix='/flask_cfg', tags=['Flask CFG'], dependencies=[Depends(DeprecationParams)])
-    app.include_router(importing.router, prefix='/importing', tags=['Importing'])
-    app.include_router(sql.router, tags=['SQL'])
+    # app.include_router(importing.router, prefix='/importing', tags=['Importing'], dependencies=[Depends(DeprecationParams)])
+    # app.include_router(sql.router, tags=['SQL'])  # LEGACY (disabled) - raw SQL select endpoint, testing only
     # app.include_router(account.router, tags=['Account'], dependencies=[Depends(DeprecationParams)])
 
     app.include_router(data_store.router, tags=['Data Store'])
@@ -38,8 +36,8 @@ def setup_routers(app: FastAPI):
 
     # app.include_router(event_device.router, tags=['Event Device'], dependencies=[Depends(DeprecationParams)])
     # app.include_router(event_exhibit.router, tags=['Event Exhibit'], dependencies=[Depends(DeprecationParams)])
-    app.include_router(event_exhibit_tracking.router, tags=['Event Exhibit Tracking'])
-    app.include_router(event_file.router, tags=['Event File'])
+    # app.include_router(event_exhibit_tracking.router, tags=['Event Exhibit Tracking'])
+    # app.include_router(event_file.router, tags=['Event File'])
     app.include_router(event_importing.router, tags=['Event Importing'])
     # app.include_router(event_location.router, tags=['Event Location'], dependencies=[Depends(DeprecationParams)])
 
@@ -47,12 +45,16 @@ def setup_routers(app: FastAPI):
     # app.include_router(event_presenter.router, prefix='/event/presenter', tags=['Event Presenter'], dependencies=[Depends(DeprecationParams)])
     # app.include_router(event_session.router, tags=['Event Session'], dependencies=[Depends(DeprecationParams)])
 
-    app.include_router(hosted_file.router, prefix='/hosted_file', tags=['Hosted File'])
+    # app.include_router(hosted_file.router, prefix='/hosted_file', tags=['Hosted File'])
     app.include_router(api_v3_actions_hosted_file.router, prefix='/v3/action/hosted_file', tags=['Hosted File (V3 Actions)'])
     app.include_router(api_v3_actions_event_file.router, prefix='/v3/action/event_file', tags=['Event File (V3 Actions)'])
+    app.include_router(api_v3_actions_event_exhibit.router, prefix='/v3/action/event_exhibit', tags=['Event Exhibit (V3 Actions)'])
     app.include_router(api_v3_actions_e_zoom.router, prefix='/v3/action/e_zoom', tags=['Zoom Events (V3 Actions)'])
     app.include_router(api_v3_actions_e_novi_mailman.router, prefix='/v3/action/e_novi_mailman', tags=['Novi-Mailman Bridge (V3 Actions)'])
-    app.include_router(lookup.router, prefix='/lu', tags=['Lookup'])
+    app.include_router(api_v3_actions_idaa.router, prefix='/v3/action/idaa', tags=['IDAA Actions (V3)'])
+    app.include_router(api_v3_actions_user.router, prefix='/v3/action/user', tags=['User (V3 Actions)'])
+    app.include_router(api_v3_actions_email.router, prefix='/v3/action/email', tags=['Email (V3 Actions)'])
+    # app.include_router(lookup.router, prefix='/lu', tags=['Lookup'])  # LEGACY (disabled) - superseded by /v3/lookup
     app.include_router(lookup_v3.router, prefix='/v3/lookup', tags=['Lookup V3'])
 
     # app.include_router(organization.router, prefix='/organization', tags=['Organization'], dependencies=[Depends(DeprecationParams)])
@@ -63,13 +65,14 @@ def setup_routers(app: FastAPI):
     # app.include_router(qr.router, tags=['QR'], dependencies=[Depends(DeprecationParams)])
     # app.include_router(site.router, tags=['Site'], dependencies=[Depends(DeprecationParams)])
     # app.include_router(site_domain.router, tags=['Site Domain'], dependencies=[Depends(DeprecationParams)])
-    app.include_router(user.router, tags=['User'])
-    app.include_router(util_email.router, tags=['Utility: Email'])
-    app.include_router(websockets.router, tags=['Websockets'])
-    app.include_router(websockets_redis.router, tags=['Websockets (Redis)'])
+    # app.include_router(user.router, tags=['User'], dependencies=[Depends(DeprecationParams)])
+    # app.include_router(util_email.router, tags=['Utility: Email'])  # LEGACY (disabled) - superseded by /v3/action/email/send
+    # app.include_router(websockets.router, tags=['Websockets'])  # LEGACY (disabled) - superseded by Websockets V3
+    # app.include_router(websockets_redis.router, tags=['Websockets (Redis)'])  # LEGACY (disabled) - superseded by Websockets V3
     app.include_router(websockets_v3.router, prefix='/v3', tags=['Websockets V3'])
 
-    app.include_router(e_confex.router, prefix='/e/confex', tags=['External Service: Confex'])
-    app.include_router(e_cvent.router, prefix='/e/cvent', tags=['External Service: Cvent'])
-    app.include_router(e_impexium.router, prefix='/e/impexium', tags=['External Service: Impexium'])
-    app.include_router(e_stripe.router, prefix='/e/stripe', tags=['External Service: Stripe'])
+    # ALERT: Temporarily commenting these out until needed for external service integrations. They can be re-enabled as needed.
+    # app.include_router(e_confex.router, prefix='/e/confex', tags=['External Service: Confex'])
+    # app.include_router(e_cvent.router, prefix='/e/cvent', tags=['External Service: Cvent'])
+    # app.include_router(e_impexium.router, prefix='/e/impexium', tags=['External Service: Impexium'])
+    # app.include_router(e_stripe.router, prefix='/e/stripe', tags=['External Service: Stripe'])

app/routers/site_domain.py

@@ -79,16 +79,18 @@ async def lookup_site_domain_obj(
         fqdn: str,
     # x_account_id: str = Header(...),
     # response: Response = Response,
-
+    access_key: Optional[str] = Query(None, min_length=4, max_length=50),
+    referrer: Optional[str] = Query(None, min_length=8, max_length=150),
     commons: Common_Route_Params_Min = Depends(common_route_params_min),
         ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-
     # Updated 2021-12-13
     if site_domain_rec_list_result := lookup_site_domain_fqdn(
             fqdn = fqdn,
+            access_key = access_key,
+            referrer = referrer,
             enabled = commons.enabled,
             limit = commons.limit,
             offset = commons.offset

app/routers/user.py

@@ -20,65 +20,67 @@ from app.models.user_models import User_Base, User_New_Base, User_Out_Base
 router = APIRouter()
 
 
-@router.post('/user', response_model=Resp_Body_Base)
-async def post_user_obj(
-        obj: User_Base,
-        return_obj: Optional[bool] = True,
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+# @router.post('/user', response_model=Resp_Body_Base)
+# async def post_user_obj(
+#         obj: User_Base,
+#         return_obj: Optional[bool] = True,
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    obj_type = 'user'
-    obj_data_dict = obj.dict(by_alias=False, exclude_unset=True)
-    result = post_obj_template(
-        obj_type = obj_type,
-        data = obj_data_dict,
-        return_obj = True,
-        by_alias = True,
-        exclude_unset = True,
-        )
-    return result
+#     obj_type = 'user'
+#     obj_data_dict = obj.dict(by_alias=False, exclude_unset=True)
+#     result = post_obj_template(
+#         obj_type = obj_type,
+#         data = obj_data_dict,
+#         return_obj = True,
+#         by_alias = True,
+#         exclude_unset = True,
+#         )
+#     return result
 
 
-# ### BEGIN ### API User ### post_user_obj_new() ###
-# Updated 2021-08-21 (complete re-write)
-@router.post('/user/new', response_model=Resp_Body_Base)
-async def post_user_obj_new(
-        user_obj: User_New_Base,
-        allow_update: bool = False,
-        avoid_dup_username: bool = False,
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+# # ### BEGIN ### API User ### post_user_obj_new() ###
+# # Updated 2021-08-21 (complete re-write)
+# @router.post('/user/new', response_model=Resp_Body_Base)
+# async def post_user_obj_new(
+#         user_obj: User_New_Base,
+#         allow_update: bool = False,
+#         avoid_dup_username: bool = False,
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    if account_id_random := user_obj.account_id_random: pass
-    else: return False
+#     if account_id_random := user_obj.account_id_random: pass
+#     else: return False
 
-    if create_user_obj_result := create_user_obj(account_id=account_id_random, user_dict_obj=user_obj, allow_update=allow_update, avoid_dup_username=avoid_dup_username): pass
-    else: return mk_resp(data=False, status_code=400, response=commons.response, status_message='The user account was not created. This is likely because that username already exists for this account.')
+#     if create_user_obj_result := create_user_obj(account_id=account_id_random, user_dict_obj=user_obj, allow_update=allow_update, avoid_dup_username=avoid_dup_username): pass
+#     else: return mk_resp(data=False, status_code=400, response=commons.response, status_message='The user account was not created. This is likely because that username already exists for this account.')
 
-    if isinstance(create_user_obj_result, int):
-        user_id = create_user_obj_result
-        if return_obj:
-            if load_user_obj_result := load_user_obj(user_id=user_id):
-                data = load_user_obj_result
-            else:
-                data = False
-        else:
-            user_id = create_user_obj_result
-            user_id_random = get_id_random(record_id=user_id, table_name='user')
-            data = {}
-            data['user_id'] = user_id
-            data['user_id_random'] = user_id_random
-        return mk_resp(data=data, response=commons.response, status_message='The user account was created.')
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response, status_message='The result from trying to create a user account was unexpected.')
-# ### END ### API User ### post_user_obj_new() ###
+#     if isinstance(create_user_obj_result, int):
+#         user_id = create_user_obj_result
+#         if return_obj:
+#             if load_user_obj_result := load_user_obj(user_id=user_id):
+#                 data = load_user_obj_result
+#             else:
+#                 data = False
+#         else:
+#             user_id = create_user_obj_result
+#             user_id_random = get_id_random(record_id=user_id, table_name='user')
+#             data = {}
+#             data['user_id'] = user_id
+#             data['user_id_random'] = user_id_random
+#         return mk_resp(data=data, response=commons.response, status_message='The user account was created.')
+#     else:
+#         return mk_resp(data=False, status_code=400, response=commons.response, status_message='The result from trying to create a user account was unexpected.')
+# # ### END ### API User ### post_user_obj_new() ###
 
 
 # ### BEGIN ### API User ### user_obj_change_password() ###
+# NOTE: This is actively in use 2026-03-24 -Scott
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 @router.patch('/user/{user_id}/change_password', response_model=Resp_Body_Base)
 async def user_obj_change_password(
         user_id: Union[int,str],
@@ -143,35 +145,37 @@ async def user_obj_change_password(
 # ### END ### API User ### user_obj_change_password() ###
 
 
-@router.patch('/user/{obj_id}', response_model=Resp_Body_Base)
-async def patch_user_obj(
-        obj: User_Base,
-        obj_id: str = Path(min_length=11, max_length=22),
-        return_obj: Optional[bool] = True,
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+# @router.patch('/user/{obj_id}', response_model=Resp_Body_Base)
+# async def patch_user_obj(
+#         obj: User_Base,
+#         obj_id: str = Path(min_length=11, max_length=22),
+#         return_obj: Optional[bool] = True,
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    obj_type = 'user'
-    obj_data_dict = obj.dict(by_alias=False, exclude_unset=True)
-    obj_data_dict['id'] = redis_lookup_id_random(record_id_random=obj_id, table_name=obj_type)
-    obj_data_dict['id_random'] = obj_id
-    result = patch_obj_template(
-        obj_type=obj_type,
-        data=obj_data_dict,
-        obj_id=obj_id,
-        return_obj=True,
-        by_alias=True,
-        exclude_unset=True,
-        )
-    return result
+#     obj_type = 'user'
+#     obj_data_dict = obj.dict(by_alias=False, exclude_unset=True)
+#     obj_data_dict['id'] = redis_lookup_id_random(record_id_random=obj_id, table_name=obj_type)
+#     obj_data_dict['id_random'] = obj_id
+#     result = patch_obj_template(
+#         obj_type=obj_type,
+#         data=obj_data_dict,
+#         obj_id=obj_id,
+#         return_obj=True,
+#         by_alias=True,
+#         exclude_unset=True,
+#         )
+#     return result
 
 
 # ### BEGIN ### API User Routers ### user_new_auth_key() ###
 # Generate a new one time use authorization key for login without password
 # Updated 2022-01-07
 # @router.get('/user/new_auth_key', response_model=Resp_Body_Base)
+# NOTE: This may be actively in use 2026-03-24
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 @router.get('/user/{user_id}/new_auth_key', response_model=Resp_Body_Base)
 async def user_new_auth_key(
         user_id: str = Path(min_length=11, max_length=22),
@@ -218,6 +222,8 @@ async def user_new_auth_key(
 # A new key will need to be requested for a particular user each time.
 # NOTE: Should this be divided into username/password and user ID/auth key endpoints? Probably vote 2x
 # Updated 2021-10-06
+# NOTE: This is actively in use 2026-03-24 -Scott
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 @router.get('/user/authenticate', response_model=Resp_Body_Base)
 async def user_authenticate(
         null_account_id: bool = False,
@@ -394,6 +400,8 @@ async def user_authenticate(
 
 
 # ### BEGIN ### API User ### user_verify_password() ###
+# NOTE: This may be actively in use 2026-03-24
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 # @router.post('/{user_id}/verify_password', response_model=Resp_Body_Base)
 @router.post('/user/verify_password', response_model=Resp_Body_Base)
 async def user_verify_password(
@@ -410,14 +418,14 @@ async def user_verify_password(
     account_id = commons.x_account_id
 
     log.debug(user_obj)
-    log.debug(user_obj.id_random)
+    log.debug(user_obj.id)
     log.debug(user_obj.current_password)
     log.debug(user_obj.username)
 
     if current_password := user_obj.current_password: pass
     else: return mk_resp(data=False, status_code=400, status_message='The current password to verify is required.', response=commons.response) # Bad Request
 
-    if user_id_random := user_obj.id_random: # Use id_random instead of user_id_random when getting from User model.
+    if user_id_random := user_obj.id: # Vision ID: User_Base uses 'id' (not 'id_random') for the random string.
         log.info(f'Using the user ID to look up the user. User ID: {user_id_random}')
         # NOTE: Not doing a redis lookup since we have to look up the record again. Redis lookup may save or add an insignificant amount of time.
         user_data = {}
@@ -487,82 +495,84 @@ async def user_verify_password(
 # ### END ### API User ### user_verify_password() ###
 
 
-# ### BEGIN ### API User ### get_account_user_obj_li() ###
-# Updated 2021-12-13
-@router.get('/account/{account_id}/user/list', response_model=Resp_Body_Base)
-async def get_account_user_obj_li(
-        account_id: str = Path(min_length=11, max_length=22),
-        hidden: str = 'not_hidden', # hidden, not_hidden, all
-        inc_address: bool = False, # Priority l1
-        inc_contact: bool = False, # Priority l1
-        inc_person: bool = False, # Priority l1
-        inc_user_role_list: bool = False, # Priority l1
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+# # ### BEGIN ### API User ### get_account_user_obj_li() ###
+# # Updated 2021-12-13
+# @router.get('/account/{account_id}/user/list', response_model=Resp_Body_Base)
+# async def get_account_user_obj_li(
+#         account_id: str = Path(min_length=11, max_length=22),
+#         hidden: str = 'not_hidden', # hidden, not_hidden, all
+#         inc_address: bool = False, # Priority l1
+#         inc_contact: bool = False, # Priority l1
+#         inc_person: bool = False, # Priority l1
+#         inc_user_role_list: bool = False, # Priority l1
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    if account_id := redis_lookup_id_random(record_id_random=account_id, table_name='account'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response)
+#     if account_id := redis_lookup_id_random(record_id_random=account_id, table_name='account'): pass
+#     else: return mk_resp(data=None, status_code=404, response=commons.response)
 
-    # Updated 2021-12-13
-    if user_rec_list_result := get_user_rec_list(
-            account_id = account_id,
-            hidden = hidden, # hidden, not_hidden, all
-            enabled = commons.enabled,
-            limit = commons.limit,
-            ):
-        user_result_list = []
-        for user_rec in user_rec_list_result:
-            if load_user_result := load_user_obj(
-                    user_id = user_rec.get('user_id', None),
-                    enabled = commons.enabled,
-                    # hidden = hidden,
-                    limit = commons.limit,
-                    inc_address = inc_address,
-                    inc_contact = inc_contact,
-                    inc_person = inc_person,
-                    inc_user_role_list = inc_user_role_list,
-                    by_alias = commons.by_alias,
-                    exclude_unset = commons.exclude_unset,
-                    # model_as_dict = model_as_dict,
-                    ):
-                user_result_list.append(load_user_result)
-            else:
-                user_result_list.append(None)
-        response_data = user_result_list
-    elif isinstance(user_rec_list_result, list) or user_rec_list_result is None: # Empty list or None
-        log.info('No results')
-        return mk_resp(data=False, status_code=404, response=commons.response) # Not Found
-    else:
-        log.warning('Likely bad request')
-        return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
+#     # Updated 2021-12-13
+#     if user_rec_list_result := get_user_rec_list(
+#             account_id = account_id,
+#             hidden = hidden, # hidden, not_hidden, all
+#             enabled = commons.enabled,
+#             limit = commons.limit,
+#             ):
+#         user_result_list = []
+#         for user_rec in user_rec_list_result:
+#             if load_user_result := load_user_obj(
+#                     user_id = user_rec.get('user_id', None),
+#                     enabled = commons.enabled,
+#                     # hidden = hidden,
+#                     limit = commons.limit,
+#                     inc_address = inc_address,
+#                     inc_contact = inc_contact,
+#                     inc_person = inc_person,
+#                     inc_user_role_list = inc_user_role_list,
+#                     by_alias = commons.by_alias,
+#                     exclude_unset = commons.exclude_unset,
+#                     # model_as_dict = model_as_dict,
+#                     ):
+#                 user_result_list.append(load_user_result)
+#             else:
+#                 user_result_list.append(None)
+#         response_data = user_result_list
+#     elif isinstance(user_rec_list_result, list) or user_rec_list_result is None: # Empty list or None
+#         log.info('No results')
+#         return mk_resp(data=False, status_code=404, response=commons.response) # Not Found
+#     else:
+#         log.warning('Likely bad request')
+#         return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
 
-    return mk_resp(data=response_data, response=commons.response)
-# ### END ### API User ### get_account_user_obj_li() ###
+#     return mk_resp(data=response_data, response=commons.response)
+# # ### END ### API User ### get_account_user_obj_li() ###
 
 
-@router.get('/user/list', response_model=Resp_Body_Base)
-async def get_user_obj_li(
-        for_obj_type: Optional[str] = Query(None, min_length=2, max_length=50),
-        for_obj_id: Optional[str] = Query(None, min_length=1, max_length=22),
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+# @router.get('/user/list', response_model=Resp_Body_Base)
+# async def get_user_obj_li(
+#         for_obj_type: Optional[str] = Query(None, min_length=2, max_length=50),
+#         for_obj_id: Optional[str] = Query(None, min_length=1, max_length=22),
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    obj_type = 'user'
-    result = get_obj_li_template(
-        obj_type=obj_type,
-        for_obj_type=for_obj_type,
-        for_obj_id=for_obj_id,
-        by_alias=True,
-        exclude_unset=True,
-        )
-    return result
+#     obj_type = 'user'
+#     result = get_obj_li_template(
+#         obj_type=obj_type,
+#         for_obj_type=for_obj_type,
+#         for_obj_id=for_obj_id,
+#         by_alias=True,
+#         exclude_unset=True,
+#         )
+#     return result
 
 
 # Look up is only for account or person records
+# NOTE: This may be actively in use 2026-03-24
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 @router.get('/user/lookup', response_model=Resp_Body_Base)
 async def lookup_user_obj(
         for_obj_id: Union[int,str],
@@ -638,6 +648,8 @@ async def lookup_user_obj(
 
 
 # Look up a user with an email address for an account
+# NOTE: This is actively in use 2026-03-24 -Scott
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 @router.get('/user/lookup_email', response_model=Resp_Body_Base)
 async def lookup_email(
         email: str = Query(..., min_length=2, max_length=50),
@@ -728,6 +740,8 @@ async def lookup_email(
 
 # Look up is only for account or person records
 # Look up a user with a username for an account
+# NOTE: This may be actively in use 2026-03-24
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 @router.get('/user/lookup_username', response_model=Resp_Body_Base)
 async def lookup_username(
         username: str = Query(..., min_length=2, max_length=50),
@@ -799,6 +813,8 @@ async def lookup_username(
 # This requires the user_id and root_url or base_url.
 # This endpoint will generate a new user auth_key and send the email to the user's email address.
 # Updated 2025-04-08
+# NOTE: This is actively in use 2026-03-24 -Scott
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 # @router.get('/user/email_auth_key_url', response_model=Resp_Body_Base)
 @router.get('/user/{user_id}/email_auth_key_url', response_model=Resp_Body_Base)
 async def email_auth_key_url(
@@ -830,69 +846,69 @@ async def email_auth_key_url(
 # ### END ### API User ### email_auth_key_url() ###
 
 
-# ### BEGIN ### API User ### get_user_obj() ###
-# Updated 2022-01-05
-@router.get('/user/{user_id}', response_model=Resp_Body_Base)
-async def get_user_obj(
-        user_id: str = Path(min_length=11, max_length=22),
-        inc_address: bool = False, # Priority l1
-        # inc_archive_list: bool = False, # Priority l3
-        inc_contact: bool = False, # Priority l1
-        inc_event_list: bool = False, # Priority l1
-        # inc_hosted_file_list: bool = False, # Priority l3
-        inc_journal_list: bool = False, # Priority l2
-        # inc_journal_entry_list: bool = False, # Priority l3
-        inc_membership_person: bool = False, # Priority l2
-        # inc_membership_list: bool = False, # ???
-        inc_order_line_list: bool = False, # Priority l1
-        inc_order_list: bool = False, # Priority l1
-        inc_order_cart_list: bool = False, # Priority l1
-        inc_organization: bool = False, # Priority l1
-        # inc_organization_list: bool = False,
-        inc_person: bool = False, # Priority l1
-        # inc_person_list: bool = False,
-        inc_post_list: bool = False, # Priority l2
-        inc_post_comment_list: bool = False, # Priority l3
-        inc_user_role_list: bool = False, # Priority l1
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+# # ### BEGIN ### API User ### get_user_obj() ###
+# # Updated 2022-01-05
+# @router.get('/user/{user_id}', response_model=Resp_Body_Base)
+# async def get_user_obj(
+#         user_id: str = Path(min_length=11, max_length=22),
+#         inc_address: bool = False, # Priority l1
+#         # inc_archive_list: bool = False, # Priority l3
+#         inc_contact: bool = False, # Priority l1
+#         inc_event_list: bool = False, # Priority l1
+#         # inc_hosted_file_list: bool = False, # Priority l3
+#         inc_journal_list: bool = False, # Priority l2
+#         # inc_journal_entry_list: bool = False, # Priority l3
+#         inc_membership_person: bool = False, # Priority l2
+#         # inc_membership_list: bool = False, # ???
+#         inc_order_line_list: bool = False, # Priority l1
+#         inc_order_list: bool = False, # Priority l1
+#         inc_order_cart_list: bool = False, # Priority l1
+#         inc_organization: bool = False, # Priority l1
+#         # inc_organization_list: bool = False,
+#         inc_person: bool = False, # Priority l1
+#         # inc_person_list: bool = False,
+#         inc_post_list: bool = False, # Priority l2
+#         inc_post_comment_list: bool = False, # Priority l3
+#         inc_user_role_list: bool = False, # Priority l1
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    if user_id := redis_lookup_id_random(record_id_random=user_id, table_name='user'): pass
-    else: return mk_resp(data=None, status_code=404, response=commons.response)
+#     if user_id := redis_lookup_id_random(record_id_random=user_id, table_name='user'): pass
+#     else: return mk_resp(data=None, status_code=404, response=commons.response)
 
-    if user_result := load_user_obj(
-            user_id = user_id,
-            limit = commons.limit,
-            model_as_dict = True, # NOTE: returning model as a dict
-            enabled = commons.enabled,
-            inc_address = inc_address,
-            # inc_archive_list = inc_archive_list,
-            inc_contact = inc_contact,
-            inc_event_list = inc_event_list,
-            # inc_hosted_file_list = inc_hosted_file_list,
-            # inc_journal_list = inc_journal_list,
-            # inc_journal_entry_list = inc_journal_entry_list,
-            # inc_membership_person = inc_membership_person,
-            # inc_membership_list = inc_membership_list, # ???
-            inc_order_line_list = inc_order_line_list,
-            inc_order_list = inc_order_list,
-            inc_order_cart_list = inc_order_cart_list,
-            # inc_organization = inc_organization,
-            # inc_organization_list = inc_organization_list,
-            inc_person = inc_person,
-            # inc_person_list = inc_person_list,
-            # inc_post_list = inc_post_list,
-            # inc_post_comment_list = inc_post_comment_list,
-            inc_user_role_list = inc_user_role_list,
-            ):
-        response_data = user_result
-    else:
-        return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
+#     if user_result := load_user_obj(
+#             user_id = user_id,
+#             limit = commons.limit,
+#             model_as_dict = True, # NOTE: returning model as a dict
+#             enabled = commons.enabled,
+#             inc_address = inc_address,
+#             # inc_archive_list = inc_archive_list,
+#             inc_contact = inc_contact,
+#             inc_event_list = inc_event_list,
+#             # inc_hosted_file_list = inc_hosted_file_list,
+#             # inc_journal_list = inc_journal_list,
+#             # inc_journal_entry_list = inc_journal_entry_list,
+#             # inc_membership_person = inc_membership_person,
+#             # inc_membership_list = inc_membership_list, # ???
+#             inc_order_line_list = inc_order_line_list,
+#             inc_order_list = inc_order_list,
+#             inc_order_cart_list = inc_order_cart_list,
+#             # inc_organization = inc_organization,
+#             # inc_organization_list = inc_organization_list,
+#             inc_person = inc_person,
+#             # inc_person_list = inc_person_list,
+#             # inc_post_list = inc_post_list,
+#             # inc_post_comment_list = inc_post_comment_list,
+#             inc_user_role_list = inc_user_role_list,
+#             ):
+#         response_data = user_result
+#     else:
+#         return mk_resp(data=False, status_code=400, response=commons.response) # Bad Request
 
-    return mk_resp(data=response_data, response=commons.response)
-# ### END ### API User ### get_user_obj() ###
+#     return mk_resp(data=response_data, response=commons.response)
+# # ### END ### API User ### get_user_obj() ###
 
 
 # # ### BEGIN ### API User ### get_user_obj_order_list() ###
@@ -962,17 +978,17 @@ async def get_user_obj(
 # # ### END ### API User ### get_user_obj_order_list() ###
 
 
-@router.delete('/user/{obj_id}', response_model=Resp_Body_Base)
-async def delete_user_obj(
-        obj_id: str = Path(min_length=11, max_length=22),
-        commons: Common_Route_Params = Depends(common_route_params),
-        ):
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
+# @router.delete('/user/{obj_id}', response_model=Resp_Body_Base)
+# async def delete_user_obj(
+#         obj_id: str = Path(min_length=11, max_length=22),
+#         commons: Common_Route_Params = Depends(common_route_params),
+#         ):
+#     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     log.debug(locals())
 
-    obj_type = 'user'
-    result = delete_obj_template(
-        obj_type=obj_type,
-        obj_id=obj_id,
-        )
-    return result
+#     obj_type = 'user'
+#     result = delete_obj_template(
+#         obj_type=obj_type,
+#         obj_id=obj_id,
+#         )
+#     return result

app/routers/util_email.py

@@ -20,6 +20,8 @@ router = APIRouter()
 
 # ### BEGIN ### API Utility: Email ### util_email_send_obj() ###
 # Updated 2023-06-27
+# NOTE: This is actively in use 2026-03-24 -Scott
+# This is marked for deprecation and must be migrated to Aether API v3 standards!
 @router.post('/util/email/send', response_model=Resp_Body_Base)
 async def util_email_send_obj(
         email_send_obj: Email_Send_Base,

documentation/ARCH__V3_DEVELOPMENT_STANDARDS.md

@@ -37,11 +37,73 @@ Finalized Jan 15, 2026, to ensure boot stability.
 -   **POST Based**: Complex filtering is handled via `POST /search` with a JSON body containing `and`, `or`, and `not` logic.
 -   **Hybrid Filtering**: (Proposed) Query parameters should append simple standard filters (e.g., `?enabled=true`) to the complex body logic.
 
-### Response Views (Proposed)
--   Implement a `view` parameter (e.g., `?view=rich`) to allow clients to request joined data without using legacy `use_alt_tbl` flags.
+### Field Evolution Checklist
+When a table or view gains, loses, or renames fields, keep the API contract and search registry in sync:
 
-## 4. Stability Rules
+1. Update the Pydantic model in `app/models/` first so CRUD serialization matches the new shape.
+2. Update the SQL view or table projection so `GET` and `SEARCH` responses actually return the field.
+3. Update `searchable_fields` in `app/object_definitions/` only for fields that should be searchable.
+4. Add write-only, virtual, or view-only fields to `fields_to_exclude_from_db` when they must not be persisted.
+5. Run the schema/search E2E tests that cover the object type before handing the change off.
+6. **Restart the Docker API containers** (`docker compose restart ae_api`) — Python file changes inside containers are not picked up until restart.
+
+For `archive_content`, the public field set now includes `external_id` and `code`, and future additions should follow the same order of operations.
+
+#### Alt-view fields (fields only in `tbl_alt`)
+Some objects have a richer alternate SQL view (`tbl_alt`) that adds JOINed/computed columns absent from the default view (`tbl_default`). For example, `event_session` uses `v_event_session_w_file_count` as its alt view (triggered by `?view=alt` on search, or `?inc_file_count=true` on GET).
+
+- Fields from `tbl_alt` **must still be declared in the Pydantic model** and in `searchable_fields` — Pydantic strips undeclared fields, and the search whitelist rejects unknown field names regardless of the view.
+- When adding such a field, add a comment noting which view provides it (e.g., `# from v_event_session_w_file_count`).
+- Searching by an alt-view field on the default endpoint returns `400 Unknown column` — this is correct behaviour. Clients must pass `?view=alt` to use those fields in a search.
+- **Known alt-view fields restored May 2026:** `event_presentation_li_qry_str`, `event_presenter_li_qry_str` (event_session); `account_name`, `account_code`, and related convenience fields (site_domain).
+
+### Response Views (`?view=` parameter)
+-   The nested search router (`api_crud_v3_nested.py`) already supports `?view=<key>` to switch between registered views. `view=default` uses `tbl_default`; `view=alt` uses `tbl_alt`; additional named views can be added to the object registry as `tbl_<name>` / `mdl_<name>`.
+-   Flat search (`api_crud_v3.py`) does not yet support `?view=` — it always uses `tbl_default`.
+
+## 4. V3 Dependency Injection Reference
+
+All V3 endpoints use granular, composable `Depends()` from `app/lib_general_v3.py`:
+
+| Dependency | Purpose |
+|---|---|
+| `get_account_context` → `AccountContext` | Resolves `account_id` with precedence: Header → Query Token → Bypass Header. Raises 403 on guest/missing context. |
+| `PaginationParams` | Standardizes `limit` and `offset`. |
+| `StatusFilterParams` | Handles `enabled` and `hidden` filtering. |
+| `SerializationParams` | Controls Pydantic serialization (`by_alias`, `exclude_unset`). |
+| `DelayParams` | Optional latency simulation (`?delay=N`) via `await asyncio.sleep()`. |
+
+`AccountContext` also carries `administrator`, `manager`, and `super` flags, populated by a deferred DB lookup when a JWT is present. These flags control whether account isolation is bypassed for support tasks.
+
+## 5. Security and Data Isolation
+
+### Fail-Closed Strategy
+If `account_id` or auth context is missing, the API defaults to a blocking filter (`account_id IS NULL`) — it does NOT fall back to returning all records. Never relax this.
+
+### Multi-Tenant Isolation
+- **Forced account filtering**: `apply_forced_account_filter` injects an `account_id` WHERE clause into every list/search query for non-super users.
+- **Post-retrieval verification**: Single-object GET, PATCH, DELETE include a secondary ownership check (`check_account_access`). A mismatch returns 403.
+- **Hierarchical verification**: Nested endpoints verify parent ownership before allowing operations on children.
+- **Creation guard**: On POST, the user's `account_id` is automatically forced onto the new record.
+
+### IDAA Privacy Baseline
+No IDAA object (Events, Files, Posts, Meetings) is public by default. All routes require `x-account-id` context. The sole exception is `site_domain` (used for site bootstrapping). This is a **Sev-1 class constraint** — violating it has happened before.
+
+### Bypass / Admin Access
+- `x-no-account-id: bypass` → grants super access, resolves to `account_id=1` (One Sky IT Demo). Use only in internal/development utilities; do not expand its use.
+- JWT query parameter (`?jwt=...`) is supported for download links and share URLs where custom headers cannot be provided.
+
+## 6. FastAPI and Pydantic Gotchas
+
+- **`response: Response` injection**: Use it as a direct type hint in function signature. `Depends(Response)` is not valid and causes router initialization failures.
+- **Parameter order**: In function signatures, arguments without defaults must come before `Depends()` arguments.
+- **`asyncio.sleep()` not `time.sleep()`**: Blocking the event loop in an async endpoint causes worker timeouts and `502 Bad Gateway` under load.
+- **Pydantic V1 only**: Do not use V2-only features (`computed_field`, `model_validator`, etc.). The migration is a separate planned project — see strategic goals in `TODO__Agents.md`.
+- **`obj_type_kv_li` in `ae_obj_types_def.py`**: Supports both modern keys (`tbl`, `mdl`) and legacy keys (`table_name`, `base_name`). Legacy V2 endpoints depend on the legacy keys — do not remove them until V1/V2 are fully retired.
+
+## 7. Stability Rules
 
 1.  **Baby Step Testing**: Restart Docker and verify root health after *every* modular change.
 2.  **Avoid Shadowing**: Never name a module part of the `app.` package the same as a common instance variable (e.g., avoid `app.middleware` package if you use `app = FastAPI()`).
 3.  **Deferred Imports**: Use `from app.db_sql import ...` *inside* functions in library modules to prevent circular dependency traps.
+4.  **Model changes require container restart**: Editing Python files on the host does not hot-reload inside Docker. Always run `docker compose restart ae_api` after model or object-definition changes, then re-run E2E tests.

documentation/BOOTSTRAP__AI_Agent_Quickstart.md

@@ -0,0 +1,420 @@
+# Aether API — AI Agent Bootstrap / Quickstart
+> **Read this first.** This doc is the fast path to being productive on this project.
+> It covers the rules, patterns, and gotchas that matter most.
+> Deep dives are in the linked docs at the bottom.
+
+---
+
+## 1. What This Project Is
+
+**Aether** is an event management platform built by One Sky IT (Scott Idem).
+This repo is the backend: **FastAPI + MariaDB**, running inside Docker.
+
+The frontend (`aether_app_sveltekit/`) talks to this API exclusively via the V3 REST API.
+There is **no standalone dev server** — the API runs only in Docker.
+
+**Key clients:**
+- **Conference organizers** — Presentation management, badges, sessions
+- **Exhibitors** — Leads capture
+- **IDAA** — International Doctors in Alcoholics Anonymous (strictly private medical/recovery community)
+
+**Stack at a glance:**
+
+| Layer | Technology |
+|---|---|
+| Framework | FastAPI + Pydantic V1 (upgrade deferred) |
+| Database | MariaDB via SQLAlchemy 1.4 (upgrade deferred) |
+| Cache | Redis |
+| Auth | Custom headers: `x-aether-api-key` + `x-account-id` |
+| Container | Docker / Gunicorn — source is volume-mounted (no rebuild for Python changes, but **restart required**) |
+
+---
+
+## 2. Critical Rules — Read Before Touching Any Code
+
+### Privacy (Sev-1 class failures if violated)
+- **IDAA content is ALWAYS private.** All routes under `/idaa/` require authentication.
+  A previous agent accidentally exposed IDAA bulletin board data publicly.
+  This is the single most serious class of mistake on this project.
+  When in doubt — it's private. Always verify with `Depends(get_account_context)`.
+- **Journals** are private personal data. Always authenticated.
+
+### File Safety
+- **Never use `rm`** to delete files. Move to `~/tmp/gemini_trash` instead.
+- **Never commit `.env`** files, API keys, or passwords of any kind.
+- Third-party credentials (Novi API key, Mailman credentials) live in the **MariaDB `site.cfg_json` column**, not in `.env` or code.
+
+### Before Every Commit
+1. `python3 -m py_compile <changed_file>` — syntax check
+2. Restart Docker and verify the API starts clean: `docker compose restart ae_api`
+3. Check logs: `docker compose logs -f ae_api` (look for startup errors or import failures)
+4. Run the relevant E2E or unit test suite
+
+### Before Starting Any Task
+- Read `documentation/TODO__Agents.md` — active tasks, known bugs, and what was recently changed and why.
+- Check `tests/README.md` — which test suite covers the area you're about to touch.
+
+### Docker Restart is Mandatory After Python Changes
+The API source is volume-mounted, so file edits appear instantly inside the container — but
+**Gunicorn does NOT hot-reload**. You MUST run:
+```bash
+docker compose restart ae_api
+```
+after any Python file change before testing. This is the #1 cause of "my change didn't take effect."
+
+---
+
+## 3. Environment & Commands Cheat Sheet
+
+```bash
+# Start full stack
+cd ~/OSIT_dev/aether_container_env && docker compose up -d
+
+# Restart API after any Python change
+docker compose restart ae_api
+
+# Follow API logs
+docker compose logs -f ae_api
+
+# Shell into the container
+docker compose exec ae_api bash
+
+# Run unit tests (from project root)
+./environment/bin/python3 -m pytest tests/unit/ -v
+
+# Run a single E2E test
+./environment/bin/python3 tests/e2e/test_e2e_v3_search_engine.py
+
+# Check a specific Python file compiles
+./environment/bin/python3 -m py_compile app/methods/my_new_methods.py
+```
+
+**Development API:** `https://dev-api.oneskyit.com`
+**Dev API secret key:** `nT0jPeiCfxSifkiDZur9jA`
+**Standard test Agent API key:** `PMM4n50teUCaOMMTN8qOJA`
+
+**Dev DB via phpMyAdmin:** `http://localhost:8081`
+
+---
+
+## 4. V3 Action Router Pattern
+
+When adding a new action endpoint (not CRUD), follow this structure:
+
+### File layout
+```
+app/methods/my_feature_methods.py   ← business logic
+app/routers/api_v3_actions_my_feature.py  ← route handler (thin)
+```
+
+Then register in `app/routers/registry.py`:
+```python
+from app.routers import api_v3_actions_my_feature
+# ...
+app.include_router(api_v3_actions_my_feature.router, prefix='/v3/action/my_feature', tags=['My Feature (V3 Actions)'])
+```
+
+### Standard route handler pattern
+```python
+from fastapi import APIRouter, Depends
+from app.lib_general_v3 import AccountContext, get_account_context, DelayParams
+from app.models.response_models import Resp_Body_Base, mk_resp
+from app.methods.my_feature_methods import my_business_logic
+
+router = APIRouter()
+
+@router.get('/my_endpoint/{obj_id}', response_model=Resp_Body_Base)
+async def get_my_endpoint(
+        obj_id: str,
+        account: AccountContext = Depends(get_account_context),
+        delay: DelayParams = Depends(),
+        ):
+    if delay.sleep_time_s > 0:
+        await asyncio.sleep(delay.sleep_time_s)
+
+    result = my_business_logic(obj_id)
+    status = result.get('status', 503)
+
+    if status == 200:
+        return mk_resp(data=result['data'])
+
+    return mk_resp(data=False, status_code=status, status_message=result.get('reason', 'Error.'))
+```
+
+### Auth dependency
+`Depends(get_account_context)` is the **standard V3 gate**:
+- Requires a valid `x-aether-api-key`
+- Requires `x-account-id` OR a valid JWT session OR bypass mode
+- Raises 403 if `auth_method == 'guest'` or no account context can be resolved
+- For IDAA/private routes: this is the minimum gate. Never relax it.
+
+---
+
+## 5. Loading Site-Based Credentials (the `_load_idaa_cfg` Pattern)
+
+Third-party credentials (Novi API key, Mailman credentials, etc.) are stored in MariaDB
+in the `site.cfg_json` column for the relevant site record. Do NOT store them in `.env`.
+
+The IDAA site record is:
+- `id_random = '58_gJESdlUh'` (site id=17)
+- Fields: `novi_api_root_url`, `novi_idaa_api_key`, `mailman_base_url`, `mailman_username`, `mailman_password`, `novi_mailman_sync`
+
+Pattern to load them (use deferred import to avoid circular deps):
+```python
+IDAA_SITE_ID_RANDOM = '58_gJESdlUh'
+
+def _load_idaa_cfg() -> dict:
+    from app.db_sql import load_site_obj
+    site = load_site_obj(IDAA_SITE_ID_RANDOM)
+    if not site:
+        raise RuntimeError('IDAA site record not found')
+    cfg = site.get('cfg_json') or {}
+    if isinstance(cfg, str):
+        import json
+        cfg = json.loads(cfg)
+    return cfg
+```
+
+---
+
+## 6. Redis Cache Pattern
+
+```python
+import json
+import datetime
+from app.lib_redis_helpers import redis_client
+
+_CACHE_TTL = datetime.timedelta(hours=4)
+
+def _cache_key(uuid: str) -> str:
+    return f'idaa:novi_member:{uuid}'
+
+# Reading
+raw = redis_client.get(_cache_key(uuid))
+if raw:
+    return json.loads(raw)
+
+# Writing (only on success — never cache error states)
+redis_client.setex(_cache_key(uuid), _CACHE_TTL, json.dumps(result))
+```
+
+**Key naming convention:** `{module}:{object_type}:{identifier}` — e.g. `idaa:novi_member:{uuid}`
+
+**Never cache:**
+- 404 responses — the member may have just joined; next call should hit the source
+- 429 / 503 errors — transient failures should not poison future callers
+
+**Cache key scoping:** If the underlying data source is the same regardless of caller
+(e.g. Novi credentials are hardcoded to the IDAA site — same UUID always returns same data),
+drop `account_id` from the key. Per-caller scoping wastes Redis space and halves hit rate.
+
+---
+
+## 7. The `@logger_reset` Decorator — Unit Test Gotcha
+
+Business logic methods use `@logger_reset` from `app.lib_general`:
+
+```python
+from app.lib_general import logger_reset
+
+@logger_reset
+def my_method(arg):
+    ...
+```
+
+**In unit tests, this decorator MUST be mocked as a passthrough.**
+If `app.lib_general` is replaced with a plain `MagicMock()`, the decorator becomes a
+MagicMock, which when applied to a function replaces it with `MagicMock()()` — the decorated
+function is gone and every call returns garbage.
+
+```python
+# WRONG — logger_reset becomes a MagicMock and swallows the function:
+sys.modules['app.lib_general'] = MagicMock()
+
+# CORRECT — make it a passthrough decorator:
+mock_lib_general = MagicMock()
+mock_lib_general.logger_reset = lambda f: f
+sys.modules['app.lib_general'] = mock_lib_general
+```
+
+This is the #1 cause of unit tests returning `<MagicMock name='...'>` instead of real dicts.
+
+---
+
+## 8. Field Evolution Checklist
+
+When a table or view gains, loses, or renames fields — **do all of these in order**:
+
+1. Update the Pydantic model in `app/models/`
+2. Update the SQL view or table projection so GET/SEARCH return the field
+3. Update `searchable_fields` in `app/object_definitions/` (only for searchable fields)
+4. Add virtual/view-only fields to `fields_to_exclude_from_db` if they must not be persisted
+5. Run the relevant schema/search E2E tests
+6. **Restart Docker:** `docker compose restart ae_api`
+
+#### Alt-view fields (in `tbl_alt` only)
+Some objects have a richer alternate SQL view triggered by `?view=alt`. These fields
+**must still be declared in the Pydantic model and `searchable_fields`** even if they only
+appear in the alt view — Pydantic strips undeclared fields silently.
+
+---
+
+## 9. Deferred Imports in Library Modules
+
+To avoid circular dependency traps, **never import from `app.db_sql` or other app modules
+at the top level of a library module**. Use deferred imports inside functions:
+
+```python
+# WRONG — causes circular import at startup:
+from app.db_sql import load_site_obj
+
+def my_func():
+    return load_site_obj('abc')
+
+# CORRECT:
+def my_func():
+    from app.db_sql import load_site_obj
+    return load_site_obj('abc')
+```
+
+---
+
+## 10. Pydantic / SQLAlchemy Version Pins — Do Not Remove
+
+Current intentional pins:
+- `pydantic==1.*`
+- `SQLAlchemy==1.4.52`
+
+A Pydantic V2 and SQLAlchemy 2.0 migration is planned but not started. Until then,
+**do not upgrade these packages** — V2 touches every model definition and the migration
+is a dedicated project.
+
+---
+
+## 11. Mistakes Agents Have Made on This Project
+
+These are real incidents — know them before you start.
+
+1. **IDAA data exposed publicly** — an agent removed an auth guard from the bulletin board
+   router. Consequence: private IDAA recovery community data was publicly accessible.
+   Always verify `Depends(get_account_context)` is present on every IDAA route.
+
+2. **"My code change has no effect"** — Python file was edited but Docker was not restarted.
+   The API runs Gunicorn inside Docker with no hot-reload. `docker compose restart ae_api`
+   is required after every Python change.
+
+3. **`@logger_reset` mock swallows functions in unit tests** — see Section 7 above.
+   Symptom: `assert result['status'] == 200` fails with `TypeError: 'MagicMock' is not subscriptable`.
+
+4. **`pytest` / `pytest-asyncio` not installed after venv rebuild** — these are dev-only
+   dependencies not in `requirements.txt`. After any OS Python update (e.g., Arch Linux
+   upgrading to a new Python minor), rebuild the venv and reinstall:
+   ```bash
+   ./environment/bin/pip install pytest pytest-asyncio
+   ```
+
+5. **Global `db` connection used instead of context manager** — `lib_sql_core.py` has a
+   global `db = engine.connect()` that is a fragile single connection, not a pool.
+   For new methods, prefer `engine.connect()` as a context manager. See `TODO__Agents.md`
+   → "[P3 full]" task for the planned migration.
+
+6. **`bypass` mode hardcodes `account_id=1`** — `x-no-account-id: bypass` resolves to
+   `account_id=1` (One Sky IT Demo). Lookup overrides from the Demo account can leak into
+   bypass sessions. Do not expand bypass usage without documenting the allowlist case.
+
+7. **Caching error states in Redis** — caching 404 or 5xx responses poisons the cache.
+   A member who just joined Novi would be denied for 4 hours if their 404 was cached.
+   Only cache verified success (200) results.
+
+8. **Not running `docker compose restart ae_api` between model changes and E2E tests** —
+   the E2E suite hits the live API, which is still running the old code until restarted.
+   Tests will pass or fail against stale behavior and the results are meaningless.
+
+---
+
+## 12. Test Patterns
+
+### Unit tests (fast, no DB/network)
+```bash
+./environment/bin/python3 -m pytest tests/unit/ -v
+```
+- Mock all DB/Redis/HTTP at the top of the file before importing the module under test
+- `@logger_reset` must be mocked as a passthrough (see Section 7)
+- Always run from the **project root** — scripts use `sys.path.append(os.getcwd())`
+
+### E2E tests (live API at dev-api.oneskyit.com)
+```bash
+./environment/bin/python3 tests/e2e/test_e2e_v3_search_engine.py
+```
+- Require the Docker stack to be running with a working DB connection
+- Use standard output format: `[✅ PASS]` / `[❌ FAIL]`
+- Verify IDs in responses are **strings** (not integers) — ID Vision compliance
+
+### Which tests to run
+| Change type | Required suites |
+|---|---|
+| Model / ID Vision changes | `test_e2e_v3_demo_parity.py`, vision parity tests |
+| Search / filter changes | `test_e2e_v3_search_engine.py` |
+| Auth / account context changes | `test_e2e_v3_security_audit.py`, `test_e2e_v3_auth_security.py` |
+| Any router or registry change | `test_e2e_v3_security_audit.py` |
+| Novi-Mailman bridge changes | `test_e2e_v3_action_novi_mailman.py` |
+| IDAA Novi verify changes | `tests/unit/test_unit_idaa_novi_verify.py` |
+
+---
+
+## 13. Source Layout (Quick Reference)
+
+```
+app/
+  main.py                    — FastAPI app, lifespan, CORS
+  config.py                  — Pydantic Settings (from .env via Docker)
+  routers/
+    registry.py              — All router registrations live here
+    api_crud_v3.py           — Generic V3 CRUD (flat)
+    api_crud_v3_nested.py    — Generic V3 CRUD (nested/parent-owned)
+    api_v3_actions_*.py      — Action endpoints (one file per domain)
+    dependencies_v3.py       — Shared FastAPI Depends() helpers
+  models/                    — Pydantic V1 models (one file per domain)
+  object_definitions/        — Per-object searchable_fields, field metadata
+  ae_obj_types_def.py        — Object type registry (all Aether types defined here)
+  methods/                   — Business logic (one file per feature/domain)
+  lib_api_crud_v3.py         — Generic CRUD handler (all object types share this)
+  lib_schema_v3.py           — Dynamic schema/field resolution per object type
+  lib_general_v3.py          — AccountContext, get_account_context, DelayParams
+  lib_general.py             — logger_reset and general utilities
+  db_sql.py                  — Import facade (always import from here)
+  lib_sql_core.py            — SQLAlchemy engine, global db connection (Source of Truth)
+  lib_sql_crud.py            — sql_insert, sql_select, sql_update, etc.
+  lib_redis_helpers.py       — redis_client global instance
+
+tests/
+  unit/                      — Isolated logic tests (mock everything, no DB)
+  integration/               — Requires local MariaDB/Redis
+  e2e/                       — Network tests against live dev API
+  tools/                     — Admin utilities (stress test, registry generator)
+  mock_config_helper.py      — Mock app.config.settings — use in all unit tests
+  README.md                  — Which tests cover what; when to run them
+
+documentation/
+  TODO__Agents.md            — Active tasks + session notes ← always read first
+  ARCH__V3_DEVELOPMENT_STANDARDS.md — Master V3 standards doc
+  ARCH__V3_CORE.md           — Module architecture (lifespan, DB layers, logging)
+  GUIDE__AE_API_V3_for_Frontend.md  — Frontend integration guide (keep current)
+  GUIDE__DEVELOPMENT.md      — Commit SOP, verification checklist
+```
+
+---
+
+## 14. Reading Order for Deeper Dives
+
+| What you need | Read |
+|---|---|
+| Active tasks + known bugs | `documentation/TODO__Agents.md` ← always first |
+| V3 standards and strategy | `documentation/ARCH__V3_DEVELOPMENT_STANDARDS.md` |
+| Module architecture | `documentation/ARCH__V3_CORE.md` |
+| Frontend API integration guide | `documentation/GUIDE__AE_API_V3_for_Frontend.md` |
+| WebSocket integration | `documentation/GUIDE__AE_API_V3_for_Frontend_websockets.md` |
+| Commit / verification SOP | `documentation/GUIDE__DEVELOPMENT.md` |
+| Which tests to run | `tests/README.md` |
+| Object type registry | `app/ae_obj_types_def.py` |
+| Shared agent docs | `~/agents_sync/aether/docs/UNIFIED_AGENT_ARCH.md` |

documentation/GUIDE__AE_API_V3_for_Frontend.md

@@ -19,8 +19,15 @@ Required for any non-public data (Journals, Badges, Users, etc.).
     *   **Header:** `x-account-id: <account_id>`
 2.  **Administrative Bypass**: For authorized scripts needing global access.
     *   **Header:** `x-no-account-id: bypass`
+  *   **Scope:** Narrow escape hatch only. Keep it limited to allowlisted bootstrap/public/global-default paths and prefer `x-account-id` or JWT-backed requests everywhere else.
 3.  **Token Access**: Provide a **JWT** in the query string.
     *   **Query Param:** `?jwt=<token>`
+4.  **Important Distinction:** A query parameter named `key` is **not** an account-context bypass signal.
+  *   `key` may be used by specific endpoints/business logic, but it must **not** cause the frontend to remove `x-account-id`.
+  *   Only explicit `x-no-account-id: bypass` should strip account context.
+
+> [!NOTE]
+> The `x-no-account-id` path should continue to shrink over time. If you need a new use, document why `x-account-id` or JWT cannot cover it and mark the use as temporary unless it is a hard bootstrap/global-default requirement.
 
 > [!CAUTION]
 > **UNSUPPORTED HEADERS:** The header `x-aether-api-token` is **NOT recognized** by the V3 API. If you send it, the backend will treat you as a guest and block access to private data.
@@ -44,6 +51,37 @@ When the frontend first loads and doesn't know the `account_id`, it performs a "
 *   Returns 200 + a list containing the `account_id` (random string ID) and `site_id` (random string ID).
 *   ** デザイン Choice:** If the domain is not found, it returns **200 OK with an empty list `[]`**. It is NOT a 404.
 
+> **Access Key Support**
+>
+> Some client deployments restrict their domain via an access key passed in the browser URL (e.g. `?key=abc123`). The frontend reads this param and forwards it as `access_key` in the POST body.
+>
+> **How to pass the key:**
+> ```json
+> {
+>   "and": [
+>     { "field": "fqdn", "op": "eq", "value": "client.example.com" },
+>     { "field": "access_key", "op": "eq", "value": "abc123" }
+>   ]
+> }
+> ```
+> If `key` is absent, empty, or falsy — **omit `access_key` from the payload entirely**. Do not send `"access_key": ""`.
+>
+> **Server behavior:**
+> - `site_access_key` (site-level key) takes priority. If set, all domains under that site require it.
+> - `site_domain_access_key` (domain-level key) is used as fallback when `site_access_key` is not set.
+> - A domain is **public** only when **both** key columns are NULL/empty.
+> - Falsy `access_key` values are ignored server-side as a safety net.
+> - Match → `200` with the record. No match → `200` with empty list `[]`.
+> - Do **not** use `access_code_kv_json` for this — that field is for UI features only.
+>
+> | Browser URL | `access_key` in payload | Result |
+> |---|---|---|
+> | `https://dev-demo.oneskyit.com` | *(omit)* | ✅ Returns record (public) |
+> | `https://client.example.com/?key=correct` | `"correct"` | ✅ Returns record |
+> | `https://client.example.com/` | *(omit)* | ❌ Empty (key required) |
+> | `https://client.example.com/?key=wrong` | `"wrong"` | ❌ Empty (wrong key) |
+> | `https://client.example.com/?key=` | *(omit — strip empty)* | ❌ Empty (key required) |
+>
 ---
 
 ## 3. Standard CRUD Patterns
@@ -68,39 +106,161 @@ Modify data in the system.
     *   **Header:** `x-ae-ignore-extra-fields: true`
     *   **Behavior:** When set to `true`, the backend will automatically strip any fields from the payload that are not defined in the object's model before attempting to save to the database.
 
+#### `*_json` field serialization — do NOT pre-stringify in route/component code
+
+The frontend API wrappers (`src/lib/ae_api/api_post__crud_obj.ts` for V3, `src/lib/api/api.ts` for legacy CRUD) automatically serialize any field whose name ends in `_json` (e.g. `cfg_json`, `data_json`) before sending. They pretty-print with 2-space indent via an internal `serialize_json_field_pretty()` helper.
+
+**Pass `*_json` fields as plain JS objects from routes and components.** The serialization layer handles the rest.
+
+```ts
+// ✅ Correct — pass as plain object; V3 wrapper serializes it
+await update_ae_obj__site({ site_id, data_kv: { cfg_json: { jitsi_token_endpoint: url } } });
+
+// ❌ Wrong — double-encodes the JSON string (the wrapper would stringify an already-stringified value)
+await update_ae_obj__site({ site_id, data_kv: { cfg_json: JSON.stringify({ jitsi_token_endpoint: url }) } });
+```
+
+The V3 wrapper (`api_post__crud_obj.ts`) only serializes when `typeof value === 'object'`, so it will not double-encode a plain string. The legacy wrapper (`api.ts`) stringifies unconditionally, so pre-stringifying there **will** produce double-encoded JSON. In both cases, the right answer is to pass the raw object and let the layer handle it.
+
+### D. ID Fields in Responses (Vision ID Convention)
+
+> [!IMPORTANT]
+> **V3 responses always use random string IDs — never database integers.**
+
+All V3 responses — `POST` create, `GET` single, `GET` list, search, and `PATCH` update — contain:
+
+| Field | Type | Use |
+| :--- | :--- | :--- |
+| `{obj_type}_id` | `string` | **Primary public ID.** Use this for subsequent `PATCH` calls and UI routing. |
+| `{obj_type}_id_random` | `string` | Legacy alias. Same value as `{obj_type}_id`. Present for backward compat only. |
+
+**Example — create then immediately PATCH:**
+```ts
+const created = await postArchiveContent(archiveId, payload);
+const newId = created.data.archive_content_id; // random string e.g. "xK9mP3qRtL2"
+
+// Use it directly in the PATCH URL — no lookup needed
+await patchArchiveContent(newId, { name: 'Updated Name' });
+// PATCH /v3/crud/archive/{archive_id}/archive_content/{newId}
+```
+
+> **Note on `_id_random` suffix:** The `{obj_type}_id_random` field is a legacy artifact from the pre-Vision model. Once you confirm `{obj_type}_id` is a random string (length 11–22), you do not need `_id_random` as a fallback. New code should only read `{obj_type}_id`.
+
 ---
 
 ## 4. V3 Uniform Lookup System
 
-The V3 Lookup system provides a hierarchical, deduplicated interface for standardized tables (Countries, Timezones, etc.). It supports global defaults, account overrides, and site-specific whitelisting.
+The V3 Lookup system provides a hierarchical, deduplicated interface for standardized reference tables (Countries, Timezones, etc.). It supports global defaults, account-level overrides, and object-level overrides, with optional site-specific whitelisting.
+
+### How the hierarchy works
+
+Each lookup table (`lu_v3_country`, `lu_v3_time_zone`, etc.) can hold multiple rows for the same logical item at different scopes:
+
+| Scope | `account_id` | `for_type` / `for_id` | Wins over |
+|---|---|---|---|
+| Global default | `NULL` | `NULL` / `NULL` | nothing |
+| Account override | set | `NULL` / `NULL` | Global default |
+| Object override | set | set | Account override + Global default |
+
+The API uses `ROW_NUMBER() PARTITION BY group` to collapse all rows for the same item down to the single highest-priority winner before returning results. **`group` is the identity key** — it is what makes two rows "the same item competing for priority."
+
+> [!IMPORTANT]
+> **The `group` field is not a display label.** It is the deduplication key. Each lookup type uses a different natural key for `group`:
+>
+> | Lookup type | `group` value | Example |
+> |---|---|---|
+> | `country` | ISO alpha-2 code | `"US"`, `"CA"`, `"GB"` |
+> | `country_subdivision` | subdivision code | `"US-NY"`, `"CA-ON"` |
+> | `time_zone` | IANA timezone name | `"America/New_York"`, `"US/Eastern"` |
+>
+> For `time_zone`, `group` and `name` must always be identical — there is no concept of "override all US timezones as a group." Each timezone is its own identity.
 
 ### A. List Lookups
-Retrieve a ranked and filtered list of lookup items.
+
+Retrieve the deduplicated, ranked list for a lookup type.
+
 *   **Endpoint:** `GET /v3/lookup/{lu_type}/list`
 *   **Available Types:** `country`, `country_subdivision`, `time_zone`
 *   **Parameters:**
-    *   `site_id` (Optional): Random ID of the site to apply a **Whitelist Policy**.
-    *   `only_priority` (Optional): Set to `true` to return only high-priority items (e.g., common time zones).
-    *   `for_type` / `for_id` (Optional): Context for object-specific overrides.
-    *   `include_disabled` (Optional): Set to `true` to see shadowed/disabled records.
+    *   `site_id` (Optional): Random ID of the site — applies a **Whitelist Policy** (see §C).
+    *   `only_priority` (Optional): `true` returns only `priority=1` items (e.g., common time zones).
+    *   `for_type` / `for_id` (Optional): Object context — activates object-level override matching.
+    *   `include_disabled` (Optional): `true` includes shadowed/disabled records (useful for admin views).
+
+**Frontend keying:** Always key Svelte `{#each}` blocks on `group`, not `id` or `name`. `group` is guaranteed unique in the response. Keying on `id` will break if an account override wins (different `id`, same logical item).
 
 ### B. Resolve Identity
-Resolves a string (code, group, or name) to a single record.
+
+Resolves a string to a single lookup record.
+
 *   **Endpoint:** `GET /v3/lookup/{lu_type}/resolve?q=VALUE`
-*   **Usage:** Use this when you have an external code (e.g., ISO "US") and need the full Aether record.
+*   **Usage:** Use when you have an external code (e.g., ISO `"US"`) and need the full Aether record. Scans `name`, `group`, and other identity fields.
 
 ### C. Site Whitelist Policy
-To limit lookups for a specific site, add a `lookup_policy` to the `site.cfg_json` field.
-**Schema:**
+
+To restrict which lookup items appear for a specific site, add a `lookup_policy` to `site.cfg_json`:
+
 ```json
 {
   "lookup_policy": {
     "country": ["US", "CA", "GB"],
-    "time_zone": ["America/New_York"]
+    "time_zone": ["America/New_York", "US/Eastern"]
   }
 }
 ```
-*Note: Whitelist values must match the `group` field in the database.*
+
+> **Whitelist values must match the `group` field** — i.e., the natural key for that type (ISO code for country, IANA name for time zone). Using a display name will silently return no results for that item.
+
+### D. Adding and managing client overrides
+
+When a client needs a customized label or wants to hide/reorder lookup items, create override records rather than modifying global defaults.
+
+**Rules:**
+1. **Never modify global default rows** (`account_id = NULL`). Those are shared across all accounts. Any change there affects every client.
+2. **Set `group` to the exact same value as the global default row** for the item you are overriding. If `group` doesn't match, the override creates a new item instead of replacing the existing one.
+3. **Set `account_id`** to the client's account ID. Leave `for_type` / `for_id` null unless the override is specific to a single object (e.g., one site).
+
+**Example — rename "US/Eastern" for one account:**
+
+```sql
+INSERT INTO lu_v3_time_zone
+  (account_id, name, name_override, `group`, enable, priority, sort)
+VALUES
+  (42, 'US/Eastern', 'Eastern Time (Client Label)', 'US/Eastern', 1, 1, 50);
+```
+
+The `name_override` field is the display label the frontend should prefer when set. `group = 'US/Eastern'` ensures this row competes with — and wins over — the global default in the `PARTITION BY group` deduplication.
+
+**To disable an item for one account** (hide it from their dropdowns):
+
+```sql
+INSERT INTO lu_v3_time_zone
+  (account_id, name, `group`, enable)
+VALUES
+  (42, 'US/Samoa', 'US/Samoa', 0);
+```
+
+Setting `enable = 0` on an account-scoped row shadows the global default for that account only.
+
+**To remove a client override** (revert to global default):
+
+Simply delete the row where `account_id = <client>` and `group = '<item>'`. The global default row is unaffected and immediately resumes winning.
+
+### E. Adding new global lookup items
+
+When seeding new lookup data (e.g., adding timezones in bulk):
+
+1. Set `group = name` for every row (for `time_zone`). This is a hard invariant — if `group` is set to a regional label like `"United States"` instead of the timezone name, the entire group collapses to a single winner and all but one entry disappear from the API response.
+2. Set `account_id = NULL` and `for_type = NULL` / `for_id = NULL` for global defaults.
+3. After seeding, verify with:
+   ```sql
+   -- Should return 0 rows; any result means multiple items will collapse into one
+   SELECT `group`, COUNT(*) AS cnt
+   FROM lu_v3_time_zone
+   WHERE account_id IS NULL
+   GROUP BY `group`
+   HAVING cnt > 1;
+   ```
 
 ---
 
@@ -154,11 +314,391 @@ Frontend guidance:
 - These endpoints run synchronously and can take time for large inputs; for heavy or batch workloads use a queued job pattern instead.
  - These endpoints may take time for large inputs. Prefer using `?background=true` to schedule work and receive a `202 Accepted` response for async processing. For heavy or batch workloads use a queued job pattern instead.
 
+---
 
-## 5. Troubleshooting 403 Forbidden
+## 8. Email Send Action
+
+Send a transactional email via the Aether API.
+
+- **Method:** `POST`
+- **Path:** `/v3/action/email/send`
+- **Auth:** `x-aether-api-key` + `x-account-id` (or `x-no-account-id` / `?jwt=`)
+
+**Request body:**
+```json
+{
+  "from_email": "noreply@example.com",
+  "from_name": "Example App",
+  "to_email": "user@example.com",
+  "to_name": "Alice Smith",
+  "subject": "Your login link",
+  "body_html": "<p>Click <a href=\"...\">here</a> to log in.</p>",
+  "body_text": "Visit ... to log in.",
+  "cc_email": null,
+  "bcc_email": null
+}
+```
+
+**Query params:**
+
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `test` | bool | `false` | Simulate send without delivering |
+
+**Response:** `data` contains `{ from_email, to_email, subject }` (first 40 chars of subject). `400` if delivery failed.
+
+> **Replaces:** `POST /util/email/send` (disabled as of May 2026).
+
+---
+
+## Axonius Zoom CSV Upload (Temporary — Apr 2026, EXPIRED)
+
+Purpose: Staff-only quick upload to upsert Event Person + Event Badge records from a Zoom Events registrant CSV.
+
+- **Endpoint:** `POST /event/{event_id}/badge/import/zoom_csv`
+- **Auth:** include `x-aether-api-key` (if required) and account context via `x-account-id: <ACCOUNT_ID>`. Admin bypass (`x-no-account-id: bypass`) or `?jwt=<token>` are accepted per site policy.
+- **Request:** `multipart/form-data` with single file field `file` (Zoom CSV). Query params:
+  - `begin_at` (int, default `0`)
+  - `end_at` (int, default `20000`)
+  - `return_detail` (bool, default `false`)
+  - Delimiter is auto-detected; Zoom CSV layout: row 1 = metadata, row 2 = blank, row 3 = headers (the backend skips the first two rows).
+
+Behavior / notes:
+- The handler forces `Registrant email` to be used as the `external_id`. `Unique identifier` is used as `external_registration_id` only when it is meaningful (placeholders like `N/A`, `NA`, `UNKNOWN` are ignored).
+- Per-ticket custom fields are parsed (Organization, Job title, Phone, Address lines, City, State/Province, Postal/Zip, Country, etc.).
+- Marketing-consent values are mapped to `agree_to_tc` and `allow_tracking`.
+- TEMP AXONIUS MAPPING: the import temporarily defaults `event_badge_template_id` to `21` and `event_badge_template_id_random` to `RKYp2HcQm9o`. Ticket-name → `badge_type_code` mapping is applied for some labels (e.g., contains "sponsor" → `sponsor`; contains "attend"/"attendee" → `attendee`). This mapping is temporary (April 2026) — surface this to staff.
+- Rows missing `Registrant email` are skipped.
+- The server upserts via existing backend methods and creates/updates `event_person`, `event_person_profile`, and `event_badge` records as needed.
+
+Frontend guidance:
+- UI must be staff-only and should validate an `event_id` is selected.
+- For large files, use `begin_at`/`end_at` to process in chunks.
+- Prefer `return_detail=false` for large imports to reduce payload size.
+
+Common errors:
+- `403` — missing/invalid account context or API key.
+- `404` — event not found.
+- `500` — file save or processing error.
+
+Example curl (replace placeholders):
+```bash
+curl -v -X POST "https://api.example.com/event/<EVENT_ID>/badge/import/zoom_csv?begin_at=0&end_at=20000&return_detail=false" \
+  -H "x-aether-api-key: <API_KEY>" \
+  -H "x-account-id: <ACCOUNT_ID>" \
+  -F "file=@/path/to/zoom_export.csv"
+```
+
+Sample success (summary mode, `return_detail=false`):
+```json
+{
+  "data": [
+    {
+      "event_id": "xK9mP3qRtL2",
+      "event_id_random": "xK9mP3qRtL2",
+      "external_id": "alice@example.com",
+      "given_name": "Alice",
+      "family_name": "Smith",
+      "email": "alice@example.com"
+    }
+  ],
+  "meta": {
+    "status_code": 200,
+    "status_name": "OK",
+    "success": true,
+    "data_type": "list",
+    "data_list_count": 1
+  }
+}
+```
+
+Sample success (detailed, `return_detail=true`) — `data` contains full `event_person` objects with nested `event_badge` (may include temporary `event_badge_template_id`: `21` and `event_badge_template_id_random`: `RKYp2HcQm9o`).
+
+Paste this section into the guide as a temporary Axonius-specific note (April 2026). Consider linking staff to a sample Zoom CSV for QA.
+
+---
+
+## 7. User Actions (`/v3/action/user/`)
+
+Stateful user account operations that are not standard CRUD. All require `x-aether-api-key`.
+
+> [!IMPORTANT]
+> **Migration from legacy `/user/*` routes:** The table below maps each legacy endpoint to its V3 replacement. Run both in parallel during transition; remove legacy routes once traffic logs confirm they are quiet.
+>
+> | Legacy | V3 Replacement |
+> |---|---|
+> | `GET /user/authenticate` | `POST /v3/action/user/authenticate` |
+> | `POST /user/verify_password` | `POST /v3/action/user/verify_password` |
+> | `PATCH /user/{id}/change_password` | `POST /v3/action/user/{id}/change_password` |
+> | `GET /user/{id}/new_auth_key` | `GET /v3/action/user/{id}/new_auth_key` |
+> | `GET /user/{id}/email_auth_key_url` | `GET /v3/action/user/{id}/email_auth_key_url` |
+> | `GET /user/lookup` | `POST /v3/crud/user/search` |
+> | `GET /user/lookup_email` | `POST /v3/crud/user/search` |
+> | `GET /user/lookup_username` | `POST /v3/crud/user/search` |
+
+### A. Authenticate
+
+Authenticate a user by **username + password** or **user_id + auth_key**.
+
+- **Method:** `POST`
+- **Path:** `/v3/action/user/authenticate`
+- **Auth:** `x-aether-api-key` + `x-account-id` (scopes username lookups to the correct account)
+- **Security improvement:** Credentials are in the **POST body**, not query params — safe from URL logging.
+
+**Request body:**
+```json
+{ "username": "scott", "password": "MyPassword123!" }
+```
+or:
+```json
+{ "user_id": "<user_id_random>", "auth_key": "<one_time_key>", "valid_email": true }
+```
+
+- `valid_email` (optional `bool`): if `true`, marks `email_verified = true` on success.
+- `inc_user_role_list` (optional query param, default `false`): include role list in the returned user object.
+
+**Response on success:** Full user object (same shape as `GET /v3/crud/user/{id}`).
+
+**Errors:** `400` missing credentials, `403` wrong password / account disabled / account not yet enabled / account expired, `404` user not found.
+
+> **Auth key flow:** Auth keys are one-time-use — the key is cleared from the DB immediately on successful authentication. Request a new one via `GET /v3/action/user/{id}/new_auth_key`.
+
+---
+
+### B. Verify Password
+
+Check a user's current password without changing it.
+
+- **Method:** `POST`
+- **Path:** `/v3/action/user/verify_password`
+- **Auth:** `x-aether-api-key` + `x-account-id`
+
+**Request body:**
+```json
+{ "user_id": "<user_id_random>", "current_password": "MyPassword123!" }
+```
+or use `"username"` instead of `"user_id"` to look up by username within the account.
+
+**Response:** `data: true` on match. `400` if the user has no password set, `403` on mismatch, `404` if user not found.
+
+---
+
+### C. Change Password
+
+Change a user's password. Optionally verify the current password first.
+
+- **Method:** `POST`
+- **Path:** `/v3/action/user/{user_id}/change_password`
+- **Auth:** `x-aether-api-key` + `x-account-id`
+
+**Request body:**
+```json
+{ "new_password": "NewPassword456!", "current_password": "MyPassword123!" }
+```
+
+- `new_password` is required (minimum 10 characters).
+- `current_password` is optional. If provided, it is verified before the change is applied. Omit it for admin-driven resets.
+
+**Response:** `data: true` on success. `403` if `current_password` provided but wrong.
+
+---
+
+### D. Generate New Auth Key
+
+Generate a fresh one-time-use auth key for the user and write it to the DB.
+
+- **Method:** `GET`
+- **Path:** `/v3/action/user/{user_id}/new_auth_key`
+- **Auth:** `x-aether-api-key` + `x-account-id`
+
+**Response:**
+```json
+{ "data": { "auth_key": "<new_key>" } }
+```
+
+The returned key can then be passed to `/authenticate` (as `auth_key`) or embedded in a login URL. The user record must have `allow_auth_key = true` for key-based authentication to work.
+
+---
+
+### E. Email Auth Key URL
+
+Generate a new auth key and email a one-time login link to the user's email address.
+
+- **Method:** `GET`
+- **Path:** `/v3/action/user/{user_id}/email_auth_key_url`
+- **Auth:** `x-aether-api-key` + `x-account-id`
+
+**Query Parameters:**
+
+| Parameter | Type | Default | Description |
+|---|---|---|---|
+| `root_url` | `string` | *(required)* | Base URL the login link is built from. Must be provided — if omitted the link in the email will be malformed (`None?...`). |
+| `key_param_name` | `string` | `auth_key` | Query param name used for the auth key in the generated link. |
+
+> [!IMPORTANT]
+> `root_url` is **required in practice**. The FastAPI query param accepts `null` but the email builder does not guard against it — omitting it produces a broken link in the email.
+
+**Magic link URL format (default `key_param_name`):**
+```
+{root_url}?user_id={user_id_random}&auth_key={auth_key}&valid_email=True
+```
+The frontend at `root_url` should read these query params and call `POST /v3/action/user/authenticate` with `{ "user_id": "...", "auth_key": "..." }`. Note that `valid_email=True` is **always** injected — authenticating via a magic link automatically marks the user's email as verified.
+
+**Response:** `data: true` on success (email sent). `404` if user not found. `500` if delivery failed — common causes: account email not configured, user `enable = false`, or `allow_auth_key = false`.
+
+---
+
+### F. User Lookups via V3 CRUD Search
+
+The three legacy lookup routes (`lookup`, `lookup_email`, `lookup_username`) are replaced by standard V3 CRUD search:
+
+```typescript
+// Look up by user_id (Vision ID)
+POST /v3/crud/user/search
+{ "and": [{ "field": "id_random", "op": "eq", "value": "<user_id>" }] }
+
+// Look up by email
+POST /v3/crud/user/search
+{ "and": [{ "field": "email", "op": "eq", "value": "user@example.com" }] }
+
+// Look up by username
+POST /v3/crud/user/search
+{ "and": [{ "field": "username", "op": "eq", "value": "scott" }] }
+```
+
+Results are automatically scoped to the `x-account-id` provided in the request.
+
+---
+
+## 10. Event Exhibit Tracking Export (Leads Export)
+
+Allows an exhibitor to download all lead-capture records for their exhibit as a CSV or XLSX file.
+
+- **Method:** `GET`
+- **Path:** `/v3/action/event_exhibit/{exhibit_id}/tracking_export`
+- **Auth:** Standard V3 headers (`x-aether-api-key` + `x-account-id` or `?jwt=`)
+
+### Query Parameters
+
+| Parameter | Type | Default | Description |
+| :--- | :--- | :--- | :--- |
+| `file_type` | `CSV` \| `XLSX` | `CSV` | Output format. |
+| `return_file` | bool | `true` | `true` → file download response. `false` → JSON body with row data. |
+
+### Response
+
+- `Content-Type: text/csv` (CSV) or `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` (XLSX)
+- `Content-Disposition: attachment; filename="leads_export_<timestamp>.csv"`
+- If there are no tracking records, a valid file with headers only is returned (not a 404).
+
+### Columns Returned
+
+Fixed columns (always present), followed by any custom question columns flattened from `responses_json`:
+
+`event_exhibit_tracking_id`, `created_on`, `updated_on`, `event_exhibit_name`, `event_badge_full_name`, `event_badge_email`, `event_badge_professional_title`, `event_badge_affiliations`, `event_badge_location`, `event_badge_country`, `external_person_id`, `exhibitor_notes`, `priority`, `enable`, `hide`, `[custom question codes…]`
+
+> **Note:** `exhibitor_notes` has HTML tags stripped automatically for clean CSV output.
+
+### Permission Requirement — `leads_api_access`
+
+> [!IMPORTANT]
+> This endpoint enforces a **per-exhibit permission flag**. The `event_exhibit` record **must** have `leads_api_access = true` set in the database, OR the caller must have manager-level account access (JWT with `manager: true`).
+>
+> If `leads_api_access` is `false` or `null` on the exhibit, the API returns:
+> ```json
+> { "detail": "Access denied: leads API access is not enabled for this exhibit." }
+> ```
+> **Fix:** Enable the flag on the exhibit record via `PATCH /v3/crud/event_exhibit/{id}` with `{ "leads_api_access": true }`, or set it directly in the database/admin panel.
+
+#### Dual purpose of `leads_api_access`
+
+This flag serves two related but distinct roles:
+
+1. **3rd-party API access (original intent):** Controls whether external systems (exhibitor apps, badge-scanning devices, etc.) are permitted to push or pull lead data for this exhibit via the API.
+2. **UI export gate (new):** The frontend should read `leads_api_access` from the exhibit record and use it to show or hide the export/download button. Only render the button when the flag is `true` — this prevents users from triggering a request that will always 403.
+
+The recommended pattern is to fetch the exhibit record first and gate the UI on this field before the user ever sees the export option. The API enforces the same check server-side as a safety net.
+
+### Example Request
+
+```ts
+const resp = await fetch(
+  `https://dev-api.oneskyit.com/v3/action/event_exhibit/${exhibitId}/tracking_export?file_type=CSV&return_file=true`,
+  {
+    headers: {
+      'x-aether-api-key': API_KEY,
+      'x-account-id': accountId,
+    },
+  }
+);
+// resp is a file blob — use URL.createObjectURL() or trigger a download
+const blob = await resp.blob();
+const url = URL.createObjectURL(blob);
+```
+
+---
+
+## 12. IDAA: Server-Side Novi Member Verification
+
+Verifies a Novi AMS member UUID by proxying the Novi API call through the Aether backend. This eliminates false "Access Denied" failures for members on hotel/conference WiFi, VPNs, and Cloudflare-filtered networks — the Novi call originates from the server's IP, not the member's browser IP.
+
+- **Method:** `GET`
+- **Path:** `/v3/action/idaa/novi_member/{uuid}`
+- **Auth:** Standard V3 (`x-aether-api-key` + `x-account-id` or `?jwt=`)
+
+### Request
+
+| Parameter | Location | Required | Description |
+|---|---|---|---|
+| `uuid` | Path | Yes | Novi member UUID (from Novi AMS) |
+
+### Response on success (`200 OK`)
+
+```json
+{
+  "data": {
+    "verified": true,
+    "full_name": "Alice S.",
+    "email": "alice+member@idaa.org"
+  }
+}
+```
+
+- `full_name`: `"{FirstName} {LastName[0]}."` format. Falls back to the Novi `Name` field if first/last are absent.
+- `email`: Novi `Email` field with space → `+` normalization applied (Novi quirk — `alice member@idaa.org` → `alice+member@idaa.org`).
+
+### Error responses
+
+| Status | Meaning | Frontend action |
+|---|---|---|
+| `404` | UUID not found in Novi, or Novi returned 200 with no identity data (empty-member anti-pattern — member may have just joined) | Treat as denied / not a member |
+| `429` | Novi rate limit hit | Surface as `'rate_limited'`; advise retry |
+| `503` | Novi unreachable or Novi 5xx error | Surface as `'api_error'`; advise retry |
+
+### Migration from direct Novi call
+
+The frontend's `+layout.svelte:verify_novi_uuid()` currently calls Novi directly from the browser. Replace that `fetch()` with this endpoint. Response code mapping:
+
+| Direct Novi result | This endpoint returns | Frontend state |
+|---|---|---|
+| `200` with identity data | `200` | `verified` |
+| `200` with no identity data | `404` | `denied` |
+| `404` | `404` | `denied` |
+| `429` | `429` | `'rate_limited'` |
+| Network error / Novi 5xx | `503` | `'api_error'` |
+
+### Caching
+
+Verified results are cached in Redis (`idaa:novi_member:{uuid}`, 4-hour TTL). `404` results are **never** cached so recently-joined members are not incorrectly denied on their next attempt.
+
+---
+
+## 11. Troubleshooting 403 Forbidden
 
 If you receive a 403 on a valid ID:
 1.  Verify `x-aether-api-key` is correct.
 2.  Ensure you are sending `x-account-id` and NOT `x-aether-api-token`.
 3.  Verify the record actually belongs to the account ID you are sending.
 4.  Check if the object is marked `public_read: True` in the registry. (Posts and Archive Content allow guest access; Journals and Badges do not).
+5.  Confirm the frontend is not treating `params.key` as an implicit bypass and stripping `x-account-id`.
+6.  If list/search endpoints work but `GET /v3/crud/{obj_type}/{id}` still returns 403, this is likely endpoint-level policy (e.g., requires stronger auth like JWT) rather than a transport/header bug.

documentation/GUIDE__AE_API_V3_for_Frontend_websockets.md

@@ -27,10 +27,7 @@ wss://[api_domain]/v3/ws/group/{group_id}/client/{client_id}
 
 **`group_id`** — identifies the shared channel (e.g., an event ID, a room name, or a Vision ID random string). All clients using the same `group_id` receive group-targeted messages together.
 
-**`client_id`** — uniquely identifies this specific connection. Common choices:
-- `Date.now()` timestamp (e.g. `1773266158823`) — simple and collision-resistant for short-lived sessions
-- A UUID (`crypto.randomUUID()`)
-- A Vision ID random string if you need the client to be addressable by a known database identity
+**`client_id`** — uniquely identifies this specific connection. The backend accepts any unique string (UUID, timestamp, Vision ID — no format validation). The **recommended pattern** is a UUID v4 generated once and persisted in `localStorage` so the same identity is reused across page reloads and sessions on that browser.
 
 > Use `ws://` for local development and `wss://` in production (any HTTPS site). The Nginx config must include the Upgrade block — see Section 6.
 
@@ -50,8 +47,13 @@ wss://dev-api.oneskyit.com/v3/ws/group/{group_id}/client/{client_id}?api_key=<ke
 
 ### C. Connection Example (TypeScript)
 ```ts
+// client_id: generated once, persisted in localStorage for stable identity across reloads
+if (!localStorage.getItem('controller_client_id')) {
+    localStorage.setItem('controller_client_id', crypto.randomUUID());
+}
+const client_id = localStorage.getItem('controller_client_id')!; // UUID v4, e.g. "550e8400-e29b-41d4-a716-446655440000"
+
 const group_id = "event_abc123"; // Any unique string identifying the shared channel
-const client_id = String(Date.now()); // Any unique string identifying this client connection
 const api_key = import.meta.env.VITE_API_KEY;
 const jwt = getSessionToken(); // your JWT helper
 

documentation/GUIDE__DEPLOYMENT_MANUAL.md

@@ -1,3 +1,9 @@
+
+# DEPRECATED: Manual Server Deployment Guide (Non-Docker)
+
+> **Notice (March 2026):**
+> This manual deployment guide is deprecated. The standard and supported method for deploying the Aether API is now via Docker Compose, as described in the main README and the `aether_container_env` documentation. Use this guide only for legacy or advanced manual setups.
+
 # Manual Server Deployment Guide (Non-Docker)
 
 This guide describes the manual process for deploying the Aether API on a Linux server using Nginx, Gunicorn, and Systemd.

documentation/PROJECT__AE_Lookups_fixes_and_docs_update.md

@@ -0,0 +1,144 @@
+# Project: V3 Lookup Bug Fix — Timezone Group Data + PARTITION BY Revert
+
+> **Status:** 🔧 Action Required
+> **Date:** 2026-03-23
+> **Related doc:** `PROJECT__V3_UNIFORM_LOOKUP_SYSTEM.md`
+> **Reported by:** Frontend Agent (Scott Idem / One Sky IT)
+
+---
+
+## 1. Summary
+
+Two bugs were discovered in the V3 Uniform Lookup System during IDAA Recovery Meetings
+timezone dropdown testing. They stem from a single root cause: the `lu_v3_time_zone`
+table was seeded with regional `group` values (`"United States"`, `"Europe"`) instead of
+individual timezone names — contrary to the design specified in Phase 2 of the lookup
+architecture doc, which explicitly states `lu_v3_time_zone (Group: name)`.
+
+An attempted fix changed `PARTITION BY group` to `PARTITION BY name` in
+`get_lookup_list_v3()`. This unintentionally broke country deduplication, which depends
+on `PARTITION BY group` being correct (country group = `alpha_2_code`, e.g. `"US"`).
+
+---
+
+## 2. Root Cause
+
+### 2.1 Timezone `group` values were set to regional names instead of timezone names
+
+The `lu_v3_time_zone` table has two groups where multiple records share a single group value:
+
+| `group` value  | Count | Example records |
+|----------------|-------|-----------------|
+| `United States` | 13   | US/Alaska, US/Arizona, US/Central, US/East-Indiana, US/Eastern, US/Hawaii, US/Indiana-Starke, US/Michigan, US/Mountain, US/Pacific, US/Pacific-New, US/Samoa, US/Aleutian |
+| `Europe`        | 63   | Europe/London, Europe/Paris, Europe/Prague, Europe/Rome, ... (all Europe/* zones) |
+
+All other timezone records already have `group = name` (e.g., `Canada/Eastern` has
+`group = "Canada/Eastern"`). The US and Europe records were loaded incorrectly.
+
+**Effect:** `PARTITION BY group` collapsed all 13 US/* records into a single winner and
+all 63 Europe/* records into a single winner. Only ~7 distinct US timezones and 1 Europe
+timezone appeared in the dropdown instead of all 76.
+
+### 2.2 Attempted fix broke country lookup deduplication
+
+Changing `PARTITION BY group` → `PARTITION BY name` in `get_lookup_list_v3()` fixed the
+timezone collapse but broke `lu_v3_country`.
+
+`lu_v3_country` has (at minimum) two records for `alpha_2_code = "US"`:
+- `id=240`: global default (`account_id=NULL`), `group="US"`
+- `id=251`: account-specific (`account_id=1`), `group="US"`
+
+With `PARTITION BY group`, both records share `group="US"` and are correctly deduped —
+the account-specific record wins per the override hierarchy. With `PARTITION BY name`,
+if the two records have different `name` values they are treated as separate identities
+and both survive, resulting in duplicate `alpha_2_code="US"` entries in the API response.
+
+The frontend's `{#each lu_country_list as country (country.alpha_2_code)}` then throws:
+> `Svelte error: each_key_duplicate — Keyed each block has duplicate key 'US'`
+
+The same risk applies to `lu_v3_country_subdivision`.
+
+---
+
+## 3. Correct Fix (Two Steps)
+
+### Step 1 — Revert `app/methods/lookup_methods.py`
+
+Change `PARTITION BY name` back to `PARTITION BY group`:
+
+```python
+# lookup_methods.py — get_lookup_list_v3()
+ROW_NUMBER() OVER (
+    PARTITION BY `group`       # <-- revert to this
+    ORDER BY
+        (for_type = :for_type AND for_id = :for_id) DESC,
+        (account_id = :account_id) DESC,
+        created_on DESC
+) as rank_priority
+```
+
+This restores correct behavior for all three active V3 lookup types
+(`country`, `country_subdivision`, `time_zone`).
+
+### Step 2 — Fix the `lu_v3_time_zone` data
+
+Set `group = name` for all records where the group is a regional label rather than the
+timezone's own name. Run once against the database:
+
+```sql
+UPDATE lu_v3_time_zone
+SET `group` = `name`
+WHERE `group` IN ('United States', 'Europe');
+```
+
+**Verification:**
+```sql
+-- Should return 0 rows after the fix
+SELECT `group`, COUNT(*) as cnt
+FROM lu_v3_time_zone
+GROUP BY `group`
+HAVING cnt > 1;
+```
+
+---
+
+## 4. Why PARTITION BY `group` Is Correct
+
+As documented in `PROJECT__V3_UNIFORM_LOOKUP_SYSTEM.md` (Section 2.1):
+
+> `group`: The primary business key/cluster key. *Note: Must be populated for hierarchy to work.*
+
+The `group` field IS the deduplication identity. Each lookup type uses a different natural
+key for `group`:
+
+| Lookup type | `group` field | Example |
+|---|---|---|
+| `country` | `alpha_2_code` | `"US"`, `"CA"`, `"GB"` |
+| `country_subdivision` | `code` | `"US-NY"`, `"CA-ON"` |
+| `time_zone` | `name` (= the IANA timezone identifier) | `"US/Eastern"`, `"Europe/London"` |
+
+For `time_zone`, `group` and `name` are intended to be the same value — each timezone
+is its own identity. There is no meaningful concept of "override all US timezones as a
+group." Each one is individually addressable.
+
+---
+
+## 5. Regression Tests to Add / Update
+
+- `test_timezone_us_dedup()` — assert all 13 US/* priority zones are present individually
+- `test_timezone_europe_dedup()` — assert all Europe/* priority zones present individually
+- `test_country_us_dedup()` — assert only one `alpha_2_code="US"` record returned;
+  account-specific override wins over global default
+- General: `GET /v3/lookup/time_zone/list?only_priority=true` should return exactly 72
+  records (the current count of priority=1 enabled timezones)
+
+---
+
+## 6. What Was NOT Changed (and Should Not Be)
+
+- The endpoint signature for `GET /v3/lookup/{lu_type}/list` — it does not and should
+  not expose `limit`, `offset`, or `order_by_li` query params. The frontend sends these
+  but they are correctly ignored. The sort order is hardcoded and correct:
+  `ORDER BY COALESCE(priority, 0) DESC, COALESCE(sort, 0) DESC, name ASC`
+- Country and country_subdivision data — no changes needed to those tables
+- Frontend code — no backend-side changes are needed on the frontend for this fix

documentation/PROJECT__AE_Site_Passcode_Security.md

@@ -0,0 +1,386 @@
+# PROJECT: Site Passcode Security — API-Verified Auth
+
+**Last updated:** 2026-04-10
+**Status:** Backend work in progress — frontend pending backend completion
+**Priority:** High — passcodes for trusted/administrator access currently in localStorage plaintext
+
+---
+
+## Problem Statement
+
+When a user loads the Aether frontend, the site bootstrap response includes `access_code_kv_json` — a JSON object containing all passcodes for all access levels (administrator, trusted, public, authenticated). The frontend stores this verbatim in `$ae_loc.site_access_code_kv`, which is persisted in localStorage.
+
+**Result:** Anyone with DevTools → Application → Local Storage can see every passcode for every access level on any Aether site. For public/authenticated this is low risk, but for trusted and administrator this is a real exposure — these passcodes can grant control over event data, badge printing, edit mode, etc.
+
+The passcode check (`handle_check_access_type_passcode` in `e_app_access_type.svelte`) is entirely local — it reads the cached values and compares directly. No API call is made. The backend already has a `/authenticate_passcode` endpoint that verifies server-side, but it needs the fixes described below before the frontend can rely on it.
+
+### Source of Truth
+
+`site.access_code_kv_json` is the single source of truth for all passcodes. The `v_site_domain` DB view joins this field from the site table — there is no separate copy. Both the bootstrap response and `/authenticate_passcode` read from the same data.
+
+---
+
+## Threat Model
+
+| Threat | Current | After Fix |
+|---|---|---|
+| Attacker inspects localStorage | Sees all passcodes in plaintext | Sees a JWT (opaque, no passcode) |
+| Attacker uses stolen trusted passcode | Trivial if they have localStorage access | Still possible if they enter the passcode — unavoidable |
+| Attacker replays an old passcode after it changes | Works forever (cached value never refreshes) | Fails — API verifies against current DB value |
+| Attacker tampers with `access_type` in localStorage | Grants apparent permission but API calls still fail | Same — `access_type` is still persisted separately |
+| Passcode reuse across sessions | Works indefinitely | JWT TTL enforces session expiry per role |
+| Offline / API-unavailable entry | Works (local cache) | **Blocked** — requires API to verify |
+
+### The fundamental constraint
+
+Passcode-based access is inherently weaker than username/password login with a hashed credential. The system's security model layers passcode access below user login, and API calls themselves are still gated by `x-aether-api-key` + `x-account-id`. The passcode primarily controls **what the frontend shows** and some API-level permission gates for trusted routes.
+
+---
+
+## Proposed Solution: API-Verified Passcode + JWT Session
+
+### Core idea
+
+1. **Never send passcodes to the client.** The frontend stops reading/storing `access_code_kv_json` from the bootstrap response.
+2. **Passcode entry triggers an API call** to `/authenticate_passcode`. API verifies server-side against the DB.
+3. **On success, the API returns a JWT** — the JWT contains the role, account context, and expiry.
+4. **Store the JWT in `$ae_loc.jwt`** (already a field, already wired into `$ae_api`).
+5. **On page reload**, check the JWT's `eat` (expires-at) claim locally (base64 decode, no signature verification needed client-side). If expired, drop to anonymous. If valid, `access_type` is already persisted in `$ae_loc`.
+
+### Session restore on reload
+
+- `access_type` still persists in localStorage (no change here)
+- The JWT is the **proof** that the access was legitimately granted and is still valid
+- On page load: decode JWT payload (base64 the middle segment), check `eat` vs `Date.now()/1000`
+- If JWT expired → reset `access_type` to anonymous, clear JWT
+- If JWT valid → no action needed, `access_type` is already correct
+
+This gives session expiry without a network call on every page load.
+
+---
+
+## TTL Per Role — Decided
+
+| Access Level | JWT TTL | Notes |
+|---|---|---|
+| `super` | 8 hours | Highest privilege |
+| `manager` | 24 hours | |
+| `administrator` | 48 hours | |
+| `trusted` | 48 hours | Onsite staff — covers multi-day events |
+| `public` | 24 hours | |
+| `authenticated` | 12 hours | |
+| `anonymous` | N/A | No passcode |
+
+---
+
+## Caching Decision
+
+**No passcode caching.** Every passcode entry makes one API call. The JWT handles session persistence — no passcode ever touches localStorage. Performance impact is only at the moment of entry (~50–150ms), which is acceptable for a once-per-session action.
+
+---
+
+## Backend Changes Required
+
+**Note:** The backend fixes described below have been implemented and tested in the `aether_api_fastapi` repository (the `/authenticate_passcode` endpoint now uses explicit role priority, returns a full passcode JWT with `auth_type: 'passcode'`, applies per-role TTLs, and validates passcode length). Frontend changes can proceed once the backend deployment with these fixes is available.
+
+**Phase 2 status:** Not started — removing `access_code_kv_json` from the public site model remains pending.
+
+**File:** `aether_api_fastapi/app/routers/api.py`
+
+The `/authenticate_passcode` endpoint exists and is structurally correct but has four issues that must be fixed before the frontend migrates to using it.
+
+### Fix 1: Passcode matching must use explicit priority order
+
+**Current (wrong):**
+```python
+for role, code in access_codes.items():  # dict insertion order — not guaranteed
+    if str(code) == str(passcode):
+        matched_role = role
+        break
+```
+
+**Required:**
+```python
+ROLE_PRIORITY = ['super', 'manager', 'administrator', 'trusted', 'public', 'authenticated']
+
+matched_role = None
+for role in ROLE_PRIORITY:
+    code = access_codes.get(role)
+    if code and str(code) == str(passcode):
+        matched_role = role
+        break
+```
+
+This ensures that if a config mistake causes two roles to share a passcode, the higher-privilege role always wins. It also makes the intent explicit and independent of JSON storage order.
+
+### Fix 2: JWT payload must include all six role flags
+
+**Current (incomplete):**
+```python
+payload = {
+    'account_id': account_id_random,
+    'administrator': (matched_role == 'administrator'),
+    'manager':       (matched_role == 'manager'),
+    'super':         (matched_role == 'super'),
+    # trusted / public / authenticated missing
+    ...
+}
+```
+
+**Required:**
+```python
+payload = {
+    'account_id':    account_id_random,
+    'super':         (matched_role == 'super'),
+    'manager':       (matched_role == 'manager'),
+    'administrator': (matched_role == 'administrator'),
+    'trusted':       (matched_role == 'trusted'),
+    'public':        (matched_role == 'public'),
+    'authenticated': (matched_role == 'authenticated'),
+    'json_str': json.dumps({
+        'auth_type': 'passcode',   # distinguishes from user login JWTs
+        'site_id':   site_id,
+        'role':      matched_role  # canonical role string — frontend uses this
+    })
+}
+```
+
+The `auth_type: 'passcode'` marker is critical — it allows the frontend and any future backend consumers to distinguish a passcode JWT from a user login JWT.
+
+### Fix 3: Per-role TTL
+
+**Current:**
+```python
+token = sign_jwt(
+    secret_key=settings.JWT_KEY,
+    ttl=3600 * 24,  # hardcoded 24h for all roles
+    **payload
+)
+```
+
+**Required:**
+```python
+ROLE_TTL = {
+    'super':         8  * 3600,   #  8 hours
+    'manager':       24 * 3600,   # 24 hours
+    'administrator': 48 * 3600,   # 48 hours
+    'trusted':       48 * 3600,   # 48 hours
+    'public':        24 * 3600,   # 24 hours
+    'authenticated': 12 * 3600,   # 12 hours
+}
+
+token = sign_jwt(
+    secret_key=settings.JWT_KEY,
+    ttl=ROLE_TTL[matched_role],
+    **payload
+)
+```
+
+### Fix 4: Add minimum length validation to `passcode` field
+
+**Current:**
+```python
+passcode: str = Field(..., description="The passcode to verify")
+```
+
+**Required:**
+```python
+passcode: str = Field(..., min_length=5, description="The passcode to verify")
+```
+
+This matches the frontend's 5-character trigger and prevents empty/trivial submissions.
+
+### Complete corrected endpoint (for reference)
+
+```python
+ROLE_PRIORITY = ['super', 'manager', 'administrator', 'trusted', 'public', 'authenticated']
+
+ROLE_TTL = {
+    'super':         8  * 3600,
+    'manager':       24 * 3600,
+    'administrator': 48 * 3600,
+    'trusted':       48 * 3600,
+    'public':        24 * 3600,
+    'authenticated': 12 * 3600,
+}
+
+class PasscodeAuthRequest(BaseModel):
+    """Request model for site-based passcode authentication."""
+    site_id:  str = Field(..., description="Random string ID of the site")
+    passcode: str = Field(..., min_length=5, description="The passcode to verify")
+
+@router.post('/authenticate_passcode', response_model=Resp_Body_Base)
+async def authenticate_passcode(
+        auth_req: PasscodeAuthRequest,
+        response: Response = Response,
+        ):
+    """
+    Passcode-to-JWT Endpoint.
+    Verifies a passcode against site.access_code_kv_json (single source of truth —
+    v_site_domain joins from the same site record).
+    Returns a signed JWT with the site's account context, full role flags, and
+    a per-role TTL. The jwt.json_str.auth_type='passcode' field distinguishes
+    this token from a user login JWT.
+    """
+    site_id  = auth_req.site_id
+    passcode = auth_req.passcode
+
+    # 1. Look up the site record
+    search_data = {'id_random': site_id}
+    if record := sql_select(table_name='site', data=search_data):
+        # 2. Parse access codes
+        access_codes_raw = record.get('access_code_kv_json')
+        access_codes = {}
+        if access_codes_raw:
+            try:
+                access_codes = json.loads(access_codes_raw) if isinstance(access_codes_raw, str) else access_codes_raw
+            except Exception as e:
+                log.error(f"Failed to parse access_code_kv_json for site {site_id}: {e}")
+
+        # 3. Verify passcode in explicit priority order (highest privilege wins)
+        matched_role = None
+        for role in ROLE_PRIORITY:
+            code = access_codes.get(role)
+            if code and str(code) == str(passcode):
+                matched_role = role
+                break
+
+        if matched_role:
+            log.info(f"Auth Success: Verified '{matched_role}' passcode for site {site_id}")
+
+            # 4. Resolve account context
+            account_id_random = record.get('account_id_random')
+            if not account_id_random:
+                if account_id_int := record.get('account_id'):
+                    account_id_random = get_id_random(record_id=account_id_int, table_name='account')
+
+            # 5. Mint JWT with complete role flags and per-role TTL
+            payload = {
+                'account_id':    account_id_random,
+                'super':         (matched_role == 'super'),
+                'manager':       (matched_role == 'manager'),
+                'administrator': (matched_role == 'administrator'),
+                'trusted':       (matched_role == 'trusted'),
+                'public':        (matched_role == 'public'),
+                'authenticated': (matched_role == 'authenticated'),
+                'json_str': json.dumps({
+                    'auth_type': 'passcode',
+                    'site_id':   site_id,
+                    'role':      matched_role
+                })
+            }
+
+            token = sign_jwt(
+                secret_key=settings.JWT_KEY,
+                ttl=ROLE_TTL[matched_role],
+                **payload
+            )
+
+            return mk_resp(
+                data={'jwt': token, 'account_id': account_id_random, 'role': matched_role},
+                response=response
+            )
+        else:
+            log.warning(f"Auth Failed: Invalid passcode for site {site_id}")
+            return mk_resp(data=False, status_code=401, response=response, status_message="Invalid passcode.")
+    else:
+        log.warning(f"Auth Failed: Site {site_id} not found.")
+        return mk_resp(data=False, status_code=404, response=response, status_message="Site not found.")
+```
+
+### Backend Phase 2 (follow-up — not blocking frontend)
+
+**Remove `access_code_kv_json` from the `Site_Domain_Base` response model** (`site_domain_models.py`). This ensures passcodes are never sent to the client even if future code reads from the bootstrap. Requires confirming no other endpoint consumers rely on `access_code_kv_json` being in the base response before making this change.
+
+---
+
+## Frontend Changes Required
+
+**These depend on the backend fixes above being deployed first.**
+
+### 1a. `src/lib/app_components/e_app_access_type.svelte`
+
+Replace `handle_check_access_type_passcode` entirely. The new version:
+
+- Is `async`
+- Adds `auth_pending: boolean = $state(false)` and `auth_error: string | null = $state(null)`
+- Uses a direct `fetch` call (NOT `post_object` — avoids triggering the session-expired banner on a 401)
+- On success: sets `$ae_loc.access_type = data.role`, stores `$ae_loc.jwt = data.jwt`, triggers `process_permission_check` as before
+- On 401: shows inline error, clears `entered_passcode`, resets `checked_passcode = null` to allow retry
+- On network error: shows inline connection error
+- Clears `auth_error` when `entered_passcode` changes
+
+API call shape:
+```http
+POST /authenticate_passcode
+Content-Type: application/json
+x-aether-api-key: <from $ae_api.headers['x-aether-api-key']>
+Body: { site_id: $ae_loc.site_id, passcode: entered_passcode }
+```
+
+Add to template (near the passcode input):
+```svelte
+{#if auth_pending}
+    <Loader size="1em" class="animate-spin text-gray-400" />
+{/if}
+{#if auth_error}
+    <span class="text-error-500 text-xs">{auth_error}</span>
+{/if}
+```
+
+### 1b. `src/routes/+layout.ts`
+
+**Stop caching passcodes from bootstrap** — remove line ~394:
+```ts
+// ae_loc_init['site_access_code_kv'] = json_data.access_code_kv_json || {};
+```
+
+**Add passcode JWT expiry check** — after the block around line 84 where `ae_loc_json.jwt` is read, add:
+```ts
+// Enforce passcode JWT TTL on page load.
+// Decodes the JWT payload (base64, no secret needed) and resets access to anonymous if expired.
+// User login JWTs (auth_type !== 'passcode') are left untouched.
+if (ae_loc_json?.jwt) {
+    try {
+        const parts = ae_loc_json.jwt.split('.');
+        if (parts.length === 3) {
+            const jwt_payload = JSON.parse(atob(parts[1]));
+            const json_str = typeof jwt_payload.json_str === 'string'
+                ? JSON.parse(jwt_payload.json_str)
+                : jwt_payload.json_str;
+            if (json_str?.auth_type === 'passcode' && jwt_payload.eat < Date.now() / 1000) {
+                // Passcode JWT has expired — revoke access
+                ae_loc_json.jwt = null;
+                ae_loc_json.access_type = 'anonymous';
+            }
+        }
+    } catch {
+        // Malformed JWT — leave untouched, let existing handling deal with it
+    }
+}
+```
+
+### 1c. `src/lib/stores/ae_stores__auth_loc_defaults.ts` (cleanup)
+
+Remove `site_access_code_kv` from the `AuthLocState` interface and the `auth_loc_defaults` object. The field is unused after 1a. Confirm no other component reads from it first (current grep: only `e_app_access_type.svelte` uses it — confirmed).
+
+---
+
+## Migration Notes
+
+- Users with existing localStorage will still have `site_access_code_kv` cached — this is harmless after the frontend stops reading it. No forced cache clear needed.
+- Existing persisted `access_type` is unaffected — users keep their current session level until their JWT expires or they manually clear storage.
+- The `$ae_loc.jwt` field is already used by the user login flow. The `auth_type: 'passcode'` marker in `json_str` ensures the expiry logic only targets passcode sessions, not user login sessions.
+
+---
+
+## Files Affected
+
+| File | Repo | Change |
+| --- | --- | --- |
+| `app/routers/api.py` | `aether_api_fastapi` | **Backend — do first.** Priority ordering, full JWT payload, per-role TTL, min_length on passcode |
+| `app/models/site_domain_models.py` | `aether_api_fastapi` | Phase 2: remove `access_code_kv_json` from public model |
+| `src/lib/app_components/e_app_access_type.svelte` | `aether_app_sveltekit` | Replace local check with async API call; loading/error UI |
+| `src/routes/+layout.ts` | `aether_app_sveltekit` | Stop caching passcodes; add JWT expiry check |
+| `src/lib/stores/ae_stores__auth_loc_defaults.ts` | `aether_app_sveltekit` | Cleanup: remove `site_access_code_kv` |
+| `documentation/AE__Permissions_and_Security.md` | `aether_app_sveltekit` | Update passcode auth section to reflect new flow |

documentation/PROJECT__AE_hosted_files_uploads_util.md

@@ -0,0 +1,124 @@
+# PROJECT: AE Hosted Files — Upload Util & V3 Actions Migration
+
+**Status:** In Progress
+**Date:** 2026-03-25
+**Affected systems:** Frontend (aether_app_sveltekit), Backend (aether_api_fastapi)
+
+---
+
+## Background
+
+The legacy `hosted_file.router` (registered at prefix `/hosted_file`) was commented out
+in `app/routers/registry.py` as part of the V3 migration:
+
+```python
+# app.include_router(hosted_file.router, prefix='/hosted_file', tags=['Hosted File'])
+app.include_router(api_v3_actions_hosted_file.router, prefix='/v3/action/hosted_file', ...)
+```
+
+This broke several frontend features that were still calling the old endpoints.
+Three endpoints have been fixed on the frontend side (already committed and pushed).
+One endpoint still needs a backend fix.
+
+---
+
+## Endpoints: Status Summary
+
+### FIXED (frontend updated to call new V3 path)
+
+| Old endpoint | New endpoint | Frontend file |
+|---|---|---|
+| `POST /hosted_file/upload_files` | `POST /v3/action/hosted_file/upload` | `src/lib/ae_core/ae_comp__hosted_files_upload.svelte`, `src/routes/events/ae_comp__event_files_upload.svelte` |
+| `GET /hosted_file/{id}/clip_video` | `GET /v3/action/hosted_file/{id}/clip_video` | `src/lib/ae_core/ae_comp__hosted_files_clip_video.svelte` |
+
+### NEEDS BACKEND ACTION — Hash Lookup Endpoint
+
+**Missing endpoint:** `GET /hosted_file/hash/{hosted_file_hash}`
+
+This endpoint existed in the legacy `hosted_file.py` router (line 233) and has **not** been
+ported to `api_v3_actions_hosted_file.py`.
+
+**What it does:**
+1. Looks up a `hosted_file` record by its `hash_sha256` field
+2. Optionally checks that the physical file actually exists on disk (`check_for_local=true`)
+3. Returns the full hosted_file object with two extra flags:
+   - `hosted_file_found_check: true` — file record exists AND physical file confirmed on disk
+   - `hosted_file_size_check: <bytes>` — file size from disk
+
+**Legacy implementation (hosted_file.py:233):**
+```python
+@router.get('/hash/{hosted_file_hash}', response_model=Resp_Body_Base)
+async def check_hosted_file_obj_w_hash(
+        hosted_file_hash: str = Path(min_length=64, max_length=64),
+        check_for_local: Optional[bool] = True,
+        commons: Common_Route_Params = Depends(common_route_params),
+        ):
+    if hfid := lookup_file_hash(file_hash=hosted_file_hash):
+        obj = load_hosted_file_obj(hosted_file_id=hfid, model_as_dict=True)
+        if check_for_local and obj:
+            if check := check_for_hosted_file_hash_file(file_hash=hosted_file_hash, sub_dir=obj.get('subdirectory_path', '')):
+                obj['hosted_file_found_check'] = True
+                obj['hosted_file_size_check'] = check['file_size']
+        return mk_resp(data=obj, response=commons.response)
+    return mk_resp(data=False, status_code=404, response=commons.response)
+```
+
+**Where it's called on the frontend:**
+- `src/lib/ae_core/core__check_hosted_file_obj_w_hash.ts` — thin wrapper, calls `GET /hosted_file/hash/{hash}`
+- `src/lib/elements/element_input_file.svelte` — calls this before uploading (dedup check)
+- `src/lib/elements/element_input_files_tbl.svelte` — same (dedup check in the table file input)
+- Exported via `src/lib/ae_core/ae_core_functions.ts` as `core_func.check_hosted_file_obj_w_hash`
+
+**Current impact:** The 404 causes a null return. The frontend checks
+`result && result.hosted_file_found_check` — so if null, it silently skips the dedup check
+and proceeds to upload anyway. Uploads still work, but duplicate files may be created rather
+than reusing existing records.
+
+**Requested fix (backend):**
+Port this endpoint to `api_v3_actions_hosted_file.py` as:
+
+```
+GET /v3/action/hosted_file/hash/{hosted_file_hash}
+```
+
+Parameters and response shape should match the legacy implementation exactly.
+The `check_for_local` query param (default `True`) must be preserved — the frontend
+passes `check_for_local=true` and expects `hosted_file_found_check` in the response.
+
+**After backend deploys the new endpoint**, the frontend needs one line changed in
+`src/lib/ae_core/core__check_hosted_file_obj_w_hash.ts`:
+```ts
+// Before:
+const endpoint = `/hosted_file/hash/${hosted_file_hash}`;
+// After:
+const endpoint = `/v3/action/hosted_file/hash/${hosted_file_hash}`;
+```
+
+---
+
+## Other Legacy Endpoints — Audit Notes
+
+The following were also in `hosted_file.py` but appear to either have V3 equivalents already
+or are not currently called by the frontend. Backend should confirm:
+
+| Legacy endpoint | V3 equivalent | Notes |
+|---|---|---|
+| `GET /hosted_file/{id}/download` | `GET /v3/action/hosted_file/{id}/download` | Exists in V3 router |
+| `DELETE /hosted_file/{id}` | `DELETE /v3/action/hosted_file/{id}` | Exists in V3 router |
+| `GET /hosted_file/{id}/convert_file` | `GET /v3/action/hosted_file/{id}/convert_file` | Exists in V3 router |
+| `GET /hosted_file/{id}/stream` | Unknown | Not confirmed in V3 router — verify |
+| `GET /hosted_file/directory_check` | Unknown | Admin/dev utility — verify if still needed |
+| `GET /hosted_file/hash/{hash}/download` (via V3) | `GET /v3/action/hosted_file/hash/{sha256}/download` | Exists in V3 router (hash-based download) |
+| `GET /hosted_file/tmp/{subdir}/{filename}/download` | Unknown | Temp file download — verify if still needed |
+| `POST /hosted_file/create_video` | Unknown | Verify if still needed |
+
+---
+
+## Coordinator Notes
+
+- Frontend commits fixing upload and clip_video are on branch `ae_app_3x_llm`
+  (commits `a5a806e2` and `362136e6`)
+- Once the backend adds the hash lookup endpoint, the frontend one-line fix in
+  `core__check_hosted_file_obj_w_hash.ts` can be committed alongside it
+- The `check_for_local` flag is important — it verifies the physical file exists on disk,
+  not just the DB record. Don't drop it in the V3 port.

documentation/TODO__Agents.md

@@ -12,6 +12,16 @@
 - [x] **Config Refactor:** Switch `app/config.py` to `pydantic-settings` to use direct Env Vars (Stop mounting config files).
 - [x] **Locking:** Generate a `requirements.lock` for bit-identical builds.
 
+## 🔌 DB Connection Hardening (April 2026 Audit)
+> Identified during pre-show review. Issues 1 and 2 likely explain observed random connection lags.
+
+- [x] **[P1] Remove zombie `db_connection.py` import** — `app/routers/api.py` imports `db` from `app/db_connection.py`, creating a parasitic second SQLAlchemy engine at startup that is never updated by `reconnect_db()` after bootstrap. The imported `db` is only used in a commented-out line (`api.py:268`). Fix: remove the import; delete or archive `db_connection.py`.
+- [x] **[P1] Fix retry mechanism in `sql_update` / `run_sql_select`** — On `OperationalError`, both call `sql_connect()` → `reconnect_db()` which calls `engine.dispose()`, nuking the entire connection pool mid-flight. Under concurrent requests this kills other in-flight connections. Fix: remove the `sql_connect()` retry call; SQLAlchemy's `pool_pre_ping=True` already handles stale connections — just open a fresh `engine.connect()` for the retry without disposing the pool.
+- [x] **[P2] Add retry logic to `sql_insert` and `sql_select`** — Added `OperationalError` retry (single fresh connection attempt) to `sql_insert`, `sql_select`, and `sql_insert_or_update`. `IntegrityError` (duplicate key, FK violation) correctly bypasses retry and returns `None` — retrying the same data would fail again.
+- [x] **[P3] Guard `db = engine.connect()` in `lib_sql_core.py` with try/except** — Wrapped in try/except; sets `db = None` on failure so Docker startup race no longer crashes the worker.
+    - [ ] **[P3 full]** Migrate `lib_schema_v3.py:39` and `lib_api_crud_v3.py:166` off the global `db` to `engine.connect()` context managers, then remove the global `db` entirely.
+- [x] **[P4] Expose `pool_size` / `max_overflow` as env vars** — `create_ae_engine()` calls `settings.DB.get('pool_size', 10)` but `settings.DB` property doesn't include those keys, so they're always hardcoded 10/20. Add `AE_DB_POOL_SIZE` / `AE_DB_POOL_MAX_OVERFLOW` to `config.py`.
+
 ## 📋 Feature Tasks
 - [x] **Core Isolation:** Harden `apply_forced_account_filter` to Fail-Closed.
 - [x] **IDAA Baseline:** Remove `public_read` from Event, CMS, and Archive objects.
@@ -24,37 +34,72 @@
     - [x] Whitelist `account_id` in all Event search definitions.
     - [x] Audit Relational "Low-Priority" Models (Address, Contact, DataStore).
     - [x] **V3 Uniform Lookup System:** Phase 1 & 2 Complete.
+    - [x] **Restore alt-view convenience fields lost in v1→v3 migration (May 2026):** `site_domain` (`account_name`, `account_code`, `account_enable`, `account_enable_from/to`, `site_enable_from/to`, `site_domain_access_key`, `logo_path`, `style_href`, `script_src`, `google_tracking_id`) and `event_session` (`event_presentation_li_qry_str`, `event_presenter_li_qry_str`). Fields added to Pydantic models and `searchable_fields`. Alt-view fields require `?view=alt` for search.
     - [ ] Verify SQL Views join in all required `_random` IDs for performance.
 - [ ] **Step 2:** Coordination (Verify Frontend uses `x-account-id` instead of token).
 - [ ] **Step 3:** Frontend V3 WebSocket integration test — queued after IDAA-specific work. Backend is ready (auth wired, heartbeat presence refresh confirmed, unit tests passing). Frontend guide updated at `GUIDE__AE_API_V3_for_Frontend_websockets.md`.
 
+## 🔌 IDAA: Server-Side Novi Verification (Mini Project)
+> **Status: P1–P4 Complete (May 2026).** Endpoint live at `GET /v3/action/idaa/novi_member/{uuid}`. P5 (frontend migration) is the remaining step.
+> Rationale and frontend integration notes: `aether_app_sveltekit/documentation/CLIENT__IDAA_and_customized_mods.md` → "Planned: Server-Side Novi Verification"
+
+**Goal:** Proxy the Novi member-verification call server-to-server (FastAPI → Novi) so members' browser IPs are no longer in the call path.
+
+- [x] **[P1] New router:** `app/routers/api_v3_actions_idaa.py`
+  - Route: `GET /v3/action/idaa/novi_member/{uuid}`
+  - Required auth: `Depends(get_account_context)` — valid API key + any account context (x-account-id, JWT, or bypass). This is the standard V3 gate.
+  - Reads `novi_idaa_api_key` / `novi_api_root_url` from site `cfg_json` via `_load_idaa_cfg()` (same as Mailman bridge)
+  - Calls Novi: `GET {novi_api_root_url}/customers/{uuid}` with `Authorization: Basic {api_key}`
+  - Normalize email: `.replace(' ', '+')` (Novi quirk — see Novi-Mailman bridge notes)
+  - Build display name: `"{FirstName} {LastName[0]}."` format, fall back to `Name` field
+  - Returns `{ "verified": true, "full_name": "...", "email": "..." }` on success
+  - Returns `404` if Novi 200 with no identity data (empty-member anti-pattern)
+  - Returns `429` if Novi rate limits; `503` if Novi unreachable or 5xx
+  - Business logic in `app/methods/idaa_novi_verify_methods.py`
+
+- [x] **[P2] Redis cache:**
+  - Key: `idaa:novi_member:{uuid}` — TTL 4 hours
+    - Note: `account_id` dropped from key — Novi credentials are hardcoded to the IDAA site; same UUID always returns the same data regardless of caller, so per-caller scoping wastes Redis space and halves hit rate.
+  - Cache only verified (200) results — do NOT cache 404 (member may have just joined)
+  - Uses `redis_client` from `lib_redis_helpers.py` directly
+
+- [x] **[P3] Register in registry:** Added to `routers/registry.py` at `/v3/action/idaa` tag `IDAA Actions (V3)`. Confirmed live — endpoint appears in `/openapi.json`.
+
+- [x] **[P4] Tests:** `tests/unit/test_unit_idaa_novi_verify.py` — 9 tests, all passing.
+  - Mock Novi responses (200/empty-200/404/429/503/unreachable)
+  - Verify Redis cache is set on 200, hit bypasses Novi call
+  - Verify email normalization (space → +)
+  - Verify display name format (5 cases)
+
+- [x] **[P5] Coordinate with Frontend Agent** — **Complete (2026-05-19)**
+  - Frontend replaced direct `fetch()` to Novi in `+layout.svelte:verify_novi_uuid()`
+  - Response codes mapped: 200 → verified, 404 → denied, 429 → `'rate_limited'`, 503 → auto-retried (3s, once) then `'api_error'`
+  - 503 auto-retry added same session to match network-error retry path
+
 ## 🛡️ Security & Privacy Baseline (IDAA)
 - **Status:** **ENFORCED**.
 - **Maintenance:** Run `tests/e2e/test_e2e_v3_security_audit.py` after ANY router or registry change.
 
+## 🔑 Credentials / Access Maintenance
+- [x] **Bitbucket API Token Migration:** Bitbucket is deprecating app passwords — all existing ones become inactive **2026-06-09**. SSH migration complete; Gitea remote also configured. Ref: https://support.atlassian.com/bitbucket-cloud/docs/api-tokens/
+
 ## 🚧 Strategic Goals (V3.5+)
 - [ ] **Pydantic V2 / SQLAlchemy 2.0:** Major framework upgrade for performance and type safety.
     - SQLAlchemy 2.0 is likely the easier migration (additive, legacy mode available).
     - Pydantic v2 touches every model definition — do this second.
     - Current pins: `pydantic==1.*`, `SQLAlchemy==1.4.52` — intentional, do not remove until migration is done.
-- [~] **Novi-Mailman Bridge:** Synchronization between Novi AMS and Mailman 3.
+- [x] **Novi-Mailman Bridge:** Cron-based mirror sync between Novi AMS and Mailman 3 — **POC complete 2026-03-17**.
     - Files: `app/methods/e_novi_mailman_methods.py`, `app/routers/api_v3_actions_e_novi_mailman.py`
     - Registered at `/v3/action/e_novi_mailman/`
-    - **Confirmed from IDAA Jitsi code:**
-        - Auth: `Authorization: Basic {api_key}` (Base64-encoded key stored in `data_store`)
-        - Novi member fields are PascalCase: `Email`, `FirstName`, `LastName`, `Name`
-        - Individual member lookup: `GET /customers/{uuid}`
-        - Group member list: `GET /groups/{guid}/members?pageSize=200` (returns `Results` or `Members` key)
-        - Emails may contain spaces instead of `+` — sanitize with `.replace(' ', '+')`
-    - **Still needs confirmation:**
-        - Bulk member list endpoint (likely `/members` or `/customers`) — hit `/novi/members` route after creds are set to inspect
-        - `MembershipStatus` field name in bulk response (may be `Status`)
-        - Webhook `EventType` values and payload shape (check Novi webhook docs)
-    - **data_store setup required (two records):**
-        - `novi_api_config` → `{"api_key": "<base64-key>", "base_url": "https://www.idaa.org/api", "mailman_list_id": "members@yourdomain.org"}`
-        - `mailman_api_config` → `{"base_url": "http://<host>:8001", "username": "restadmin", "password": "<password>"}`
-    - **Outstanding TODO in code:** Webhook HMAC signature verification once Novi webhook secret is known.
+    - **Confirmed Novi API shape:** No flat member list. Fetch via `/groups/{guid}/members` → UUIDs, then `/customers/{uuid}` for full record. Fields: `Email`, `FirstName`, `LastName`, `Active` (bool), `UnsubscribeFromEmails` (bool). Emails may contain spaces instead of `+` — sanitized with `.replace(' ', '+')`.
+    - **Credentials:** All in IDAA site `cfg_json` (`id_random='58_gJESdlUh'`, site id=17). Keys: `novi_api_root_url`, `novi_idaa_api_key`, `mailman_base_url`, `mailman_username`, `mailman_password`, `novi_mailman_sync` (array).
+    - **Mailman 3 REST API:** `https://lists.idaa.org/mailman-api` (Nginx proxy → `127.0.0.1:8008` → Docker container). Roster: `/3.1/lists/{list_id_dot}/roster/member`.
+    - **Sync logic:** Full mirror — subscribe Novi-only addresses, unsubscribe Mailman-only addresses. Respects `Active=false` and `UnsubscribeFromEmails=true`.
+    - **Cron target:** `POST /v3/action/e_novi_mailman/sync` — runs all `novi_mailman_sync` mappings.
+    - **Webhook approach abandoned** — cron is simpler; Novi webhook payload format is unknown and Novi hasn't been configured to send webhooks.
+    - **Remaining:** Set production group→list mappings in `cfg_json`, configure cron schedule, rotate Mailman `restadmin` password.
 - [ ] **Lookup System Batch 2:** Migration of `post_topic`, `user_status`, `file_purpose`.
+- [ ] **Post Topic Storage Review:** Revisit whether IDAA BB posts should keep the lookup-based `topic_id`/`topic_name` shape or eventually flatten to saved topic text if the list stays fixed.
 - [ ] **Zoom Events Integration:** Implement cron synchronization for OAuth2 ticket retrieval.
 
 ## 📝 Session Notes (March 11, 2026)

documentation/archive/GUIDE__DEPLOYMENT_MANUAL.md

@@ -0,0 +1,158 @@
+# DEPRECATED: Manual Server Deployment Guide (Non-Docker)
+
+> **Notice (March 2026):**
+> This manual deployment guide is deprecated. The standard and supported method for deploying the Aether API is now via Docker Compose, as described in the main README and the `aether_container_env` documentation. Use this guide only for legacy or advanced manual setups.
+
+# Manual Server Deployment Guide (Non-Docker)
+
+This guide describes the manual process for deploying the Aether API on a Linux server using Nginx, Gunicorn, and Systemd.
+
+## 1. Initial Server Setup
+
+### Clone the Repository
+```bash
+sudo git clone https://scott_idem@bitbucket.org/oneskyit/one-sky-it-api-fastapi.git /srv/http/dev_fastapi.oneskyit.com
+```
+
+### Configure Permissions
+```bash
+sudo mkdir admin/log
+sudo chown http:http -R /srv/http/dev_fastapi.oneskyit.com/
+sudo chmod 775 -R /srv/http/dev_fastapi.oneskyit.com/
+```
+
+### Environment Preparation
+```bash
+cd /srv/http/dev_fastapi.oneskyit.com/
+git switch development
+
+virtualenv environment
+source environment/bin/activate
+pip install -U -r admin/requirements.txt
+```
+
+## 2. Gunicorn Configuration (Systemd)
+
+### Socket Configuration (`/etc/systemd/system/gunicorn.socket`)
+```ini
+[Unit]
+Description=gunicorn socket
+
+[Socket]
+ListenStream=/run/gunicorn.sock
+User=http
+
+[Install]
+WantedBy=sockets.target
+```
+
+### Service Configuration (`/etc/systemd/system/gunicorn.service`)
+```ini
+[Unit]
+Description=gunicorn daemon
+Requires=gunicorn.socket
+After=network.target
+
+[Service]
+Type=notify
+User=root
+Group=root
+RuntimeDirectory=gunicorn
+WorkingDirectory=/srv/http/dev_fastapi.oneskyit.com
+Environment="PATH=/srv/http/dev_fastapi.oneskyit.com/environment/bin"
+ExecStart=/srv/http/dev_fastapi.oneskyit.com/environment/bin/gunicorn \
+    --bind unix:/srv/http/dev_fastapi.oneskyit.com/gunicorn.sock \
+    -m 007 app.main:app \
+    --workers 4 \
+    -k uvicorn.workers.UvicornWorker \
+    --access-logfile admin/log/access.log \
+    --error-logfile admin/log/error.log \
+    --capture-output --keep-alive 5
+
+ExecReload=/bin/kill -s HUP $MAINPID
+KillMode=mixed
+TimeoutStopSec=5
+PrivateTmp=true
+
+[Install]
+WantedBy=multi-user.target
+```
+
+### Activation
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable gunicorn.socket
+sudo systemctl start gunicorn.socket
+```
+
+## 3. Nginx Configuration
+
+Create a site configuration file at `/etc/nginx/sites-available/dev_fastapi.oneskyit.com`:
+
+```nginx
+server {
+  access_log  /var/log/nginx/access_dev_fastapi.oneskyit.com.log;
+
+  listen 443 ssl;
+  listen [::]:443 ssl http2;
+  server_name dev-fastapi.oneskyit.com;
+
+  ssl_certificate /etc/letsencrypt/live/oneskyit.com-0001/fullchain.pem;
+  ssl_certificate_key /etc/letsencrypt/live/oneskyit.com-0001/privkey.pem;
+
+  client_max_body_size 4096M;
+
+  location / {
+        proxy_set_header Host $http_host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        proxy_redirect off;
+        proxy_buffering off;
+
+        proxy_pass http://unix:/run/gunicorn.sock;
+  }
+
+  # WebSocket Support
+  location ~ ^/(ws|ws_redis) {
+        proxy_set_header Host $http_host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+
+        proxy_http_version 1.1;
+        proxy_redirect off;
+        proxy_buffering off;
+
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection "upgrade";
+
+        proxy_pass http://unix:/run/gunicorn.sock;
+  }
+}
+
+server {
+  listen 80;
+  listen [::]:80;
+  server_name dev-fastapi.oneskyit.com;
+  return 301 https://$host$request_uri;
+}
+```
+
+Enable and restart Nginx:
+```bash
+sudo ln -s /etc/nginx/sites-available/dev_fastapi.oneskyit.com /etc/nginx/sites-enabled/
+sudo systemctl restart nginx.service
+```
+
+## 4. Troubleshooting
+```bash
+# Check status
+sudo systemctl status gunicorn.socket
+sudo systemctl status gunicorn.service
+sudo systemctl status nginx.service
+
+# List active units
+systemctl list-units --type=service --state=running
+```

documentation/archive/PLAN__SECURITY_HARDENING.md

@@ -1,9 +1,11 @@
 # Project: API Security Hardening (V3)
 
-**Status:** Draft / Planning
+**Status:** Complete / Archived (Reviewed March 18, 2026)
 **Date:** Jan 18, 2026
 **Owner:** Scott / Aether API Team
 
+> This plan was fully implemented and reviewed on March 18, 2026. All critical and high vulnerabilities described have been addressed in the current codebase. See dependencies in `app/routers/dependencies_v3.py` and JWT logic in `app/lib_jwt.py` for details.
+
 ## 1. Executive Summary
 This project aims to close a critical security vulnerability in the Aether API V3 dependencies where the `x_no_account_id` header allows unauthorized "Superuser/Bypass" access without validation. Additionally, it addresses the lack of cryptographic verification for JWTs in the V3 CRUD layer.
 
@@ -53,7 +55,7 @@ This project aims to close a critical security vulnerability in the Aether API V
     *   Any external scripts using the "Bypass" header must be updated to send a valid API Key.
 
 ## 5. Action Items
-- [ ] Create `PROJECT_SECURITY_HARDENING.md` (This document).
-- [ ] Refactor `app/routers/dependencies_v3.py`.
-- [ ] Verify fix with `curl` tests.
-- [ ] Commit changes.
+- [x] Create `PROJECT_SECURITY_HARDENING.md` (This document).
+- [x] Refactor `app/routers/dependencies_v3.py`.
+- [x] Verify fix with `curl` tests.
+- [x] Commit changes.

requirements.txt

@@ -15,7 +15,7 @@ email-validator
 et-xmlfile
 fastapi>=0.115.5
 # greenlet
-gunicorn
+gunicorn==23.0.0
 h11
 html2text
 httpcore

tests/README.md

@@ -7,7 +7,7 @@ This directory contains the automated and manual test scripts for the Aether Fas
 - **`unit/`**: Isolated logic tests. These use heavy mocking to bypass database and network requirements. Fast and safe to run in any environment.
 - **`integration/`**: Local environment tests. These verify component interactions, often requiring a connection to the local MariaDB/Redis instance.
 - **`e2e/` (End-to-End)**: Network-based API tests. these use the `requests` library to call the live API endpoints at `https://dev-api.oneskyit.com`.
-- **`tools/`**: Utility scripts for administrative tasks like registry generation or Docker exploration.
+- **`tools/`**: Utility scripts for administrative tasks like registry generation, Docker exploration, and performance stress testing.
 - **`archive/`**: Legacy or deprecated scripts kept for historical reference.
 
 ## 📜 Standardized E2E Suite (`tests/e2e/`)
@@ -19,15 +19,21 @@ These consolidated scripts are the primary verification tool for the V3 API.
 | `test_e2e_v3_search_engine.py` | **Primary Search**: Basic operators, Registry fields, Nested search, and Filter bypass. |
 | `test_e2e_v3_security_audit.py` | **Core Security**: Verifies multi-tenant isolation, cross-account write blocking, and ID Vision compliance. |
 | `test_e2e_v3_auth_security.py` | **Primary Auth**: Site bootstrap, Passcode-to-JWT, and permission boundaries. |
+| `test_e2e_v3_user_action_routes.py` | **V3 User Actions**: Sign-in (username+password and auth-key flow), verify password, change password, new auth key, email magic link, and auth guards. |
+| `test_e2e_v3_user_auth_routes.py` | **Legacy User Routes**: Tests the pre-V3 `/user/*` endpoints (change_password, new_auth_key, verify_password, lookup, email_auth_key_url, authenticate). |
 | `test_e2e_v3_actions_file_lifecycle.py` | **Primary Actions**: Upload, Download (ID/Hash/Streaming), and physical Deletion. |
 | `test_e2e_v3_data_store_lookup.py` | **V3 Parity**: Verifies code-based lookups and latency simulation. |
 | `test_e2e_redis_extensive.py` | **Redis Stress**: Benchmarks bidirectional ID caching across thousands of records. |
 | `test_e2e_v3_event_vision_parity.py`| **Vision ID**: Verifies string-ID enforcement across event models. |
 | `test_e2e_v3_cms_vision_parity.py`| **Vision ID**: Verifies string-ID enforcement across CMS (post/comment) models. |
 | `test_e2e_v3_core_vision_parity.py`| **Vision ID**: Verifies string-ID and polymorphic resolution across core models (Account, Person, Address, Contact, DataStore). |
-| `test_e2e_v3_demo_parity.py` | **Demo Parity**: Comprehensive check for Badge, Exhibit, Tracking, and nested Journal Entries. |
+| `test_e2e_v3_demo_parity.py` | **Demo Parity + Nested Create Regression**: Vision ID check for Badge, Exhibit, Tracking; nested create lifecycle (POST+DELETE) for `journal/journal_entry` and `event/event_session`; alias resolution. **Run after any model or nested-router change.** |
 | `test_e2e_v3_action_event_file.py` | **Event Actions**: Specialized atomic upload and linking for event files. |
 | `test_e2e_v3_action_zoom.py` | **Zoom Integration**: Verifies OAuth and ticket sync logic for Zoom Events. |
+| `test_e2e_v3_action_novi_mailman.py` | **Novi-Mailman Bridge — Connections**: Verifies Novi AMS and Mailman 3 API credentials are valid (IDAA). Run first before the lists test. |
+| `test_e2e_v3_action_novi_mailman_lists.py` | **Novi-Mailman Bridge — List Operations**: Full member lifecycle — read roster, subscribe, verify, unsubscribe — against `mm3@idaa.org`, `mm3@dgrzone.com`, `mm3@oneskyit.com`. |
+| `test_e2e_v3_action_event_exhibit_tracking_export.py` | **Exhibit Leads Export**: Auth/permission guards, CSV column structure, XLSX bytes, and `return_file` mode for the V3 tracking export action. |
+| `test_e2e_v3_action_idaa_novi_verify.py` | **IDAA Novi Member Verify**: Auth guard, 200 verified, 404 not-found, 429 rate-limit, 503 unreachable, Redis cache hit, email normalization. (not yet written — add when endpoint is stable) |
 | `test_e2e_v3_accounts.py` | CRUD verification for the core Account object. |
 | `test_e2e_v3_schema.py` | Network verification of the V3 metadata discovery endpoint. |
 | `test_e2e_agent_bridge.py` | Verifies container diagnostics and log streaming routes. |
@@ -35,12 +41,53 @@ These consolidated scripts are the primary verification tool for the V3 API.
 
 ---
 
+## 🔧 Tools (`tests/tools/`)
+
+| Script | Description |
+| :--- | :--- |
+| `stress_list_queries.py` | **Read-only concurrency stress test.** Fires N worker threads making R sequential requests across all V3 list endpoints. Reports per-endpoint p50/p95/max latency and error counts. CLI: `--workers` (default 10), `--requests` (default 5), `--limit` (default 20), `--base-url` (default dev API). Exit code 1 on any error. |
+| `tool_generate_registry.py` | Generates the object type registry from source definitions. |
+| `tool_mcp_docker_explorer.py` | Explores running Docker containers via the MCP bridge. |
+
+**Stress test quick reference:**
+```bash
+# Baseline (10 workers, 5 rounds, 400 total requests)
+./environment/bin/python3 tests/tools/stress_list_queries.py
+
+# Heavy load (35 workers, 5 rounds, 1400 total requests)
+./environment/bin/python3 tests/tools/stress_list_queries.py --workers 35 --requests 5
+
+# Target a different environment
+./environment/bin/python3 tests/tools/stress_list_queries.py --base-url https://api.oneskyit.com --workers 5
+```
+
+---
+
 ## 🛠️ Shared Helpers
 
 - **`mock_config_helper.py`**: A critical utility that mocks `app.config.settings` before other modules are imported. Use this in unit tests.
 
 ---
 
+## 🚦 When to Run Tests
+
+Tests exist to be used — run the relevant suite whenever you touch backend code, not just when something breaks.
+
+| Change type | Required suites |
+| :--- | :--- |
+| Model `root_validator` / ID Vision changes | `test_e2e_v3_demo_parity.py`, `test_e2e_v3_event_vision_parity.py`, `test_e2e_v3_core_vision_parity.py` |
+| Nested router (`api_crud_v3_nested.py`) changes | `test_e2e_v3_demo_parity.py` |
+| Search / filter changes | `test_e2e_v3_search_engine.py` |
+| Auth / account context changes | `test_e2e_v3_security_audit.py`, `test_e2e_v3_auth_security.py` |
+| User action route changes (sign-in, password, magic link) | `test_e2e_v3_user_action_routes.py` |
+| File upload / download changes | `test_e2e_v3_actions_file_lifecycle.py` |
+| Novi-Mailman bridge changes | `test_e2e_v3_action_novi_mailman.py`, `test_e2e_v3_action_novi_mailman_lists.py` |
+| IDAA Novi member verify changes | `tests/unit/test_unit_idaa_novi_verify.py`, `test_e2e_v3_action_idaa_novi_verify.py` (e2e pending) |
+| Event exhibit tracking export changes | `test_e2e_v3_action_event_exhibit_tracking_export.py` |
+| Any backend change before frontend hand-off | All of the above |
+
+---
+
 ## 🧹 Maintenance Policy
 
 1.  **Standardization**: All E2E tests should use the standard Agent API Key (`PMM4n50teUCaOMMTN8qOJA`) and provide clean `[✅ PASS]` or `[❌ FAIL]` output.
@@ -66,5 +113,73 @@ To maintain a "nice" and readable test suite, follow these patterns in all new P
 ./environment/bin/python3 tests/e2e/test_e2e_v3_search_engine.py
 ```
 
+### Running unit tests with pytest
+```bash
+./environment/bin/python3 -m pytest tests/unit/ -v
+```
+
+`pytest` and `pytest-asyncio` are dev-only dependencies (not in `requirements.txt`). After rebuilding the venv (e.g. following an OS Python update), reinstall them:
+```bash
+./environment/bin/pip install pytest pytest-asyncio
+```
+
 ### Path Requirements
 Always run test scripts from the **project root** directory. Most scripts include `sys.path.append(os.getcwd())` to ensure local imports work correctly.
+
+
+---
+
+## Development / Testing / Demo environment information
+* Use snake_case (or Snake_Case or Snake_case or test_NASA_example or test_API_key)
+* Aether test/demo base URL: 'http://demo.localhost:5173'
+* Aether development API: 'https://dev-api.oneskyit.com'
+* Aether development API "secret" key: 'nT0jPeiCfxSifkiDZur9jA'
+
+These are IDs for records that we can use for testing. Please do not delete them. They are also used for demo purposes with clients.
+
+### Core Modules
+* Aether test/demo Account: '_XY7DXtc9MY' (1) "One Sky IT Demo"
+* Aether test/demo Site: '92vkYC4fVEl' (12) "One Sky IT Demo"
+* Aether test/demo Site Domain: '_6jcTbnJk-o' (12) "demo.localhost:5173"
+* Aether test/demo Site Domain: 'heXRgHOs4ns' (30) "sk-demo.oneskyit.com"
+* Aether test/demo Site Domain: 'DASm8fP92yw' (69) "dev-demo.oneskyit.com"
+* Aether test/demo Site Domain: '2i_0Za6yRPo' (2) "demo.oneskyit.com"
+* Aether test/demo Person: 'QWODAPCNLQU' (49) "Osiris Idem"
+* Aether test/demo Person: 'HMQRNPIXQMK' (48) "Cleo Idem"
+
+### Events Modules
+* Aether test/demo Event: 'pjrcghqwert' (1) "Demo One Sky IT Conference"
+* Aether test/demo Event Session: 'DOW3h7v6H42' (703) "How To Do Things"
+* Aether test/demo Event Session (Digital Posters): "K8cxUIEWyQk" "The Beginning of Digital Posters!"
+* Aether test/demo Event Session (Digital Posters): "1Un1xI1Rgk8" "Poster Session 99: All about posters!"
+* Aether test/demo Event Presentation: '7U2eXSjR6H4' (1670) "Build a House"
+* Aether test/demo Event Presenter: 'gT-hxnifb-0' (2202) "Bob The Builder"
+* Aether test/demo Event File: 'OOsHXtng5mr' (2985) "1 Quick Test for macOS.mp4"
+* Aether test/demo Event Badge: 'UIJT-73-63-61' (37163) "Scott Idem"
+* Aether test/demo Event Person: 'ffkKxiHpOEC' (16603) "Scott Idem"
+* Aether test/demo Event Badge Template: 'jgfixEpYp1B' (18) "Dev Demo 202x"
+* Aether test/demo Event Badge Template: 'rzmUgsk7mkq' (19) "Dev Demo 202x Workshops"
+* Aether test/demo Event Location: 'VXXY-98-46-14' (26) "Ballroom 1"
+* Aether test/demo Event Location: 'FGRN-67-92-45' (298) "Ballroom AB"
+* Aether test/demo Event Location: 'PQKB-15-39-81' (78) "Poster Display Station A"
+
+### Journals Module
+* Aether test/demo Journal: 'BVYE-94-46-29' (42) "Testing Things"
+* Aether test/demo Journal Entry: 'xRx-Y4-h3-fU' (233) "Another Journal Entry in the Test Journal"
+
+### Archives Module (IDAA Archives)
+* Aether test/demo Archive: 'nAA2bHLv8RK' (1) "One Sky Test Archive"
+* Aether test/demo Archive Content: 'UjKzrk-GKu5' (1) "Hosted File Test"
+
+### Posts Module (IDAA Bulletin Board)
+* Aether test/demo Post:
+* Aether test/demo Post:
+
+### Events Module (IDAA Recovery Meetings)
+* Aether test/demo Event: '1Pkd025vvxU' (36) "IDAA Recovery Meeting Test"
+* Aether test/demo Event: 'gIZgAjISkf8' (43) "IDAA Recovery Meeting Test"
+
+### IDAA and Novi AMS
+Scott Idem (test 1)
+* Novi (customer) UUID: "1dadf11c-b74b-4582-8a0a-7ec738a033dc"
+* Novi Email: "stidem+test1@gmail.com"

tests/archive/reproduce_event_file_bug.py

@@ -1,35 +0,0 @@
-import sys
-import os
-
-# Set up project root for imports
-sys.path.append(os.getcwd())
-
-# 1. Initialize Mock Config Helper BEFORE other imports
-import tests.mock_config_helper
-from app.config import settings
-
-# Now set some REAL values for DB connection so it actually works
-import os
-settings.DB_SERVER = "vpn-db.oneskyit.com"
-settings.DB_USER = "aether_dev"
-settings.DB_PASS = "$1sky.AE_dev.2023"
-settings.DB_NAME = "aether_dev"
-settings.DB_PORT = 3306
-settings.REDIS = {"server": "127.0.0.1", "port": 6379}
-settings.FILES_PATH = {"hosted_files_root": "/home/scott/tmp/gemini_trash"} # Dummy
-
-from app.methods.event_file_methods import load_event_file_obj
-from app.db_sql import get_id_random
-
-print("--- Testing get_id_random directly ---")
-print(f"event ID 1 -> {get_id_random(1, 'event')}")
-print(f"session ID 543 -> {get_id_random(543, 'event_session')}")
-print(f"presenter ID 1629 -> {get_id_random(1629, 'event_presenter')}")
-
-print("\n--- Testing load_event_file_obj for a2pPIT_W28o ---")
-res = load_event_file_obj('a2pPIT_W28o', model_as_dict=True)
-if res:
-    import json
-    print(json.dumps(res, indent=4))
-else:
-    print("Failed to load object.")

tests/archive/test_novi_webhook_ARCHIVED.py

@@ -0,0 +1,51 @@
+"""
+ARCHIVED 2026-03-17
+The webhook endpoint (/webhook/novi) was removed — sync is cron-based.
+If Novi webhook support is added in future, restore the endpoint in
+api_v3_actions_e_novi_mailman.py and move this file back to tests/e2e/.
+The webhook secret is stored as novi_webhook_secret in IDAA site cfg_json.
+
+Original: Integration test — send a signed Novi webhook payload to the API.
+"""
+
+import os
+import json
+import hmac
+import hashlib
+import requests
+
+BASE_URL = os.environ.get('AE_API_BASE', 'https://dev-api.oneskyit.com')
+ENDPOINT = f"{BASE_URL}/v3/action/e_novi_mailman/webhook/novi"
+SECRET = os.environ.get('NOVI_WEBHOOK_SECRET', 'test-secret')
+
+payload = {
+    "EventType": "MembershipActivated",
+    "Member": {
+        "Email": "test+webhook@example.com",
+        "FirstName": "Test",
+        "LastName": "Webhook",
+        "MembershipStatus": "Active"
+    }
+}
+
+body = json.dumps(payload).encode('utf-8')
+signature = hmac.new(SECRET.encode('utf-8'), body, hashlib.sha256).hexdigest()
+
+headers = {
+    'Content-Type': 'application/json',
+    'X-Novi-Signature': signature
+}
+
+print('Posting to', ENDPOINT)
+resp = requests.post(ENDPOINT, headers=headers, data=body, timeout=30)
+print('Status:', resp.status_code)
+try:
+    print('JSON:', resp.json())
+except Exception:
+    print('Body:', resp.text)
+
+if resp.status_code == 200:
+    print('\u2705 PASS: webhook accepted')
+else:
+    print('\u274C FAIL: webhook rejected')
+    raise SystemExit(1)

tests/e2e/test_e2e_jitsi_token.py

@@ -0,0 +1,185 @@
+"""
+Jitsi JWT Token E2E Test Suite
+
+Tests the /api/jitsi_token endpoint to verify:
+  - Moderator tokens contain moderator=true in the JWT payload
+  - Attendee tokens contain moderator=false in the JWT payload
+  - Room claim is correctly scoped per request
+  - Basic validation rejects malformed input
+
+Run from project root:
+    ./environment/bin/python3 tests/e2e/test_e2e_jitsi_token.py
+"""
+
+import sys
+import os
+import json
+import base64
+import time
+import requests
+
+sys.path.append(os.getcwd())
+
+# --- Configuration ---
+API_ROOT = "https://dev-api.oneskyit.com"
+JITSI_ENDPOINT = f"{API_ROOT}/api/jitsi_token"
+
+TEST_ROOM = "idaa-test-room-001"
+TEST_NAME = "E2E Test User"
+TEST_EMAIL = "e2e-test@oneskyit.com"
+
+
+def print_result(label, success, message=""):
+    status = "✅ PASS" if success else "❌ FAIL"
+    suffix = f" — {message}" if message else ""
+    print(f"  [{status}] {label}{suffix}")
+
+
+def decode_jwt_payload(token: str) -> dict:
+    """Decode a JWT payload without signature verification (for inspection)."""
+    try:
+        parts = token.split(".")
+        if len(parts) != 3:
+            return {}
+        # Add padding
+        padded = parts[1] + "=" * (4 - len(parts[1]) % 4)
+        return json.loads(base64.urlsafe_b64decode(padded))
+    except Exception:
+        return {}
+
+
+def test_moderator_token():
+    """Request a moderator JWT and verify the claim is set correctly."""
+    print("\n--- Test: Moderator Token ---")
+    payload = {
+        "room": TEST_ROOM,
+        "name": TEST_NAME,
+        "email": TEST_EMAIL,
+        "is_moderator": True,
+    }
+    resp = requests.post(JITSI_ENDPOINT, json=payload)
+    print_result("HTTP 200", resp.status_code == 200, f"status={resp.status_code}")
+
+    if resp.status_code != 200:
+        print(f"    Response: {resp.text}")
+        return None
+
+    token = resp.json().get("token")
+    print_result("Token returned", bool(token))
+    if not token:
+        return None
+
+    decoded = decode_jwt_payload(token)
+    print(f"    Decoded payload: {json.dumps(decoded, indent=6)}")
+
+    moderator_claim = decoded.get("context", {}).get("user", {}).get("moderator")
+    room_claim = decoded.get("room")
+
+    print_result("moderator == True", moderator_claim is True, f"got: {moderator_claim!r}")
+    print_result("room scoped correctly", room_claim == TEST_ROOM, f"got: {room_claim!r}")
+
+    return token
+
+
+def test_attendee_token():
+    """Request a non-moderator JWT and verify the claim is False."""
+    print("\n--- Test: Attendee Token (is_moderator=False) ---")
+    payload = {
+        "room": TEST_ROOM,
+        "name": TEST_NAME,
+        "email": TEST_EMAIL,
+        "is_moderator": False,
+    }
+    resp = requests.post(JITSI_ENDPOINT, json=payload)
+    print_result("HTTP 200", resp.status_code == 200, f"status={resp.status_code}")
+
+    if resp.status_code != 200:
+        print(f"    Response: {resp.text}")
+        return None
+
+    token = resp.json().get("token")
+    print_result("Token returned", bool(token))
+    if not token:
+        return None
+
+    decoded = decode_jwt_payload(token)
+    print(f"    Decoded payload: {json.dumps(decoded, indent=6)}")
+
+    moderator_claim = decoded.get("context", {}).get("user", {}).get("moderator")
+    print_result("moderator == False", moderator_claim is False, f"got: {moderator_claim!r}")
+
+    return token
+
+
+def test_room_isolation():
+    """Verify two requests for different rooms produce different room claims."""
+    print("\n--- Test: Room Isolation ---")
+    rooms = ["room-alpha", "room-beta"]
+    tokens = []
+    for room in rooms:
+        resp = requests.post(JITSI_ENDPOINT, json={
+            "room": room, "name": TEST_NAME, "email": TEST_EMAIL, "is_moderator": False
+        })
+        if resp.status_code == 200:
+            tokens.append((room, decode_jwt_payload(resp.json().get("token", ""))))
+
+    if len(tokens) == 2:
+        match_0 = tokens[0][1].get("room") == tokens[0][0]
+        match_1 = tokens[1][1].get("room") == tokens[1][0]
+        print_result("room-alpha scoped", match_0, f"got: {tokens[0][1].get('room')!r}")
+        print_result("room-beta scoped", match_1, f"got: {tokens[1][1].get('room')!r}")
+        print_result("Rooms differ", tokens[0][1].get("room") != tokens[1][1].get("room"))
+    else:
+        print_result("Both requests succeeded", False, "could not get both tokens")
+
+
+def test_invalid_email():
+    """Verify that a malformed email is rejected with 422."""
+    print("\n--- Test: Input Validation (bad email) ---")
+    payload = {
+        "room": TEST_ROOM,
+        "name": TEST_NAME,
+        "email": "not-an-email",
+        "is_moderator": False,
+    }
+    resp = requests.post(JITSI_ENDPOINT, json=payload)
+    print_result("422 on bad email", resp.status_code == 422, f"status={resp.status_code}")
+
+
+def test_token_expiry():
+    """Verify the exp claim is approximately 1 hour from now."""
+    print("\n--- Test: Token Expiry (exp claim) ---")
+    payload = {
+        "room": TEST_ROOM, "name": TEST_NAME, "email": TEST_EMAIL, "is_moderator": False
+    }
+    resp = requests.post(JITSI_ENDPOINT, json=payload)
+    if resp.status_code != 200:
+        print_result("HTTP 200 (skipping exp check)", False)
+        return
+
+    decoded = decode_jwt_payload(resp.json().get("token", ""))
+    exp = decoded.get("exp")
+    now = int(time.time())
+    ttl = exp - now if exp else 0
+    # Should be ~3600s (allow 30s window for test runtime)
+    ok = 3550 < ttl <= 3600
+    print_result("exp ≈ now + 3600s", ok, f"ttl={ttl}s")
+
+
+if __name__ == "__main__":
+    suite_start = time.time()
+    print("=" * 55)
+    print("  Jitsi JWT Token — E2E Test Suite")
+    print(f"  Endpoint: {JITSI_ENDPOINT}")
+    print("=" * 55)
+
+    test_moderator_token()
+    test_attendee_token()
+    test_room_isolation()
+    test_invalid_email()
+    test_token_expiry()
+
+    elapsed = time.time() - suite_start
+    print(f"\n{'=' * 55}")
+    print(f"  Suite completed in {elapsed:.2f}s")
+    print("=" * 55)

tests/e2e/test_e2e_v3_action_event_exhibit_tracking_export.py

@@ -0,0 +1,231 @@
+"""
+E2E Test: V3 Action — Event Exhibit Tracking Export
+Route: GET /v3/action/event_exhibit/{exhibit_id}/tracking_export
+
+Tests:
+  1. Auth guard — rejected when API key is missing
+  2. Auth guard — rejected when account context is missing
+  3. Permission guard — rejected when leads_api_access is not enabled (without manager bypass)
+  4. Success (bypass) — CSV file returned with correct headers
+  5. Success (bypass) — XLSX file returned
+  6. Column structure — expected fixed columns present in CSV
+  7. 404 for a bogus exhibit ID
+"""
+
+import csv
+import io
+import sys
+import time
+
+import requests
+
+# ---------------------------------------------------------------------------
+# Configuration
+# ---------------------------------------------------------------------------
+BASE_URL = "https://dev-api.oneskyit.com/v3/action/event_exhibit"
+API_KEY = "PMM4n50teUCaOMMTN8qOJA"  # Agent API Key
+
+# This exhibit is the stable demo record (from tests/README.md "event_exhibit")
+# xK_9yEj1bQY is verified to exist in the demo environment (TARGETS list in demo_parity).
+EXHIBIT_ID = "xK_9yEj1bQY"
+
+BYPASS_HEADERS = {
+    "x-aether-api-key": API_KEY,
+    "x-no-account-id": "bypass",
+}
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+def print_result(label: str, success: bool, message: str = ""):
+    status = "✅ PASS" if success else "❌ FAIL"
+    line = f"  {status} | {label}"
+    if message:
+        line += f" — {message}"
+    print(line)
+    return success
+
+
+# ---------------------------------------------------------------------------
+# Tests
+# ---------------------------------------------------------------------------
+
+def test_missing_api_key():
+    """No auth at all → 403."""
+    resp = requests.get(f"{BASE_URL}/{EXHIBIT_ID}/tracking_export")
+    ok = resp.status_code == 403
+    print_result("Missing API key → 403", ok, f"got {resp.status_code}")
+    return ok
+
+
+def test_missing_account_context():
+    """API key present but no account context → 403."""
+    resp = requests.get(
+        f"{BASE_URL}/{EXHIBIT_ID}/tracking_export",
+        headers={"x-aether-api-key": API_KEY},
+    )
+    ok = resp.status_code == 403
+    print_result("Missing account context → 403", ok, f"got {resp.status_code}")
+    return ok
+
+
+def test_leads_api_access_gate():
+    """
+    A real (non-bypass) account that does NOT own this exhibit should be blocked.
+    Demo account '_XY7DXtc9MY' is used here; if it happens to own the exhibit and
+    have leads_api_access, this test will get a 200 — that's still acceptable data
+    (the endpoint is working). The main value is confirming no 500 error.
+    """
+    resp = requests.get(
+        f"{BASE_URL}/{EXHIBIT_ID}/tracking_export",
+        headers={
+            "x-aether-api-key": API_KEY,
+            "x-account-id": "_XY7DXtc9MY",  # Demo account
+        },
+    )
+    # Accept 403 (correct gate) or 200 (demo account owns exhibit with access enabled)
+    ok = resp.status_code in (200, 403)
+    note = "blocked (correct)" if resp.status_code == 403 else "allowed (demo account owns exhibit)"
+    print_result("leads_api_access gate", ok, f"got {resp.status_code} — {note}")
+    return ok
+
+
+def test_bogus_exhibit_id():
+    """Non-existent exhibit ID → 404."""
+    resp = requests.get(
+        f"{BASE_URL}/AAAAAAAAAAA/tracking_export",
+        headers=BYPASS_HEADERS,
+    )
+    ok = resp.status_code == 404
+    print_result("Bogus exhibit ID → 404", ok, f"got {resp.status_code}")
+    return ok
+
+
+def test_csv_export_bypass():
+    """Bypass auth → CSV file returned with correct content-type and columns."""
+    resp = requests.get(
+        f"{BASE_URL}/{EXHIBIT_ID}/tracking_export",
+        params={"file_type": "CSV", "return_file": "true"},
+        headers=BYPASS_HEADERS,
+    )
+    if resp.status_code != 200:
+        print_result("CSV export (bypass)", False, f"got {resp.status_code}: {resp.text[:200]}")
+        return False
+
+    # Content-Type check
+    ct = resp.headers.get("content-type", "")
+    ct_ok = "text/csv" in ct
+    print_result("CSV content-type header", ct_ok, ct)
+
+    # Content-Disposition check
+    cd = resp.headers.get("content-disposition", "")
+    cd_ok = "attachment" in cd and ".csv" in cd
+    print_result("CSV content-disposition header", cd_ok, cd)
+
+    # Parse and check columns
+    try:
+        reader = csv.DictReader(io.StringIO(resp.text))
+        fieldnames = reader.fieldnames or []
+        expected_fixed = [
+            "event_exhibit_tracking_id",
+            "created_on",
+            "updated_on",
+            "event_exhibit_name",
+            "event_badge_full_name",
+            "event_badge_email",
+            "event_badge_professional_title",
+            "event_badge_affiliations",
+            "event_badge_location",
+            "event_badge_country",
+            "external_person_id",
+            "exhibitor_notes",
+            "priority",
+            "enable",
+            "hide",
+        ]
+        missing = [c for c in expected_fixed if c not in fieldnames]
+        cols_ok = len(missing) == 0
+        print_result(
+            "CSV expected columns present",
+            cols_ok,
+            f"missing: {missing}" if missing else f"{len(fieldnames)} columns total",
+        )
+
+        rows = list(reader)
+        print_result("CSV parseable", True, f"{len(rows)} data rows")
+    except Exception as e:
+        print_result("CSV parse", False, str(e))
+        return False
+
+    return ct_ok and cd_ok and cols_ok
+
+
+def test_xlsx_export_bypass():
+    """Bypass auth → XLSX file returned with correct content-type."""
+    resp = requests.get(
+        f"{BASE_URL}/{EXHIBIT_ID}/tracking_export",
+        params={"file_type": "XLSX", "return_file": "true"},
+        headers=BYPASS_HEADERS,
+    )
+    if resp.status_code != 200:
+        print_result("XLSX export (bypass)", False, f"got {resp.status_code}: {resp.text[:200]}")
+        return False
+
+    ct = resp.headers.get("content-type", "")
+    ct_ok = "spreadsheetml" in ct or "openxmlformats" in ct or "octet-stream" in ct
+    print_result("XLSX content-type header", ct_ok, ct)
+
+    cd = resp.headers.get("content-disposition", "")
+    cd_ok = "attachment" in cd and ".xlsx" in cd
+    print_result("XLSX content-disposition header", cd_ok, cd)
+
+    # Basic magic-bytes check (XLSX starts with PK zip header)
+    magic_ok = resp.content[:2] == b"PK"
+    print_result("XLSX magic bytes (PK zip)", magic_ok, "")
+
+    return ct_ok and cd_ok and magic_ok
+
+
+def test_return_file_false():
+    """return_file=false → JSON response body instead of a file download."""
+    resp = requests.get(
+        f"{BASE_URL}/{EXHIBIT_ID}/tracking_export",
+        params={"file_type": "CSV", "return_file": "false"},
+        headers=BYPASS_HEADERS,
+    )
+    if resp.status_code != 200:
+        print_result("return_file=false", False, f"got {resp.status_code}: {resp.text[:200]}")
+        return False
+
+    ct = resp.headers.get("content-type", "")
+    json_ok = "json" in ct
+    print_result("return_file=false → JSON response", json_ok, ct)
+    return json_ok
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+if __name__ == "__main__":
+    print("=" * 60)
+    print("E2E: V3 Action — Event Exhibit Tracking Export")
+    print("=" * 60)
+    t_start = time.time()
+
+    results = [
+        test_missing_api_key(),
+        test_missing_account_context(),
+        test_leads_api_access_gate(),
+        test_bogus_exhibit_id(),
+        test_csv_export_bypass(),
+        test_xlsx_export_bypass(),
+        test_return_file_false(),
+    ]
+
+    elapsed = time.time() - t_start
+    passed = sum(results)
+    total = len(results)
+    print()
+    print(f"Results: {passed}/{total} passed  ({elapsed:.2f}s)")
+    sys.exit(0 if passed == total else 1)

tests/e2e/test_e2e_v3_action_novi_mailman.py

@@ -0,0 +1,99 @@
+"""
+E2E tests for the Novi-Mailman Bridge — API connection checks.
+
+Verifies that both the Novi AMS API and Mailman 3 REST API are reachable
+and that the credentials stored in IDAA site cfg_json are valid.
+
+Run from project root:
+    ./environment/bin/python3 tests/e2e/test_e2e_v3_action_novi_mailman.py
+
+Environment:
+    AE_API_BASE  — override the target API base URL (default: https://dev-api.oneskyit.com)
+
+Related tests:
+    test_e2e_v3_action_novi_mailman_lists.py  — member read/subscribe/unsubscribe lifecycle
+    test_e2e_v3_action_novi_mailman_sync.py   — full Novi → Mailman mirror sync (TODO)
+
+Credential storage:
+    All Novi and Mailman credentials live in IDAA site cfg_json (id_random='58_gJESdlUh').
+    See project memory for key names.
+"""
+
+import os
+import sys
+import time
+import requests
+
+BASE_URL    = os.environ.get('AE_API_BASE', 'https://dev-api.oneskyit.com')
+ACTION_BASE = f"{BASE_URL}/v3/action/e_novi_mailman"
+API_KEY     = "PMM4n50teUCaOMMTN8qOJA"
+
+AUTH_HEADERS = {
+    "X-Aether-API-Key": API_KEY,
+    "x-no-account-id":  "bypass",
+}
+
+pass_count = 0
+fail_count = 0
+
+
+def print_result(label: str, success: bool, message: str = ""):
+    global pass_count, fail_count
+    status = "✅ PASS" if success else "❌ FAIL"
+    msg = f"   {message}" if message else ""
+    print(f"  [{status}] {label}{msg}")
+    if success:
+        pass_count += 1
+    else:
+        fail_count += 1
+
+
+def test_novi_connection():
+    print("\n[1] Novi API Connection")
+    try:
+        resp = requests.get(f"{ACTION_BASE}/test_connection/novi", headers=AUTH_HEADERS, timeout=15)
+        data = resp.json().get('data', {})
+        if resp.status_code == 200 and data.get('ok'):
+            print_result("Novi credentials valid", True)
+        elif resp.status_code == 401:
+            print_result("Novi credentials valid", False, f"401 — {data.get('error', resp.text[:120])}")
+        else:
+            print_result("Novi credentials valid", False, f"HTTP {resp.status_code}: {resp.text[:120]}")
+    except Exception as e:
+        print_result("Novi credentials valid", False, f"Exception: {e}")
+
+
+def test_mailman_connection():
+    print("\n[2] Mailman 3 API Connection")
+    try:
+        resp = requests.get(f"{ACTION_BASE}/test_connection/mailman", headers=AUTH_HEADERS, timeout=15)
+        data = resp.json().get('data', {})
+        if resp.status_code == 200 and data.get('ok'):
+            print_result("Mailman credentials valid", True, f"version={data.get('version', 'unknown')}")
+        elif resp.status_code == 401:
+            print_result("Mailman credentials valid", False,
+                         f"401 — {data.get('error', resp.text[:120])}  "
+                         f"(add mailman_* keys to IDAA cfg_json)")
+        else:
+            print_result("Mailman credentials valid", False, f"HTTP {resp.status_code}: {resp.text[:120]}")
+    except Exception as e:
+        print_result("Mailman credentials valid", False, f"Exception: {e}")
+
+
+if __name__ == "__main__":
+    print("=" * 60)
+    print(" Novi-Mailman Bridge — Connection Tests")
+    print(f" Target: {ACTION_BASE}")
+    print("=" * 60)
+
+    t_start = time.time()
+
+    test_novi_connection()
+    test_mailman_connection()
+
+    elapsed = time.time() - t_start
+    print(f"\n{'=' * 60}")
+    print(f" Results: {pass_count} passed, {fail_count} failed  ({elapsed:.2f}s)")
+    print("=" * 60)
+
+    sys.exit(0 if fail_count == 0 else 1)

tests/e2e/test_e2e_v3_action_novi_mailman_lists.py

@@ -0,0 +1,166 @@
+"""
+E2E tests for the Novi-Mailman Bridge — Mailman list member operations.
+
+Covers the full member lifecycle:
+  1. Read current members of all TEST_LISTS
+  2. Subscribe TEST_EMAIL to the primary test list (TEST_LISTS[0])
+  3. Verify the address appears in the member roster
+  4. Unsubscribe TEST_EMAIL (cleanup)
+  5. Verify the address is removed
+
+Run from project root:
+    ./environment/bin/python3 tests/e2e/test_e2e_v3_action_novi_mailman_lists.py
+
+Configuration:
+    TEST_LISTS[0]  — list used for the subscribe/unsubscribe lifecycle test
+    TEST_EMAIL     — address used as the test subscriber (safe to add/remove)
+    TEST_LISTS[1:] — additional lists read-only (member count + roster check)
+
+Notes:
+    - Uses dot-notation for list IDs in URL paths (mm3@idaa.org → mm3.idaa.org)
+      because @ is a special character in URL paths.
+    - Mailman pre-confirms subscriptions (no confirmation email sent to TEST_EMAIL).
+    - Run test_e2e_v3_action_novi_mailman.py first to confirm both APIs are reachable.
+"""
+
+import sys
+import time
+import requests
+
+BASE_URL    = "https://dev-api.oneskyit.com/v3/action/e_novi_mailman"
+API_KEY     = "PMM4n50teUCaOMMTN8qOJA"
+TEST_EMAIL  = "scott.idem+mm3api@gmail.com"
+TEST_NAME   = "Scott Idem (MM3 API Test)"
+
+TEST_LISTS = [
+    "mm3@idaa.org",
+    "mm3@dgrzone.com",
+    "mm3@oneskyit.com",
+]
+
+HEADERS = {
+    "X-Aether-API-Key": API_KEY,
+    "x-no-account-id":  "bypass",
+}
+
+pass_count = 0
+fail_count = 0
+
+
+def print_result(label: str, success: bool, message: str = ""):
+    global pass_count, fail_count
+    status = "✅ PASS" if success else "❌ FAIL"
+    msg = f"   {message}" if message else ""
+    print(f"  [{status}] {label}{msg}")
+    if success:
+        pass_count += 1
+    else:
+        fail_count += 1
+
+
+def get_members(list_id: str) -> list | None:
+    """Return member list for a given list_id, or None on failure."""
+    list_id_dot = list_id.replace('@', '.')
+    resp = requests.get(f"{BASE_URL}/mailman/lists/{list_id_dot}/members",
+                        headers=HEADERS, timeout=15)
+    if resp.status_code == 200 and resp.json().get('meta', {}).get('success'):
+        return resp.json()['data'].get('members', [])
+    return None
+
+
+def subscribe(list_id: str, email: str, display_name: str = '') -> bool:
+    list_id_dot = list_id.replace('@', '.')
+    resp = requests.post(
+        f"{BASE_URL}/mailman/lists/{list_id_dot}/subscribe",
+        headers=HEADERS,
+        params={"email": email, "display_name": display_name},
+        timeout=15,
+    )
+    return resp.status_code == 200 and resp.json().get('meta', {}).get('success')
+
+
+def unsubscribe(list_id: str, email: str) -> bool:
+    list_id_dot = list_id.replace('@', '.')
+    resp = requests.delete(
+        f"{BASE_URL}/mailman/lists/{list_id_dot}/subscribe",
+        headers=HEADERS,
+        params={"email": email},
+        timeout=15,
+    )
+    return resp.status_code == 200 and resp.json().get('meta', {}).get('success')
+
+
+# ── Tests ─────────────────────────────────────────────────────────────────
+
+def test_read_members():
+    print("\n[1] Read Members")
+    for list_id in TEST_LISTS:
+        members = get_members(list_id)
+        if members is not None:
+            emails = [m['email'] for m in members]
+            print_result(f"Read {list_id}", True, f"{len(members)} member(s): {', '.join(emails) or 'none'}")
+        else:
+            print_result(f"Read {list_id}", False, "request failed")
+
+
+def test_subscribe():
+    print(f"\n[2] Subscribe {TEST_EMAIL}")
+    list_id = TEST_LISTS[0]  # mm3@idaa.org
+    ok = subscribe(list_id, TEST_EMAIL, TEST_NAME)
+    print_result(f"Subscribe to {list_id}", ok)
+
+
+def test_verify_subscription():
+    print(f"\n[3] Verify {TEST_EMAIL} appears in member list")
+    list_id = TEST_LISTS[0]
+    members = get_members(list_id)
+    if members is None:
+        print_result(f"Verify subscription in {list_id}", False, "could not fetch members")
+        return
+    found = any(m['email'] == TEST_EMAIL for m in members)
+    print_result(f"Found {TEST_EMAIL} in {list_id}", found,
+                 "" if found else "not found after subscribe")
+
+
+def test_unsubscribe():
+    print(f"\n[4] Unsubscribe {TEST_EMAIL} (cleanup)")
+    list_id = TEST_LISTS[0]
+    ok = unsubscribe(list_id, TEST_EMAIL)
+    print_result(f"Unsubscribe from {list_id}", ok)
+
+
+def test_verify_unsubscription():
+    print(f"\n[5] Verify {TEST_EMAIL} removed from member list")
+    list_id = TEST_LISTS[0]
+    members = get_members(list_id)
+    if members is None:
+        print_result(f"Verify removal from {list_id}", False, "could not fetch members")
+        return
+    still_present = any(m['email'] == TEST_EMAIL for m in members)
+    print_result(f"{TEST_EMAIL} removed from {list_id}", not still_present,
+                 "still present after unsubscribe" if still_present else "")
+
+
+# ── Main ──────────────────────────────────────────────────────────────────
+
+if __name__ == "__main__":
+    print("=" * 60)
+    print(" Novi-Mailman Bridge — List Member Operations E2E")
+    print(f" Target: {BASE_URL}")
+    print(f" Test email: {TEST_EMAIL}")
+    print("=" * 60)
+
+    t_start = time.time()
+
+    test_read_members()
+    test_subscribe()
+    test_verify_subscription()
+    test_unsubscribe()
+    test_verify_unsubscription()
+
+    elapsed = time.time() - t_start
+    print(f"\n{'=' * 60}")
+    print(f" Results: {pass_count} passed, {fail_count} failed  ({elapsed:.2f}s)")
+    print("=" * 60)
+
+    sys.exit(0 if fail_count == 0 else 1)

tests/e2e/test_e2e_v3_demo_parity.py

@@ -1,10 +1,19 @@
 import requests
 import json
+import time
 
 # --- Configuration ---
 BASE_URL = "https://dev-api.oneskyit.com/v3/crud"
 API_KEY = "PMM4n50teUCaOMMTN8qOJA" # Agent API Key
-# ACCOUNT_ID = "_XY7DXtc9MY" 
+
+# Stable parent IDs used for nested create regression tests.
+# journal account: nqOzejLCDXM  |  event account: GpLf_bnywCs
+JOURNAL_PARENT_ID = "OGQK-02-04-94"
+EVENT_PARENT_ID = "vfzVJF0LH1O"
+# event_person: ffkKxiHpOEC (16603) "Scott Idem" under Demo event
+EVENT_PERSON_PARENT_ID = "ffkKxiHpOEC"
+# event_badge_template: jgfixEpYp1B (18) "Dev Demo 202x"
+EVENT_BADGE_TEMPLATE_ID = "jgfixEpYp1B"
 
 # Test Targets: (Object Type, Valid ID Random)
 # Note: These IDs are extracted from real active records.
@@ -69,6 +78,128 @@ def verify_demo_parity(obj_type, record_id):
         print(f"  💥 [EXCEPTION] {e}")
         return False
 
+def test_nested_create_lifecycle(parent_type, parent_id, child_type, payload):
+    """
+    Regression test for nested POST create (parent FK injection).
+
+    Bug: root_validators on child models stripped integer parent FKs before
+    INSERT, causing MariaDB 1364 errors. Fixed in api_crud_v3_nested.py by
+    re-injecting resolved_parent_id into data_to_insert after serialization.
+
+    Verifies:
+      1. POST /{parent_type}/{parent_id}/{child_type}/ returns 200
+      2. Response data has a string 'id' (Vision Standard)
+      3. Cleanup: DELETE the created record
+    """
+    label = f"Nested Create ({parent_type}/{child_type})"
+    print(f"\n--- Regression: {label} ---")
+    url = f"{BASE_URL}/{parent_type}/{parent_id}/{child_type}/"
+    headers = get_headers()
+
+    # --- CREATE ---
+    resp = requests.post(url, headers=headers, json=payload)
+    if resp.status_code != 200:
+        print(f"  ❌ [FAIL] POST returned {resp.status_code}: {resp.text[:300]}")
+        return False
+
+    data = resp.json().get('data', {})
+    new_id = data.get('id') or data.get('obj_id_random')
+
+    if not new_id or not isinstance(new_id, str):
+        print(f"  ❌ [FAIL] No string 'id' in response. Got: {data}")
+        return False
+
+    print(f"  ✅ [PASS] Created {child_type} with id: {new_id}")
+
+    # --- VISION COMPLIANCE: parent FK must not appear as integer ---
+    for key, val in data.items():
+        if (key == 'id' or key.endswith('_id')) and not key.endswith('external_id'):
+            if val is not None and not isinstance(val, str):
+                print(f"  ❌ [FAIL] Vision violation: {key} is {type(val).__name__} ({val})")
+                return False
+
+    print(f"  ✅ [PASS] Vision Standard: all ID fields are strings.")
+
+    # --- CLEANUP ---
+    delete_url = f"{BASE_URL}/{parent_type}/{parent_id}/{child_type}/{new_id}"
+    del_resp = requests.delete(delete_url, headers=headers)
+    if del_resp.status_code == 200:
+        print(f"  ✅ [PASS] Cleanup: deleted {new_id}")
+    else:
+        print(f"  ⚠️  [WARN] Cleanup failed ({del_resp.status_code}) — manual cleanup may be needed for {new_id}")
+
+    return True
+
+
+def test_nested_create_secondary_fk(parent_type, parent_id, child_type, payload, required_fk_fields):
+    """
+    Regression test for secondary FK resolution in nested POST create.
+
+    Bug: sanitize_payload ran BEFORE model instantiation in the nested POST handler.
+    For FKs other than the parent FK (e.g. event_badge_template_id on event_badge),
+    sanitize_payload resolved the string → integer, then the model's root_validator
+    stripped the integer back to None (Vision ID anti-leakage guard).  The parent FK
+    survived only because it was explicitly re-injected; secondary FKs were silently lost.
+
+    Fix (api_crud_v3_nested.py): moved sanitize_payload to run on data_to_insert AFTER
+    model serialization, matching the flat V3 POST pattern.
+
+    Verifies:
+      1. POST returns 200.
+      2. Each field in required_fk_fields is present AND non-None in the response.
+      3. All *_id fields are strings (Vision Standard).
+      4. Cleanup: DELETE the created record.
+    """
+    label = f"Nested Secondary FK ({parent_type}/{child_type})"
+    print(f"\n--- Regression: {label} ---")
+    url = f"{BASE_URL}/{parent_type}/{parent_id}/{child_type}/"
+    headers = get_headers()
+
+    resp = requests.post(url, headers=headers, json=payload)
+    if resp.status_code != 200:
+        print(f"  ❌ [FAIL] POST returned {resp.status_code}: {resp.text[:300]}")
+        return False
+
+    data = resp.json().get('data', {})
+    new_id = data.get('id') or data.get('obj_id_random')
+    if not new_id or not isinstance(new_id, str):
+        print(f"  ❌ [FAIL] No string 'id' in response. Got: {data}")
+        return False
+    print(f"  ✅ [PASS] Created {child_type} with id: {new_id}")
+
+    # Check required secondary FK fields are present and non-None
+    for field in required_fk_fields:
+        val = data.get(field)
+        if val is None:
+            print(f"  ❌ [FAIL] Secondary FK '{field}' is None — was not saved to DB.")
+            # Still attempt cleanup
+            requests.delete(f"{BASE_URL}/{parent_type}/{parent_id}/{child_type}/{new_id}", headers=headers)
+            return False
+        if not isinstance(val, str):
+            print(f"  ❌ [FAIL] Secondary FK '{field}' is {type(val).__name__} ({val}) — must be string (Vision Standard).")
+            requests.delete(f"{BASE_URL}/{parent_type}/{parent_id}/{child_type}/{new_id}", headers=headers)
+            return False
+        print(f"  ✅ [PASS] Secondary FK '{field}' = {val}")
+
+    # Vision compliance: all *_id fields must be strings
+    for key, val in data.items():
+        if (key == 'id' or key.endswith('_id')) and not key.endswith('external_id'):
+            if val is not None and not isinstance(val, str):
+                print(f"  ❌ [FAIL] Vision violation: {key} is {type(val).__name__} ({val})")
+                requests.delete(f"{BASE_URL}/{parent_type}/{parent_id}/{child_type}/{new_id}", headers=headers)
+                return False
+    print(f"  ✅ [PASS] Vision Standard: all ID fields are strings.")
+
+    # Cleanup
+    del_resp = requests.delete(f"{BASE_URL}/{parent_type}/{parent_id}/{child_type}/{new_id}", headers=headers)
+    if del_resp.status_code == 200:
+        print(f"  ✅ [PASS] Cleanup: deleted {new_id}")
+    else:
+        print(f"  ⚠️  [WARN] Cleanup failed ({del_resp.status_code}) — manual cleanup may be needed for {new_id}")
+
+    return True
+
+
 def test_nested_alias_resolution():
     """
     Verifies that the 'entry' alias and nested resolution works for journals.
@@ -88,6 +219,7 @@ def test_nested_alias_resolution():
         return False
 
 if __name__ == "__main__":
+    suite_start = time.time()
     print("🚀 Starting Aether V3 Demo Parity Suite\n")
 
     results = []
@@ -97,7 +229,39 @@ if __name__ == "__main__":
 
     results.append(test_nested_alias_resolution())
 
+    # --- Nested Create Regression Tests ---
+    # These guard against the Jan 2026 bug where child model root_validators
+    # stripped the parent FK integer before INSERT, causing MariaDB 1364 errors.
+    results.append(test_nested_create_lifecycle(
+        parent_type='journal',
+        parent_id=JOURNAL_PARENT_ID,
+        child_type='journal_entry',
+        payload={'name': '[e2e-test] nested create regression', 'enable': False},
+    ))
+    results.append(test_nested_create_lifecycle(
+        parent_type='event',
+        parent_id=EVENT_PARENT_ID,
+        child_type='event_session',
+        payload={'name': '[e2e-test] nested create regression', 'enable': False},
+    ))
+    # Secondary FK regression: event_badge_template_id must survive nested POST
+    # (was silently dropped as NULL before the sanitize_payload order fix)
+    results.append(test_nested_create_secondary_fk(
+        parent_type='event_person',
+        parent_id=EVENT_PERSON_PARENT_ID,
+        child_type='event_badge',
+        payload={
+            'event_badge_template_id': EVENT_BADGE_TEMPLATE_ID,
+            'given_name': '[e2e-test]',
+            'family_name': 'secondary-fk-regression',
+            'enable': False,
+            'hide': True,
+        },
+        required_fk_fields=['event_badge_template_id'],
+    ))
+
+    elapsed = time.time() - suite_start
     if all(results):
-        print("\n🏆 DEMO SUITE SUCCESS: All critical endpoints are verified stable.")
+        print(f"\n🏆 DEMO SUITE SUCCESS: All critical endpoints are verified stable. ({elapsed:.2f}s)")
     else:
-        print("\n🚨 DEMO SUITE FAILURE: Some critical checks failed.")
+        print(f"\n🚨 DEMO SUITE FAILURE: Some critical checks failed. ({elapsed:.2f}s)")

tests/e2e/test_e2e_v3_lookup.py

@@ -16,6 +16,19 @@ HEADERS = {
 # TODO: SET THIS to your demo site's random ID
 SITE_ID_RANDOM = "92vkYC4fVEl"
 
+# All US/* priority timezones — group must equal name in lu_v3_time_zone for these to survive
+# PARTITION BY group dedup. If group="United States" for these, only 1 survives.
+US_TIMEZONES = [
+    "US/Alaska", "US/Aleutian", "US/Arizona", "US/Central", "US/East-Indiana",
+    "US/Eastern", "US/Hawaii", "US/Indiana-Starke", "US/Michigan",
+    "US/Mountain", "US/Pacific", "US/Pacific-New", "US/Samoa",
+]
+
+# Spot-check a subset of Europe/* priority timezones — same root cause as US/*
+EUROPE_TIMEZONES_SAMPLE = [
+    "Europe/London", "Europe/Paris", "Europe/Prague", "Europe/Rome",
+]
+
 def print_result(label, success, message=""):
     status = "✅ PASS" if success else "❌ FAIL"
     print(f"{status} | {label} {': ' + message if message else ''}")
@@ -75,27 +88,99 @@ def test_lookup_resolve(lu_type, query):
         print_result(f"GET /{lu_type}/resolve?q={query}", False, str(e))
         return False
 
+def test_timezone_us_dedup(data):
+    """
+    Regression: lu_v3_time_zone group data fix.
+    All 13 US/* priority zones must appear individually.
+    Root cause: group was seeded as 'United States' instead of name — PARTITION BY group
+    collapsed all 13 into one winner.
+    """
+    label = "time_zone: all 13 US/* zones present (group=name data fix)"
+    if data is None:
+        print_result(label, False, "No data")
+        return
+    names = {item.get("name") for item in data}
+    missing = [tz for tz in US_TIMEZONES if tz not in names]
+    if missing:
+        print_result(label, False, f"Missing (group data not yet fixed?): {missing}")
+    else:
+        print_result(label, True, f"All {len(US_TIMEZONES)} US/* timezones present")
+
+def test_timezone_europe_dedup(data):
+    """
+    Regression: same root cause as US/* — group was 'Europe' for all Europe/* zones.
+    Spot-check that the priority ones appear individually after data fix.
+    """
+    label = "time_zone: Europe/* spot-check (group=name data fix)"
+    if data is None:
+        print_result(label, False, "No data")
+        return
+    names = {item.get("name") for item in data}
+    missing = [tz for tz in EUROPE_TIMEZONES_SAMPLE if tz not in names]
+    if missing:
+        print_result(label, False, f"Missing (group data not yet fixed?): {missing}")
+    else:
+        print_result(label, True, f"Europe/* spot-check passed ({len(EUROPE_TIMEZONES_SAMPLE)} zones found)")
+
+def test_country_us_dedup(data):
+    """
+    Regression: PARTITION BY group must NOT produce duplicate alpha_2_code values.
+    Two records exist for alpha_2_code='US' (global default + account override) — only one
+    should survive. If PARTITION BY name were used, both would appear and Svelte would
+    throw each_key_duplicate on alpha_2_code='US'.
+    """
+    label = "country: no duplicate alpha_2_code (PARTITION BY group dedup)"
+    if data is None:
+        print_result(label, False, "No data")
+        return
+    codes = [item.get("alpha_2_code") for item in data if item.get("alpha_2_code")]
+    duplicates = [c for c in set(codes) if codes.count(c) > 1]
+    if duplicates:
+        print_result(label, False, f"Duplicate alpha_2_codes: {duplicates}")
+    else:
+        print_result(label, True, f"No duplicates across {len(data)} countries")
+
+def test_priority_only_count(data, expected=72):
+    """priority=1 enabled timezones: should be exactly {expected} after data fix."""
+    label = f"time_zone priority-only count == {expected}"
+    if data is None:
+        print_result(label, False, "No data")
+        return
+    if len(data) == expected:
+        print_result(label, True, f"{len(data)} records")
+    else:
+        print_result(label, False, f"Got {len(data)}, expected {expected} (data fix pending?)")
+
 if __name__ == "__main__":
     print(f"🚀 Starting V3 Lookup E2E Suite ({BASE_URL})\n")
     start_suite = time.time()
 
-    # 1. Basic Lists (Phase 1)
-    test_lookup_list("country")
+    # 1. Country — basic list + dedup regression
+    print("--- Country ---")
+    country_data = test_lookup_list("country")
+    test_country_us_dedup(country_data)
 
-    print("\n--- Testing Priority Only ---")
-    test_lookup_list("time_zone", only_priority=True)
+    # 2. Timezone — full list + group data fix regressions
+    print("\n--- Timezone (full list) ---")
+    tz_data = test_lookup_list("time_zone")
+    test_timezone_us_dedup(tz_data)
+    test_timezone_europe_dedup(tz_data)
 
-    # 2. Whitelist Test (Phase 2)
+    # 3. Timezone — priority only
+    print("\n--- Timezone (priority only) ---")
+    tz_priority_data = test_lookup_list("time_zone", only_priority=True)
+    test_priority_only_count(tz_priority_data, expected=72)
+
+    # 4. Whitelist Test
     if SITE_ID_RANDOM != "SET_ME_TO_SITE_ID":
-        print("\n--- Testing Site Whitelist Policy ---")
-        # Should return only whitelisted items
+        print("\n--- Site Whitelist Policy ---")
         test_lookup_list("country", site_id=SITE_ID_RANDOM)
         test_lookup_list("time_zone", site_id=SITE_ID_RANDOM)
     else:
-        print("\n⚠️ Skipping Phase 2 test: SITE_ID_RANDOM not set.")
+        print("\n⚠️ Skipping whitelist test: SITE_ID_RANDOM not set.")
 
-    # 3. Resolve Test
-    print("\n--- Testing Resolve ---")
+    # 5. Resolve
+    print("\n--- Resolve ---")
     test_lookup_resolve("country", "US")
 
     print(f"\n⏱️ Suite completed in {time.time() - start_suite:.2f}s")

tests/e2e/test_e2e_v3_nested_create_event_badge.py

@@ -0,0 +1,91 @@
+"""
+E2E: Nested-create regression test
+- Creates an `event_person` under demo event `pjrcghqwert` then creates an
+  `event_badge` under that person using the nested CRUD endpoints.
+- Cleans up created records on success.
+
+Usage:
+    ./environment/bin/python3 tests/e2e/test_e2e_v3_nested_create_event_badge.py
+
+This test uses the standard Agent API Key defined in the project README.
+"""
+
+import os
+import requests
+import sys
+import time
+
+BASE = os.environ.get('AE_API_BASE', 'https://dev-api.oneskyit.com')
+API_BASE = BASE.rstrip('/') + '/v3/crud'
+AGENT_API_KEY = os.environ.get('AE_AGENT_API_KEY', 'nT0jPeiCfxSifkiDZur9jA')
+EVENT_ID = os.environ.get('AE_TEST_EVENT', 'pjrcghqwert')
+ACCOUNT_ID = os.environ.get('AE_ACCOUNT', '_XY7DXtc9MY')
+
+HEADERS = {'x-aether-api-key': AGENT_API_KEY, 'x-account-id': ACCOUNT_ID, 'Content-Type': 'application/json'}
+
+
+def print_result(label, success, message=""):
+    mark = '✅ PASS' if success else '❌ FAIL'
+    print(f"{mark} - {label}: {message}")
+
+
+def run():
+    created = {}
+    try:
+        # 1) Create event_person under event
+        url = f"{API_BASE}/event/{EVENT_ID}/event_person/?return_obj=false"
+        r = requests.post(url, headers=HEADERS, json={})
+        if r.status_code != 200:
+            print_result('create event_person', False, f'status={r.status_code} body={r.text}')
+            return 2
+        data = r.json().get('data') or {}
+        person_id = data.get('obj_id') or data.get('obj_id_random')
+        if not person_id:
+            print_result('create event_person', False, f'missing obj_id in response {r.json()}')
+            return 2
+        created['person'] = person_id
+        print_result('create event_person', True, f'person_id={person_id}')
+
+        # small delay to let DB/indexing settle on remote dev
+        time.sleep(0.5)
+
+        # 2) Create event_badge under event_person
+        url = f"{API_BASE}/event_person/{person_id}/event_badge/?return_obj=false"
+        r2 = requests.post(url, headers=HEADERS, json={})
+        if r2.status_code != 200:
+            print_result('create event_badge', False, f'status={r2.status_code} body={r2.text}')
+            return 2
+        data2 = r2.json().get('data') or {}
+        badge_id = data2.get('obj_id') or data2.get('obj_id_random')
+        if not badge_id:
+            print_result('create event_badge', False, f'missing obj_id in response {r2.json()}')
+            return 2
+        created['badge'] = badge_id
+        print_result('create event_badge', True, f'badge_id={badge_id}')
+
+        # 3) Cleanup: delete badge then person
+        # Delete badge
+        del_url = f"{API_BASE}/event_person/{person_id}/event_badge/{badge_id}?method=delete"
+        rd = requests.delete(del_url, headers=HEADERS)
+        if rd.status_code == 200:
+            print_result('delete event_badge', True, '')
+        else:
+            print_result('delete event_badge', False, f'status={rd.status_code} body={rd.text}')
+
+        # Delete person (as child of event)
+        delp_url = f"{API_BASE}/event/{EVENT_ID}/event_person/{person_id}?method=delete"
+        rp = requests.delete(delp_url, headers=HEADERS)
+        if rp.status_code == 200:
+            print_result('delete event_person', True, '')
+        else:
+            print_result('delete event_person', False, f'status={rp.status_code} body={rp.text}')
+
+        return 0
+
+    except Exception as e:
+        print_result('exception', False, str(e))
+        return 2
+
+
+if __name__ == '__main__':
+    sys.exit(run())

tests/e2e/test_e2e_v3_search_engine.py

@@ -64,6 +64,60 @@ def test_extra_filters():
     resp = requests.get(f"{API_BASE}/user/?enabled=all&hidden=all", headers=get_headers())
     print_result("Bypass Filters (enabled=all)", resp.status_code == 200)
 
+def test_event_session_qry_str_fields():
+    """
+    Regression test for event_presentation_li_qry_str and event_presenter_li_qry_str.
+    These fields were lost during the v1/v2 -> v3 migration and restored May 2026.
+    They live in v_event_session_w_file_count (triggered by ?inc_file_count=true).
+
+    Demo session: DOW3h7v6H42 "How To Do Things" under Demo event pjrcghqwert
+    """
+    print("\n--- Testing event_session qry_str fields (regression: May 2026) ---")
+
+    EVENT_ID = "pjrcghqwert"
+    SESSION_ID = "DOW3h7v6H42"
+
+    headers = {
+        "Content-Type": "application/json",
+        "X-Aether-API-Key": API_KEY,
+        "x-no-account-id": "bypass"
+    }
+
+    # 1. Verify fields are returned in the GET response when inc_file_count=true
+    url = f"{API_BASE}/event_session/{SESSION_ID}?inc_file_count=true"
+    resp = requests.get(url, headers=headers)
+    ok = resp.status_code == 200
+    print_result("GET event_session with inc_file_count", ok, f"(status={resp.status_code})")
+    if ok:
+        data = resp.json().get("data", {})
+        has_pres = "event_presentation_li_qry_str" in data
+        has_presenter = "event_presenter_li_qry_str" in data
+        print_result("Field present: event_presentation_li_qry_str", has_pres,
+                     f"(value={data.get('event_presentation_li_qry_str')!r})")
+        print_result("Field present: event_presenter_li_qry_str", has_presenter,
+                     f"(value={data.get('event_presenter_li_qry_str')!r})")
+
+    # 2. Verify searching by event_presentation_li_qry_str via ?view=alt (v_event_session_w_file_count)
+    # These fields only exist in the alt view, so ?view=alt is required.
+    search_url = f"{API_BASE}/event/{EVENT_ID}/event_session/search?view=alt"
+    query = {"and": [{"field": "event_presentation_li_qry_str", "op": "like", "value": "%"}]}
+    resp = requests.post(search_url, headers=headers, json=query)
+    print_result("Search by event_presentation_li_qry_str (?view=alt)", resp.status_code == 200,
+                 f"(status={resp.status_code})")
+
+    # 3. Verify searching by event_presenter_li_qry_str via ?view=alt
+    query = {"and": [{"field": "event_presenter_li_qry_str", "op": "like", "value": "%"}]}
+    resp = requests.post(search_url, headers=headers, json=query)
+    print_result("Search by event_presenter_li_qry_str (?view=alt)", resp.status_code == 200,
+                 f"(status={resp.status_code})")
+
+    # 4. Confirm search on default view still rejects these fields (expected 400 — not in v_event_session)
+    search_url_default = f"{API_BASE}/event/{EVENT_ID}/event_session/search"
+    query = {"and": [{"field": "event_presentation_li_qry_str", "op": "like", "value": "%"}]}
+    resp = requests.post(search_url_default, headers=headers, json=query)
+    print_result("Search on default view correctly rejects qry_str field (expect 400)", resp.status_code == 400,
+                 f"(status={resp.status_code})")
+
 if __name__ == "__main__":
     print(f"Starting Consolidated Search Engine E2E Suite")
     print(f"Target: {API_BASE}")
@@ -74,6 +128,7 @@ if __name__ == "__main__":
         test_registry_fields()
         test_nested_search()
         test_extra_filters()
+        test_event_session_qry_str_fields()
     except Exception as e:
         print(f"💥 Suite Error: {e}")
 

tests/e2e/test_e2e_v3_user_action_routes.py

@@ -0,0 +1,498 @@
+"""
+E2E Tests: V3 User Action Routes (app/routers/api_v3_actions_user.py)
+======================================================================
+Covers the new V3 action endpoints under /v3/action/user/:
+  - POST  /v3/action/user/authenticate
+  - POST  /v3/action/user/verify_password
+  - POST  /v3/action/user/{user_id}/change_password
+  - GET   /v3/action/user/{user_id}/new_auth_key
+  - GET   /v3/action/user/{user_id}/email_auth_key_url
+
+Setup: creates a temporary test user via V3 CRUD; tears down on completion.
+
+Run from project root:
+    ./environment/bin/python3 tests/e2e/test_e2e_v3_user_action_routes.py
+"""
+
+import os
+import sys
+import time
+import requests
+
+sys.path.append(os.getcwd())
+
+# --- Configuration ---
+API_ROOT = "https://dev-api.oneskyit.com"
+API_KEY = "PMM4n50teUCaOMMTN8qOJA"
+ACCOUNT_ID = "_XY7DXtc9MY"  # One Sky IT Demo account
+
+V3_HEADERS = {
+    "x-aether-api-key": API_KEY,
+    "x-account-id": ACCOUNT_ID,
+}
+
+TEST_PASSWORD = "TestAction1234!"   # >= 10 chars
+NEW_PASSWORD  = "NewAction5678!"    # used after change_password tests
+
+# Populated during setup
+_test_user_id = None    # Vision ID (random string)
+_test_username = None
+_test_email    = None
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+def print_result(label, success, message=""):
+    status = "✅ PASS" if success else "❌ FAIL"
+    print(f"  [{status}] {label}" + (f" — {message}" if message else ""))
+
+
+def assert_vision_id(obj, field_name="user_id"):
+    """Returns True if field is a non-empty string of length 11–22 (Vision ID)."""
+    val = obj.get(field_name) if isinstance(obj, dict) else None
+    return isinstance(val, str) and 11 <= len(val) <= 22
+
+
+# ---------------------------------------------------------------------------
+# Setup / Teardown
+# ---------------------------------------------------------------------------
+
+def setup_test_user():
+    """Create a temporary test user via V3 CRUD. Returns the Vision ID or None."""
+    global _test_user_id, _test_username, _test_email
+
+    ts = int(time.time())
+    _test_username = f"test_v3act_e2e_{ts}"
+    _test_email    = f"test_v3act_e2e_{ts}@test.invalid"
+
+    payload = {
+        "account_id":    ACCOUNT_ID,
+        "username":      _test_username,
+        "name":          "E2E V3 Action Test User",
+        "email":         _test_email,
+        "new_password":  TEST_PASSWORD,
+        "enable":        True,
+        "allow_auth_key": True,
+    }
+
+    resp = requests.post(f"{API_ROOT}/v3/crud/user/", json=payload, headers=V3_HEADERS)
+
+    if resp.status_code != 200:
+        print(f"  [SETUP ❌] Failed to create test user — HTTP {resp.status_code}")
+        print(f"             {resp.text[:300]}")
+        return None
+
+    data = resp.json().get("data", {})
+    _test_user_id = data.get("user_id") or data.get("id")
+
+    if not _test_user_id:
+        print(f"  [SETUP ❌] Test user created but no Vision ID returned: {data}")
+        return None
+
+    print(f"  [SETUP ✅] Test user created — user_id={_test_user_id}  username={_test_username}")
+    return _test_user_id
+
+
+def teardown_test_user(user_id):
+    """Delete the test user via V3 CRUD."""
+    if not user_id:
+        return
+    resp = requests.delete(f"{API_ROOT}/v3/crud/user/{user_id}", headers=V3_HEADERS)
+    if resp.status_code == 200:
+        print(f"  [TEARDOWN ✅] Test user deleted — user_id={user_id}")
+    else:
+        print(f"  [TEARDOWN ❌] Failed to delete test user — HTTP {resp.status_code} {resp.text[:200]}")
+
+
+# ---------------------------------------------------------------------------
+# authenticate
+# ---------------------------------------------------------------------------
+
+def test_authenticate_username_password():
+    """POST /v3/action/user/authenticate — valid username + password."""
+    print("\n--- authenticate ---")
+
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/authenticate",
+        json={"username": _test_username, "password": TEST_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    data = resp.json().get("data", {})
+    vision_ok = assert_vision_id(data, "user_id")
+    success = resp.status_code == 200 and vision_ok
+    print_result("Valid username+password", success,
+                 f"HTTP {resp.status_code}" + ("" if vision_ok else " — missing Vision ID"))
+    return success
+
+
+def test_authenticate_wrong_password():
+    """POST /v3/action/user/authenticate — wrong password → 403."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/authenticate",
+        json={"username": _test_username, "password": "WrongPassword999!"},
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 403
+    print_result("Wrong password → 403", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_authenticate_unknown_user():
+    """POST /v3/action/user/authenticate — unknown username → 404."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/authenticate",
+        json={"username": "no_such_user_xyzzy", "password": TEST_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 404
+    print_result("Unknown username → 404", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_authenticate_missing_fields():
+    """POST /v3/action/user/authenticate — no credentials → 400."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/authenticate",
+        json={"username": _test_username},  # password missing
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 400
+    print_result("Missing credentials → 400", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_authenticate_auth_key_flow():
+    """
+    Full auth-key flow:
+      1. GET new_auth_key  → get a key
+      2. POST authenticate with user_id + auth_key → success
+      3. POST authenticate again with same key → 404 (key cleared)
+    """
+    print("\n--- authenticate (auth_key flow) ---")
+
+    # Step 1: generate key
+    resp1 = requests.get(
+        f"{API_ROOT}/v3/action/user/{_test_user_id}/new_auth_key",
+        headers=V3_HEADERS,
+    )
+    if resp1.status_code != 200:
+        print_result("Auth key flow — generate key", False, f"HTTP {resp1.status_code}")
+        return False
+    key = resp1.json().get("data", {}).get("auth_key")
+    if not key:
+        print_result("Auth key flow — generate key", False, "No auth_key in response")
+        return False
+
+    # Step 2: authenticate with key
+    resp2 = requests.post(
+        f"{API_ROOT}/v3/action/user/authenticate",
+        json={"user_id": _test_user_id, "auth_key": key},
+        headers=V3_HEADERS,
+    )
+    data2 = resp2.json().get("data", {})
+    step2_ok = resp2.status_code == 200 and assert_vision_id(data2, "user_id")
+    print_result("Auth key flow — first use succeeds", step2_ok,
+                 f"HTTP {resp2.status_code}")
+
+    # Step 3: replay must fail (key is cleared)
+    resp3 = requests.post(
+        f"{API_ROOT}/v3/action/user/authenticate",
+        json={"user_id": _test_user_id, "auth_key": key},
+        headers=V3_HEADERS,
+    )
+    step3_ok = resp3.status_code == 404
+    print_result("Auth key flow — replay → 404 (one-time-use)", step3_ok,
+                 f"HTTP {resp3.status_code}")
+
+    return step2_ok and step3_ok
+
+
+# ---------------------------------------------------------------------------
+# verify_password
+# ---------------------------------------------------------------------------
+
+def test_verify_password_by_user_id():
+    """POST /v3/action/user/verify_password — correct password by user_id."""
+    print("\n--- verify_password ---")
+
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/verify_password",
+        json={"user_id": _test_user_id, "current_password": TEST_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    data = resp.json().get("data")
+    # Primitive True is wrapped as {"result": True}
+    result = data.get("result") if isinstance(data, dict) else data
+    success = resp.status_code == 200 and result is True
+    print_result("Correct password by user_id → True", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_verify_password_by_username():
+    """POST /v3/action/user/verify_password — correct password by username."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/verify_password",
+        json={"username": _test_username, "current_password": TEST_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    data = resp.json().get("data")
+    result = data.get("result") if isinstance(data, dict) else data
+    success = resp.status_code == 200 and result is True
+    print_result("Correct password by username → True", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_verify_password_wrong():
+    """POST /v3/action/user/verify_password — wrong password → 403."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/verify_password",
+        json={"user_id": _test_user_id, "current_password": "WrongPassword999!"},
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 403
+    print_result("Wrong password → 403", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_verify_password_no_identifier():
+    """POST /v3/action/user/verify_password — no user_id or username → 400."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/verify_password",
+        json={"current_password": TEST_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 400
+    print_result("No identifier → 400", success, f"HTTP {resp.status_code}")
+    return success
+
+
+# ---------------------------------------------------------------------------
+# change_password
+# ---------------------------------------------------------------------------
+
+def test_change_password_no_verification():
+    """POST /v3/action/user/{id}/change_password — no current_password (admin reset)."""
+    print("\n--- change_password ---")
+
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/{_test_user_id}/change_password",
+        json={"new_password": NEW_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    data = resp.json().get("data")
+    result = data.get("result") if isinstance(data, dict) else data
+    success = resp.status_code == 200 and result is True
+    print_result("Change password (no verification)", success, f"HTTP {resp.status_code}")
+
+    # Verify the new password works
+    resp2 = requests.post(
+        f"{API_ROOT}/v3/action/user/verify_password",
+        json={"user_id": _test_user_id, "current_password": NEW_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    data2 = resp2.json().get("data")
+    r2 = data2.get("result") if isinstance(data2, dict) else data2
+    verify_ok = resp2.status_code == 200 and r2 is True
+    print_result("New password accepted by verify_password", verify_ok,
+                 f"HTTP {resp2.status_code}")
+
+    return success and verify_ok
+
+
+def test_change_password_with_verification():
+    """POST /v3/action/user/{id}/change_password — with correct current_password."""
+    # Password is currently NEW_PASSWORD (set by previous test)
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/{_test_user_id}/change_password",
+        json={"current_password": NEW_PASSWORD, "new_password": TEST_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    data = resp.json().get("data")
+    result = data.get("result") if isinstance(data, dict) else data
+    success = resp.status_code == 200 and result is True
+    print_result("Change password with correct current_password", success,
+                 f"HTTP {resp.status_code}")
+    return success
+
+
+def test_change_password_wrong_current():
+    """POST /v3/action/user/{id}/change_password — wrong current_password → 403."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/{_test_user_id}/change_password",
+        json={"current_password": "WrongPassword999!", "new_password": NEW_PASSWORD},
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 403
+    print_result("Wrong current_password → 403", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_change_password_too_short():
+    """POST /v3/action/user/{id}/change_password — new_password < 10 chars → 422."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/{_test_user_id}/change_password",
+        json={"new_password": "short"},
+        headers=V3_HEADERS,
+    )
+    # Pydantic validation rejects min_length constraint with 422 Unprocessable Entity
+    success = resp.status_code == 422
+    print_result("new_password too short → 422", success, f"HTTP {resp.status_code}")
+    return success
+
+
+def test_change_password_bad_user():
+    """POST /v3/action/user/{id}/change_password — invalid user_id → 404."""
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/AAAAAAAAAAA/change_password",
+        json={"new_password": "ValidPassword123!"},
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 404
+    print_result("Invalid user_id → 404", success, f"HTTP {resp.status_code}")
+    return success
+
+
+# ---------------------------------------------------------------------------
+# new_auth_key
+# ---------------------------------------------------------------------------
+
+def test_new_auth_key():
+    """GET /v3/action/user/{user_id}/new_auth_key — generates and returns key."""
+    print("\n--- new_auth_key ---")
+
+    resp = requests.get(
+        f"{API_ROOT}/v3/action/user/{_test_user_id}/new_auth_key",
+        headers=V3_HEADERS,
+    )
+    data = resp.json().get("data", {})
+    key = data.get("auth_key") if isinstance(data, dict) else None
+    success = resp.status_code == 200 and isinstance(key, str) and len(key) >= 11
+    print_result("Returns new auth_key string", success,
+                 f"HTTP {resp.status_code}" + (f" key={key!r}" if success else ""))
+    return success
+
+
+def test_new_auth_key_bad_user():
+    """GET /v3/action/user/{user_id}/new_auth_key — invalid user → 404."""
+    resp = requests.get(
+        f"{API_ROOT}/v3/action/user/AAAAAAAAAAA/new_auth_key",
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 404
+    print_result("Invalid user_id → 404", success, f"HTTP {resp.status_code}")
+    return success
+
+
+# ---------------------------------------------------------------------------
+# email_auth_key_url
+# ---------------------------------------------------------------------------
+
+def test_email_auth_key_url():
+    """GET /v3/action/user/{user_id}/email_auth_key_url — sends or fails gracefully."""
+    print("\n--- email_auth_key_url ---")
+
+    resp = requests.get(
+        f"{API_ROOT}/v3/action/user/{_test_user_id}/email_auth_key_url",
+        params={"root_url": "https://test.invalid/login"},
+        headers=V3_HEADERS,
+    )
+    # 200 = email sent; 500 = delivery failed (.invalid domain) — both are acceptable.
+    success = resp.status_code in (200, 500)
+    print_result(
+        "email_auth_key_url (200=sent, 500=delivery failed — both OK for .invalid domain)",
+        success, f"HTTP {resp.status_code}"
+    )
+    return success
+
+
+def test_email_auth_key_url_bad_user():
+    """GET /v3/action/user/{user_id}/email_auth_key_url — invalid user → 404."""
+    resp = requests.get(
+        f"{API_ROOT}/v3/action/user/AAAAAAAAAAA/email_auth_key_url",
+        params={"root_url": "https://test.invalid/login"},
+        headers=V3_HEADERS,
+    )
+    success = resp.status_code == 404
+    print_result("Invalid user_id → 404", success, f"HTTP {resp.status_code}")
+    return success
+
+
+# ---------------------------------------------------------------------------
+# Auth guard checks
+# ---------------------------------------------------------------------------
+
+def test_no_api_key():
+    """All V3 action endpoints require x-aether-api-key — missing → 403."""
+    print("\n--- auth guards ---")
+
+    resp = requests.post(
+        f"{API_ROOT}/v3/action/user/authenticate",
+        json={"username": _test_username, "password": TEST_PASSWORD},
+        headers={"x-account-id": ACCOUNT_ID},  # no API key
+    )
+    success = resp.status_code == 403
+    print_result("No API key → 403", success, f"HTTP {resp.status_code}")
+    return success
+
+
+# ---------------------------------------------------------------------------
+# Runner
+# ---------------------------------------------------------------------------
+
+def run_suite():
+    start = time.time()
+    print("=" * 60)
+    print("E2E: V3 User Action Routes")
+    print("=" * 60)
+
+    if not setup_test_user():
+        print("\n[ABORT] Setup failed — cannot run tests.\n")
+        return
+
+    results = []
+
+    # authenticate
+    results.append(test_authenticate_username_password())
+    results.append(test_authenticate_wrong_password())
+    results.append(test_authenticate_unknown_user())
+    results.append(test_authenticate_missing_fields())
+    results.append(test_authenticate_auth_key_flow())
+
+    # verify_password
+    results.append(test_verify_password_by_user_id())
+    results.append(test_verify_password_by_username())
+    results.append(test_verify_password_wrong())
+    results.append(test_verify_password_no_identifier())
+
+    # change_password (order matters — each test assumes the password left by the previous)
+    results.append(test_change_password_no_verification())    # TEST → NEW
+    results.append(test_change_password_with_verification())  # NEW → TEST
+    results.append(test_change_password_wrong_current())      # bad → 403 (no change)
+    results.append(test_change_password_too_short())          # bad → 422
+    results.append(test_change_password_bad_user())           # 404
+
+    # new_auth_key
+    results.append(test_new_auth_key())
+    results.append(test_new_auth_key_bad_user())
+
+    # email_auth_key_url
+    results.append(test_email_auth_key_url())
+    results.append(test_email_auth_key_url_bad_user())
+
+    # auth guards
+    results.append(test_no_api_key())
+
+    teardown_test_user(_test_user_id)
+
+    elapsed = time.time() - start
+    passed  = sum(1 for r in results if r)
+    total   = len(results)
+    print(f"\n{'=' * 60}")
+    print(f"Results: {passed}/{total} passed  ({elapsed:.2f}s)")
+    print("=" * 60)
+
+
+if __name__ == "__main__":
+    run_suite()

tests/e2e/test_e2e_v3_user_auth_routes.py

@@ -0,0 +1,511 @@
+"""
+E2E Tests: User Auth Routes (app/routers/user.py)
+==================================================
+Covers the active legacy user routes that are marked for migration to V3:
+  - PATCH  /user/{user_id}/change_password
+  - GET    /user/{user_id}/new_auth_key
+  - GET    /user/authenticate              ← KNOWN BUG: decorator accidentally commented out
+  - POST   /user/verify_password
+  - GET    /user/lookup
+  - GET    /user/lookup_email
+  - GET    /user/lookup_username
+  - GET    /user/{user_id}/email_auth_key_url
+
+Run from project root:
+    ./environment/bin/python3 tests/e2e/test_e2e_v3_user_auth_routes.py
+"""
+
+import os
+import sys
+import time
+import requests
+
+sys.path.append(os.getcwd())
+
+# --- Configuration ---
+API_ROOT = "https://dev-api.oneskyit.com"
+API_KEY = "PMM4n50teUCaOMMTN8qOJA"
+ACCOUNT_ID = "_XY7DXtc9MY"  # One Sky IT Demo account
+
+# Standard headers for V3 CRUD (create/delete the test user)
+V3_HEADERS = {
+    "x-aether-api-key": API_KEY,
+    "x-account-id": ACCOUNT_ID,
+}
+# Legacy routes use the same headers (Common_Route_Params reads x-account-id)
+LEGACY_HEADERS = V3_HEADERS
+
+TEST_PASSWORD = "TestAuth1234!"   # >= 10 chars
+NEW_PASSWORD  = "NewTestPwd5678!" # used after change_password
+
+# Populated during setup
+_test_user_id = None       # Vision ID (random string)
+_test_username = None
+_test_email = None
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+def print_result(label, success, message=""):
+    status = "✅ PASS" if success else "❌ FAIL"
+    print(f"  [{status}] {label}" + (f" — {message}" if message else ""))
+
+
+def assert_vision_id(obj_dict, field_name="user_id"):
+    """Returns True if the given field is a string (Vision ID), not an int."""
+    val = obj_dict.get(field_name)
+    return isinstance(val, str) and 11 <= len(val) <= 22
+
+
+# ---------------------------------------------------------------------------
+# Setup / Teardown
+# ---------------------------------------------------------------------------
+
+def setup_test_user():
+    """Create a temporary test user via V3 CRUD. Returns the Vision ID or None."""
+    global _test_user_id, _test_username, _test_email
+
+    ts = int(time.time())
+    _test_username = f"test_auth_e2e_{ts}"
+    _test_email = f"test_auth_e2e_{ts}@test.invalid"
+
+    payload = {
+        "account_id": ACCOUNT_ID,
+        "username": _test_username,
+        "name": "E2E Auth Test User",
+        "email": _test_email,
+        "new_password": TEST_PASSWORD,
+        "enable": True,
+        "allow_auth_key": True,  # needed for new_auth_key / email_auth_key_url tests
+    }
+
+    resp = requests.post(
+        f"{API_ROOT}/v3/crud/user/",
+        json=payload,
+        headers=V3_HEADERS,
+    )
+
+    if resp.status_code != 200:
+        print(f"  [SETUP ❌] Failed to create test user — HTTP {resp.status_code}")
+        print(f"             {resp.text[:300]}")
+        return None
+
+    data = resp.json().get("data", {})
+    _test_user_id = data.get("user_id") or data.get("id")
+
+    if not _test_user_id:
+        print(f"  [SETUP ❌] Test user created but no Vision ID returned: {data}")
+        return None
+
+    print(f"  [SETUP ✅] Test user created — user_id={_test_user_id}  username={_test_username}")
+    return _test_user_id
+
+
+def teardown_test_user(user_id):
+    """Delete the test user via V3 CRUD."""
+    if not user_id:
+        return
+    resp = requests.delete(
+        f"{API_ROOT}/v3/crud/user/{user_id}",
+        headers=V3_HEADERS,
+    )
+    if resp.status_code == 200:
+        print(f"  [TEARDOWN ✅] Test user deleted — user_id={user_id}")
+    else:
+        print(f"  [TEARDOWN ❌] Failed to delete test user — HTTP {resp.status_code} {resp.text[:200]}")
+
+
+# ---------------------------------------------------------------------------
+# change_password
+# ---------------------------------------------------------------------------
+
+def test_change_password():
+    """PATCH /user/{user_id}/change_password — valid new password."""
+    print("\n--- change_password ---")
+
+    resp = requests.patch(
+        f"{API_ROOT}/user/{_test_user_id}/change_password",
+        json={"password": NEW_PASSWORD},
+        headers=LEGACY_HEADERS,
+    )
+    success = resp.status_code == 200 and resp.json().get("data") is not False
+    print_result("Valid password change", success,
+                 f"HTTP {resp.status_code}" if not success else "")
+    return success
+
+
+def test_change_password_too_short():
+    """PATCH /user/{user_id}/change_password — password < 10 chars → 400."""
+    resp = requests.patch(
+        f"{API_ROOT}/user/{_test_user_id}/change_password",
+        json={"password": "short"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Short password rejected (400)", resp.status_code == 400,
+                 f"HTTP {resp.status_code}")
+
+
+def test_change_password_missing_field():
+    """PATCH /user/{user_id}/change_password — no password field → 400."""
+    resp = requests.patch(
+        f"{API_ROOT}/user/{_test_user_id}/change_password",
+        json={"not_password": "whatever"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Missing password field rejected (400)", resp.status_code == 400,
+                 f"HTTP {resp.status_code}")
+
+
+def test_change_password_invalid_user():
+    """PATCH /user/{invalid_id}/change_password → 404."""
+    resp = requests.patch(
+        f"{API_ROOT}/user/NotARealUserID99/change_password",
+        json={"password": "ValidPassword123!"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Invalid user_id rejected (404)", resp.status_code == 404,
+                 f"HTTP {resp.status_code}")
+
+
+# ---------------------------------------------------------------------------
+# new_auth_key
+# ---------------------------------------------------------------------------
+
+def test_new_auth_key():
+    """GET /user/{user_id}/new_auth_key — generates and returns a new key."""
+    print("\n--- new_auth_key ---")
+    resp = requests.get(
+        f"{API_ROOT}/user/{_test_user_id}/new_auth_key",
+        headers=LEGACY_HEADERS,
+    )
+    data = resp.json().get("data", {})
+    has_key = isinstance(data, dict) and bool(data.get("auth_key"))
+    print_result("New auth_key generated", resp.status_code == 200 and has_key,
+                 f"HTTP {resp.status_code}")
+    return data.get("auth_key") if has_key else None
+
+
+def test_new_auth_key_invalid_user():
+    """GET /user/{invalid_id}/new_auth_key → 404."""
+    resp = requests.get(
+        f"{API_ROOT}/user/NotARealUserID99/new_auth_key",
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Invalid user_id rejected (404)", resp.status_code == 404,
+                 f"HTTP {resp.status_code}")
+
+
+# ---------------------------------------------------------------------------
+# verify_password
+# ---------------------------------------------------------------------------
+
+def _verify_result(resp) -> bool:
+    """Extract the boolean result from a legacy mk_resp response.
+    Primitive data is wrapped as {"data": {"result": value}}.
+    """
+    data = resp.json().get("data", {})
+    if isinstance(data, dict):
+        return data.get("result")
+    return data
+
+
+def test_verify_password_by_username_correct():
+    """POST /user/verify_password — correct password via username → result True."""
+    print("\n--- verify_password ---")
+    resp = requests.post(
+        f"{API_ROOT}/user/verify_password",
+        json={"username": _test_username, "current_password": NEW_PASSWORD},
+        headers=LEGACY_HEADERS,
+    )
+    result = _verify_result(resp)
+    success = resp.status_code == 200 and result is True
+    print_result("Correct password (username path)", success,
+                 f"HTTP {resp.status_code} result={result}")
+
+
+def test_verify_password_by_username_wrong():
+    """POST /user/verify_password — wrong password → result not True."""
+    resp = requests.post(
+        f"{API_ROOT}/user/verify_password",
+        json={"username": _test_username, "current_password": "WrongPassword999!"},
+        headers=LEGACY_HEADERS,
+    )
+    result = _verify_result(resp)
+    success = result is not True
+    print_result("Wrong password rejected", success,
+                 f"HTTP {resp.status_code} result={result}")
+
+
+def test_verify_password_by_user_id():
+    """
+    POST /user/verify_password — correct password via Vision ID ('id' field).
+
+    The handler reads user_obj.id (User_Base Vision ID field). Send the
+    Vision ID as 'id' in the request body.
+    """
+    resp = requests.post(
+        f"{API_ROOT}/user/verify_password",
+        json={"id": _test_user_id, "current_password": NEW_PASSWORD},
+        headers=LEGACY_HEADERS,
+    )
+    result = _verify_result(resp)
+    success = resp.status_code == 200 and result is True
+    print_result("Correct password (Vision ID / 'id' path)", success,
+                 f"HTTP {resp.status_code} result={result}")
+
+
+def test_verify_password_missing_fields():
+    """POST /user/verify_password — no user_id or username → 400."""
+    resp = requests.post(
+        f"{API_ROOT}/user/verify_password",
+        json={"current_password": NEW_PASSWORD},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Missing user fields rejected (400)", resp.status_code == 400,
+                 f"HTTP {resp.status_code}")
+
+
+# ---------------------------------------------------------------------------
+# lookup, lookup_email, lookup_username
+# ---------------------------------------------------------------------------
+
+def test_lookup_by_account():
+    """GET /user/lookup?for_obj_type=account&for_obj_id={account_id} — returns user list."""
+    print("\n--- lookup ---")
+    resp = requests.get(
+        f"{API_ROOT}/user/lookup",
+        params={"for_obj_type": "account", "for_obj_id": ACCOUNT_ID},
+        headers=LEGACY_HEADERS,
+    )
+    data = resp.json().get("data")
+    success = resp.status_code == 200 and isinstance(data, list) and len(data) > 0
+    print_result("Lookup by account (list)", success, f"HTTP {resp.status_code} count={len(data) if isinstance(data, list) else 'n/a'}")
+
+    # Vision ID check on first result
+    if success and isinstance(data, list) and data:
+        has_vision_id = assert_vision_id(data[0], "user_id")
+        print_result("Vision ID compliance (user_id is string)", has_vision_id,
+                     f"user_id={data[0].get('user_id')!r}")
+
+
+def test_lookup_by_person_invalid():
+    """GET /user/lookup?for_obj_type=person&for_obj_id={bad_id} → 404."""
+    resp = requests.get(
+        f"{API_ROOT}/user/lookup",
+        params={"for_obj_type": "person", "for_obj_id": "NotARealUID999"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Invalid person ID rejected (404)", resp.status_code == 404,
+                 f"HTTP {resp.status_code}")
+
+
+def test_lookup_bad_obj_type():
+    """GET /user/lookup?for_obj_type=invalid → 404.
+    The redis lookup for for_obj_id against an unknown table returns None,
+    which triggers the 404 before the 400 type-check is reached.
+    """
+    resp = requests.get(
+        f"{API_ROOT}/user/lookup",
+        params={"for_obj_type": "invoice", "for_obj_id": ACCOUNT_ID},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Unsupported for_obj_type returns 404", resp.status_code == 404,
+                 f"HTTP {resp.status_code}")
+
+
+def test_lookup_email():
+    """GET /user/lookup_email?email={email} — finds the test user."""
+    print("\n--- lookup_email ---")
+    resp = requests.get(
+        f"{API_ROOT}/user/lookup_email",
+        params={"email": _test_email},
+        headers=LEGACY_HEADERS,
+    )
+    data = resp.json().get("data")
+    found = (
+        resp.status_code == 200
+        and isinstance(data, dict)
+        and data.get("email") == _test_email
+    )
+    print_result("Lookup by email (found)", found, f"HTTP {resp.status_code}")
+
+    if found:
+        has_vision_id = assert_vision_id(data, "user_id")
+        print_result("Vision ID compliance (user_id is string)", has_vision_id,
+                     f"user_id={data.get('user_id')!r}")
+
+
+def test_lookup_email_not_found():
+    """GET /user/lookup_email?email={nonexistent} → 404."""
+    resp = requests.get(
+        f"{API_ROOT}/user/lookup_email",
+        params={"email": "nobody_at_all@test.invalid"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Nonexistent email → 404", resp.status_code == 404,
+                 f"HTTP {resp.status_code}")
+
+
+def test_lookup_username():
+    """GET /user/lookup_username?username={username} — finds the test user."""
+    print("\n--- lookup_username ---")
+    resp = requests.get(
+        f"{API_ROOT}/user/lookup_username",
+        params={"username": _test_username},
+        headers=LEGACY_HEADERS,
+    )
+    data = resp.json().get("data")
+    found = (
+        resp.status_code == 200
+        and isinstance(data, dict)
+        and data.get("username") == _test_username
+    )
+    print_result("Lookup by username (found)", found, f"HTTP {resp.status_code}")
+
+    if found:
+        has_vision_id = assert_vision_id(data, "user_id")
+        print_result("Vision ID compliance (user_id is string)", has_vision_id,
+                     f"user_id={data.get('user_id')!r}")
+
+
+def test_lookup_username_not_found():
+    """GET /user/lookup_username?username={nonexistent} → 404."""
+    resp = requests.get(
+        f"{API_ROOT}/user/lookup_username",
+        params={"username": "no_such_user_xyz_99999"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Nonexistent username → 404", resp.status_code == 404,
+                 f"HTTP {resp.status_code}")
+
+
+# ---------------------------------------------------------------------------
+# email_auth_key_url
+# ---------------------------------------------------------------------------
+
+def test_email_auth_key_url():
+    """
+    GET /user/{user_id}/email_auth_key_url — generates auth key and sends email.
+
+    NOTE: The test user email uses '@test.invalid' domain, so actual mail
+    delivery will fail. This test verifies the route responds correctly;
+    expect HTTP 500 if the mail server rejects the send. The auth key IS
+    generated and stored regardless of email success.
+    """
+    print("\n--- email_auth_key_url ---")
+    resp = requests.get(
+        f"{API_ROOT}/user/{_test_user_id}/email_auth_key_url",
+        params={"root_url": "https://dev-app.oneskyit.com"},
+        headers=LEGACY_HEADERS,
+    )
+    # 200 = email sent; 500 = route hit but email delivery failed (acceptable for .invalid)
+    route_hit = resp.status_code in [200, 500]
+    print_result("Route reachable", route_hit, f"HTTP {resp.status_code}"
+                 + (" (email delivery failed — expected for .invalid domain)" if resp.status_code == 500 else ""))
+
+
+def test_email_auth_key_url_invalid_user():
+    """GET /user/{invalid_id}/email_auth_key_url → 404."""
+    resp = requests.get(
+        f"{API_ROOT}/user/NotARealUserID99/email_auth_key_url",
+        params={"root_url": "https://dev-app.oneskyit.com"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Invalid user_id rejected (404)", resp.status_code == 404,
+                 f"HTTP {resp.status_code}")
+
+
+# ---------------------------------------------------------------------------
+# BUG VERIFICATION: user_authenticate route
+# ---------------------------------------------------------------------------
+
+def test_authenticate():
+    """
+    GET /user/authenticate — authenticate with username + password.
+
+    Note: The @router.get() decorator was accidentally commented out in a
+    prior version (user.py line 226). That bug has been fixed. This test
+    verifies the route is reachable and returns user data on success.
+    """
+    print("\n--- authenticate ---")
+    resp = requests.get(
+        f"{API_ROOT}/user/authenticate",
+        params={"username": _test_username, "password": NEW_PASSWORD},
+        headers=LEGACY_HEADERS,
+    )
+    data = resp.json().get("data")
+    success = resp.status_code == 200 and isinstance(data, dict) and bool(data.get("user_id") or data.get("id"))
+    print_result("authenticate (username+password)", success, f"HTTP {resp.status_code}")
+
+    if success:
+        has_vision_id = assert_vision_id(data, "user_id")
+        print_result("Vision ID compliance (user_id is string)", has_vision_id,
+                     f"user_id={data.get('user_id')!r}")
+
+    # Wrong password should be rejected
+    resp2 = requests.get(
+        f"{API_ROOT}/user/authenticate",
+        params={"username": _test_username, "password": "WrongPassword000!"},
+        headers=LEGACY_HEADERS,
+    )
+    print_result("Wrong password rejected", resp2.status_code in [200, 404] and resp2.json().get("data") is not True,
+                 f"HTTP {resp2.status_code}")
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+
+if __name__ == "__main__":
+    suite_start = time.time()
+    print("=" * 60)
+    print("User Auth Routes E2E Test Suite")
+    print(f"API: {API_ROOT}")
+    print("=" * 60)
+
+    # --- Setup ---
+    print("\n[Setup]")
+    user_id = setup_test_user()
+    if not user_id:
+        print("\n❌ Setup failed — cannot run tests. Aborting.")
+        sys.exit(1)
+
+    # --- Tests ---
+    test_change_password()
+    test_change_password_too_short()
+    test_change_password_missing_field()
+    test_change_password_invalid_user()
+
+    test_new_auth_key()
+    test_new_auth_key_invalid_user()
+
+    test_verify_password_by_username_correct()
+    test_verify_password_by_username_wrong()
+    test_verify_password_by_user_id()
+    test_verify_password_missing_fields()
+
+    test_lookup_by_account()
+    test_lookup_by_person_invalid()
+    test_lookup_bad_obj_type()
+
+    test_lookup_email()
+    test_lookup_email_not_found()
+
+    test_lookup_username()
+    test_lookup_username_not_found()
+
+    test_email_auth_key_url()
+    test_email_auth_key_url_invalid_user()
+
+    test_authenticate()
+
+    # --- Teardown ---
+    print("\n[Teardown]")
+    teardown_test_user(user_id)
+
+    elapsed = time.time() - suite_start
+    print(f"\n{'=' * 60}")
+    print(f"Suite completed in {elapsed:.2f}s")
+    print("=" * 60)

tests/e2e/verify_v3_security_hardening.py

@@ -75,6 +75,65 @@ def test_restored_access():
     success = (resp.status_code == 200)
     print_result("Access Restored (Journal with Header)", success, f"- Status: {resp.status_code}")
 
+
+def test_site_domain_access_key():
+    """
+    Verify site_domain lookup respects access_key.
+
+    The frontend reads the 'key' query param from the browser URL and forwards it
+    as 'access_key' in the POST body.  No key means a public domain is expected.
+
+    Valid (should return a result):
+        https://dev-demo.oneskyit.com                              — public, no key needed
+        http://idaa.localhost:5173/?key=restricted                 — correct key
+        https://dev-idaa.oneskyit.com/?key=restricted-access       — correct key
+        https://sk-idaa.oneskyit.com/?key=8VTOJ0X5hvT6JdiTJsGEzQ  — correct key
+
+    Invalid (should return empty):
+        http://idaa.localhost:5173/                                — key required, none given
+        http://idaa.localhost:5173/?key=bad-key-example            — wrong key
+        https://dev-idaa.oneskyit.com/                             — key required, none given
+        https://dev-idaa.oneskyit.com/?key=                        — empty key treated as none
+        https://dev-idaa.oneskyit.com/?key=any-wrong-key           — wrong key
+        https://sk-idaa.oneskyit.com/                              — key required, none given
+        https://sk-idaa.oneskyit.com/?key=another-bad-key-example  — wrong key
+    """
+    print("\n--- Test 5: Site Domain Access Key Behavior ---")
+    url = f"{API_ROOT}/v3/crud/site_domain/search"
+    headers = {"x-aether-api-key": API_KEY}
+
+    cases = [
+        # (fqdn, key, should_pass, label)
+        # --- valid ---
+        ("dev-demo.oneskyit.com",   None,                      True,  "public domain, no key"),
+        ("idaa.localhost:5173",     "restricted",              True,  "correct key"),
+        ("dev-idaa.oneskyit.com",   "restricted-access",       True,  "correct key"),
+        ("sk-idaa.oneskyit.com",    "8VTOJ0X5hvT6JdiTJsGEzQ", True,  "correct key"),
+        # --- invalid ---
+        ("idaa.localhost:5173",     None,                      False, "key required, none given"),
+        ("idaa.localhost:5173",     "bad-key-example",         False, "wrong key"),
+        ("dev-idaa.oneskyit.com",   None,                      False, "key required, none given"),
+        ("dev-idaa.oneskyit.com",   "",                        False, "empty key treated as none"),
+        ("dev-idaa.oneskyit.com",   "any-wrong-key",           False, "wrong key"),
+        ("sk-idaa.oneskyit.com",    None,                      False, "key required, none given"),
+        ("sk-idaa.oneskyit.com",    "another-bad-key-example", False, "wrong key"),
+    ]
+
+    for fqdn, key, should_pass, label in cases:
+        payload = {"and": [{"field": "fqdn", "op": "eq", "value": fqdn}]}
+        # Omit access_key entirely when None (no key in URL); send it when present (even if empty)
+        if key is not None:
+            payload["and"].append({"field": "access_key", "op": "eq", "value": key})
+
+        try:
+            resp = requests.post(url, headers=headers, json=payload)
+            data = resp.json().get('data', []) if resp.status_code == 200 else []
+            success = (resp.status_code == 200 and ((len(data) > 0) == should_pass))
+            tag = "VALID  " if should_pass else "INVALID"
+            print_result(f"[{tag}] {fqdn} key={key!r:30} ({label})", success, f"- Count: {len(data)}")
+        except Exception as e:
+            print_result(f"[{'VALID  ' if should_pass else 'INVALID'}] {fqdn} key={key!r}", False, f"- Exception: {e}")
+
 if __name__ == "__main__":
     print(f"Starting V3 Security Hardening Verification")
     print(f"Target: {API_ROOT}")
@@ -84,6 +143,7 @@ if __name__ == "__main__":
         test_strict_id_block()
         test_bootstrap_exception()
         test_restored_access()
+        test_site_domain_access_key()
     except Exception as e:
         print(f"\n❌ ERROR during test execution: {e}")
 

tests/tools/stress_list_queries.py

@@ -0,0 +1,152 @@
+"""
+Read-only concurrent stress test against V3 list endpoints.
+Fires N workers each making R sequential requests across a set of
+list endpoints, then prints per-endpoint latency stats and an
+overall error summary.
+
+Usage (from project root):
+    ./environment/bin/python3 tests/tools/stress_list_queries.py
+    ./environment/bin/python3 tests/tools/stress_list_queries.py --workers 20 --requests 10
+    ./environment/bin/python3 tests/tools/stress_list_queries.py --base-url https://api.oneskyit.com --workers 5
+"""
+import argparse
+import math
+import statistics
+import sys
+import time
+from concurrent.futures import ThreadPoolExecutor, as_completed
+
+import requests
+
+DEFAULT_BASE_URL = "https://test-api.oneskyit.com"
+API_KEY          = "nT0jPeiCfxSifkiDZur9jA"
+ACCOUNT_ID       = "_XY7DXtc9MY"  # One Sky IT Demo
+
+HEADERS = {
+    "x-aether-api-key": API_KEY,
+    "x-account-id":     ACCOUNT_ID,
+}
+
+# Read-only list endpoints to hammer. Each is a (label, path) tuple.
+ENDPOINTS = [
+    ("event list",              "/v3/crud/event/"),
+    ("event_session list",      "/v3/crud/event_session/"),
+    ("event_badge list",        "/v3/crud/event_badge/"),
+    ("event_file list",         "/v3/crud/event_file/"),
+    ("person list",             "/v3/crud/person/"),
+    ("journal list",            "/v3/crud/journal/"),
+    ("hosted_file list",        "/v3/crud/hosted_file/"),
+    ("data_store list",         "/v3/crud/data_store/"),
+]
+
+
+def percentile(sorted_times: list[float], pct: float) -> float:
+    """Return the pct-th percentile of a pre-sorted list (0–100)."""
+    if not sorted_times:
+        return 0.0
+    k = (len(sorted_times) - 1) * pct / 100
+    lo, hi = int(math.floor(k)), int(math.ceil(k))
+    return sorted_times[lo] + (sorted_times[hi] - sorted_times[lo]) * (k - lo)
+
+
+def do_request(label: str, url: str, session: requests.Session) -> dict:
+    t0 = time.perf_counter()
+    try:
+        r = session.get(url, headers=HEADERS, timeout=15)
+        elapsed = (time.perf_counter() - t0) * 1000
+        return {"label": label, "status": r.status_code, "ms": elapsed, "error": None}
+    except Exception as e:
+        elapsed = (time.perf_counter() - t0) * 1000
+        return {"label": label, "status": 0, "ms": elapsed, "error": str(e)}
+
+
+def worker(worker_id: int, requests_per_worker: int, base_url: str, limit: int) -> list[dict]:
+    results = []
+    with requests.Session() as session:
+        for _ in range(requests_per_worker):
+            for label, path in ENDPOINTS:
+                url = f"{base_url}{path}?limit={limit}"
+                results.append(do_request(label, url, session))
+    return results
+
+
+def print_result(label, success, message=""):
+    icon = "✅" if success else "❌"
+    suffix = f" — {message}" if message else ""
+    print(f"  [{icon}] {label}{suffix}")
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Concurrent read-only stress test")
+    parser.add_argument("--workers",  type=int, default=10,             help="Concurrent worker threads (default: 10)")
+    parser.add_argument("--requests", type=int, default=5,              help="Requests per worker per endpoint (default: 5)")
+    parser.add_argument("--limit",    type=int, default=20,             help="?limit= param on each list request (default: 20)")
+    parser.add_argument("--base-url", type=str, default=DEFAULT_BASE_URL, help=f"API base URL (default: {DEFAULT_BASE_URL})")
+    args = parser.parse_args()
+
+    total_requests = args.workers * args.requests * len(ENDPOINTS)
+    print(f"\n🔥 Stress Test: {args.workers} workers × {args.requests} rounds × {len(ENDPOINTS)} endpoints = {total_requests} total requests")
+    print(f"   Target: {args.base_url}  limit={args.limit}\n")
+
+    all_results: list[dict] = []
+    suite_start = time.perf_counter()
+
+    with ThreadPoolExecutor(max_workers=args.workers) as pool:
+        futures = [pool.submit(worker, wid, args.requests, args.base_url, args.limit) for wid in range(args.workers)]
+        for f in as_completed(futures):
+            all_results.extend(f.result())
+
+    suite_elapsed = time.perf_counter() - suite_start
+
+    # --- Per-endpoint stats ---
+    print("─" * 60)
+    print(f"{'Endpoint':<35} {'OK':>5} {'ERR':>5} {'p50ms':>7} {'p95ms':>7} {'maxms':>7}")
+    print("─" * 60)
+
+    by_label: dict[str, list[dict]] = {}
+    for r in all_results:
+        by_label.setdefault(r["label"], []).append(r)
+
+    any_fail = False
+    for label, _ in ENDPOINTS:
+        rows = by_label.get(label, [])
+        ok   = [r for r in rows if r["status"] in (200, 201, 404) and not r["error"]]
+        err  = [r for r in rows if r not in ok]
+        times = sorted(r["ms"] for r in ok)
+        p50 = statistics.median(times) if times else 0
+        p95 = percentile(times, 95)
+        mx  = max(times) if times else 0
+        flag = "" if not err else " ⚠"
+        if err:
+            any_fail = True
+        print(f"  {label:<33} {len(ok):>5} {len(err):>5} {p50:>7.0f} {p95:>7.0f} {mx:>7.0f}{flag}")
+
+    print("─" * 60)
+
+    # --- Error detail ---
+    errors = [r for r in all_results if r["error"] or r["status"] not in (200, 201, 404)]
+    if errors:
+        print(f"\n⚠  {len(errors)} errors encountered:")
+        seen = set()
+        for r in errors:
+            key = (r["label"], r["status"], r["error"])
+            if key not in seen:
+                seen.add(key)
+                print(f"   [{r['status']}] {r['label']}: {r['error'] or 'non-2xx/404'}")
+    else:
+        print("\n✅ Zero errors.")
+
+    # --- Overall summary ---
+    all_times = sorted(r["ms"] for r in all_results if not r["error"])
+    rps = total_requests / suite_elapsed
+    print(f"\n🏁 {total_requests} requests in {suite_elapsed:.2f}s  ({rps:.1f} req/s)")
+    if all_times:
+        print(f"   p50={statistics.median(all_times):.0f}ms  "
+              f"p95={percentile(all_times, 95):.0f}ms  "
+              f"max={max(all_times):.0f}ms\n")
+
+    sys.exit(1 if any_fail else 0)
+
+
+if __name__ == "__main__":
+    main()

tests/unit/test_unit_errors.py

@@ -41,6 +41,22 @@ def test_null_error_handling():
     else:
         print("❌ Null error check FAILED.")
 
+def test_1364_schema_mismatch():
+    print("\n--- Testing 1364 Schema Mismatch ---")
+    raw = "(MySQLdb.OperationalError) (1364, \"Field 'account' doesn't have a default value\")"
+    formatted = format_db_error(raw)
+    print(f"Raw: {raw}")
+    print(f"Formatted: {formatted}")
+
+    if (formatted.category == "database_schema"
+            and formatted.code == 1364
+            and "account" in formatted.message
+            and "NOT NULL" in formatted.message):
+        print("✅ 1364 schema mismatch handled correctly.")
+    else:
+        print("❌ 1364 schema mismatch check FAILED.")
+
 if __name__ == "__main__":
     test_error_formatting()
     test_null_error_handling()
+    test_1364_schema_mismatch()

tests/unit/test_unit_idaa_novi_verify.py

@@ -0,0 +1,219 @@
+import sys
+import os
+import json
+from unittest.mock import MagicMock, patch
+
+# Add project root to path
+sys.path.append(os.getcwd())
+
+# Mock low-level deps BEFORE importing the target module.
+# logger_reset must be a passthrough — if it stays a MagicMock the decorator
+# replaces the decorated function with a MagicMock and tests get garbage results.
+mock_lib_general = MagicMock()
+mock_lib_general.logger_reset = lambda f: f
+sys.modules['app.config'] = MagicMock()
+sys.modules['app.lib_general'] = mock_lib_general
+sys.modules['app.db_sql'] = MagicMock()
+sys.modules['app.lib_redis_helpers'] = MagicMock()
+
+from app.methods import idaa_novi_verify_methods as m
+
+
+# ── Helpers ───────────────────────────────────────────────────────────────
+
+def _make_cfg():
+    return {
+        'novi_api_root_url': 'https://www.idaa.org/api',
+        'novi_idaa_api_key': 'dGVzdGtleQ==',
+    }
+
+
+def _novi_resp(email='alice@idaa.org', first='Alice', last='Smith', name=None):
+    d = {'Email': email, 'FirstName': first, 'LastName': last}
+    if name is not None:
+        d['Name'] = name
+    return d
+
+
+def _set_redis(cached_value=None):
+    """Set redis_client on the already-imported module's imported name."""
+    r = MagicMock()
+    r.get.return_value = cached_value
+    sys.modules['app.lib_redis_helpers'].redis_client = r
+    return r
+
+
+# ── Cache hit bypasses Novi ───────────────────────────────────────────────
+
+def test_cache_hit_bypasses_novi():
+    print('--- test_cache_hit_bypasses_novi ---')
+    cached = json.dumps({'status': 200, 'verified': True, 'full_name': 'Bob J.', 'email': 'bob@idaa.org'})
+    redis_mock = _set_redis(cached_value=cached)
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get') as mock_get:
+        result = m.verify_novi_member('some-uuid')
+
+    print('Result:', result)
+    assert result['status'] == 200
+    assert result['full_name'] == 'Bob J.'
+    mock_get.assert_not_called()  # Novi was never contacted
+    print('PASS')
+
+
+# ── Verified 200 ──────────────────────────────────────────────────────────
+
+def test_verified_member_200():
+    print('--- test_verified_member_200 ---')
+    mock_resp = MagicMock()
+    mock_resp.status_code = 200
+    mock_resp.json.return_value = _novi_resp()
+    redis_mock = _set_redis()
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get', return_value=mock_resp):
+        result = m.verify_novi_member('abc-123')
+
+    print('Result:', result)
+    assert result['status'] == 200
+    assert result['verified'] is True
+    assert result['full_name'] == 'Alice S.'
+    assert result['email'] == 'alice@idaa.org'
+    redis_mock.setex.assert_called_once()  # verified result cached
+    print('PASS')
+
+
+# ── Email normalization: space → + ────────────────────────────────────────
+
+def test_email_space_normalization():
+    print('--- test_email_space_normalization ---')
+    mock_resp = MagicMock()
+    mock_resp.status_code = 200
+    mock_resp.json.return_value = _novi_resp(email='alice member@idaa.org')
+    _set_redis()
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get', return_value=mock_resp):
+        result = m.verify_novi_member('abc-123')
+
+    print('Result:', result)
+    assert result['status'] == 200
+    assert result['email'] == 'alice+member@idaa.org'
+    print('PASS')
+
+
+# ── Display name format ───────────────────────────────────────────────────
+
+def test_display_name_format():
+    print('--- test_display_name_format ---')
+    cases = [
+        (_novi_resp(first='Alice', last='Smith'),              'Alice S.'),
+        (_novi_resp(first='Alice', last=''),                   'Alice'),
+        (_novi_resp(first='', last='Smith', name='Dr. Alice'), 'Dr. Alice'),
+        (_novi_resp(first='', last='', name='Dr. Alice'),      'Dr. Alice'),
+        (_novi_resp(first='', last='', name=''),               'Member'),
+    ]
+
+    for novi_data, expected_name in cases:
+        mock_resp = MagicMock()
+        mock_resp.status_code = 200
+        mock_resp.json.return_value = novi_data
+        _set_redis()
+
+        with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+             patch('requests.get', return_value=mock_resp):
+            result = m.verify_novi_member('abc-123')
+
+        assert result['status'] == 200
+        assert result['full_name'] == expected_name, \
+            f"Expected '{expected_name}', got '{result['full_name']}' for input {novi_data}"
+
+    print('All display name cases PASS')
+
+
+# ── Empty-member anti-pattern: Novi 200, no Email ─────────────────────────
+
+def test_empty_member_returns_404():
+    print('--- test_empty_member_returns_404 ---')
+    mock_resp = MagicMock()
+    mock_resp.status_code = 200
+    mock_resp.json.return_value = {}  # Novi 200 with no identity data
+    redis_mock = _set_redis()
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get', return_value=mock_resp):
+        result = m.verify_novi_member('ghost-uuid')
+
+    print('Result:', result)
+    assert result['status'] == 404
+    redis_mock.setex.assert_not_called()  # 404 must NOT be cached
+    print('PASS')
+
+
+# ── Novi 404 ──────────────────────────────────────────────────────────────
+
+def test_novi_404_returns_404():
+    print('--- test_novi_404_returns_404 ---')
+    mock_resp = MagicMock()
+    mock_resp.status_code = 404
+    redis_mock = _set_redis()
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get', return_value=mock_resp):
+        result = m.verify_novi_member('missing-uuid')
+
+    print('Result:', result)
+    assert result['status'] == 404
+    redis_mock.setex.assert_not_called()
+    print('PASS')
+
+
+# ── Novi 429 ──────────────────────────────────────────────────────────────
+
+def test_novi_429_returns_429():
+    print('--- test_novi_429_returns_429 ---')
+    mock_resp = MagicMock()
+    mock_resp.status_code = 429
+    redis_mock = _set_redis()
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get', return_value=mock_resp):
+        result = m.verify_novi_member('any-uuid')
+
+    print('Result:', result)
+    assert result['status'] == 429
+    redis_mock.setex.assert_not_called()
+    print('PASS')
+
+
+# ── Novi 5xx → 503 ────────────────────────────────────────────────────────
+
+def test_novi_5xx_returns_503():
+    print('--- test_novi_5xx_returns_503 ---')
+    mock_resp = MagicMock()
+    mock_resp.status_code = 502
+    _set_redis()
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get', return_value=mock_resp):
+        result = m.verify_novi_member('any-uuid')
+
+    print('Result:', result)
+    assert result['status'] == 503
+    print('PASS')
+
+
+# ── Novi unreachable → 503 ────────────────────────────────────────────────
+
+def test_novi_unreachable_returns_503():
+    print('--- test_novi_unreachable_returns_503 ---')
+    import requests as req_lib
+    _set_redis()
+
+    with patch.object(m, '_load_idaa_cfg', return_value=_make_cfg()), \
+         patch('requests.get', side_effect=req_lib.exceptions.ConnectionError('refused')):
+        result = m.verify_novi_member('any-uuid')
+
+    print('Result:', result)
+    assert result['status'] == 503
+    print('PASS')