Update searchable fields for event_exhibit

Added account_id and event_exhibit_id to the searchable_fields list to allow V3 search filtering by integer IDs.

.ae_brief

@@ -0,0 +1,15 @@
+# Aether Project Brief: aether_api_fastapi
+**Last Updated:** 2026-01-16 17:22:55
+**Current Agent:** mcp_agent
+
+## 🛠️ What I Just Did
+1. Resolved 'Bootstrap Paradox' bug in lib_config_v3.py (hosted file path fix). 2. Developed Aether Field Manager (ae_field_manage.py) with table/view snapshotting and complex view detection. 3. Verified infrastructure and dry-run logic for vertical-slice field management.
+
+## 🚧 Current Blockers
+None. Awaiting user verification of the first 'execute' run for the Field Manager.
+
+## ➡️ Exact Next Steps
+1. Execute real-world test of ae_field_manage.py with user. 2. Proceed with Journal Management architecture review (Task 155435511). 3. Initiate Pydantic V2 migration impact analysis.
+
+---
+*Generated by ae_brief*

.gitignore

@@ -83,6 +83,7 @@ celerybeat-schedule
 
 # Environments
 .env
+.env.dev
 .venv
 env/
 venv/
@@ -111,24 +112,32 @@ environment/
 Thumbs.db
 
 # Added by Scott Idem
+# Updated 2024-10-09
 # https://github.com/github/gitignore
+
 *.sock
-*.csv
-*.xlsx
-#*.pdf
+*.bak
 *.cfg
 *.ini
-*.bak
+*.kate-swp
 *.pid
+
+*.csv
+# *.pdf
+*.xlsx
+
+.directory
+.vscode
 flask_config.py
 config.py
-#config.cfg
-#users.cfg
-.directory
-tmp/
-temp/
-log/
+# config.cfg
+# users.cfg
+
+backups/
 development/
+log/
+logs/
 myapp/files/
 myapp/file_distribution/
-.vscode
+temp/
+tmp/

GEMINI.md

@@ -0,0 +1,110 @@
+# Gemini Agent Context: Aether API Orchestrator
+
+> **Template Version:** 1.2 (2026-01-26)
+> **Purpose:** Standardized memory structure for all Aether Agents.
+> **Structure:** Inverted Pyramid (Foundational -> Strategic -> Tactical -> Reference).
+
+## 1. 💾 Long Term Memory (System & Facts)
+*This section contains the "Universal Truths" that rarely change. It grounds the agent in the user's reality.*
+
+### 🤖 Agent Identity & Role
+- **Agent Name:** Aether API Orchestrator (mcp_agent)
+- **Primary Role:** Backend Development, System Orchestration, and API Stabilization.
+- **Scope:** `/home/scott/OSIT_dev/aether_api_fastapi` and Aether Platform backend infrastructure.
+
+### 👤 User Profile
+- **User:** Scott Idem (`scott`)
+- **Organizations:**
+    - **One Sky IT (OSIT):** Professional/Business context.
+    - **Danger Zone (DgrZone):** Personal/Home context.
+    - **Aether Platform (AE):** Scott's (One Sky IT) platform developed for OSIT.
+- **Preferences:**
+    - **Editor:** `vim` (Terminal), VS Code (GUI).
+    - **Communication:** Direct, concise, professional CLI tone.
+    - **Safety:** "Recycle Bin" (`~/tmp/gemini_trash`) instead of `rm`. Explain destructive actions first.
+- **Hardware/OS:**
+    - **Host:** Linux (Ubuntu/Arch context)
+
+### 🏗️ Aether Architecture (V3)
+- **Concept:** Unified AI-driven platform for business/personal management.
+- **Backend:** FastAPI (v4.9.0) + Pydantic V1 + SQLAlchemy + MariaDB (Remote).
+- **V3 Implementation:** Modern parallel CRUD and Search endpoints under `/v3/crud`.
+- **Core Principle:** "Agent Bridge" - Distributed agents coordinating via file-based messaging (`~/agents_sync`).
+
+### 📜 Core Protocols
+- **RAR Protocol:** Request -> Ack -> Result.
+- **V3 CRUD Paradigm:** JSON metadata via `/v3/crud/`, binary actions (Upload/Download) via `/v3/action/`.
+- **Fail Fast & Transparently:** API returns `500` on hard errors; avoid silent failures (confirmed in `sql_select`).
+- **Bite-Sized Data:** Avoid monolithic files (>1MB).
+- **Source of Truth:** `~/agents_sync` is the shared brain. `~/OSIT_dev` is the local development environment.
+
+### 🛡️ Security & Secrets Guardrails
+- **Secrets:** NEVER read/display content from `.env` files unless explicitly debugging configuration logic.
+- **PII:** Scrub personally identifiable information if sharing logs or data across the bridge.
+- **Hiding Internal Paths:** `subdirectory_path` is hidden from public-facing API responses via Pydantic `Field(exclude=True)`.
+
+### 🧠 Key Technical Learnings (Cumulative)
+- **Circular Dependencies Fixed**: Successfully resolved the fragile startup dependency chain by isolating Auth models and using strictly deferred DB imports in a dedicated `dependencies_v3.py` module.
+- **Bootstrap Paradox Solved**: Implemented a guest-access exception for `site_domain` search, allowing the frontend to resolve site context without a JWT.
+- **V3 Searchable Fields**: `searchable_fields` must explicitly include integer ID fields (e.g., `event_id`) to ensure valid numeric filters are not blocked by the V3 search security layer.
+- **NULL Logic in Filters**: Confirmed that explicit frontend filters like `hide: false` will FAIL to match `NULL` database values. Rely on the API's built-in `hidden=not_hidden` parameter for robust handling.
+- **Vision ID Safety Net**: Enhanced `lookup_id_random_pop` to resolve random string IDs found in any `*_id` field, ensuring "Vision" style payloads are correctly converted to integers.
+
+---
+
+## 2. 🗓️ Near Term Memory (Strategic Context)
+*This section tracks active projects (1-2 weeks scope). It answers "Why are we doing this?"*
+
+### 📩 In-Flight RAR Requests
+- [ ] **mcp_agent**: Real-world test of `ae_field_manage.py` (ID: 153357623).
+- [ ] **codebase_investigator**: Review report for Aether extension and journal management (ID: 155435511).
+
+### 🎯 Strategic Goals (Current Sprint)
+- **Primary:** OSIT_dev Environment Optimization & Context Stabilization (Template v1.2 Adoption).
+- **Secondary:** ID Vision Phase 2 Migration and V3 API Migration (Contacts/Clients).
+
+### 🚧 Active Workstreams
+- **[ID Vision]:** Phase 2 complete. Strictly enforced string-ID standardization for Page, Post, Person, Journal, Contact, and User models. (ID: 161311118 - DONE).
+- **[Infrastructure]:** Restore AE Events Presentation Launcher (Electron) (ID: 221513945).
+- **[Infrastructure]:** Pydantic V2 Migration Impact Analysis (Technical Debt).
+- **[Journals]:** UI: Implementation of Quick Add & Append/Prepend (ID: 185821382).
+
+### 🧠 Recent Decisions
+- **ID Hardening:** Modified the `map_v3_ids` root validator across core models to explicitly delete aliased integer IDs (e.g., `post_id`, `journal_id`) to prevent Pydantic coercion of legacy integers into strings.
+- **Search Optimization:** Standardized on `default_qry_str` for optimized fulltext searching. `Event_Badge_Base` is noted as a temporary outlier (`default_qry_string`) awaiting frontend alignment.
+- **Privacy & Information Hiding:** Centralized `public_read` flag in object definitions and excluded internal file sharding paths from responses.
+
+---
+
+## 3. 🧠 Short Term Memory (Session Context)
+*This section is the "Scratchpad" for the current interaction. It is cleared or summarized often.*
+
+- **Status:** Online
+- **Last Action:** Successfully refactored `GEMINI.md` to v1.2 structure.
+- **Current Blocker:** None.
+- **Immediate Next Step:** Check for new messages in the inbox or proceed with high-priority tasks.
+
+---
+
+## 4. 📂 Reference: Directory & Whitelist
+*Low-density reference data. Keep at the bottom to avoid cluttering the prompt's "hot zone".*
+
+### 🛡️ File Whitelist
+- `~/tmp`
+- `~/OSIT_dev/aether_api_fastapi`
+- `~/agents_sync`
+
+### 🗺️ Standard Directory Map
+- **`app/methods/`**: Object-specific business logic.
+- **`app/models/`**: Pydantic schemas.
+- **`app/object_definitions/`**: V3 Metadata definitions.
+- **`app/routers/`**: API endpoints.
+
+### 📋 Aether API Development Protocol
+0. **Pre-Flight Check:** Verify `git status`. Ensure all previous working changes are committed.
+1. **Strategic Plan:** Write a concise plan identifying the issue, specific files, and verification steps (curl commands/test scripts).
+2. **Implementation:** Perform atomic code modifications using `replace` or `write_file`.
+3. **Syntax Validation:** Run `python3 -m py_compile <modified_file>` immediately.
+4. **Process Cycle:** Restart the Docker FastAPI service: `docker restart aether_container_env-ae_api-2`.
+5. **Empirical Testing:** Execute `curl` commands and inspect logs: `tail -n 20 ~/OSIT_dev/aether_container_env/logs/ae_api/aether_api.log`.
+6. **Finalize:** Commit changes with a descriptive message and sync documentation.

admin/documentation/gunicorn.service.default

@@ -1,24 +0,0 @@
-[Unit]
-Description=gunicorn daemon
-Requires=gunicorn.socket
-After=network.target
-
-[Service]
-Type=notify
-# the specific user that our service will run as
-User=root
-Group=root
-# another option for an even more restricted service is
-# DynamicUser=yes
-# see http://0pointer.net/blog/dynamic-users-with-systemd.html
-RuntimeDirectory=gunicorn
-WorkingDirectory=/srv/http/dev_fastapi.oneskyit.com
-Environment="PATH=/srv/http/dev_fastapi.oneskyit.com/environment/bin"
-ExecStart=/srv/http/dev_fastapi.oneskyit.com/environment/bin/gunicorn --bind unix:/srv/http/dev_fastapi.oneskyit.com/gunicorn.sock -m 007 app.main:app --workers 4 -k uvicorn.workers.UvicornWorker --access-logfile admin/log/access.log --error-logfile admin/log/error.log, --log-file admin/log/log.log --capture-output --keep-alive 5
-ExecReload=/bin/kill -s HUP $MAINPID
-KillMode=mixed
-TimeoutStopSec=5
-PrivateTmp=true
-
-[Install]
-WantedBy=multi-user.target

admin/documentation/gunicorn.socket.default

@@ -1,14 +0,0 @@
-[Unit]
-Description=gunicorn socket
-
-[Socket]
-ListenStream=/run/gunicorn.sock
-# Our service won't need permissions for the socket, since it
-# inherits the file descriptor by socket activation
-# only the nginx daemon will need access to the socket
-User=http
-# Optionally restrict the socket permissions even more.
-# Mode=600
-
-[Install]
-WantedBy=sockets.target

admin/documentation/nginx_fastapi_server.example.default

@@ -1,85 +0,0 @@
-server {
-  access_log  /var/log/nginx/access_dev_fastapi.oneskyit.com.log;
-
-  listen 443 ssl; # managed by Certbot
-  listen [::]:443 ssl http2; # managed by Certbot
-  #listen 443 http3 reuseport;  # UDP listener for QUIC+HTTP/3
-  server_name dev-fastapi.oneskyit.com;
-
-  ssl_certificate /etc/letsencrypt/live/oneskyit.com-0001/fullchain.pem; # managed by Certbot
-  ssl_certificate_key /etc/letsencrypt/live/oneskyit.com-0001/privkey.pem; # managed by Certbot
-
-  include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
-  ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
-
-  #add_header Alt-Svc 'quic=":443"'; # Advertise that QUIC is available
-  #add_header QUIC-Status $quic;     # Sent when QUIC was used
-
-  include brotli.conf;
-  include gzip.conf;
-
-  client_max_body_size 4096M; # or 4G
-
-  location / {
-        proxy_set_header Host $http_host;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Forwarded-Proto $scheme;
-
-        proxy_redirect off;
-        proxy_buffering off;
-
-        proxy_pass http://unix:/run/gunicorn.sock;
-  }
-
-  location /ws {
-        proxy_set_header Host $http_host;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Forwarded-Proto $scheme;
-
-        proxy_http_version 1.1;
-
-        proxy_redirect off;
-        proxy_buffering off;
-
-        proxy_set_header Upgrade $http_upgrade;
-        proxy_set_header Connection "upgrade";
-
-        #proxy_read_timeout 600;
-        #proxy_headers_hash_max_size 1024;
-
-        proxy_pass http://unix:/run/gunicorn.sock;
-  }
-
-  location /ws_redis {
-        proxy_set_header Host $http_host;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Forwarded-Proto $scheme;
-
-        proxy_http_version 1.1;
-
-        proxy_redirect off;
-        proxy_buffering off;
-
-        proxy_set_header Upgrade $http_upgrade;
-        proxy_set_header Connection "upgrade";
-
-        #proxy_read_timeout 600;
-        #proxy_headers_hash_max_size 1024;
-
-        proxy_pass http://unix:/run/gunicorn.sock;
-  }
-}
-
-server {
-  if ($host = dev-fastapi.oneskyit.com) {
-    return 301 https://$host$request_uri;
-  } # managed by Certbot
-
-  listen 80;
-  listen [::]:80;
-  server_name dev-fastapi.oneskyit.com;
-  return 404; # managed by Certbot
-}

admin/documentation/run locally.txt

@@ -1,55 +0,0 @@
-# Go to root of application
-cd ~/path/to/directory/my_application/
-
-# Create new application environment
-virtualenv environment
-
-# Activate application environment
-source environment/bin/activate
-
-# Install application requirements
-pip install -r admin/requirements.txt
-pip install --upgrade --force-reinstall -r admin/requirements.txt
-pip install --ignore-installed -r admin/requirements.txt
-pip list
-
-# Start application
-uvicorn app.main:app --host 0.0.0.0 --port 5005 --reload
-
-# View app
-http://localhost:5005
-
-# Deactivate environment when done
-deactivate
-
-
-# Use git
-# Go to root of application
-cd ~/path/to/directory/my_application/
-
-git init
-
-git remote add origin https://scott_idem@bitbucket.org/oneskyit/one-sky-it-api.git
-git add .
-git commit -m 'Initial commit'
-
-git push -u origin master
-git push -u origin development
-git push -u origin new-branch-name
-
-# List branches
-git branch -a
-
-# Create new branch
-git branch new-branch-name
-
-# Switch branch
-git switch new-branch-name
-
-
-# Clone from Bitbucket:
-git clone https://scott_idem@bitbucket.org/oneskyit/one-sky-it-api-fastapi.git /srv/http/the_path_to_create
-
-
-
-gunicorn --bind unix:/home/scott/OSIT_dev/aether_api_fastapi/gunicorn.sock --umask 007 app.main:app --workers 2 --worker-class uvicorn.workers.UvicornWorker --access-logfile admin/log/access.log --error-logfile admin/log/error.log, --log-file admin/log/log.log --capture-output --keep-alive 5 --reload

admin/documentation/setup_server.txt

@@ -1,42 +0,0 @@
-sudo git clone https://scott_idem@bitbucket.org/oneskyit/one-sky-it-api-fastapi.git /srv/http/dev_fastapi.oneskyit.com
-
-sudo mkdir admin/log
-
-sudo ls -lha /srv/http/
-sudo chown http:http -R /srv/http/dev_fastapi.oneskyit.com/
-sudo chmod 775 -R /srv/http/dev_fastapi.oneskyit.com/
-sudo ls -lha /srv/http/
-
-cd /srv/http/dev_fastapi.oneskyit.com/
-rm .gitignore
-
-git branch -a
-git switch development
-
-virtualenv environment
-source environment/bin/activate
-pip install -U -r admin/requirements.txt
-
-sudo vim /etc/systemd/system/gunicorn.socket
-sudo vim /etc/systemd/system/gunicorn.service
-sudo systemctl daemon-reload
-sudo systemctl enable gunicorn.socket
-sudo systemctl start gunicorn.socket
-sudo systemctl status gunicorn.socket
-sudo systemctl status gunicorn.service
-
-# Do not: sudo systemctl enable gunicorn.service
-# Do not? sudo systemctl start gunicorn.service
-
-sudo vim /etc/nginx/sites-available/dev_fastapi.oneskyit.com
-sudo ln -s /etc/nginx/sites-available/dev_fastapi.oneskyit.com /etc/nginx/sites-enabled/dev_fastapi.oneskyit.com
-
-sudo systemctl restart nginx.service
-sudo systemctl status nginx.service
-
-# Troubleshooting:
-systemctl list-units --type=service --state=active
-systemctl list-units --type=service --state=running
-
-sudo systemctl | grep running
-sudo systemctl list-unit-files | grep enabled

admin/requirements.txt

@@ -1,63 +0,0 @@
-# aioredis # BAD! Not maintained!
-anyio
-argon2-cffi
-argon2-cffi-bindings
-asgiref
-async-timeout
-certifi
-cffi
-charset-normalizer
-click
-Deprecated
-dnspython
-email-validator
-et-xmlfile
-fastapi
-greenlet
-gunicorn
-h11
-html2text
-httpcore
-httptools
-httpx
-idna
-itsdangerous
-Jinja2
-MarkupSafe
-mysqlclient
-numpy
-openpyxl
-orjson
-packaging
-pandas
-passlib
-# pdf2image
-Pillow
-pycparser
-pydantic
-PyJWT
-pyparsing
-python-dateutil
-python-dotenv
-python-multipart
-pytz
-PyYAML
-qrcode
-redis[hiredis]
-requests
-rfc3986
-six
-sniffio
-SQLAlchemy==1.4.47 # 1.4.47 is the newest I am working with
-starlette
-stripe
-typing_extensions
-ujson
-urllib3
-uvicorn
-uvloop
-watchfiles
-watchgod
-websockets
-wrapt
-xlrd

aether_api_fastapi.code-workspace

@@ -4,5 +4,9 @@
 			"path": "."
 		}
 	],
-	"settings": {}
+	"settings": {
+		"cSpell.words": [
+			"poolclass"
+		]
+	}
 }

app/ae_obj_types_def.py

@@ -0,0 +1,84 @@
+"""
+This file centralizes the object type definitions for the Aether API.
+It merges definitions from modular files in app/object_definitions/ to support
+both V2 (legacy) and V3 CRUD operations.
+"""
+
+# Restore blanket imports for legacy compatibility (V1 and V2 rely on these)
+from app.models.response_models import *
+from app.models.api_crud_models import *
+from app.models.account_models import *
+from app.models.account_cfg_models import *
+from app.models.activity_log_models import *
+from app.models.address_models import *
+from app.models.archive_models import *
+from app.models.archive_content_models import *
+from app.models.contact_models import *
+from app.models.cont_edu_cert_models import *
+from app.models.cont_edu_cert_person_models import *
+from app.models.data_store_models import *
+from app.models.event_models import *
+from app.models.event_abstract_models import *
+from app.models.event_badge_models import *
+from app.models.event_badge_template_models import *
+from app.models.event_device_models import *
+from app.models.event_exhibit_models import *
+from app.models.event_exhibit_tracking_models import *
+from app.models.event_file_models import *
+from app.models.event_location_models import *
+from app.models.event_person_models import *
+from app.models.event_person_tracking_models import *
+from app.models.event_presentation_models import *
+from app.models.event_presenter_models import *
+from app.models.event_registration_models import *
+from app.models.event_session_models import *
+from app.models.event_track_models import *
+from app.models.grant_models import *
+from app.models.hosted_file_models import *
+from app.models.journal_models import *
+from app.models.journal_entry_models import *
+from app.models.log_client_viewing_models import Log_Client_Viewing_Base
+from app.models.membership_cfg_models import *
+from app.models.membership_group_models import *
+from app.models.membership_person_group_models import *
+from app.models.membership_person_models import *
+from app.models.membership_person_profile_models import *
+from app.models.membership_type_models import *
+from app.models.membership_person_type_models import *
+from app.models.order_models import *
+from app.models.order_cart_models import *
+from app.models.organization_models import *
+from app.models.page_models import *
+from app.models.person_models import *
+from app.models.product_models import *
+from app.models.post_models import *
+from app.models.post_comment_models import *
+from app.models.site_models import *
+from app.models.site_domain_models import *
+from app.models.sponsorship_cfg_models import *
+from app.models.sponsorship_models import *
+from app.models.user_models import *
+from app.models.user_role_models import *
+from app.models.e_stripe_models import *
+
+# Modularized definitions
+from app.object_definitions.core import core_obj_li
+from app.object_definitions.events import event_obj_li
+from app.object_definitions.journals import journal_obj_li
+from app.object_definitions.orders import order_obj_li
+from app.object_definitions.cms import cms_obj_li
+from app.object_definitions.lookups import lu_obj_li
+from app.object_definitions.membership import membership_obj_li
+from app.object_definitions.other import other_obj_li
+
+# Merge all modular definitions into the main registry
+obj_type_kv_li = {
+    **core_obj_li,
+    **event_obj_li,
+    **journal_obj_li,
+    **order_obj_li,
+    **cms_obj_li,
+    **lu_obj_li,
+    **membership_obj_li,
+    **other_obj_li,
+}

app/config.py.default

@@ -1,85 +1,71 @@
 # Configuration file for this FastAPI app.
 import os
-from pydantic import AnyHttpUrl, BaseSettings, EmailStr, HttpUrl, PostgresDsn, validator
+from pydantic import BaseSettings
 from typing import Any, Dict, List, Optional, Union
 
-
-# ### ### #
-
-
 class Settings(BaseSettings):
-    AETHER_CFG = {}
-    AETHER_CFG['id'] = 0
-    # AETHER_CFG['api_id'] = 0 # NOT CURRENTLY NEED OR USED
-
-    JWT_KEY = '' # 22 characters; super secret Aether JWT signing key
-
-    # APP_NAME: str = "Aether API (FastAPI)"
-    # SUPER_EMAIL: EmailStr = 'Aether.Super@oneskyit.com'
-
+    AETHER_CFG: Dict[str, Any] = {
+        "id": os.getenv('AE_CFG_ID', '0')
+    }
+    
+    JWT_KEY: str = os.getenv('AE_API_JWT_KEY', 'fake-super-secret-token')
 
     # Database Connection
-    DB = {}
-    DB['server'] = 'db.oneskyit.com'
-    DB['port'] = '3306' # default = 3306
-    DB['name'] = 'aether_default'
-    DB['username'] = ''
-    DB['password'] = ''
-    SQLALCHEMY_DB_URI = 'mysql://'+DB['username']+':'+DB['password']+'@'+DB['server']+'/'+DB['name']
+    DB_SERVER: str = os.getenv('AE_DB_SERVER', 'mariadb')
+    DB_PORT: str = os.getenv('AE_DB_PORT', '3306')
+    DB_NAME: str = os.getenv('AE_DB_NAME', 'aether_dev')
+    DB_USER: str = os.getenv('AE_DB_USERNAME', 'aether_dev')
+    DB_PASS: str = os.getenv('AE_DB_PASSWORD', '')
 
-    DB['wait_timeout'] = int(os.getenv('AE_DB_WAIT_TIMEOUT', 1800)) # default = 28800; Time (seconds) that the server waits for a connection to become active before closing it.
-    DB['connect_timeout'] = int(os.getenv('AE_DB_CONNECTION_TIMEOUT', 20)) # default = 10; Time (seconds) that the server waits for a connection to become active before closing it.
-    DB['pool_recycle'] = int(os.getenv('AE_DB_POOL_RECYCLE', 1800)) # default = ?; Related to SQLAlchemy
+    @property
+    def SQLALCHEMY_DB_URI(self) -> str:
+        return f"mysql://{self.DB_USER}:{self.DB_PASS}@{self.DB_SERVER}:{self.DB_PORT}/{self.DB_NAME}"
 
+    @property
+    def DB(self) -> Dict[str, Any]:
+        return {
+            "server": self.DB_SERVER,
+            "port": self.DB_PORT,
+            "name": self.DB_NAME,
+            "username": self.DB_USER,
+            "password": self.DB_PASS,
+            "connect_timeout": int(os.getenv('AE_DB_CONNECTION_TIMEOUT', 20)),
+            "pool_recycle": int(os.getenv('AE_DB_POOL_RECYCLE', 1800))
+        }
 
-    # Aether API log files paths
-    LOG_PATH = {}
-    LOG_PATH['app'] = '/logs/aether_api.log' # 'admin/log/app.log', '../../logs/aether_api.log'
-    # LOG_PATH['app_warning'] = '/logs/aether_api_warning.log' # 'admin/log/app_warning.log' '../../logs/aether_api_warning.log'
-
+    # Logging
+    LOG_PATH: Dict[str, str] = {
+        "app": os.getenv('AE_API_LOG_PATH', '/logs/aether_api.log')
+    }
 
     # Redis
-    REDIS = {}
-    REDIS['server'] = 'localhost' # 'localhost' 'redis'
-    REDIS['port'] = '6379'
-
+    REDIS: Dict[str, str] = {
+        "server": os.getenv('AE_REDIS_SERVER', 'redis'),
+        "port": os.getenv('AE_REDIS_PORT', '6379')
+    }
 
+    # --- CRITICAL CONFIGURATIONS ---
     # Send SMTP Email
-    SMTP = {}
-    # server
-    # port
-    # username
-    # password
-
+    SMTP: Dict[str, str] = {
+        "server": os.getenv('AE_SMTP_SERVER', ''),
+        "port": os.getenv('AE_SMTP_PORT', '465'),
+        "username": os.getenv('AE_SMTP_USERNAME', ''),
+        "password": os.getenv('AE_SMTP_PASSWORD', '')
+    }
 
     # Server Hosted File Paths
-    FILES_PATH = {}
-    # hosted_files_root
-    # hosted_tmp_root
+    FILES_PATH: Dict[str, str] = {
+        "hosted_files_root": os.getenv('AE_FILES_PATH_ROOT', '/srv/hosted_files'),
+        "hosted_tmp_root": os.getenv('AE_FILES_PATH_TMP', '/srv/hosted_tmp')
+    }
+    # --- END CRITICAL CONFIGURATIONS ---
 
-
-    # CORS Origins
-    ORIGINS_REGEX = '(https://.*\.oneskyit\.com)|(http://.*\.oneskyit\.com:8181)|(https://.*\.oneskyit\.com:4443)|(https://.*\.oneskyit\.com:8443)'
-    # A reasonable, but fairly open example regular expression for the CORS origins:
-    # '(https://.*\.oneskyit\.com)|(http://.*\.oneskyit\.com)|(http://.*\.oneskyit\.com:8181)|(https://.*\.oneskyit\.com:8443)|(http://.*\.oneskyit\.local)|(http://.*\.oneskyit\.local:5000)|(http://.*.localhost)|(http://.*.localhost:5000)|(http://.*.localhost:8181)'
-
-    ORIGINS = [
+    # CORS
+    ORIGINS_REGEX: str = os.getenv('AE_API_ORIGINS_REGEX', '(https://.*\.oneskyit\.com)|(https://.*\.oneskyit\.com:4443)')
+    ORIGINS: List[str] = [
         'https://oneskyit.com',
-        # 'http://app-local.oneskyit.com',
-        # 'http://192.168.32.20:3000',
-        # 'http://192.168.32.20:8080',
-
-        # 'http://localhost',
-        # 'http://localhost:3000',
-        # 'http://localhost:5000',
-        # 'http://localhost:8080',
-        # 'http://localhost:7800',
-        # 'http://localhost:8888',
-
         'http://fastapi.localhost',
-
         'http://svelte.oneskyit.local:5555',
-        ]
-
+    ]
 
 settings = Settings()

app/db_connection.py

@@ -0,0 +1,28 @@
+"""
+Independent database connection module to prevent circular imports.
+"""
+import logging
+from sqlalchemy import create_engine
+from app.config import settings
+
+# Use local logger to avoid importing app.log (which might create cycles)
+log = logging.getLogger(__name__)
+
+db_uri = settings.SQLALCHEMY_DB_URI
+engine = create_engine(
+    url = db_uri,
+    echo = False,
+    pool_use_lifo = True,
+    pool_pre_ping = True,
+    pool_recycle = settings.DB['pool_recycle'],
+    isolation_level = 'READ COMMITTED',
+    connect_args = {'connect_timeout': settings.DB['connect_timeout']}
+)
+
+log.info('DB Connection initializing...')
+db = None
+try:
+    db = engine.connect()
+    log.info(f'Connected to database: {db_uri}')
+except Exception:
+    log.exception('Could not connect to database.')

app/lib_api_crud_v3.py

@@ -0,0 +1,240 @@
+from typing import Any, Dict, Optional, Union
+import json
+import logging
+import re
+
+from app.lib_general_v3 import AccountContext, StatusFilterParams
+from app.models.error_models import StandardError
+
+log = logging.getLogger(__name__)
+
+def format_db_error(raw_error: str) -> StandardError:
+    """
+    Parses raw SQLAlchemy/MariaDB errors into structured StandardError objects.
+    """
+    if not raw_error:
+        return StandardError(
+            category="unknown",
+            message="An unspecified database error occurred."
+        )
+    
+    # 1. Extract Error Code and Message using regex
+    # Standard MariaDB pattern: (code, "message")
+    code = None
+    message = raw_error
+    recoverable = False
+    
+    match = re.search(r'\((\d+),\s*["\'](.*?)["\']\s*\)', raw_error)
+    if match:
+        code = int(match.group(1))
+        message = match.group(2).strip()
+    else:
+        # Fallback: remove all (parenthesized) blocks which often contain codes
+        message = re.sub(r'\(.*?\)', '', raw_error).strip()
+
+    # 2. Categorize based on known MariaDB codes
+    # Ref: https://mariadb.com/kb/en/mariadb-error-codes/
+    if code in [1062]: # Duplicate Entry
+        category = "database_duplicate"
+    elif code in [1451, 1452]: # Foreign Key Constraint
+        category = "database_constraint"
+    elif code in [1045, 2002, 2003, 2006]: # Connection / Auth issues
+        category = "database_connection"
+        recoverable = True
+    elif code in [1054, 1146]: # Unknown column / Table
+        category = "database_schema"
+    else:
+        category = "database"
+
+    return StandardError(
+        category=category,
+        code=code,
+        message=message,
+        recoverable=recoverable,
+        details=raw_error if category == "database" else None # Only include raw details for uncategorized errors
+    )
+
+def check_account_access(sql_result: Any, account: AccountContext, obj_name: str = None) -> bool:
+    """
+    Enforce Multi-Tenant Data Isolation.
+    
+    Verifies that the requested record belongs to the authenticated user's account.
+    Returns True if:
+    - User is a Super User or System (Bypass).
+    - The record's `account_id` matches the user's `account_id`.
+    """
+    if account.super or account.auth_method == 'bypass':
+        return True
+    if not account.account_id:
+        return False
+    
+    res_account_id = None
+    if isinstance(sql_result, dict):
+        if obj_name == 'account':
+            res_account_id = sql_result.get('id')
+        else:
+            res_account_id = sql_result.get('account_id')
+    
+    if res_account_id is not None and res_account_id != account.account_id:
+        return False
+    return True
+
+def apply_forced_account_filter(and_qry_dict: Optional[Dict], account: AccountContext, model: Any, obj_name: str, table_name: str = None) -> Dict:
+    """
+    Secure Search Filtering.
+    
+    Automatically appends an `account_id` filter to database queries to ensure 
+    users only retrieve records associated with their own account.
+    
+    Now schema-aware: checks if the column actually exists in the DB before applying.
+    """
+    forced = and_qry_dict or {}
+    if account.super or account.auth_method == 'bypass':
+        return forced
+    
+    # 1. Determine the target column
+    target_col = 'account_id'
+    if obj_name == 'account':
+        target_col = 'id'
+    
+    # 2. Check if the model even supports it
+    if model and hasattr(model, '__fields__') and target_col not in model.__fields__:
+        return forced
+
+    # 3. If we have a table name, verify the column exists in the physical DB schema
+    # (Important for Views that might exclude account_id for performance/privacy)
+    if table_name:
+        from app import lib_sql_core
+        from sqlalchemy import text
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                conn.execute(text(f"SELECT `{target_col}` FROM `{table_name}` LIMIT 0"))
+            has_col = True
+        except:
+            has_col = False
+        
+    forced[target_col] = account.account_id
+    return forced
+
+def filter_order_by(order_by_li: Any, model: Any, table_name: str = None) -> Optional[Dict[str, str]]:
+    """
+    Sanitize Sorting Parameters.
+    
+    Prevents SQL injection and logic errors by validating that requested sort columns
+    actually exist in the Pydantic model and/or the database table.
+    """
+    if not order_by_li or not isinstance(order_by_li, dict) or not model:
+        return order_by_li
+    if not hasattr(model, '__fields__'):
+        return order_by_li
+    
+    model_fields = set(model.__fields__.keys())
+    model_fields.update({f.alias for f in model.__fields__.values() if f.alias})
+    filtered = {k: v for k, v in order_by_li.items() if k in model_fields}
+    
+    if table_name and filtered:
+        from app.db_sql import db
+        from sqlalchemy import text
+        final_filtered = {}
+        for column in filtered:
+            try:
+                # Lightweight check to see if column exists in SQL
+                db.execute(text(f"SELECT `{column}` FROM `{table_name}` LIMIT 0"))
+                final_filtered[column] = filtered[column]
+            except Exception:
+                pass
+        filtered = final_filtered
+    return filtered
+
+def get_supported_filters(model: Any, status_filter: StatusFilterParams) -> StatusFilterParams:
+    """
+    Adaptive Status Filtering.
+    
+    Adjusts the default filters (enabled/hidden) based on whether the target object
+    actually supports those concepts (i.e., has those columns).
+    """
+    if not model or not hasattr(model, "__fields__"):
+        return status_filter
+    # We create a new instance to avoid side effects on the dependency object
+    from app.routers.dependencies_v3 import StatusFilterParams as SF
+    adjusted = SF()
+    adjusted.enabled = status_filter.enabled
+    adjusted.hidden = status_filter.hidden
+    
+    if 'enable' not in model.__fields__:
+        adjusted.enabled = 'all'
+    if 'hide' not in model.__fields__:
+        adjusted.hidden = 'all'
+    return adjusted
+
+def safe_json_loads(json_str: Optional[str]) -> Any:
+    if not json_str or json_str == 'undefined': return None
+    try: return json.loads(json_str)
+    except: return None
+
+def sanitize_payload(data: dict, model: Any, ignore_extra: bool = False) -> None:
+    """
+    Sanitizes an input payload before database insertion or update.
+    
+    1. Resolves ID strings to integers:
+       - Handles legacy `*_id_random` fields.
+       - Handles Vision `*_id` fields where the value is a string (e.g., account_id: "random_str").
+    2. Removes virtual lookup fields (ending in `_id_random`) after resolution.
+    3. Removes fields explicitly marked for exclusion in the model's 
+       `fields_to_exclude_from_db` ClassVar (e.g., view-only fields).
+    4. If `ignore_extra` is True, removes all fields NOT present in the model definition.
+       
+    Modifies the `data` dictionary in-place.
+    """
+    if not isinstance(data, dict):
+        return
+
+    from app.db_sql import redis_lookup_id_random
+
+    # Resolve ID strings to integers
+    for k, v in list(data.items()):
+        if not v or not isinstance(v, str):
+            continue
+            
+        target_id_field = None
+        obj_type_lookup = None
+        
+        # Scenario A: Legacy suffix (e.g., account_id_random: "abc")
+        if k.endswith('_id_random') and k != 'id_random':
+            target_id_field = k.replace('_id_random', '_id')
+            obj_type_lookup = k.replace('_id_random', '')
+            
+        # Scenario B: Vision naming (e.g., account_id: "abc")
+        # We only resolve if it's a string of the correct length (random ID format)
+        elif k.endswith('_id') and 11 <= len(v) <= 22:
+            target_id_field = k
+            obj_type_lookup = k.replace('_id', '')
+            
+        if target_id_field and obj_type_lookup:
+            # Special table mapping if needed
+            if obj_type_lookup == 'address_location': obj_type_lookup = 'address'
+            
+            resolved_id = redis_lookup_id_random(record_id_random=v, table_name=obj_type_lookup)
+            if resolved_id:
+                data[target_id_field] = resolved_id
+                # If we were handling Scenario A, remove the original random key
+                if k.endswith('_id_random'):
+                    del data[k]
+
+    # Filter out model-specific excluded fields (e.g., view-only fields)
+    if hasattr(model, 'fields_to_exclude_from_db'):
+        for k in model.fields_to_exclude_from_db:
+             if k in data:
+                 del data[k]
+
+    # If permissive mode is on, remove any field not in the Pydantic model
+    if ignore_extra and model and hasattr(model, '__fields__'):
+        model_fields = set(model.__fields__.keys())
+        # Also check for aliases
+        for f in model.__fields__.values():
+            if f.alias:
+                model_fields.add(f.alias)
+        
+        extra_keys = [k for k in data.keys() if k not in model_fields]
+        for k in extra_keys:
+            del data[k]

app/lib_config_v3.py

@@ -0,0 +1,88 @@
+import logging
+from typing import Any
+
+log = logging.getLogger('root')
+
+def validate_critical_config(settings: Any):
+    """
+    Validates that essential settings are populated and not using placeholders.
+    Logs warnings or errors for missing critical infrastructure.
+    """
+    log.info("Checking critical system configuration...")
+    
+    # 1. Database Check
+    db = getattr(settings, 'DB', {})
+    if not db.get('server') or db.get('server') == 'mariadb':
+        # 'mariadb' is the default in .env, usually fine, but worth noting
+        log.info(f"Database server: {db.get('server')}")
+    
+    # 2. SMTP Check
+    smtp = getattr(settings, 'SMTP', {})
+    if not smtp.get('server'):
+        log.warning("CRITICAL: SMTP server not configured. Email features will fail.")
+    if smtp.get('password') == 'set-in-ae-sql-db-cnf-tbl':
+        log.error("CRITICAL: SMTP password is still set to placeholder. Email authentication will fail.")
+
+    # 3. Security Check
+    jwt_key = getattr(settings, 'JWT_KEY', '')
+    if not jwt_key or jwt_key == 'fake-super-secret-token':
+        log.error("SECURITY: JWT_KEY is missing or using a known fake token!")
+
+    log.info("Configuration validation complete.")
+
+def bootstrap_db_config(settings: Any) -> bool:
+    """
+    Loads dynamic settings from the 'cfg' table and updates the settings object.
+    Uses deferred import of sql_select to avoid circular dependencies.
+    """
+    # CRITICAL: Deferred import to prevent boot-time circular dependencies
+    from app.db_sql import sql_select 
+    
+    cfg_id = settings.AETHER_CFG.get('id', '0')
+    log.info(f"Bootstrapping system configuration from DB (cfg_id={cfg_id})...")
+    
+    try:
+        # Fetch the config record
+        aether_cfg_sql = sql_select(
+            table_name='cfg',
+            record_id=int(cfg_id),
+            as_list=False,
+            max_count=1,
+        )
+        
+        # In some cases sql_select might return a single-item list even with as_list=False
+        if isinstance(aether_cfg_sql, list):
+            if len(aether_cfg_sql) > 0:
+                aether_cfg_sql = aether_cfg_sql[0]
+            else:
+                aether_cfg_sql = None
+
+        if not aether_cfg_sql or not isinstance(aether_cfg_sql, dict):
+            log.error(f"FAILED to load system config from DB for ID {cfg_id}. Table 'cfg' might be empty or ID missing.")
+            return False
+
+        # --- Update Database settings ---
+        # Safety: Only update if the values are provided in the DB record
+        if aether_cfg_sql.get('db_server'): settings.DB_SERVER = aether_cfg_sql.get('db_server')
+        if aether_cfg_sql.get('db_port'): settings.DB_PORT = str(aether_cfg_sql.get('db_port'))
+        if aether_cfg_sql.get('db_name'): settings.DB_NAME = aether_cfg_sql.get('db_name')
+        if aether_cfg_sql.get('db_username'): settings.DB_USER = aether_cfg_sql.get('db_username')
+        if aether_cfg_sql.get('db_password'): settings.DB_PASS = aether_cfg_sql.get('db_password')
+        
+        # --- Update SMTP Settings ---
+        if aether_cfg_sql.get('smtp_server'): settings.SMTP['server'] = aether_cfg_sql.get('smtp_server')
+        if aether_cfg_sql.get('smtp_port'): settings.SMTP['port'] = str(aether_cfg_sql.get('smtp_port'))
+        if aether_cfg_sql.get('smtp_username'): settings.SMTP['username'] = aether_cfg_sql.get('smtp_username')
+        if aether_cfg_sql.get('smtp_password'): settings.SMTP['password'] = aether_cfg_sql.get('smtp_password')
+
+        # --- Update File Paths ---
+        # DEPRECATED: Filesystem paths should be controlled by the Environment/Docker, not the DB.
+        # if aether_cfg_sql.get('path_hosted_files_root'): settings.FILES_PATH['hosted_files_root'] = aether_cfg_sql.get('path_hosted_files_root')
+        # if aether_cfg_sql.get('path_hosted_tmp_root'): settings.FILES_PATH['hosted_tmp_root'] = aether_cfg_sql.get('path_hosted_tmp_root')
+        
+        log.info("System configuration successfully synchronized with DB.")
+        return True
+
+    except Exception as e:
+        log.exception(f"Unexpected error during system bootstrap: {e}")
+        return False

app/lib_email.py

@@ -0,0 +1,195 @@
+import html2text
+import smtplib, ssl
+import logging
+from email.message import EmailMessage
+from email.headerregistry import Address
+from typing import Optional
+
+from app.log import logger_reset
+from app.config import settings
+
+log = logging.getLogger(__name__)
+
+# ### BEGIN ### API Lib Email ### send_email() ###
+# Moved from lib_general.py 2026-01-07
+@logger_reset
+def send_email(
+        from_email: str,
+        to_email: str,
+        subject: str,
+        body_html: str,
+
+        from_name: str = '',
+        reply_to_email: str = '',
+        reply_to_name: str = '',
+        to_name: str = '',
+        cc_email: str = '',
+        cc_name: str = '',
+        bcc_email: str = '',
+        bcc_name: str = '',
+        body_text: str = '',
+
+        test: bool = False,
+        log_lvl: int = logging.WARNING, # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        ):
+    log.setLevel(log_lvl)
+    log.debug(locals())
+
+    if test:
+        log.setLevel(logging.DEBUG)
+        log.debug('[TESTING] Running with send_email() in TEST mode')
+
+    message = EmailMessage()
+    if subject:
+        message['Subject'] = subject
+    else:
+        return False
+
+    if from_email and from_name:
+        try:
+            message['From'] = Address(display_name=from_name, addr_spec=from_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+    elif from_email:
+        try:
+            message['From'] = Address(display_name=from_email, addr_spec=from_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+    else:
+        return False
+
+    if reply_to_email and reply_to_name:
+        try:
+            message['Reply-To'] = Address(display_name=reply_to_name, addr_spec=reply_to_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+    elif reply_to_email:
+        try:
+            message['Reply-To'] = Address(display_name=reply_to_email, addr_spec=reply_to_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+
+    if to_email and to_name:
+        try:
+            message['To'] = Address(display_name=to_name, addr_spec=to_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+    elif to_email:
+        try:
+            message['To'] = Address(display_name=to_email, addr_spec=to_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+    else:
+        return False
+
+    if cc_email and cc_name:
+        try:
+            message['Cc'] = Address(display_name=cc_name, addr_spec=cc_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+    elif cc_email:
+        try:
+            message['Cc'] = Address(display_name=cc_email, addr_spec=cc_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+
+    if bcc_email and bcc_name:
+        try:
+            message['Bcc'] = Address(display_name=bcc_name, addr_spec=bcc_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+    elif bcc_email:
+        try:
+            message['Bcc'] = Address(display_name=bcc_email, addr_spec=bcc_email)
+        except Exception as e:
+            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
+            return False
+
+    html_version = """
+<html>
+<body>
+
+"""+body_html+"""
+
+</body>
+</html>
+    """
+
+    if body_text:
+        text_version = body_text
+    else:
+        text_version = html2text.html2text(html_version)
+
+    message.set_content(text_version)
+    message.add_alternative(html_version, subtype='html')
+
+    log.info('Sending email...')
+    
+    # Safe access to SMTP settings
+    smtp_settings = getattr(settings, 'SMTP', {})
+    if not smtp_settings:
+        log.error('SMTP settings not found in configuration. Returning False.')
+        return False
+
+    log.debug(smtp_settings)
+
+    log.info(f'Subject: {subject}')
+    log.info(f'From: {from_email} Reply To: {reply_to_email} To: {to_email} CC: {cc_email} BCC: {bcc_email}')
+
+    log.debug('Message:')
+    log.debug(message.as_string())
+
+    log.info('Creating SMTP SSL connection...')
+    context = ssl.create_default_context()
+
+    # Validate SMTP settings
+    smtp_server = smtp_settings.get('server')
+    smtp_port = smtp_settings.get('port')
+    smtp_username = smtp_settings.get('username')
+    smtp_password = smtp_settings.get('password')
+
+    if not smtp_server or not smtp_port:
+        log.error(f'Error: SMTP server or port not configured. Server: {smtp_server}, Port: {smtp_port}')
+        return False
+    
+    try:
+        smtp_port = int(smtp_port)
+    except ValueError:
+        log.error(f'Error: Invalid SMTP port: {smtp_port}')
+        return False
+
+    log.info('SMTP configuration, connect, and send')
+    log.info(f'Server: {smtp_server} Port: {smtp_port} Username: {smtp_username}')
+
+    log.info('Trying smtplib.SMTP_SSL in send_email()...')
+    if test:
+        log.info('[TESTING] Email will NOT actually be sent! [TEST MODE]')
+    try:
+        with smtplib.SMTP_SSL(smtp_server, smtp_port, context=context) as server:
+            log.info('SMTP log in...')
+            # Avoid logging password in debug
+            log.debug(f'Server: {smtp_server} Port: {smtp_port} Username: {smtp_username}')
+            
+            if smtp_username and smtp_password:
+                server.login(smtp_username, smtp_password)
+            
+            log.info('SMTP send message...')
+            if not test:
+                log.info('Email sent! Returning True')
+                server.send_message(message)
+            else:
+                log.info('[TESTING] Email (NOT) sent! Returning True [TEST MODE]')
+        return True
+    except Exception as e:
+        log.error(f'Error: Unable to send email. Exception: {e}')
+        return False
+# ### END ### API Lib Email ### send_email() ###

app/lib_export.py

@@ -0,0 +1,116 @@
+import os
+import pandas
+import pathlib
+import logging
+from typing import Optional, Union
+
+from app.log import logger_reset
+from app.config import settings
+
+log = logging.getLogger(__name__)
+
+# ### BEGIN ### API Lib Export ### create_export_file() ###
+# Moved from lib_general.py 2026-01-07
+@logger_reset
+def create_export_file(
+        data_dict_list: list,
+        subdir_path: str,
+        filename: str,
+        column_name_li: list = [],
+        rm_id: bool = True,
+        export_type: str = 'CSV', # CSV, Excel
+        ) -> Union[bool, str]:
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    hosted_tmp_path = settings.FILES_PATH['hosted_tmp_root']
+    log.info(f'Hosted Temp Path: {hosted_tmp_path}')
+
+    subdirectory_dest = os.path.join(hosted_tmp_path, subdir_path)
+    log.debug(subdirectory_dest)
+    pathlib.Path(subdirectory_dest).mkdir(parents=True, exist_ok=True)
+    file_dest_w_subdir = os.path.join(subdirectory_dest, filename)
+    log.info(f'File Dest With Subdir: {file_dest_w_subdir}')
+
+    if column_name_li:
+        log.info('Using column name list passed')
+    else:
+        log.info('Using an auto generated column name list')
+        column_name_li = list(data_dict_list[0].keys())
+    log.debug(column_name_li)
+
+    if rm_id:
+        for column_name in list(column_name_li):
+            if column_name.endswith('_id'):
+                column_name_li.remove(column_name)
+                log.info(f'Removing column name: {column_name}')
+    log.info(column_name_li)
+
+    data_dataframe = pandas.DataFrame(data_dict_list)
+    log.debug(data_dataframe)
+
+    missing_cols = [col for col in column_name_li if col not in data_dataframe.columns]
+    if missing_cols:
+        column_name_li = [col for col in column_name_li if col not in missing_cols]
+
+    try:
+        if export_type == 'CSV':
+            log.info('Saving dataframe to CSV file')
+            full_dest_path = file_dest_w_subdir+'.csv'
+            filename_w_ext = filename+'.csv'
+            tmp_file_path = os.path.join(subdir_path,filename_w_ext)
+            data_dataframe.to_csv(
+                full_dest_path,
+                na_rep='NULL',
+                columns=column_name_li,
+                index=False,
+                )
+        elif export_type == 'Excel':
+            log.info('Saving dataframe to Excel file')
+            full_dest_path = file_dest_w_subdir+'.xlsx'
+            filename_w_ext = filename+'.xlsx'
+            tmp_file_path = os.path.join(subdir_path,filename_w_ext)
+            data_dataframe.to_excel(
+                full_dest_path,
+                na_rep='NULL',
+                columns=column_name_li,
+                index=False,
+                )
+    except:
+        log.exception('Something went wrong while trying to save the export file.')
+        return False
+
+    log.info(f'Temp File Path: {tmp_file_path}')
+
+    return tmp_file_path 
+# ### END ### API Lib Export ### create_export_file() ###
+
+# ### BEGIN ### API Lib Export ### return_full_tmp_path() ###
+# This is for using with return FileResponse(path=full_tmp_path, filename=filename)
+# Moved from lib_general.py 2026-01-07
+@logger_reset
+def return_full_tmp_path(
+        full_tmp_path: str = None,
+        subdir_path: str = None,
+        filename: str = None,
+        ) -> Union[bool, str]:
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    hosted_tmp_path = settings.FILES_PATH['hosted_tmp_root']
+    log.info(f'Hosted Temp Path: {hosted_tmp_path}')
+
+    if full_tmp_path:
+        file_dest = os.path.join(hosted_tmp_path, full_tmp_path)
+        return file_dest
+    elif subdir_path and filename:
+        subdirectory_dest = os.path.join(hosted_tmp_path, subdir_path)
+        log.debug(subdirectory_dest)
+        pathlib.Path(subdirectory_dest).mkdir(parents=True, exist_ok=True)
+        file_dest_w_subdir = os.path.join(subdirectory_dest, filename)
+        log.info(f'File Dest With Subdir: {file_dest_w_subdir}')
+
+        return file_dest_w_subdir
+    else:
+        return False
+# ### END ### API Lib Export ### return_full_tmp_path() ###

app/lib_general.py

@@ -1,23 +1,20 @@
-# from __future__ import annotations
-import datetime, html2text, jwt, os, pandas, pathlib, pytz, redis, time
-from passlib.hash import argon2
-
-# Import smtplib for the actual sending function
-import smtplib, ssl
-
-# Import the email package modules needed
-from email.message import EmailMessage
-from email.headerregistry import Address
-from email.utils import make_msgid
-
-from fastapi import APIRouter, Depends, Header, HTTPException, Response, status
-from pydantic import BaseModel, EmailStr, Field
 from typing import Dict, List, Optional, Set, Union
+import logging
 
-from app.log import log, logging, logger_reset
+from fastapi import Header, HTTPException, Response, status
+
+from app.log import logger_reset
 from app.config import settings
 from app.db_sql import redis_lookup_id_random, sql_select
 
+from app.lib_email import send_email
+from app.lib_export import create_export_file, return_full_tmp_path
+from app.lib_jwt import sign_jwt, decode_jwt
+from app.lib_hash import secure_hash_string, verify_secure_hash_string
+
+log = logging.getLogger(__name__)
+
+
 
 # ### BEGIN ### API Lib General ### async get_token_header() ###
 def get_token_header(x_token: str = Header(...)):
@@ -31,8 +28,9 @@ def get_token_header(x_token: str = Header(...)):
 class Common_Route_Params_No_Account_ID:
     def __init__(
             self,
-            x_account_id: int = None,
-            x_account_id_random: str = None,
+            x_account_id: int|None = None,
+            x_account_id_random: str|None = None,
+            x_no_account_id_token: str|None = None,
             enabled: str = 'enabled',
             limit: int = 10,
             offset: int = 0,
@@ -42,6 +40,7 @@ class Common_Route_Params_No_Account_ID:
             ):
         self.x_account_id = x_account_id
         self.x_account_id_random = x_account_id_random
+        self.x_no_account_id_token = x_no_account_id_token
         self.enabled = enabled
         self.limit = limit
         self.offset = offset
@@ -126,6 +125,7 @@ class Common_Route_Params:
 def common_route_params(
         # x_account_id: str = Header(..., min_length=11, max_length=22), # NOTE WARNING: Commented out 2023-08-17
         x_account_id: str = Header(None, min_length=11, max_length=22), # NOTE WARNING: Changed to this 2023-08-17
+        x_no_account_id: str = Header(None, min_length=11, max_length=22), # NOTE WARNING: Changed to this 2023-08-17
         # x_aether_api_key: Optional[str] = Header(..., min_length=11, max_length=22),
         # x_aether_api_token: Optional[str] = Header(..., min_length=11, max_length=22),
         # x_aether_jwt_token: Optional[str] = Header(..., min_length=11, max_length=50),
@@ -142,7 +142,7 @@ def common_route_params(
         # include: Optional[list] = [], # Leaving this and exclude commented out
         response: Response = Response,
         log_lvl: int = logging.WARNING, # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-        ) -> Common_Route_Params:
+        ) -> Common_Route_Params|Common_Route_Params_No_Account_ID:
     log.setLevel(log_lvl)
     log.debug(locals())
 
@@ -150,10 +150,20 @@ def common_route_params(
 
     x_account_id_random = x_account_id
 
-    if x_account_id := redis_lookup_id_random(table_name='account', record_id_random=x_account_id):
-        log.info(f'Found the x-account-id header with the value: {x_account_id}')
+    if x_account_id:
+        if x_account_id := redis_lookup_id_random(table_name='account', record_id_random=x_account_id):
+            log.info(f'Found the x-account-id header with the value: {x_account_id}')
+        else:
+            log.warning(f'The x-account-id header was found, but the Account ID was not found or is not valid. Account ID: {x_account_id}')
+            raise HTTPException(status_code=403, detail='The x-account-id Account ID was not found.') # Forbidden
+    elif x_no_account_id and len(x_no_account_id) > 10:
+        log.warning(f'Found the x_no_account_id header param with the value: {x_no_account_id}')
+
+        x_account_id = None
+        x_account_id_random = '--- NOT SET ---'
+
     elif x_no_account_id_token and len(x_no_account_id_token) > 10: # NOTE: Not a header value!
-        # NOTE WARNING: This token should be varified and able to be disabled quickly.
+        # NOTE WARNING: This token should be verified and able to be disabled quickly.
         log.warning(f'Found the x_no_account_id_token URL param with the value: {x_no_account_id_token}')
 
         if x_account_id := redis_lookup_id_random(table_name='account', record_id_random=x_no_account_id_token):
@@ -162,11 +172,17 @@ def common_route_params(
         else:
             x_account_id = 0
             x_account_id_random = ''
-    else:
-        log.warning(f'The x-account-id header was found, but the Account ID was not found or is not valid. Account ID: {x_account_id}')
-        raise HTTPException(status_code=403, detail='The x-account-id Account ID was not found.') # Forbidden
 
-    commons = Common_Route_Params( x_account_id=x_account_id, x_account_id_random=x_account_id_random, x_no_account_id_token=x_no_account_id_token, limit=limit, offset=offset, enabled=enabled, by_alias=by_alias, exclude_unset=exclude_unset, response=response )
+        x_account_id = 0
+        x_account_id_random = '--- NOT SET ---'
+    else:
+        log.warning(f'The x-account-id and x-no-account-id-token headers were not found.')
+        raise HTTPException(status_code=403, detail='The x-account-id and x-no-account-id-token headers were not found.') # Forbidden
+
+    if x_account_id:
+        commons = Common_Route_Params( x_account_id=x_account_id, x_account_id_random=x_account_id_random, x_no_account_id_token=x_no_account_id_token, limit=limit, offset=offset, enabled=enabled, by_alias=by_alias, exclude_unset=exclude_unset, response=response )
+    else:
+        commons = Common_Route_Params_No_Account_ID( x_account_id=None, x_account_id_random=None, x_no_account_id_token=x_no_account_id_token, limit=limit, offset=offset, enabled=enabled, by_alias=by_alias, exclude_unset=exclude_unset, response=response )
 
     log.debug(commons)
 
@@ -245,347 +261,4 @@ def common_route_params_min(
 # ### END ### API Lib General ### async common_route_params_min() ###
 
 
-def secure_hash_string(string: str):
-    string_hash = argon2.using(rounds=14, memory_cost=1536, parallelism=2).hash(string)
 
-    return string_hash
-
-
-def verify_secure_hash_string(string: str, string_hash: str):
-    if argon2.verify(string, string_hash):
-        return True
-    else:
-        return False
-
-
-# ### BEGIN ### API Lib General ### sign_jwt() ###
-# Updated 2021-07-14
-@logger_reset
-def sign_jwt(
-        secret_key: str, # Secret/Private/Password
-        ttl: int = 60, # Default to 60 seconds
-        max_renew: int = 0, # Default to 0
-        public_key: str = None, # Will be part of the token. Use to look up secret when verifying.???
-        account_id: str = None,
-        person_id: str = None,
-        user_id: str = None,
-        json_str: str = None,
-        b64_str: str = None,
-        ) -> Dict[str, str]:
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    payload = {
-        'iat': time.time(), # Issued at
-        'eat': time.time() + ttl, # Expires at
-        'max_renew': max_renew, # Number of times allowed to request renew without API secret key
-        'public_key': public_key, # Use to lookup the secret/private/password key when verifying
-        'account_id': account_id,
-        'person_id': person_id,
-        'user_id': user_id,
-        'json_str': json_str,
-        'b64_str': b64_str,
-    }
-    secret = secret_key
-    algorithm = 'HS256'
-    token = jwt.encode(payload, secret, algorithm=algorithm)
-
-    log.debug(token)
-
-    return token
-# ### END ### API Lib General ### sign_jwt() ###
-
-
-# ### BEGIN ### API Lib General ### decode_jwt() ###
-# Updated 2021-07-14
-@logger_reset
-def decode_jwt(
-        secret_key: str,
-        token: str,
-        ) -> dict:
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    secret = secret_key
-    algorithm = 'HS256'
-
-    try:
-        decoded_token = jwt.decode(token, secret, algorithms=[algorithm])
-        log.debug(decoded_token)
-        if decoded_token['eat'] >= time.time(): return decoded_token
-        else: return False
-    except:
-        return None
-# ### END ### API Lib General ### decode_jwt() ###
-
-
-# ### BEGIN ### API Lib General ### create_export() ###
-# Updated 2021-11-23
-@logger_reset
-def create_export_file(
-        data_dict_list: list,
-        subdir_path: str,
-        filename: str,
-        column_name_li: list = [],
-        rm_id: bool = True,
-        export_type: str = 'CSV', # CSV, Excel
-        ) -> bool|str:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    hosted_tmp_path = settings.FILES_PATH['hosted_tmp_root']
-    # hosted_tmp_path = 'admin/temp'
-    log.info(f'Hosted Temp Path: {hosted_tmp_path}')
-
-    subdirectory_dest = os.path.join(hosted_tmp_path, subdir_path)
-    log.debug(subdirectory_dest)
-    pathlib.Path(subdirectory_dest).mkdir(parents=True, exist_ok=True)
-    file_dest_w_subdir = os.path.join(subdirectory_dest, filename)
-    log.info(f'File Dest With Subdir: {file_dest_w_subdir}')
-
-    if column_name_li:
-        log.info('Using column name list passed')
-    else:
-        log.info('Using an auto generated column name list')
-        column_name_li = list(data_dict_list[0].keys())
-    log.debug(column_name_li)
-
-    if rm_id:
-        for column_name in list(column_name_li):
-            # log.info(f'Checking column name: {column_name}')
-            if column_name.endswith('_id'):
-                column_name_li.remove(column_name)
-                log.info(f'Removing column name: {column_name}')
-        log.debug(column_name_li)
-
-
-    # column_name_li = ['order_line_id_random', 'order_id_random', 'product_id_random', 'product_type', 'product_name', 'quantity', 'amount', 'dollar_amount', 'message', 'person_id_random', 'person_given_name', 'person_family_name', 'person_display_name', 'person_full_name', 'person_contact_email', 'person_contact_cc_email', 'person_contact_address_name', 'person_contact_address_organization_name', 'person_contact_address_line_1', 'person_contact_address_line_2', 'person_contact_address_line_3', 'person_contact_address_city', 'person_contact_address_country_subdivision_code', 'person_contact_address_state_province', 'person_contact_address_postal_code', 'person_contact_address_country_alpha_2_code', 'person_contact_address_country_name', 'person_contact_address_country', 'order_status', 'order_created_on', 'order_updated_on']
-
-    data_dataframe = pandas.DataFrame(data_dict_list)
-    log.debug(data_dataframe)
-
-    try:
-        if export_type == 'CSV':
-            log.info('Saving dataframe to CSV file')
-            full_dest_path = file_dest_w_subdir+'.csv'
-            filename_w_ext = filename+'.csv'
-            tmp_file_path = os.path.join(subdir_path,filename_w_ext)
-            data_dataframe.to_csv(full_dest_path, columns=column_name_li, index=False)
-        elif export_type == 'Excel':
-            log.info('Saving dataframe to Excel file')
-            full_dest_path = file_dest_w_subdir+'.xlsx'
-            filename_w_ext = filename+'.xlsx'
-            tmp_file_path = os.path.join(subdir_path,filename_w_ext)
-            data_dataframe.to_excel(full_dest_path, columns=column_name_li, index=False) # sheet_name='Sheet_name_1'
-    except:
-        log.exception('Something went wrong while trying to save the export file.')
-        return False
-
-    log.info(f'Temp File Path: {tmp_file_path}')
-
-    return tmp_file_path # True
-# ### END ### API Lib General ### create_export() ###
-
-# ### BEGIN ### API Lib General ### return_full_tmp_path() ###
-# This is for using with return FileResponse(path=full_tmp_path, filename=filename)
-# Updated 2022-04-22
-@logger_reset
-def return_full_tmp_path(
-        full_tmp_path: str = None,
-        subdir_path: str = None,
-        filename: str = None,
-        ) -> bool|str:
-    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    hosted_tmp_path = settings.FILES_PATH['hosted_tmp_root']
-    # hosted_tmp_path = 'admin/temp'
-    log.info(f'Hosted Temp Path: {hosted_tmp_path}')
-
-    if full_tmp_path:
-        file_dest = os.path.join(hosted_tmp_path, full_tmp_path)
-        return file_dest
-    elif subdir_path and filename:
-        subdirectory_dest = os.path.join(hosted_tmp_path, subdir_path)
-        log.debug(subdirectory_dest)
-        pathlib.Path(subdirectory_dest).mkdir(parents=True, exist_ok=True)
-        file_dest_w_subdir = os.path.join(subdirectory_dest, filename)
-        log.info(f'File Dest With Subdir: {file_dest_w_subdir}')
-
-        return file_dest_w_subdir
-    else:
-        return False
-# ### END ### API Lib General ### return_full_tmp_path() ###
-
-
-# ### BEGIN ### API Lib General ### send_email() ###
-# Updated 2021-12-02
-@logger_reset
-def send_email(
-        from_email: str,
-        to_email: str,
-        subject: str,
-        body_html: str,
-
-        from_name: str = '',
-        reply_to_email: str = '',
-        reply_to_name: str = '',
-        to_name: str = '',
-        cc_email: str = '',
-        cc_name: str = '',
-        bcc_email: str = '',
-        bcc_name: str = '',
-        body_text: str = '',
-
-        test: bool = False
-        ):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    message = EmailMessage()
-    if subject:
-        message['Subject'] = subject
-    else:
-        return False
-
-    if from_email and from_name:
-        #message['From'] = Address(display_name=from_name, username=from_email.split('@')[0], domain=from_email.split('@')[1])
-        try:
-            message['From'] = Address(display_name=from_name, addr_spec=from_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-    elif from_email:
-        try:
-            message['From'] = Address(display_name=from_email, addr_spec=from_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-    else:
-        return False
-
-    if reply_to_email and reply_to_name:
-        try:
-            message['Reply-To'] = Address(display_name=reply_to_name, addr_spec=reply_to_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-    elif reply_to_email:
-        try:
-            message['Reply-To'] = Address(display_name=reply_to_email, addr_spec=reply_to_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-
-    if to_email and to_name:
-        try:
-            message['To'] = Address(display_name=to_name, addr_spec=to_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-    elif to_email:
-        try:
-            message['To'] = Address(display_name=to_email, addr_spec=to_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-    else:
-        return False
-
-    if cc_email and cc_name:
-        try:
-            message['Cc'] = Address(display_name=cc_name, addr_spec=cc_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-    elif cc_email:
-        try:
-            message['Cc'] = Address(display_name=cc_email, addr_spec=cc_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-
-    if bcc_email and bcc_name:
-        try:
-            message['Bcc'] = Address(display_name=bcc_name, addr_spec=bcc_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-    elif bcc_email:
-        try:
-            message['Bcc'] = Address(display_name=bcc_email, addr_spec=bcc_email)
-        except Exception as e:
-            log.exception('**** *** ** * ### BEGIN ### Exception Happened: Returning False * ** *** ****')
-            return False
-
-    html_version = """\
-<html>
-<body>
-
-"""+body_html+"""
-
-</body>
-</html>
-    """
-
-    # html_version = """\
-    #     <html>
-    #     <body>
-    #         <div>
-    #         """+body_html+"""
-    #         </div>
-    #     </body>
-    #     </html>
-    # """
-
-    if body_text:
-        text_version = body_text
-    else:
-        text_version = html2text.html2text(html_version)
-
-    message.set_content(text_version)
-
-    message.add_alternative(html_version, subtype='html')
-
-    log.info('Sending email...')
-    log.debug(settings.SMTP)
-
-    log.info(f'Subject: {subject}')
-    log.info(f'From: {from_email} Reply To: {reply_to_email} To: {to_email} CC: {cc_email} BCC: {bcc_email}')
-
-    log.debug('Message:')
-    log.debug(message.as_string())
-
-    log.info('Creating SMTP SSL connection...')
-    context = ssl.create_default_context()
-
-    # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.info('SMTP configuration, connect, and send')
-
-    if test:
-        try:
-            with smtplib.SMTP_SSL(settings.SMTP['server'], settings.SMTP['port'], context=context) as server:
-                log.debug('[TEST] SMTP log in...')
-                server.login(settings.SMTP['username'], settings.SMTP['password'])
-                log.debug('[TEST] SMTP send message... [WILL NOT SEND IN TEST MODE]')
-                # server.send_message(message)
-            log.info('[TEST] Email sent!')
-            return True
-        except:
-            #except SMTPException:
-            log.error('[TEST] Error: unable to send email')
-            return False
-
-    try:
-        with smtplib.SMTP_SSL(settings.SMTP['server'], settings.SMTP['port'], context=context) as server:
-            log.debug('SMTP log in...')
-            server.login(settings.SMTP['username'], settings.SMTP['password'])
-            log.debug('SMTP send message...')
-            server.send_message(message)
-        log.info('Email sent!')
-        return True
-    except:
-        #except SMTPException:
-        log.error('Error: unable to send email')
-        return False
-# ### END ### API Lib General ### send_email() ###

app/lib_general_v3.py

@@ -0,0 +1,47 @@
+"""
+This file contains general utility functions and helpers specifically for API v3.
+Refactored 2026-01-07 to move Auth logic to dependencies_v3.py to fix circular dependencies.
+"""
+
+import logging
+from typing import (
+    Any,
+    Dict,
+    List,
+    Optional,
+    Union,
+)
+
+from fastapi import (
+    APIRouter,
+    Depends,
+    Header,
+    HTTPException,
+    Query,
+    Request,
+    Response,
+    status,
+)
+from pydantic import (
+    BaseModel,
+    Field,
+)
+
+# Re-import from the new central auth models
+from app.models.auth_models import AccountContext
+# Import the dependency functions for backward compatibility in existing v3 routes
+from app.routers.dependencies_v3 import (
+    get_account_context,
+    get_account_context_optional,
+    PaginationParams,
+    StatusFilterParams,
+    SerializationParams,
+    DelayParams
+)
+
+from app.config import settings
+from app.log import get_logger
+
+logger = get_logger(__name__)
+
+# Note: Dependency function implementations have moved to app/routers/dependencies_v3.py

app/lib_general_v3.py.snapshot

@@ -0,0 +1,171 @@
+"""
+This file contains general utility functions and helpers specifically for API v3.
+It aims to provide a clean slate for new methods and refactor existing ones from lib_general.py
+that are relevant to the v3 API, while removing unused or outdated functionalities.
+"""
+
+# Standard library imports
+import time
+import logging
+from typing import (
+    Any,
+    Dict,
+    List,
+    Optional,
+    Union,
+)
+
+# Third-party imports
+from fastapi import (
+    APIRouter,
+    Depends,
+    Header,
+    HTTPException,
+    Query,
+    Request,
+    Response,
+    status,
+)
+from pydantic import (
+    BaseModel,
+    Field,
+    ValidationError,
+    computed_field,
+    model_validator,
+)
+
+# Internal imports (from this project)
+from app.config import settings
+from app.db_sql import redis_lookup_id_random
+from app.log import get_logger
+
+logger = get_logger(__name__)
+
+
+# --- Pydantic Model for Account Context ---
+class AccountContext(BaseModel):
+    account_id: Optional[int]
+    account_id_random: Optional[str]
+
+
+# --- Dependency Function for Account Context ---
+def get_account_context(
+    x_account_id: Optional[str] = Header(None, min_length=11, max_length=22),
+    x_no_account_id: Optional[str] = Header(None, min_length=3, max_length=100), # Assuming 'bypass' or similar string
+    x_no_account_id_token: Optional[str] = Query(None, min_length=11, max_length=22),
+) -> AccountContext:
+    """
+    Resolves the account context from headers/query parameters with defined precedence.
+    Precedence: x_account_id (header) > x_no_account_id_token (query) > x_no_account_id (header flag)
+    Raises HTTPException 403 if no valid account is found and no bypass is indicated.
+    """
+    logger.setLevel(logging.DEBUG) # Adjust as needed
+    logger.debug(locals())
+
+    resolved_account_id = None
+    resolved_account_id_random = None
+
+    if x_account_id:
+        # Primary check: x_account_id header
+        resolved_account_id_random = x_account_id
+        if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_account_id):
+            resolved_account_id = looked_up_id
+            logger.info(f'Found account from x_account_id header: {resolved_account_id}')
+        else:
+            logger.warning(f'Invalid x_account_id header provided: {x_account_id}')
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Invalid X-Account-ID header.')
+    elif x_no_account_id_token:
+        # Secondary check: x_no_account_id_token query parameter
+        resolved_account_id_random = x_no_account_id_token
+        if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_no_account_id_token):
+            resolved_account_id = looked_up_id
+            logger.info(f'Found account from x_no_account_id_token query: {resolved_account_id}')
+        else:
+            logger.warning(f'Invalid x_no_account_id_token query provided: {x_no_account_id_token}')
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Invalid X-No-Account-ID-Token query parameter.')
+    elif x_no_account_id:
+        # Tertiary check: x_no_account_id header for bypass
+        # For now, just presence indicates bypass. Can add a specific value check later if needed.
+        logger.info(f'X-No-Account-ID header found: {x_no_account_id}. Proceeding without specific account context.')
+        resolved_account_id = None # Explicitly None for "no specific account"
+        resolved_account_id_random = '--- NO ACCOUNT ---'
+    else:
+        logger.warning('No valid account context provided via X-Account-ID, X-No-Account-ID-Token, or X-No-Account-ID.')
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Account context required. Please provide X-Account-ID, X-No-Account-ID-Token, or X-No-Account-ID.')
+
+    return AccountContext(account_id=resolved_account_id, account_id_random=resolved_account_id_random)
+
+
+# --- Pydantic Model for Pagination ---
+class PaginationParams(BaseModel):
+    limit: int = 100 # Default limit
+    offset: int = 0
+
+# --- Dependency Function for Pagination ---
+def get_pagination_params(
+    limit: int = Query(100, ge=0, description="Maximum number of items to return"),
+    offset: int = Query(0, ge=0, description="Number of items to skip (for pagination)"),
+) -> PaginationParams:
+    return PaginationParams(limit=limit, offset=offset)
+
+
+# --- Pydantic Model for Status Filtering ---
+class StatusFilterParams(BaseModel):
+    enabled: str = 'enabled' # 'enabled', 'disabled', 'all'
+    hidden: str = 'not_hidden' # 'hidden', 'not_hidden', 'all'
+
+# --- Dependency Function for Status Filtering ---
+def get_status_filter_params(
+    enabled: str = Query('enabled', description="Filter by object enabled status ('enabled', 'disabled', 'all')"),
+    hidden: str = Query('not_hidden', description="Filter by object hidden status ('hidden', 'not_hidden', 'all')"),
+) -> StatusFilterParams:
+    allowed_enabled_values = {'enabled', 'disabled', 'all'}
+    allowed_hidden_values = {'hidden', 'not_hidden', 'all'}
+
+    if enabled not in allowed_enabled_values:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Invalid value for 'enabled'. Must be one of {list(allowed_enabled_values)}."
+        )
+    if hidden not in allowed_hidden_values:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Invalid value for 'hidden'. Must be one of {list(allowed_hidden_values)}."
+        )
+    return StatusFilterParams(enabled=enabled, hidden=hidden)
+
+
+# --- Pydantic Model for Serialization Options ---
+class SerializationParams(BaseModel):
+    by_alias: bool = True
+    exclude_unset: bool = False
+    exclude_defaults: bool = False # Added based on common_route_params
+    exclude_none: bool = False # Added based on common_route_params
+
+# --- Dependency Function for Serialization Options ---
+def get_serialization_params(
+    by_alias: bool = Query(True, description="Whether to use field aliases for serialization"),
+    exclude_unset: bool = Query(False, description="Whether to exclude unset fields from the response"),
+    exclude_defaults: bool = Query(False, description="Whether to exclude fields with their default values from the response"),
+    exclude_none: bool = Query(False, description="Whether to exclude fields that are None from the response"),
+) -> SerializationParams:
+    return SerializationParams(
+        by_alias=by_alias,
+        exclude_unset=exclude_unset,
+        exclude_defaults=exclude_defaults,
+        exclude_none=exclude_none,
+    )
+
+
+# --- Pydantic Model for Delay ---
+class DelayParams(BaseModel):
+    sleep_time_ms: int = 0 # Raw delay value in ms
+    sleep_time_s: float = 0.0 # Converted to seconds for time.sleep()
+
+# --- Dependency Function for Delay ---
+def get_delay_params(
+    x_delay_ms: Optional[int] = Header(0, alias='X-Delay-ms', description="Delay response for X milliseconds (header)"),
+    delay_ms: Optional[int] = Query(0, description="Delay response for X milliseconds (query parameter)"),
+) -> DelayParams:
+    calculated_delay_ms = max(x_delay_ms or 0, delay_ms or 0)
+    return DelayParams(sleep_time_ms=calculated_delay_ms, sleep_time_s=calculated_delay_ms / 1000.0)

app/lib_hash.py

@@ -0,0 +1,16 @@
+from passlib.hash import argon2
+import logging
+
+log = logging.getLogger(__name__)
+
+# Moved from lib_general.py 2026-01-07
+def secure_hash_string(string: str) -> str:
+    string_hash = argon2.using(rounds=14, memory_cost=1536, parallelism=2).hash(string)
+    return string_hash
+
+# Moved from lib_general.py 2026-01-07
+def verify_secure_hash_string(string: str, string_hash: str) -> bool:
+    if argon2.verify(string, string_hash):
+        return True
+    else:
+        return False

app/lib_id_resolver.py

@@ -0,0 +1,130 @@
+"""
+Centralized ID random to integer ID resolution.
+"""
+import logging
+import datetime
+import random
+import redis
+from app.config import settings
+from app.db_connection import db
+
+log = logging.getLogger(__name__)
+
+def redis_lookup_id_random(
+        record_id_random: int|str,
+        table_name: str,
+        check_int_id: bool = False,
+        log_lvl: int = logging.WARNING,
+        minutes: int = 30,
+        reset_rate: int = 10,
+        ) -> str|int|bool|None:
+    """
+    Looks up a record ID in Redis, falling back to SQL if not found.
+    Resolves 'id_random' (URL-safe string) to internal integer 'id'.
+    """
+    from app.db_sql import sql_select, get_id_random
+    
+    log.setLevel(log_lvl)
+
+    if isinstance(record_id_random, str) and 11 <= len(record_id_random) <= 22:
+        pass
+    elif isinstance(record_id_random, int):
+        if check_int_id:
+            if get_id_random(record_id=record_id_random, table_name=table_name):
+                return record_id_random
+            return False
+        return record_id_random
+    elif record_id_random is None:
+        return None
+    else:
+        log.error(f'Unexpected data type: {type(record_id_random)}. Expected string (11-22 chars) or int.')
+        return False
+
+    if not table_name:
+        log.error(f'Missing table_name for id_random lookup: {record_id_random}')
+        return False
+
+    r = redis.Redis(host=settings.REDIS['server'], port=settings.REDIS['port'], db=7, password=None, decode_responses=True)
+    key_name = f'{table_name}:{record_id_random}'
+
+    record_id = r.get(key_name)
+
+    # Periodic cache refresh
+    if record_id and random.randint(1, reset_rate) == 1:
+        log.warning(f'Redis: Randomly (1/{reset_rate}) refreshing cache for Key="{key_name}"')
+        record_id = None
+
+    if record_id:
+        r.setex(key_name, datetime.timedelta(minutes=minutes), value=record_id)
+        return int(record_id)
+    else:
+        data = { 'id_random': record_id_random }
+        sql = f"SELECT id FROM `{table_name}` WHERE id_random = :id_random;"
+
+        if select_results := sql_select(sql=sql, data=data):
+            if isinstance(select_results, dict):
+                if rid := select_results.get('id'):
+                    r.setex(key_name, datetime.timedelta(minutes=minutes), value=rid)
+                    return int(rid)
+                log.error('SQL result missing ID field.')
+                return False
+            else:
+                log.error(f'SQL: Duplicate id_random found in "{table_name}". Retrying...')
+                return redis_lookup_id_random(record_id_random=record_id_random, table_name=table_name)
+        else:
+            log.warning(f'SQL: ID Random "{record_id_random}" not found in "{table_name}".')
+            return None
+
+def lookup_id_random_pop(
+        obj_data: dict,
+        log_lvl: int = logging.WARNING,
+        ):
+    """
+    Resolves any *_id_random fields in a dict to their integer IDs and removes the random keys.
+    """
+    log.setLevel(log_lvl)
+    
+    # List of common prefix patterns to resolve
+    id_patterns = [
+        'account', 'activity_log', 'address', 'archive', 'contact', 'cont_edu_cert',
+        'cont_edu_cert_person', 'event', 'event_abstract', 'event_badge',
+        'event_badge_template', 'event_exhibit', 'event_file', 'event_location',
+        'event_person', 'event_person_profile', 'event_presentation',
+        'event_presenter', 'event_registration', 'event_session', 'event_track',
+        'grant', 'hosted_file', 'journal', 'journal_entry', 'membership_group',
+        'membership_person_group', 'membership_person', 'membership_type',
+        'membership_person_type', 'order', 'order_line', 'order_cart',
+        'order_cart_line', 'organization', 'page', 'person', 'post', 'product',
+        'sponsorship', 'sponsorship_cfg', 'site', 'user'
+    ]
+
+    for prefix in id_patterns:
+        key = f'{prefix}_id_random'
+        if key in obj_data:
+            table = prefix
+            if prefix == 'address_location': table = 'address'
+            if prefix in ['contact_1', 'contact_2']: table = 'contact'
+            if prefix == 'event_id_random_only': table = 'event'
+            
+            resolved_id = redis_lookup_id_random(record_id_random=obj_data[key], table_name=table)
+            obj_data[f'{prefix}_id'] = resolved_id
+            obj_data.pop(key)
+
+    # Handle polymorphic link fields
+    polymorphic = [
+        ('for_type', 'for_id_random', 'for_id'),
+        ('link_to_type', 'link_to_id_random', 'link_to_id'),
+        ('object_type', 'object_id_random', 'object_id'),
+        ('to_object_type', 'to_object_id_random', 'to_object_id'),
+        ('from_object_type', 'from_object_id_random', 'from_object_id')
+    ]
+
+    for type_key, rand_key, id_key in polymorphic:
+        if type_key in obj_data and rand_key in obj_data:
+            obj_data[id_key] = redis_lookup_id_random(
+                record_id_random=obj_data[rand_key], 
+                table_name=obj_data[type_key]
+            )
+            obj_data.pop(rand_key)
+
+    return obj_data

app/lib_jwt.py

@@ -0,0 +1,82 @@
+import jwt
+import time
+import logging
+from typing import Dict, Optional
+
+from app.log import logger_reset
+
+log = logging.getLogger(__name__)
+
+# ### BEGIN ### API Lib JWT ### sign_jwt() ###
+# Moved from lib_general.py 2026-01-07
+@logger_reset
+def sign_jwt(
+        secret_key: str, # Secret/Private/Password
+        ttl: int = 60, # Default to 60 seconds
+        max_renew: int = 0, # Default to 0
+        public_key: str = None, # Will be part of the token. Use to look up secret when verifying.???
+        account_id: str = None,
+        person_id: str = None,
+        user_id: str = None,
+        json_str: str = None,
+        b64_str: str = None,
+        **kwargs # Allow arbitrary claims
+        ) -> str:
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    # SECURITY CHECK: Ensure we are not signing numeric IDs
+    for label, val in [('account_id', account_id), ('person_id', person_id), ('user_id', user_id)]:
+        if val is not None:
+            if isinstance(val, int) or (isinstance(val, str) and val.isdigit()):
+                log.critical(f"SECURITY BREACH: Attempted to sign a numeric ID for {label}='{val}'. Only random string IDs allowed.")
+                # For now we log and proceed, but in Phase 3 we should raise an Exception
+                # raise ValueError(f"Numeric IDs cannot be signed in JWTs.")
+
+    payload = {
+        'iat': time.time(), # Issued at
+        'eat': time.time() + ttl, # Expires at
+        'max_renew': max_renew, # Number of times allowed to request renew without API secret key
+        'public_key': public_key, # Use to lookup the secret/private/password key when verifying
+        'account_id': account_id,
+        'person_id': person_id,
+        'user_id': user_id,
+        'json_str': json_str,
+        'b64_str': b64_str,
+    }
+    
+    # Merge additional claims
+    if kwargs:
+        payload.update(kwargs)
+
+    secret = secret_key
+    algorithm = 'HS256'
+    token = jwt.encode(payload, secret, algorithm=algorithm)
+
+    log.debug(token)
+
+    return token
+# ### END ### API Lib JWT ### sign_jwt() ###
+
+
+# ### BEGIN ### API Lib JWT ### decode_jwt() ###
+# Moved from lib_general.py 2026-01-07
+@logger_reset
+def decode_jwt(
+        secret_key: str,
+        token: str,
+        ) -> Optional[dict]:
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    secret = secret_key
+    algorithm = 'HS256'
+
+    try:
+        decoded_token = jwt.decode(token, secret, algorithms=[algorithm])
+        log.debug(decoded_token)
+        if decoded_token['eat'] >= time.time(): return decoded_token
+        else: return False
+    except:
+        return None
+# ### END ### API Lib JWT ### decode_jwt() ###

app/lib_log_v3.py

@@ -0,0 +1,77 @@
+import functools
+import logging
+import logging.config
+from typing import Any
+
+# Global logger instance used throughout the app
+log = logging.getLogger('root')
+
+def get_logger(name: str):
+    """Returns a logger instance by name."""
+    return logging.getLogger(name)
+
+def setup_logging(settings: Any):
+    """
+    Configures logging based on provided settings.
+    Moving this here prevents immediate execution on module import.
+    """
+    log_file_path = getattr(settings, 'LOG_PATH', {}).get('app', '/logs/aether_api.log')
+    
+    try:
+        logging.config.dictConfig({
+            'version': 1,
+            'disable_existing_loggers': False, # Critical to not kill FastAPI/Uvicorn loggers
+            'formatters': {
+                'default': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s'},
+                'long': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s', 'datefmt': '%Y-%m-%d %H:%M:%S'},
+                'short': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s', 'datefmt': '%H:%M:%S'},
+            },
+            'handlers': {
+                'console': {
+                    'class': 'logging.StreamHandler',
+                    'stream': 'ext://sys.stderr',
+                    'formatter': 'short',
+                },
+                'log_file_all': {
+                    'level': 'NOTSET',
+                    'class': 'logging.handlers.RotatingFileHandler',
+                    'formatter': 'long',
+                    'filename': log_file_path,
+                    'maxBytes': 10485760, # 10 MB
+                    'backupCount': 9
+                },
+            },
+            'loggers': {
+                'uvicorn': {'handlers': ['console'], 'level': 'INFO'},
+            },
+            'root': {
+                'handlers': ['log_file_all'],
+                'level': 'WARNING',
+            }
+        })
+        log.info(f"Logging successfully configured. Path: {log_file_path}")
+    except Exception as e:
+        print(f"Error configuring logging: {e}")
+        logging.basicConfig(level=logging.WARNING)
+
+def logger_reset(func):
+    """
+    Decorator to log function entry/exit and reset log levels.
+    """
+    @functools.wraps(func)
+    def wrapper(*args, **kwargs):
+        # Local safer access to root logger
+        root_log = logging.getLogger('root')
+
+        if func.__name__ not in ['redis_lookup_id_random', 'sql_enable_part', 'sql_hidden_part']:
+            root_log.info(f'*** Function: "{func.__name__}()"')
+        
+        root_log.debug(f'*** Function Positional Args: {args}\nFunction Key Args: {kwargs}')
+        init_log_level = root_log.level
+        
+        returned_result = func(*args, **kwargs)
+        
+        # Reset level in case it was changed during func execution
+        root_log.setLevel(init_log_level)
+        return returned_result
+    return wrapper

app/lib_redis_helpers.py

@@ -0,0 +1,247 @@
+"""
+Redis-based ID resolution and caching helpers for Aether.
+"""
+import datetime
+import random
+import redis
+import logging
+
+from app.config import settings
+
+log = logging.getLogger(__name__)
+
+def redis_lookup_id_random(
+        record_id_random: int|str,
+        table_name: str,
+        check_int_id: bool = False,
+        log_lvl: int = logging.WARNING, # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        minutes: int = 30, # Expire the Redis key after 8 minutes
+        reset_rate: int = 10, # 1 in 10 chance of resetting the Redis key
+        ) -> str|int|bool|None:
+    """
+    Looks up a record ID in Redis, falling back to SQL if not found.
+    Resolves 'id_random' (URL-safe string) to internal integer 'id'.
+    """
+    from app.db_sql import sql_select, get_id_random
+    log.setLevel(log_lvl)
+
+    if isinstance(record_id_random, str) and len(record_id_random) >= 11 and len(record_id_random) <= 22: pass
+    elif isinstance(record_id_random, int):
+        record_id = record_id_random
+        if check_int_id:
+            log.info(f'Checking the int ID if exists. Table Name: {table_name} ID: {record_id}')
+            if get_id_random_result := get_id_random(
+                    record_id = record_id,
+                    table_name = table_name,
+                    ):
+                log.info(f'The int ID exists. Returning the int ID. ID Random: {get_id_random_result}')
+                return record_id
+            else:
+                log.info(f'The int ID does not exists. Returning False. Table Name: {table_name} ID: {record_id}')
+                return False
+        else:
+            log.debug(f'Not checking if the int ID exists. Returning the int ID. ID: {record_id}')
+            return record_id
+    elif record_id_random is None:
+        log.info(f'No record ID was passed. Returning None')
+        return None
+    else:
+        log.error(f'Unexpected data type or string format: {type(record_id_random)} Expected type is a string 11 or 22 characters long.')
+        return False
+
+    if record_id_random and table_name:
+        if len(record_id_random) < 11:
+            log.error(f'The length of id_random is too short: {record_id_random} ({len(record_id_random)} chars)')
+            return False
+        elif len(record_id_random) > 22:
+            log.error(f'The length of id_random is too long: {record_id_random} ({len(record_id_random)} chars)')
+            return False
+    elif record_id_random:
+        log.error(f'Missing table_name to select from for id_random "{record_id_random}"')
+        return False
+    elif table_name:
+        log.error(f'Missing id_random to select from table "{table_name}"')
+        return False
+    else:
+        log.error('Missing table_name and record_id_random')
+        return False
+
+    r = redis.Redis(host=settings.REDIS['server'], port=settings.REDIS['port'], db=7, password=None, decode_responses=True)
+    key_name = f'{table_name}:{record_id_random}'
+
+    record_id = r.get(key_name)
+
+    if record_id and random.randint(1, reset_rate) == 1:
+        log.warning(f'Redis: Randomly (1/{reset_rate}) setting record_id to None. Key="{key_name}" value="{record_id}" TTL={r.ttl(key_name)} seconds')
+        record_id = None
+
+    if record_id:
+        r.setex(key_name, datetime.timedelta(minutes=minutes), value=record_id)
+        log.info(f'Redis: Entry found for: Key="{key_name}" value="{record_id}" TTL={r.ttl(key_name)} seconds')
+        return int(record_id)
+    elif table_name:
+        data = { 'id_random': record_id_random }
+        sql = f"SELECT id FROM `{table_name}` AS `table` WHERE `table`.id_random = :id_random;"
+
+        if select_results := sql_select(sql=sql, data=data):
+            log.debug(f'SQL: SELECT result: {select_results}')
+            if isinstance(select_results, dict):
+                log.info(f"""SQL: Found ID Random for: {str(record_id_random)} = {str(select_results.get('id'))}""")
+                if record_id := select_results.get('id'):
+                    r.setex(key_name, datetime.timedelta(minutes=minutes), value=record_id)
+                    return int(record_id)
+                else:
+                    log.error('The SQL result was not what was expected. The ID field was not found.')
+                    return False
+            else:
+                log.error(f'SQL: More than one record found in "{table_name}". Duplicate id_random!')
+                return redis_lookup_id_random(record_id_random=record_id_random, table_name=table_name)
+        else:
+            log.warning(f'SQL: ID Random "{record_id_random}" not found in "{table_name}". Returning None.')
+            return None
+
+    log.error('Unexpected state in redis_lookup_id_random.')
+    return False
+
+
+def get_id_random(
+        record_id: int,
+        table_name: str,
+        log_lvl: int = logging.WARNING, # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        ) -> str|bool|None:
+    """
+    Looks up the 'id_random' for a given internal integer ID.
+    """
+    from app.db_sql import sql_select
+    log.setLevel(log_lvl)
+
+    data = { 'id': record_id }
+    sql = f"SELECT id_random FROM `{table_name}` AS `table` WHERE `table`.id = :id;"
+
+    if select_results := sql_select(sql=sql, data=data):
+        if isinstance(select_results, dict):
+            if record_id_random := select_results.get('id_random'):
+                return str(record_id_random)
+            else:
+                log.error('The SQL result was not what was expected.')
+                return False
+        elif isinstance(select_results, list):
+            log.exception('More than one record may have been found. Duplicate ID!')
+            return False
+        else:
+            log.exception(f'Got an unexpected result while trying to look up the ID.')
+            return False
+    elif select_results is None:
+        return None
+    else:
+        return False
+
+
+def reset_redis():
+    """Flushes the Redis database used for ID caching."""
+    r = redis.Redis(host=settings.REDIS['server'], port=settings.REDIS['port'], db=7, password=None, decode_responses=True)
+    r.flushdb()
+    return True
+
+
+def lookup_id_random_pop(
+        obj_data: dict,
+        log_lvl: int = logging.WARNING, # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        ):
+    """
+    Look up and resolve id_random values to their id
+    Remove the unneeded *_id_random key from the dict
+    """
+    log.setLevel(log_lvl)
+    
+    # Common prefixes for ID resolution
+    id_prefixes = [
+        'account', 'activity_log', 'address', 'address_location', 'archive', 
+        'contact', 'contact_1', 'contact_2', 'cont_edu_cert', 'cont_edu_cert_person',
+        'event', 'event_id_random_only', 'event_abstract', 'event_badge', 
+        'event_badge_template', 'event_exhibit', 'event_file', 'event_location', 
+        'event_person', 'event_person_profile', 'event_presentation', 
+        'event_presenter', 'event_registration', 'event_session', 'event_track', 
+        'grant', 'hosted_file', 'journal', 'journal_entry', 'membership_group', 
+        'membership_person_group', 'membership_person', 'membership_type', 
+        'membership_person_type', 'order', 'order_line', 'order_cart', 
+        'order_cart_line', 'organization', 'page', 'person', 'poc_event_person', 
+        'poc_person', 'post', 'product', 'sponsorship', 'sponsorship_cfg', 
+        'site', 'user'
+    ]
+
+    for prefix in id_prefixes:
+        key_random = f'{prefix}_id_random'
+        key_id = f'{prefix}_id'
+        
+        # Table name mapping
+        table = prefix
+        if prefix == 'address_location': table = 'address'
+        elif prefix in ['contact_1', 'contact_2']: table = 'contact'
+        elif prefix == 'event_id_random_only': table = 'event'
+        elif prefix == 'poc_event_person': table = 'event_person'
+        elif prefix == 'poc_person': table = 'person'
+
+        resolved_id = None
+
+        # Scenario A: Legacy suffix (e.g., account_id_random: "abc")
+        if key_random in obj_data:
+            resolved_id = redis_lookup_id_random(record_id_random=obj_data[key_random], table_name=table)
+            obj_data.pop(key_random)
+            
+        # Scenario B: Vision naming (e.g., account_id: "abc")
+        # Only resolve if it's a string of the correct length (random ID format)
+        elif key_id in obj_data and isinstance(obj_data[key_id], str) and 11 <= len(obj_data[key_id]) <= 22:
+            resolved_id = redis_lookup_id_random(record_id_random=obj_data[key_id], table_name=table)
+
+        if resolved_id is not None:
+            # Set the target ID field
+            target_id_key = key_id
+            if prefix == 'event_id_random_only': target_id_key = 'event_id_only'
+            obj_data[target_id_key] = resolved_id
+            
+            # Removed the short prefix version (e.g., obj_data['account'] = 1) 
+            # as it causes 'Unknown column' errors in direct table inserts.
+
+    # Polymorphic links
+    polymorphic = [
+        ('for_type', 'for_id_random', 'for_id'),
+        ('link_to_type', 'link_to_id_random', 'link_to_id'),
+        ('object_type', 'object_id_random', 'object_id'),
+        ('to_object_type', 'to_object_id_random', 'to_object_id'),
+        ('from_object_type', 'from_object_id_random', 'from_object_id')
+    ]
+
+    for type_key, rand_key, id_key in polymorphic:
+        # Handle random key if present
+        if type_key in obj_data and rand_key in obj_data:
+            obj_data[id_key] = redis_lookup_id_random(
+                record_id_random=obj_data.get(rand_key), 
+                table_name=obj_data.get(type_key)
+            )
+            obj_data.pop(rand_key)
+        # Handle Vision naming (id_key contains the string)
+        elif type_key in obj_data and id_key in obj_data and isinstance(obj_data[id_key], str) and 11 <= len(obj_data[id_key]) <= 22:
+            obj_data[id_key] = redis_lookup_id_random(
+                record_id_random=obj_data.get(id_key), 
+                table_name=obj_data.get(type_key)
+            )
+
+    return obj_data
+
+
+def get_account_id_w_for_type_id(
+        for_type: str, # This is the table name
+        for_id: int|str,
+        ) -> bool|int|None:
+    """Helper to find an account_id associated with an object."""
+    from app.db_sql import sql_select
+    log.setLevel(logging.WARNING)
+    
+    if fid := redis_lookup_id_random(record_id_random=for_id, table_name=for_type):
+        data = {'for_id': fid}
+        sql = f"SELECT account_id FROM `{for_type}` WHERE id = :for_id LIMIT 1;"
+        if result := sql_select(data=data, sql=sql):
+            return result.get('account_id')
+        return False
+    return None

app/lib_schema_v3.py

@@ -0,0 +1,67 @@
+from typing import Any, Dict
+from sqlalchemy import text
+from app.db_sql import db
+from app.ae_obj_types_def import obj_type_kv_li
+
+def get_object_schema_info(obj_type: str, view: str = 'default', variant: str = 'base') -> Dict[str, Any]:
+    """
+    Introspects an object type to return its database and model structure.
+    
+    Args:
+        obj_type: The name of the object (e.g., 'person').
+        view: The SQL view to describe (default, detail, etc.).
+        variant: The model variant to describe (base, in, out).
+        
+    Returns:
+        A dictionary containing database column info and Pydantic field info.
+    """
+    if obj_type not in obj_type_kv_li:
+        return {"error": f"Object type '{obj_type}' not found."}
+
+    obj_cfg = obj_type_kv_li[obj_type]
+    table_name = obj_cfg.get(f'tbl_{view}', obj_cfg.get('tbl_default', obj_cfg.get('tbl')))
+    model_key = f'mdl_{variant}' if variant != 'base' else 'mdl'
+    model = obj_cfg.get(model_key, obj_cfg.get('mdl_default', obj_cfg.get('mdl')))
+
+    if not table_name:
+        return {"error": f"Table configuration for '{obj_type}' is missing."}
+
+    schema_info = {
+        "object_type": obj_type,
+        "view": view,
+        "variant": variant,
+        "database": {"table_name": table_name, "columns": []},
+        "model": {"name": model.__name__ if hasattr(model, '__name__') else str(model), "fields": {}}
+    }
+
+    # 1. Database Introspection
+    try:
+        db_result = db.execute(text(f"DESCRIBE `{table_name}`"))
+        for row in db_result.fetchall():
+            # row format: (Field, Type, Null, Key, Default, Extra)
+            schema_info["database"]["columns"].append({
+                "field": row[0], 
+                "db_type": row[1], 
+                "nullable": row[2] == 'YES',
+                "required": row[2] == 'NO', # Explicitly capture NOT NULL
+                "key": row[3], 
+                "db_default": row[4], 
+                "extra": row[5]
+            })
+    except Exception as e:
+        schema_info["database"]["error"] = str(e)
+
+    # 2. Pydantic Model Introspection
+    if model and hasattr(model, "__fields__"):
+        for field_name, field in model.__fields__.items():
+            field_info = {
+                "alias": field.alias, 
+                "type": str(field.outer_type_), 
+                "required": field.required, 
+                "default": field.default
+            }
+            if field.field_info.description: 
+                field_info["description"] = field.field_info.description
+            schema_info["model"]["fields"][field_name] = field_info
+
+    return schema_info

app/lib_sql_core.py

@@ -0,0 +1,82 @@
+"""
+Foundational SQL connection management for the Aether API.
+Isolates the SQLAlchemy engine and global connection state to prevent circular imports.
+"""
+import logging
+import threading
+from typing import Any, Optional
+from sqlalchemy import create_engine
+from app.config import settings
+
+log = logging.getLogger('root')
+log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+
+# 1. Thread-local storage for capturing last SQL error message
+_sql_error_state = threading.local()
+
+def get_last_sql_error() -> Optional[str]:
+    """Retrieves and clears the last captured SQL error message."""
+    error = getattr(_sql_error_state, 'last_error', None)
+    _sql_error_state.last_error = None
+    return error
+
+def set_last_sql_error(error: Any):
+    """Sets the last captured SQL error message."""
+    _sql_error_state.last_error = str(error)
+
+
+# 2. Initial Engine Setup
+db_uri = settings.SQLALCHEMY_DB_URI
+
+def create_ae_engine(uri: str):
+    return create_engine(
+        url = uri,
+        echo = False,
+        pool_size = settings.DB.get('pool_size', 10),
+        max_overflow = settings.DB.get('max_overflow', 20),
+        pool_use_lifo = True,
+        pool_pre_ping = True,
+        pool_recycle = settings.DB['pool_recycle'],
+        isolation_level = 'READ COMMITTED',
+        connect_args = {'connect_timeout': settings.DB['connect_timeout']}
+    )
+
+engine = create_ae_engine(db_uri)
+
+# DEPRECATED: Global shared 'db' connection. Use engine.connect() in context managers instead.
+# Keeping for legacy compatibility but will phase out usage in crud lib.
+db = engine.connect()
+
+log.info('DB SQL Core: Initializing engine...')
+
+
+# 3. Connection Management Logic
+def reconnect_db() -> bool:
+    """
+    Re-initializes the global database engine using current settings.
+    Useful after bootstrapping new credentials from the 'cfg' table.
+    """
+    global engine, db, db_uri
+
+    log.info("DB SQL Core: Refreshing database connection engine...")
+    try:
+        if engine:
+            engine.dispose()
+            log.info("DB SQL Core: Disposed of previous database engine.")
+
+        db_uri = settings.SQLALCHEMY_DB_URI
+        engine = create_ae_engine(db_uri)
+        db = engine.connect()
+
+        safe_uri = db_uri.split('@')[-1] if '@' in db_uri else db_uri
+        log.info(f"DB SQL Core: Database engine re-established successfully: {safe_uri}")
+        return True
+    except Exception:
+        log.exception("DB SQL Core: FAILED to refresh database engine!")
+        return False
+
+def sql_connect(current_db=None, log_lvl: int = logging.INFO) -> bool:
+    """Refreshes the global database connection."""
+    log.setLevel(log_lvl)
+    log.info('DB SQL Core: Refreshing database connection via sql_connect...')
+    return reconnect_db()

app/lib_sql_crud.py

@@ -0,0 +1,392 @@
+"""
+Standardized SQL CRUD operations for the Aether API.
+Provides high-level helpers for INSERT, UPDATE, SELECT, and DELETE.
+"""
+import logging
+import json
+from typing import Any, List, Optional
+from sqlalchemy import text, Time
+from sqlalchemy.exc import IntegrityError, OperationalError, ProgrammingError
+
+from app.log import log, logger_reset
+# CRITICAL: Import the core module to access current global state
+from app import lib_sql_core
+from app.lib_sql_core import sql_connect, set_last_sql_error
+
+log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+
+# Helper for resolving random IDs
+from app.lib_redis_helpers import lookup_id_random_pop
+
+# ### BEGIN ### API DB SQL ### sql_insert() ###
+@logger_reset
+def sql_insert(
+        sql: str|None = None,
+        data: dict|None = None,
+        table_name: str|None = None,
+        rm_id_random: bool = False,
+        id_random_length: int = 8,
+        log_lvl: int = logging.WARNING,
+        ) -> None|bool|int:
+    log.setLevel(log_lvl)
+
+    if sql:
+        sql_insert_stmt = text(sql)
+    elif table_name and data:
+        if rm_id_random:
+            data = lookup_id_random_pop(obj_data=data)
+        if not data.get('id_random', None) and id_random_length:
+            import secrets
+            data['id_random'] = secrets.token_urlsafe(id_random_length)
+
+        fields = []
+        values = []
+        for key, value in data.items():
+            if key != 'id':
+                fields.append('`'+str(key)+'`')
+                values.append(':'+str(key))
+                if isinstance(value, (dict, list)):
+                    data[key] = json.dumps(value)
+
+        sql_insert_stmt = text(f"INSERT INTO `{table_name}` ({', '.join(fields)}) VALUES ({', '.join(values)});")
+    else:
+        log.error('SQL INSERT statement could not be created. Missing params.')
+        return False
+
+    trans = None
+    try:
+        with lib_sql_core.engine.connect() as conn:
+            trans = conn.begin()
+            result_insert = conn.execute(sql_insert_stmt, data)
+            trans.commit()
+            if result_insert.rowcount == 1 and result_insert.lastrowid > 0:
+                return result_insert.lastrowid
+            return False
+    except IntegrityError as e:
+        if trans: trans.rollback()
+        log.error('Integrity error (likely duplicate). Returning None')
+        log.debug(e)
+        set_last_sql_error(e)
+        return None
+    except Exception as e:
+        if trans: trans.rollback()
+        log.error('Unknown exception in sql_insert. Returning False')
+        log.exception(e)
+        set_last_sql_error(e)
+        return False
+# ### END ### API DB SQL ### sql_insert() ###
+
+
+# ### BEGIN ### API DB SQL ### sql_update() ###
+@logger_reset
+def sql_update(
+        sql: str|None = None,
+        data: dict|None = None,
+        table_name: str|None = None,
+        record_id: int|None = None,
+        record_id_random: str|None = None,
+        rm_id_random: bool = False,
+        id_random_length: None|int = None,
+        log_lvl: int = logging.WARNING,
+        ):
+    log.setLevel(log_lvl)
+
+    if sql:
+        sql_update_stmt = text(sql)
+    elif table_name and data:
+        if rm_id_random:
+            data = lookup_id_random_pop(obj_data=data)
+        if not data.get('id_random', None) and id_random_length:
+            import secrets
+            data['id_random'] = secrets.token_urlsafe(id_random_length)
+
+        field_list = []
+        for key, value in data.items():
+            if key != 'id':
+                field_list.append('`'+str(key) + '` = :' + str(key))
+                if isinstance(value, (dict, list)):
+                    data[key] = json.dumps(value)
+
+        sql_set = ', '.join(field_list)
+        if len(sql_set) < 4:
+            return None
+
+        if record_id:
+            data['id'] = record_id
+            sql_update_stmt = text(f'UPDATE `{table_name}` SET {sql_set} WHERE id = :id')
+        elif record_id_random:
+            data['id_random'] = record_id_random
+            sql_update_stmt = text(f'UPDATE `{table_name}` SET {sql_set} WHERE id_random = :id_random')
+        elif 'id' in data:
+            sql_update_stmt = text(f'UPDATE `{table_name}` SET {sql_set} WHERE id = :id')
+        elif 'id_random' in data:
+            sql_update_stmt = text(f'UPDATE `{table_name}` SET {sql_set} WHERE id_random = :id_random')
+        else:
+            return False
+    else:
+        return False
+
+    trans = None
+    try:
+        with lib_sql_core.engine.connect() as conn:
+            trans = conn.begin()
+            result_update = conn.execute(sql_update_stmt, data)
+            trans.commit()
+            if result_update.rowcount >= 1:
+                return True
+            return None
+    except OperationalError:
+        if trans: trans.rollback()
+        log.error('Operational error (gone away?). Retrying once...')
+        sql_connect()
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                trans = conn.begin()
+                result_update = conn.execute(sql_update_stmt, data)
+                trans.commit()
+                if result_update.rowcount >= 1:
+                    return True
+                return None
+        except Exception as e:
+            set_last_sql_error(e)
+            return False
+    except Exception as e:
+        if trans: trans.rollback()
+        log.exception(e)
+        set_last_sql_error(e)
+        return False
+# ### END ### API DB SQL ### sql_update() ###
+
+
+# ### BEGIN ### Core Help CRUD ### sql_insert_or_update() ###
+@logger_reset
+def sql_insert_or_update(
+        sql: str|None = None,
+        data: dict|None = None,
+        table_name: str|None = None,
+        rm_id_random: bool = False,
+        id_random_length: int|None = None,
+        log_lvl: int = logging.DEBUG,
+        ):
+    log.setLevel(log_lvl)
+
+    if sql:
+        stmt = text(sql)
+    elif table_name and data:
+        if rm_id_random:
+            data = lookup_id_random_pop(obj_data=data)
+        if not data.get('id_random', None) and id_random_length:
+            import secrets
+            data['id_random'] = secrets.token_urlsafe(id_random_length)
+
+        fields = [f'`{k}`' for k in data.keys() if k != 'id']
+        placeholders = [f':{k}' for k in data.keys() if k != 'id']
+        updates = [f'`{k}` = :{k}' for k in data.keys() if k != 'id']
+
+        for k, v in data.items():
+            if isinstance(v, (dict, list)):
+                data[k] = json.dumps(v)
+
+        stmt = text(f"INSERT INTO `{table_name}` ({', '.join(fields)}) VALUES ({', '.join(placeholders)}) "
+                    f"ON DUPLICATE KEY UPDATE {', '.join(updates)};")
+    else:
+        return False
+
+    trans = None
+    try:
+        with lib_sql_core.engine.connect() as conn:
+            trans = conn.begin()
+            res = conn.execute(stmt, data)
+            trans.commit()
+            return res.lastrowid if res.lastrowid > 0 else True
+    except Exception as e:
+        if trans: trans.rollback()
+        log.exception(e)
+        return False
+# ### END ### Core Help CRUD ### sql_insert_or_update() ###
+
+
+# ### BEGIN ### Core Help CRUD ### sql_select() ###
+@logger_reset
+def sql_select(
+        table_name: str|None = None,
+        record_id: int|None = None,
+        record_id_random: str|None = None,
+        field_name: str|None = None,
+        field_value = None,
+        enabled: str|None = None,
+        hidden: str|None = None,
+        qry_dict_li: dict|None = None,
+        fulltext_qry_dict: dict|None = None,
+        and_qry_dict: dict|None = None,
+        and_like_dict: dict|None = None,
+        or_like_dict: dict|None = None,
+        and_in_dict_li: dict|None = None,
+        search_query: Any|None = None,
+        searchable_fields: List[str]|None = None,
+        order_by_li: dict|None = None,
+        limit: int = 9999999,
+        offset: int = 0,
+        sql: str|None = None,
+        data: dict|None = None,
+        rm_id_random: bool = False,
+        as_dict: bool|None = True,
+        as_list: bool|None = False,
+        max_count: int = 100000,
+        log_lvl: int = logging.WARNING,
+        ) -> None|bool|dict|list:
+    from app.lib_sql_search import (
+        sql_enable_part, sql_hidden_part, sql_search_qry_part,
+        sql_where_qry_part, sql_fulltext_qry_part, sql_and_qry_part,
+        sql_and_like_part, sql_or_like_part, sql_and_in_dict_li_part
+    )
+
+    log.setLevel(log_lvl)
+
+    sql_limit_offset = f'LIMIT {limit} OFFSET {offset}' if limit >= 0 and offset >= 0 else ''
+
+    sql_order_by = ''
+    if order_by_li and isinstance(order_by_li, dict):
+        order_by_str_li = [f'`{table_name}`.`{k}` {v}' for k, v in order_by_li.items()]
+        sql_order_by = f"ORDER BY {', '.join(order_by_str_li)}"
+
+    if table_name and not (record_id or record_id_random or field_name or field_value or sql or data):
+        data = {}
+        s_en, d_en = sql_enable_part(table_name, enabled) if enabled else ('', None)
+        s_hi, d_hi = sql_hidden_part(table_name, hidden) if hidden else ('', None)
+        if d_en is not None: data['enabled'] = d_en
+        if d_hi is not None: data['hidden'] = d_hi
+
+        s_search, d_search = ('', {})
+        if search_query:
+            s_search, d_search = sql_search_qry_part(search_query, searchable_fields, table_name=table_name)
+            data.update(d_search)
+
+        stmt = text(f"SELECT * FROM `{table_name}` WHERE 1=1 {s_search} {s_en} {s_hi} {sql_order_by} {sql_limit_offset};")
+
+    elif table_name and (record_id or record_id_random) and not (field_name or field_value or sql or data):
+        data = {'rid': record_id} if record_id else {'ridr': record_id_random}
+        where = f"`{table_name}`.id = :rid" if record_id else f"`{table_name}`.id_random = :ridr"
+        stmt = text(f"SELECT * FROM `{table_name}` WHERE {where} {sql_order_by} {sql_limit_offset};")
+
+    elif table_name and field_name and field_value and not (record_id or record_id_random or sql or data):
+        data = {field_name: field_value}
+        s_where, d_where = sql_where_qry_part(qry_dict_li) if qry_dict_li else ('', {})
+        s_ft, d_ft = sql_fulltext_qry_part(fulltext_qry_dict) if fulltext_qry_dict else ('', {})
+        s_and, d_and = sql_and_qry_part(and_qry_dict) if and_qry_dict else ('', {})
+        s_alike, d_alike = sql_and_like_part(and_like_dict) if and_like_dict else ('', {})
+        s_olike, d_olike = sql_or_like_part(or_like_dict) if or_like_dict else ('', {})
+        s_in, d_in = sql_and_in_dict_li_part(and_in_dict_li) if and_in_dict_li else ('', {})
+        s_search, d_search = sql_search_qry_part(search_query, searchable_fields, table_name=table_name) if search_query else ('', {})
+        s_en, d_en = sql_enable_part(table_name, enabled) if enabled else ('', None)
+        s_hi, d_hi = sql_hidden_part(table_name, hidden) if hidden else ('', None)
+
+        data.update(d_where); data.update(d_ft); data.update(d_and); data.update(d_alike)
+        data.update(d_olike); data.update(d_in); data.update(d_search)
+        if d_en is not None: data['enabled'] = d_en
+        if d_hi is not None: data['hidden'] = d_hi
+
+        stmt = text(f"SELECT * FROM `{table_name}` WHERE `{table_name}`.{field_name} = :{field_name} "
+                     f"{s_where} {s_ft} {s_and} {s_alike} {s_olike} {s_in} {s_search} {s_en} {s_hi} {sql_order_by} {sql_limit_offset};")
+    elif table_name and data and not (record_id or record_id_random or field_name or field_value or sql):
+        if rm_id_random: data = lookup_id_random_pop(obj_data=data)
+        where_clauses = [f"`{table_name}`.{k} = :{k}" for k in data.keys()]
+        stmt = text(f"SELECT * FROM `{table_name}` WHERE {' AND '.join(where_clauses)} {sql_order_by} {sql_limit_offset};")
+    elif sql:
+        stmt = text(sql)
+    else:
+        return False
+
+    try:
+        with lib_sql_core.engine.connect() as conn:
+            result = conn.execute(stmt, data)
+            if not result:
+                return [] if as_list else None
+
+            # Fetch all rows first to determine actual count reliably
+            if hasattr(result, 'returns_rows') and not result.returns_rows:
+                log.warning("SQL Result does not return rows (ResourceClosedError prevented).")
+                return [] if as_list else None
+                
+            rows = result.all()
+    except Exception as e:
+        log.error(f"SQL Fetch Error: {e}")
+        set_last_sql_error(e)
+        return False
+
+    count = len(rows)
+
+    if count == 0:
+        return [] if as_list else None
+
+    if count == 1:
+        record = dict(rows[0]) if as_dict else rows[0]
+        return [record] if as_list else record
+
+    # count > 1
+    records = [dict(r) for r in rows] if as_dict else rows
+    return records
+# ### END ### Core Help CRUD ### sql_select() ###
+
+
+# ### BEGIN ### API DB SQL ### run_sql_select() ###
+@logger_reset
+def run_sql_select(
+        sql: text,
+        data: dict|None = None,
+        log_lvl: int = logging.WARNING,
+        ) -> Any:
+    log.setLevel(log_lvl)
+
+    try:
+        with lib_sql_core.engine.connect() as conn:
+            return conn.execute(sql, data)
+    except (OperationalError, ProgrammingError) as e:
+        log.error(f'DB Error: {e}. Retrying once...')
+        sql_connect()
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                return conn.execute(sql, data)
+        except Exception as e2:
+            set_last_sql_error(e2)
+            raise e2 # RAISING instead of returning False
+    except Exception as e:
+        log.exception(e)
+        set_last_sql_error(e)
+        raise e # RAISING instead of returning False
+# ### END ### API DB SQL ### run_sql_select() ###
+
+# ### BEGIN ### Core Help CRUD ### sql_delete() ###
+@logger_reset
+def sql_delete(
+        table_name: str|None = None,
+        record_id: int|None = None,
+        record_id_random: str|None = None,
+        field_name: str|None = None,
+        field_value = None,
+        sql: str|None = None,
+        data: dict|None = None,
+        log_lvl: int = logging.INFO,
+        ) -> None|bool:
+    log.setLevel(log_lvl)
+
+    if table_name and (record_id or record_id_random) and not (field_name or field_value or sql or data):
+        data = {'rid': record_id} if record_id else {'ridr': record_id_random}
+        where = f"`{table_name}`.id = :rid" if record_id else f"`{table_name}`.id_random = :ridr"
+        stmt = text(f"DELETE FROM `{table_name}` WHERE {where}")
+    elif table_name and field_name and field_value and not (record_id or record_id_random or sql or data):
+        data = {field_name: field_value}
+        stmt = text(f"DELETE FROM `{table_name}` WHERE `{table_name}`.{field_name} = :{field_name}")
+    elif sql:
+        stmt = text(sql)
+    else:
+        return False
+
+    try:
+        with lib_sql_core.engine.connect() as conn:
+            result = conn.execute(stmt, data) if data else conn.execute(stmt)
+            return True if result.rowcount >= 1 else None
+    except Exception as e:
+        log.exception(e)
+        return False
+# ### END ### Core Help CRUD ### sql_delete() ###

app/lib_sql_search.py

@@ -0,0 +1,275 @@
+"""
+Modular search builder and query generators for Aether.
+"""
+import logging
+from typing import Any, List, Optional
+from fastapi import HTTPException
+from sqlalchemy import text
+
+log = logging.getLogger(__name__)
+
+def sql_limit_offset_part(limit: int, offset: int = 0) -> str:
+    """Creates a partial SQL string for LIMIT and OFFSET."""
+    if limit >= 0 and offset >= 0:
+        log.info(f'Creating partial SQL string for LIMIT and OFFSET. Limit: {limit}; Offset: {offset}')
+        return f'LIMIT {limit} OFFSET {offset}'
+    else:
+        return ''
+
+def sql_and_like_part(and_like_dict_obj: dict) -> tuple[str, dict]:
+    """Creates a partial SQL string for AND LIKE queries."""
+    data = {}
+    if and_like_dict_obj and isinstance(and_like_dict_obj, dict):
+        log.info('Creating partial SQL string for additional AND LIKE queries.')
+        clauses = []
+        for key, value in and_like_dict_obj.items():
+            clauses.append(f"{key} LIKE :and_like_{key}")
+            data[f'and_like_{key}'] = value
+        return f"AND ({' AND '.join(clauses)})", data
+    return '', {}
+
+def sql_or_like_part(or_like_dict_obj: dict) -> tuple[str, dict]:
+    """Creates a partial SQL string for OR LIKE queries."""
+    data = {}
+    if or_like_dict_obj and isinstance(or_like_dict_obj, dict):
+        log.info('Creating partial SQL string for additional OR LIKE queries.')
+        clauses = []
+        for key, value in or_like_dict_obj.items():
+            clauses.append(f"{key} LIKE :or_like_{key}")
+            data[f'or_like_{key}'] = value
+        return f"AND ({' OR '.join(clauses)})", data
+    return '', {}
+
+def sql_and_in_dict_li_part(and_in_dict_li_dict_obj: dict) -> tuple[str, dict]:
+    """Creates a partial SQL string for AND IN queries."""
+    data = {}
+    if and_in_dict_li_dict_obj and isinstance(and_in_dict_li_dict_obj, dict):
+        log.info('Creating partial SQL string for additional AND IN queries.')
+        clauses = []
+        for key, value in and_in_dict_li_dict_obj.items():
+            clauses.append(f"{key} IN :and_in_{key}")
+            data[f'and_in_{key}'] = value
+        return f"AND ({' AND '.join(clauses)})", data
+    return '', {}
+
+def sql_and_qry_part(and_qry_dict_obj: dict) -> tuple[str, dict]:
+    """Creates a partial SQL string for additional AND queries (equals)."""
+    data = {}
+    if and_qry_dict_obj and isinstance(and_qry_dict_obj, dict):
+        log.info('Creating partial SQL string for additional AND queries.')
+        clauses = []
+        for key, value in and_qry_dict_obj.items():
+            clauses.append(f"{key} = :and_{key}")
+            data[f'and_{key}'] = value
+        return f"AND ({' AND '.join(clauses)})", data
+    return '', {}
+
+def sql_fulltext_qry_part(fulltext_qry_dict: dict) -> tuple[str, dict]:
+    """Creates a partial SQL string for fulltext search."""
+    data = {}
+    if fulltext_qry_dict and isinstance(fulltext_qry_dict, dict):
+        log.info('Creating partial SQL string for fulltext search.')
+        clauses = []
+        for key, value in fulltext_qry_dict.items():
+            clauses.append(f"MATCH( {key} ) AGAINST( :ft_{key} IN BOOLEAN MODE )")
+            data[f'ft_{key}'] = value
+        return f"AND ({' OR '.join(clauses)})", data
+    return '', {}
+
+def sql_enable_part(table_name: str, enabled: str) -> tuple[str, bool|None]:
+    """Handles enabled/disabled status filtering with schema check."""
+    from app import lib_sql_core
+    if not table_name: return '', None
+    if enabled in ['enabled', 'disabled', 'not_enabled', 'all']:
+        if enabled == 'all': return '', None
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                conn.execute(text(f"SELECT enable FROM `{table_name}` LIMIT 0"))
+        except:
+            log.warning(f"Table '{table_name}' missing 'enable' column. Skipping filter.")
+            return '', None
+        val = (enabled == 'enabled')
+        return f"AND `{table_name}`.enable = {str(val).lower()}", val
+    return '', None
+
+def sql_hidden_part(table_name: str, hidden: str) -> tuple[str, bool|None]:
+    """Handles hidden status filtering with schema check."""
+    from app import lib_sql_core
+    if not table_name: return '', None
+    if hidden in ['hidden', 'not_hidden', 'all']:
+        if hidden == 'all': return '', None
+        try:
+            with lib_sql_core.engine.connect() as conn:
+                conn.execute(text(f"SELECT hide FROM `{table_name}` LIMIT 0"))
+        except:
+            log.warning(f"Table '{table_name}' missing 'hide' column. Skipping filter.")
+            return '', None
+        if hidden == 'hidden':
+            return f"AND `{table_name}`.hide = true", True
+        return f"AND (`{table_name}`.hide = false OR `{table_name}`.hide IS NULL)", False
+    return '', None
+
+def sql_where_qry_part(qry_dict_li: list) -> tuple[str, dict]:
+    """Standard v2 style WHERE clause builder."""
+    data = {}
+    if qry_dict_li and isinstance(qry_dict_li, list):
+        log.info('Creating partial SQL string for WHERE queries.')
+        clauses = []
+        for qry in qry_dict_li:
+            field = qry.get('field')
+            op = qry.get('operator')
+            val = qry.get('value')
+            type_ = qry.get('type', 'AND') or 'AND'
+            if op == 'MATCH':
+                clauses.append(f'{type_} MATCH( {field} ) AGAINST( :{field} IN BOOLEAN MODE )')
+            else:
+                clauses.append(f'{type_} {field} {op} :{field}')
+            data[field] = val
+        return ' '.join(clauses), data
+    return '', {}
+
+def sql_search_qry_part(
+        search_query: Any,
+        searchable_fields: List[str]|None = None,
+        max_depth: int = 5,
+        table_name: str|None = None,
+        ) -> tuple[str, dict]:
+    """Recursively builds a SQL WHERE clause from a SearchQuery model."""
+    from app import lib_sql_core
+    data = {}
+    param_counter = [0]
+
+    def get_param_name():
+        param_counter[0] += 1
+        return f"sp_{param_counter[0]}"
+
+    operator_map = {
+        "eq": "=", "ne": "!=", "gt": ">", "gte": ">=", "lt": "<", "lte": "<=",
+        "like": "LIKE", "in": "IN", "is_null": "IS NULL", "is_not_null": "IS NOT NULL",
+        "contains": "LIKE", "icontains": "LIKE", "startswith": "LIKE", "istartswith": "LIKE",
+        "endswith": "LIKE", "iendswith": "LIKE"
+    }
+
+    def process_node(query_node, current_depth: int) -> str:
+        if current_depth > max_depth:
+            raise HTTPException(status_code=400, detail=f"Search query too complex.")
+        clauses = []
+        if hasattr(query_node, 'query_string') and query_node.query_string:
+            if query_node.query_string == '%': pass
+            else:
+                use_match = True
+                if table_name:
+                    try:
+                        with lib_sql_core.engine.connect() as conn:
+                            conn.execute(text(f"SELECT default_qry_str FROM `{table_name}` LIMIT 0"))
+                    except:
+                        use_match = False
+                else:
+                    use_match = False
+
+                if use_match:
+                    p_name = get_param_name()
+                    clauses.append(f"MATCH( default_qry_str ) AGAINST( :{p_name} IN BOOLEAN MODE )")
+                    data[p_name] = query_node.query_string
+                elif searchable_fields:
+                    like_clauses = []
+                    # Fields to exclude from a generic text 'q' search (numeric, technical, or date fields)
+                    exclude_patterns = [
+                        'enable', 'hide', 'priority', 'sort', 'group',
+                        'created_on', 'updated_on'
+                    ]
+                    for field in searchable_fields:
+                        # Exclude internal integer IDs specifically
+                        if field.endswith('_id') or field == 'id':
+                            continue
+
+                        # Exclude other technical/meta fields
+                        if any(x == field for x in exclude_patterns):
+                            continue
+
+                        f_p_name = get_param_name()
+                        like_clauses.append(f"`{field}` LIKE :{f_p_name}")
+                        data[f_p_name] = f"%{query_node.query_string}%"
+
+                    if like_clauses: clauses.append(f"({' OR '.join(like_clauses)})")
+        for filter_attr in ['and_filters', 'or_filters']:
+            if hasattr(query_node, filter_attr) and getattr(query_node, filter_attr):
+                node_clauses = []
+                for item in getattr(query_node, filter_attr):
+                    if hasattr(item, 'field'):
+                        clause, item_data = process_filter(item)
+                        node_clauses.append(clause); data.update(item_data)
+                    else: node_clauses.append(f"({process_node(item, current_depth + 1)})")
+                if node_clauses:
+                    joiner = ' AND ' if 'and' in filter_attr else ' OR '
+                    clauses.append(f"({joiner.join(node_clauses)})")
+        return ' AND '.join(clauses)
+
+    def process_filter(f) -> tuple[str, dict]:
+        # --- ID VISION MAPPING ---
+        # If the frontend uses clean names (id, account_id),
+        # map them to the database columns (id_random, account_id_random)
+        # ONLY if those columns actually exist in this table/view.
+        target_field = f.field
+        vision_fields = [
+            'id', 'account_id', 'site_id', 'person_id', 'user_id',
+            'archive_id', 'archive_content_id',
+            'event_id',
+            'event_session_id', 'event_presentation_id', 'event_presenter_id',
+            'event_device_id', 'event_location_id', 'event_track_id',
+            'event_exhibit_id',
+            'event_person_id', 'event_registration_id',
+            'order_id', 'product_id', 'order_cart_id', 'membership_id', 'sponsorship_id',
+            'journal_id', 'journal_entry_id', 'page_id',
+            'post_id', 'post_comment_id',
+            'organization_id', 'address_id', 'contact_id',
+            'hosted_file_id'
+        ]
+
+        if target_field in vision_fields:
+            # ONLY map to _random if the value is a string (looks like a random ID)
+            # If it's an integer, we want to query the original integer column.
+            is_int_val = isinstance(f.value, int) or (isinstance(f.value, str) and f.value.isdigit())
+
+            if not is_int_val:
+                candidate_field = 'id_random' if target_field == 'id' else f"{target_field}_random"
+
+                # Schema Check: Verify if the random version exists in the current table/view
+                use_random = False
+                if table_name:
+                    try:
+                        with lib_sql_core.engine.connect() as conn:
+                            conn.execute(text(f"SELECT `{candidate_field}` FROM `{table_name}` LIMIT 0"))
+                        use_random = True
+                    except Exception:
+                        pass
+
+                if use_random:
+                    target_field = candidate_field
+                    # print(f"Search Trace: Mapping filter field '{f.field}' -> '{target_field}'", flush=True)
+                else:
+                    # If random doesn't exist, we must stick to the integer column
+                    # but we'll need to resolve the string value to an integer elsewhere
+                    # or rely on the user providing an integer for now.
+                    pass
+
+        if searchable_fields is not None and target_field not in searchable_fields:
+            # Fallback check for original field just in case
+            if f.field not in searchable_fields:
+                raise HTTPException(status_code=400, detail=f"Unauthorized search field '{f.field}' (mapped to '{target_field}')")
+
+        sql_op = operator_map.get(f.op.lower())
+        if not sql_op: raise HTTPException(status_code=400, detail=f"Unsupported operator: {f.op}")
+        filter_data = {}
+        if f.op.lower() in ['is_null', 'is_not_null']: clause = f"`{target_field}` {sql_op}"
+        else:
+            p_name = get_param_name()
+            if f.op.lower() == 'in': clause = f"`{target_field}` IN (:{p_name})"; filter_data[p_name] = f.value
+            elif f.op.lower() in ['contains', 'icontains']: clause = f"`{target_field}` LIKE :{p_name}"; filter_data[p_name] = f"%{f.value}%"
+            elif f.op.lower() in ['startswith', 'istartswith']: clause = f"`{target_field}` LIKE :{p_name}"; filter_data[p_name] = f"{f.value}%"
+            elif f.op.lower() in ['endswith', 'iendswith']: clause = f"`{target_field}` LIKE :{p_name}"; filter_data[p_name] = f"%{f.value}"
+            else: clause = f"`{target_field}` {sql_op} :{p_name}"; filter_data[p_name] = f.value
+        return clause, filter_data
+
+    sql_where = process_node(search_query, 1)
+    return (f"AND ({sql_where})", data) if sql_where else ("", {})

app/log.py

@@ -1,97 +1,5 @@
-import functools, logging
+import logging
+from app.lib_log_v3 import log, logger_reset, get_logger, setup_logging
 
-from app.config import settings
-
-# stream options: 'ext://sys.stderr' or 'ext://sys.stdout'
-
-# NOTE: This log config is confusing and may need work... 2022-10-07
-# 'uvicorn' under 'loggers' creates an output to the 'console' handler
-# Do not also add 'console' handler to the 'root' 'handlers' list
-# For now just using that to add or remove file logging options.
-logging.config.dictConfig({
-    'version': 1,
-    'formatters': {
-        'default': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s'},
-        'long': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s', 'datefmt': '%Y-%m-%d %H:%M:%S'},
-        'short': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s', 'datefmt': '%H:%M:%S', 'use_colors': True},
-    },
-    #'filename': 'example.log',
-    # 'level': logging.ERROR,
-    'handlers': {
-        'console': {
-            'class': 'logging.StreamHandler',
-            'stream': 'ext://sys.stderr',
-            'formatter': 'short',
-        },
-        'log_file_all': {
-            'level': 'NOTSET',
-            'class': 'logging.handlers.RotatingFileHandler',
-            'formatter': 'long',
-            'filename': settings.LOG_PATH['app'],
-            'maxBytes': 10485760, # 5,242,880 = 5 MB; 10,485,760 = 10 MB
-            'backupCount': 9
-        },
-        # 'log_file_warning': {
-        #     'level': 'WARNING',
-        #     'class': 'logging.handlers.RotatingFileHandler',
-        #     'formatter': 'long',
-        #     'filename': settings.LOG_PATH['app_warning'],
-        #     'maxBytes': 512000, # 524,288 =  512KB
-        #     'backupCount': 9
-        # },
-        # 'test_handler': {
-        #     'class': 'logging.StreamHandler',
-        #     'level': 'INFO',
-        #     'formatter': 'short',
-        # },
-        # 'test_handler_all_rotate': {
-        #     'class': 'logging.handlers.RotatingFileHandler',
-        #     'level': 'NOTSET',
-        #     'formatter': 'short',
-        #     'filename': '/logs/test_rotate.log',
-        #     'maxBytes': 100000, # 5120000 = 5 MB
-        #     'backupCount': 2,
-        # }
-    },
-    'loggers': {
-        # 'uvicorn': {'handlers': ['default'], 'level': 'INFO'},
-        'uvicorn': {'handlers': ['console'], 'level': 'INFO'},
-        # 'uvicorn.error': {'level': 'INFO', 'handlers': ['default'], 'propagate': True},
-        # 'uvicorn.error': {'level': 'INFO', 'handlers': ['console'], 'propagate': True},
-        # 'uvicorn.access': {'handlers': ['access'], 'level': 'INFO', 'propagate': False},
-        # 'gunicorn': {'handlers': ['console'], 'level': 'INFO'},
-    },
-    'root': {
-        'handlers': ['log_file_all'], #, 'log_file_all', 'log_file_warning'],
-        # 'handlers': ['console', 'log_file_all'], #, 'log_file_all', 'log_file_warning'],
-        'level': 'WARNING', # WARNING
-    }
-})
-
-
-log = logging.getLogger('root')
-# log.setLevel(logging.INFO) # DEBUG > INFO > WARNING > ERROR > CRITICAL
-# logging.basicConfig(
-#     format='[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s'
-#     )
-
-
-
-# ### BEGIN ### Log ### logger_reset() ###
-# https://realpython.com/primer-on-python-decorators/
-# Updated 2022-02-15
-def logger_reset(func):
-    # log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    # log.info(locals())
-    @functools.wraps(func)
-    def wrapper(*args, **kwargs):
-        if func.__name__ not in ['redis_lookup_id_random']:
-            log.info(f'*** Function: "{func.__name__}()"')
-        log.debug(f'*** Function Positional Args: {args}\nFunction Key Args: {kwargs}')
-        init_log_level = log.level
-        returned_result = func(*args, **kwargs)
-        log.debug(f'*** Function finished: "{func.__name__}()". Resetting logger level to level: {log.level} ***')
-        log.setLevel(init_log_level)
-        return returned_result
-    return wrapper
-# ### END ### Log ### logger_reset() ###
+# Re-exporting for backward compatibility with ~200 existing imports
+__all__ = ['log', 'logging', 'logger_reset', 'get_logger', 'setup_logging']

app/log.py.snapshot

@@ -0,0 +1,100 @@
+import functools, logging
+
+from app.config import settings
+
+# stream options: 'ext://sys.stderr' or 'ext://sys.stdout'
+
+# NOTE: This log config is confusing and may need work... 2022-10-07
+# 'uvicorn' under 'loggers' creates an output to the 'console' handler
+# Do not also add 'console' handler to the 'root' 'handlers' list
+# For now just using that to add or remove file logging options.
+# logging.config.dictConfig({
+#     'version': 1,
+#     'formatters': {
+#         'default': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s'},
+#         'long': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s', 'datefmt': '%Y-%m-%d %H:%M:%S'},
+#         'short': {'format': '[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s', 'datefmt': '%H:%M:%S', 'use_colors': True},
+#     },
+#     #'filename': 'example.log',
+#     # 'level': logging.ERROR,
+#     'handlers': {
+#         'console': {
+#             'class': 'logging.StreamHandler',
+#             'stream': 'ext://sys.stderr',
+#             'formatter': 'short',
+#         },
+#         'log_file_all': {
+#             'level': 'NOTSET',
+#             'class': 'logging.handlers.RotatingFileHandler',
+#             'formatter': 'long',
+#             'filename': settings.LOG_PATH['app'],
+#             'maxBytes': 10485760, # 5,242,880 = 5 MB; 10,485,760 = 10 MB
+#             'backupCount': 9
+#         },
+#         # 'log_file_warning': {
+#         #     'level': 'WARNING',
+#         #     'class': 'logging.handlers.RotatingFileHandler',
+#         #     'formatter': 'long',
+#         #     'filename': settings.LOG_PATH['app_warning'],
+#         #     'maxBytes': 512000, # 524,288 =  512KB
+#         #     'backupCount': 9
+#         # },
+#         # 'test_handler': {
+#         #     'class': 'logging.StreamHandler',
+#         #     'level': 'INFO',
+#         #     'formatter': 'short',
+#         # },
+#         # 'test_handler_all_rotate': {
+#         #     'class': 'logging.handlers.RotatingFileHandler',
+#         #     'level': 'NOTSET',
+#         #     'formatter': 'short',
+#         #     'filename': '/logs/test_rotate.log',
+#         #     'maxBytes': 100000, # 5120000 = 5 MB
+#         #     'backupCount': 2,
+#         # }
+#     },
+#     'loggers': {
+#         # 'uvicorn': {'handlers': ['default'], 'level': 'INFO'},
+#         'uvicorn': {'handlers': ['console'], 'level': 'INFO'},
+#         # 'uvicorn.error': {'level': 'INFO', 'handlers': ['default'], 'propagate': True},
+#         # 'uvicorn.error': {'level': 'INFO', 'handlers': ['console'], 'propagate': True},
+#         # 'uvicorn.access': {'handlers': ['access'], 'level': 'INFO', 'propagate': False},
+#         # 'gunicorn': {'handlers': ['console'], 'level': 'INFO'},
+#     },
+#     'root': {
+#         'handlers': ['log_file_all'], #, 'log_file_all', 'log_file_warning'],
+#         # 'handlers': ['console', 'log_file_all'], #, 'log_file_all', 'log_file_warning'],
+#         'level': 'WARNING', # WARNING
+#     }
+# })
+
+
+# log = logging.getLogger('root')
+# # log.setLevel(logging.INFO) # DEBUG > INFO > WARNING > ERROR > CRITICAL
+# # logging.basicConfig(
+# #     format='[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s'
+# #     )
+
+
+
+# ### BEGIN ### Log ### logger_reset() ###
+# https://realpython.com/primer-on-python-decorators/
+# Updated 2022-02-15
+# def logger_reset(func):
+#     # log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+#     # log.info(locals())
+#     @functools.wraps(func)
+#     def wrapper(*args, **kwargs):
+#         if func.__name__ not in ['redis_lookup_id_random', 'sql_enable_part', 'sql_hidden_part']:
+#             log.info(f'*** Function: "{func.__name__}()"')
+#         log.debug(f'*** Function Positional Args: {args}\nFunction Key Args: {kwargs}')
+#         init_log_level = log.level
+#         returned_result = func(*args, **kwargs)
+#         log.debug(f'*** Function finished: "{func.__name__}()". Resetting logger level to level: {log.level} ***')
+#         log.setLevel(init_log_level)
+#         return returned_result
+#     return wrapper
+# ### END ### Log ### logger_reset() ###
+
+def get_logger(name: str) -> logging.Logger:
+    return logging.getLogger(name)

app/main.py

@@ -1,4 +1,4 @@
-import datetime, json, os, pytz, random, secrets # , uvicorn
+import datetime, json, os, pytz, random, secrets, contextlib # , uvicorn
 
 from enum import Enum
 #from datetime import datetime, time, timedelta
@@ -10,78 +10,98 @@ from functools import lru_cache
 from pydantic import BaseModel, EmailStr, Field
 from typing import Dict, List, Optional, Set, Union
 
-# from sqlalchemy import create_engine, text
-# from sqlalchemy.exc import IntegrityError, OperationalError
-
 from . import config
 # from app.lib_general import common_route_params, Common_Route_Params
-from app.log import log, logging
+import logging
+import app.log
+from app.log import setup_logging
 
-# Import the routers here first:
-from app.routers import aether_cfg, api_crud, api, importing, sql, account, activity_log, address, archive, archive_content, contact, cont_edu_cert, cont_edu_cert_person, data_store, event, event_abstract, event_badge, event_badge_importing, event_badge_template, event_device, event_exhibit, event_exhibit_tracking, event_file, event_importing, event_location, event_person, event_person_detail, event_person_tracking, event_presentation, event_presenter, event_registration, event_session, flask_cfg, fundraising, grant, hosted_file, journal, journal_entry, log_client_viewing, lookup, membership_cfg, membership_group, membership_person_group, membership_person, membership_person_profile, membership_type, membership_person_type, order, order_v3, order_line, order_cart, organization, page, person, person_user, post, post_comment, product, qr, site, site_domain, user, util_email, websockets_redis, e_confex, e_cvent, c_idaa, e_impexium, e_stripe
+# Import middleware with alias to avoid shadowing 'app' FastAPI instance
+from app.middleware import add_process_time_header as process_time_middleware
 
-# from app.routers import aether_cfg, sql
+# Centralized router registry
+from app.routers.registry import setup_routers
 
-from app.db_sql import sql_select # , sql_connect
+from app.db_sql import sql_select, reset_redis, reconnect_db
+from app.lib_config_v3 import bootstrap_db_config, validate_critical_config
 
 
 print('### **** *** ** * The Aether API v4 using FastAPI is loading... * ** *** **** ###')
 
 
-#log = logging.getLogger('root')
+log = logging.getLogger(__name__)
 # log.setLevel(logging.DEBUG) # DEBUG > INFO > WARNING > ERROR > CRITICAL
 #logging.basicConfig(
     #format='[%(asctime)s] %(levelname)s @ %(module)s.%(funcName)s()#%(lineno)d: %(message)s'
     #)
 
+@contextlib.asynccontextmanager
+async def lifespan(app: FastAPI):
+    """
+    Handles application startup and shutdown lifecycle.
+    """
+    # 1. Initialize Logging early but safely
+    setup_logging(config.settings)
+    log.info('### **** *** ** * Aether API v4 using FastAPI - Startup Lifespan Initiated * ** *** **** ###')
+    
+    # 2. Bootstrapping Configuration from DB with robust error handling
+    log.info("Bootstrapping Configuration...")
+    
+    # Save original settings for fallback
+    orig_db_server = config.settings.DB_SERVER
+    orig_db_user = config.settings.DB_USER
+    orig_db_pass = config.settings.DB_PASS
+    orig_db_name = config.settings.DB_NAME
+    orig_db_port = config.settings.DB_PORT
 
+    try:
+        if bootstrap_db_config(config.settings):
+            log.info("Successfully bootstrapped configuration from database.")
+            # Re-initialize the database engine with new credentials/URI
+            if reconnect_db():
+                log.info("Database connection re-established with production configuration.")
+            else:
+                log.warning("FAILED to re-establish database connection after bootstrap. Reverting to .env settings.")
+                config.settings.DB_SERVER = orig_db_server
+                config.settings.DB_USER = orig_db_user
+                config.settings.DB_PASS = orig_db_pass
+                config.settings.DB_NAME = orig_db_name
+                config.settings.DB_PORT = orig_db_port
+                reconnect_db()
+        else:
+            log.warning("System bootstrap from DB returned no results. Using environment defaults.")
+    except Exception as e:
+        log.error(f"Unexpected error during configuration bootstrap: {e}. Falling back to .env settings.")
+        config.settings.DB_SERVER = orig_db_server
+        config.settings.DB_USER = orig_db_user
+        config.settings.DB_PASS = orig_db_pass
+        config.settings.DB_NAME = orig_db_name
+        config.settings.DB_PORT = orig_db_port
+        reconnect_db()
+
+    # 3. Final validation of critical infrastructure
+    validate_critical_config(config.settings)
+    
+    log.info('### **** *** ** * Aether API v4 using FastAPI - Startup Sequence Complete * ** *** **** ###')
+    
+    yield
+    
+    # Shutdown logic
+    log.info('### **** *** ** * Aether API v4 using FastAPI - Shutdown Lifespan Initiated * ** *** **** ###')
+    log.info('The Aether FastAPI API is shutting down...')
+
+
+print('### **** *** ** * Aether API v4 using FastAPI - About to try FastAPI() while loading... * ** *** **** ###')
 app = FastAPI(
     # debug = True,
     title = 'Aether API',
     description = 'One Sky IT\'s Aether API v4 using FastAPI.',
     version = '4.9.0',
     operationsSorter = 'method',
+    lifespan = lifespan,
     )
 
 
-log.setLevel(logging.INFO)
-# log.debug(config.settings)
-
-if aether_cfg_sql_result := sql_select(
-        table_name = 'cfg',
-        record_id = config.settings.AETHER_CFG['id'],
-        as_list = False,
-        max_count = 1,
-        ):
-    aether_cfg_sql = aether_cfg_sql_result
-
-    config.settings.DB['server'] = aether_cfg_sql.get('db_server')
-    config.settings.DB['port'] = aether_cfg_sql.get('db_port')
-    config.settings.DB['name'] = aether_cfg_sql.get('db_name')
-    config.settings.DB['username'] = aether_cfg_sql.get('db_username')
-    config.settings.DB['password'] = aether_cfg_sql.get('db_password')
-
-    DB = config.settings.DB
-    config.settings.SQLALCHEMY_DB_URI = 'mysql://'+DB['username']+':'+DB['password']+'@'+DB['server']+'/'+DB['name']
-    # db_result = sql_connect(config.settings.SQLALCHEMY_DB_URI)
-    log.debug(config.settings.DB)
-
-    config.settings.SMTP['server'] = aether_cfg_sql.get('smtp_server')
-    config.settings.SMTP['port'] = aether_cfg_sql.get('smtp_port')
-    config.settings.SMTP['username'] = aether_cfg_sql.get('smtp_username')
-    config.settings.SMTP['password'] = aether_cfg_sql.get('smtp_password')
-
-    # config.settings.FILES_PATH['hosted_files_root'] = aether_cfg_sql.get('PATH_HOSTED_FILES_ROOT')
-    # config.settings.FILES_PATH['hosted_tmp_root'] = aether_cfg_sql.get('PATH_HOSTED_TMP_ROOT')
-
-    config.settings.FILES_PATH['hosted_files_root'] = aether_cfg_sql.get('path_hosted_files_root')
-    config.settings.FILES_PATH['hosted_tmp_root'] = aether_cfg_sql.get('path_hosted_tmp_root')
-else:
-    # aether_cfg_sql_result
-    pass
-log.debug(aether_cfg_sql_result)
-log.debug(config.settings)
-
 # @lru_cache()
 # def get_settings():
 #     return config.Settings()
@@ -90,348 +110,8 @@ log.debug(config.settings)
 app.mount('/static', StaticFiles(directory='static'), name='static')
 
 
-# Set up each route once the router has been imported
-app.include_router(
-    aether_cfg.router,
-    tags=['Aether Config'],
-)
-app.include_router(
-    api_crud.router,
-    prefix='/crud',
-    tags=['CRUD'],
-    #dependencies=[Depends(get_token_header)],
-    #dependencies=[Depends(get_account_header)],
-    #responses={404: {'description': 'Not found'}},
-)
-app.include_router(
-    api.router,
-    prefix='/api',
-    tags=['API'],
-)
-app.include_router(
-    flask_cfg.router,
-    prefix='/flask_cfg',
-    tags=['Flask CFG'],
-)
-app.include_router(
-    importing.router,
-    prefix='/importing',
-    tags=['Importing'],
-)
-app.include_router(
-    sql.router,
-    # prefix='/sql',
-    tags=['SQL'],
-)
-# # app.include_router(
-# #     flask_cfg.router,
-# #     prefix='/redis',
-# #     tags=['Redis'],
-# # )
-
-app.include_router(
-    account.router,
-    # prefix='/account',
-    tags=['Account'],
-)
-app.include_router(
-    activity_log.router,
-    prefix='/activity_log',
-    tags=['Activity Log'],
-)
-app.include_router(
-    address.router,
-    prefix='/address',
-    tags=['Address'],
-)
-app.include_router(
-    archive.router,
-    # prefix='/archive',
-    tags=['Archive'],
-)
-app.include_router(
-    archive_content.router,
-    prefix='/archive/content',
-    tags=['Archive Content'],
-)
-app.include_router(
-    contact.router,
-    prefix='/contact',
-    tags=['Contact'],
-)
-app.include_router(
-    cont_edu_cert.router,
-    tags=['Cont Edu Cert'],
-)
-app.include_router(
-    cont_edu_cert_person.router,
-    tags=['Cont Edu Cert Person'],
-)
-app.include_router(
-    data_store.router,
-    # prefix='/data_store',
-    tags=['Data Store'],
-)
-app.include_router(
-    event.router,
-    # prefix='/event',
-    tags=['Event'],
-)
-app.include_router(
-    event_abstract.router,
-    tags=['Event Abstract'],
-)
-app.include_router(
-    event_badge.router,
-    tags=['Event Badge'],
-)
-app.include_router(
-    event_badge_importing.router,
-    tags=['Event Badge Importing'],
-)
-app.include_router(
-    event_badge_template.router,
-    # prefix='/event/badge/template',
-    tags=['Event Badge Template'],
-)
-app.include_router(
-    event_device.router,
-    # prefix='/event/device',
-    tags=['Event Device'],
-)
-app.include_router(
-    event_exhibit.router,
-    # prefix='/event/exhibit',
-    tags=['Event Exhibit'],
-)
-app.include_router(
-    event_exhibit_tracking.router,
-    # prefix='/event/exhibit/tracking',
-    tags=['Event Exhibit Tracking'],
-)
-app.include_router(
-    event_file.router,
-    # prefix='/event/file',
-    tags=['Event File'],
-)
-app.include_router(
-    event_importing.router,
-    # prefix='/event/importing',
-    tags=['Event Importing'],
-)
-app.include_router(
-    event_location.router,
-    # prefix='/event/location',
-    tags=['Event Location'],
-)
-app.include_router(
-    event_person.router,
-    # prefix='/event/person',
-    tags=['Event Person'],
-)
-app.include_router(
-    event_person.router,
-    prefix='/event/person/detail',
-    tags=['Event Person Detail'],
-)
-app.include_router(
-    event_person_tracking.router,
-    tags=['Event Person Tracking'],
-)
-app.include_router(
-    event_presentation.router,
-    # prefix='/event/presentation',
-    tags=['Event Presentation'],
-)
-app.include_router(
-    event_presenter.router,
-    prefix='/event/presenter',
-    tags=['Event Presenter'],
-)
-app.include_router(
-    event_registration.router,
-    prefix='/event/registration',
-    tags=['Event Registration'],
-)
-app.include_router(
-    event_session.router,
-    # prefix='/event/session',
-    tags=['Event Session'],
-)
-app.include_router(
-    fundraising.router,
-    tags=['Fundraising'],
-)
-app.include_router(
-    grant.router,
-    tags=['Grant'],
-)
-app.include_router(
-    hosted_file.router,
-    prefix='/hosted_file',
-    tags=['Hosted File'],
-)
-app.include_router(
-    journal.router,
-    prefix='/journal',
-    tags=['Journal'],
-)
-app.include_router(
-    journal_entry.router,
-    # prefix='/journal/entry',
-    tags=['Journal Entry'],
-)
-app.include_router(
-    log_client_viewing.router,
-    # prefix='/log/client_viewing',
-    tags=['Log Client Viewing'],
-)
-app.include_router(
-    lookup.router,
-    prefix='/lu',
-    tags=['Lookup'],
-)
-app.include_router(
-    membership_cfg.router,
-    tags=['Membership Config'],
-)
-app.include_router(
-    membership_group.router,
-    tags=['Membership Group'],
-)
-app.include_router(
-    membership_person_group.router,
-    tags=['Membership Group Person'],
-)
-app.include_router(
-    membership_person_profile.router,
-    tags=['Membership Person Profile'],
-)
-app.include_router(
-    membership_person.router,
-    tags=['Membership Person'],
-)
-app.include_router(
-    membership_type.router,
-    tags=['Membership Type'],
-)
-app.include_router(
-    membership_person_type.router,
-    tags=['Membership Type Person'],
-)
-app.include_router(
-    order.router,
-    # prefix='/order',
-    tags=['Order'],
-)
-app.include_router(
-    order_v3.router,
-    # prefix='/order',
-    tags=['Order v3'],
-)
-app.include_router(
-    order_line.router,
-    # prefix='/order',
-    tags=['Order Line'],
-)
-app.include_router(
-    order_cart.router,
-    prefix='/order/cart',
-    tags=['Order Cart'],
-)
-app.include_router(
-    organization.router,
-    prefix='/organization',
-    tags=['Organization'],
-)
-app.include_router(
-    page.router,
-    prefix='/page',
-    tags=['Page'],
-)
-app.include_router(
-    person.router,
-    tags=['Person'],
-)
-app.include_router(
-    person_user.router,
-    prefix='/person_user',
-    tags=['Person User'],
-)
-app.include_router(
-    post.router,
-    # prefix='/post',
-    tags=['Post'],
-)
-app.include_router(
-    post_comment.router,
-    prefix='/post/comment',
-    tags=['Post Comment'],
-)
-app.include_router(
-    product.router,
-    # prefix='/product',
-    tags=['Product'],
-)
-app.include_router(
-    qr.router,
-    tags=['QR'],
-)
-app.include_router(
-    site.router,
-    # prefix='/site',
-    tags=['Site'],
-)
-app.include_router(
-    site_domain.router,
-    # prefix='/site/domain',
-    tags=['Site Domain'],
-)
-app.include_router(
-    user.router,
-    tags=['User'],
-)
-app.include_router(
-    util_email.router,
-    tags=['Utility: Email'],
-)
-# app.include_router(
-#     websockets.router,
-#     # prefix='/websocket',
-#     tags=['Websockets'],
-#     # dependencies=[Depends(get_token_header)],
-#     # responses={404: {'description': 'Not found'}},
-# )
-app.include_router(
-    websockets_redis.router,
-    tags=['Websockets (Redis)'],
-)
-app.include_router(
-    e_confex.router,
-    prefix='/e/confex',
-    tags=['External Service: Confex'],
-)
-app.include_router(
-    e_cvent.router,
-    prefix='/e/cvent',
-    tags=['External Service: Cvent'],
-)
-app.include_router(
-    e_impexium.router,
-    prefix='/e/impexium',
-    tags=['External Service: Impexium'],
-)
-app.include_router(
-    e_stripe.router,
-    prefix='/e/stripe',
-    tags=['External Service: Stripe'],
-)
-
-app.include_router(
-    c_idaa.router,
-    prefix='/c/idaa',
-    tags=['Client: IDAA'],
-)
+# Register all application routes
+setup_routers(app)
 
 
 # BEGIN: CORS
@@ -452,33 +132,8 @@ app.add_middleware(
 # END: CORS
 
 
-@app.on_event('startup')
-async def startup():
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    log.info('The Aether FastAPI API is starting up...')
-    #await database.connect()
-
-
-@app.on_event('shutdown')
-async def shutdown():
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    log.info('The Aether FastAPI API is shutting down...')
-    #await database.disconnect()
-
-
-#Add the processing time to the response header.
-@app.middleware('http')
-async def add_process_time_header(request: Request, call_next):
-    import time
-    start_time = time.time()
-    response = await call_next(request)
-    process_time = time.time() - start_time
-    response.headers['X-Process-Time'] = str(process_time)
-    return response
+# Register utility middleware from external module
+app.middleware('http')(process_time_middleware)
 
 
 # ### BEGIN ### API Main ### fastapi_root() ###
@@ -499,6 +154,10 @@ async def fastapi_root(response: Response = Response):
     log.critical('This is critical') # 50 CRITICAL
     log.info('^^^')
 
+    log.warning('Resetting Redis...')
+    reset_redis()
+    log.info('Reset Redis')
+
     response_data = {}
     response_data['message'] = 'This is One Sky IT\'s Aether API root (FastAPI).'
 
@@ -566,45 +225,3 @@ async def generate_id_random(response: Response = Response):
 
     return HTMLResponse(content=html_list, status_code=200)
 # ### END ### API Main ### generate_id_random() ###
-
-
-# ### BEGIN ### API Main ### sql_test() ###
-# ### TEST TEST TEST ### #
-@app.get('/sql_test', tags=['Testing'], response_class=PlainTextResponse)
-async def sql_test(response: Response = Response):
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    return mk_resp(data=False, status_code=501, response=response)
-
-    log.info('Getting all accounts from DB...')
-
-    sql = text(
-        """
-        SELECT id, id_random, name, enable
-        FROM `account`
-        """
-        )
-    try:
-        result = db.execute(sql)
-    except Exception as e:
-        log.error('*** An exception happened. ***')
-        log.error(repr(e))
-        log.error('***')
-        log.error(str(e))
-        log.error('^^^ exception ^^^')
-    else:
-        if result.rowcount:
-            record_li = [dict(record) for record in result.fetchall()]
-            log.debug(record_li)
-        else:
-            log.error('No records found. Something went wrong.')
-
-    log.info('Got the account list')
-
-    response_data = {}
-    response_data['message'] = 'This is the Aether API using FastAPI.'
-    response_data['data'] = record_li
-
-    return json.dumps(response_data, indent=4) # , sort_keys=True
-# ### END ### API Main ### sql_test() ###

app/main.py.snapshot

@@ -0,0 +1,625 @@
+import datetime, json, os, pytz, random, secrets # , uvicorn
+
+from enum import Enum
+#from datetime import datetime, time, timedelta
+from fastapi import Body, Cookie, Depends, FastAPI, File, Form, Header, HTTPException, Path, Query, Request, Response, status, UploadFile
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, PlainTextResponse
+from fastapi.staticfiles import StaticFiles
+from functools import lru_cache
+from pydantic import BaseModel, EmailStr, Field
+from typing import Dict, List, Optional, Set, Union
+
+from . import config
+
+from app.log import log, logging
+
+# Import the routers here first:
+from app.routers import ae_obj, aether_cfg, api_crud, api_crud_v2, api_crud_v3, api, importing, sql, account, activity_log, address, archive, archive_content, contact, cont_edu_cert, cont_edu_cert_person, data_store, event, event_abstract, event_badge, event_badge_importing, event_badge_template, event_device, event_exhibit, event_exhibit_tracking, event_file, event_importing, event_location, event_person, event_person_detail, event_person_tracking, event_presentation, event_presenter, event_registration, event_session, flask_cfg, fundraising, grant, hosted_file, journal, journal_entry, log_client_viewing, lookup, membership_cfg, membership_group, membership_person_group, membership_person, membership_person_profile, membership_type, membership_person_type, order, order_v3, order_line, order_cart, organization, page, person, person_user, post, post_comment, product, qr, site, site_domain, user, util_email, websockets_redis, e_confex, e_cvent, c_idaa, e_impexium, e_stripe
+
+# from app.routers import aether_cfg, sql
+
+from app.db_sql import sql_select, reset_redis # , sql_connect
+
+
+print('### **** *** ** * The Aether API v4 using FastAPI is loading... * ** *** **** ###')
+
+
+
+
+
+app = FastAPI(
+    # debug = True,
+    title = 'Aether API',
+    description = 'One Sky IT\'s Aether API v4 using FastAPI.',
+    version = '4.9.0',
+    operationsSorter = 'method',
+    )
+
+
+log.setLevel(logging.INFO)
+# log.debug(config.settings)
+
+if aether_cfg_sql_result := sql_select(
+        table_name = 'cfg',
+        record_id = config.settings.AETHER_CFG['id'],
+        as_list = False,
+        max_count = 1,
+        ):
+    aether_cfg_sql = aether_cfg_sql_result
+
+    config.settings.DB['server'] = aether_cfg_sql.get('db_server')
+    config.settings.DB['port'] = aether_cfg_sql.get('db_port')
+    config.settings.DB['name'] = aether_cfg_sql.get('db_name')
+    config.settings.DB['username'] = aether_cfg_sql.get('db_username')
+    config.settings.DB['password'] = aether_cfg_sql.get('db_password')
+
+    DB = config.settings.DB
+    config.settings.SQLALCHEMY_DB_URI = 'mysql://'+DB['username']+':'+DB['password']+'@'+DB['server']+'/'+DB['name']
+    # db_result = sql_connect(config.settings.SQLALCHEMY_DB_URI)
+    log.debug(config.settings.DB)
+
+    config.settings.SMTP['server'] = aether_cfg_sql.get('smtp_server')
+    config.settings.SMTP['port'] = aether_cfg_sql.get('smtp_port')
+    config.settings.SMTP['username'] = aether_cfg_sql.get('smtp_username')
+    config.settings.SMTP['password'] = aether_cfg_sql.get('smtp_password')
+
+    # config.settings.FILES_PATH['hosted_files_root'] = aether_cfg_sql.get('PATH_HOSTED_FILES_ROOT')
+    # config.settings.FILES_PATH['hosted_tmp_root'] = aether_cfg_sql.get('PATH_HOSTED_TMP_ROOT')
+
+    config.settings.FILES_PATH['hosted_files_root'] = aether_cfg_sql.get('path_hosted_files_root')
+    config.settings.FILES_PATH['hosted_tmp_root'] = aether_cfg_sql.get('path_hosted_tmp_root')
+else:
+    # aether_cfg_sql_result
+    pass
+log.debug(aether_cfg_sql_result)
+log.debug(config.settings)
+
+# @lru_cache()
+# def get_settings():
+#     return config.Settings()
+
+
+app.mount('/static', StaticFiles(directory='static'), name='static')
+
+
+# Set up each route once the router has been imported
+app.include_router(
+    ae_obj.router,
+    prefix='/ae_obj',
+    tags=['AE Object'],
+)
+app.include_router(
+    aether_cfg.router,
+    tags=['Aether Config'],
+)
+app.include_router(
+    api_crud.router,
+    prefix='/crud',
+    tags=['CRUD v1.2 (Legacy)'],
+    #dependencies=[Depends(get_token_header)],
+    #dependencies=[Depends(get_account_header)],
+    #responses={404: {'description': 'Not found'}},
+)
+app.include_router(
+    api_crud_v2.router,
+    prefix='/v2/crud',
+    tags=['CRUD v2.5'],
+    #dependencies=[Depends(get_token_header)],
+    #dependencies=[Depends(get_account_header)],
+    #responses={404: {'description': 'Not found'}},
+)
+app.include_router(
+    api_crud_v3.router,
+    prefix='/v3/crud',
+    tags=['CRUD v3'],
+)
+app.include_router(
+    api.router,
+    prefix='/api',
+    tags=['API'],
+)
+app.include_router(
+    flask_cfg.router,
+    prefix='/flask_cfg',
+    tags=['Flask CFG'],
+)
+app.include_router(
+    importing.router,
+    prefix='/importing',
+    tags=['Importing'],
+)
+app.include_router(
+    sql.router,
+    # prefix='/sql',
+    tags=['SQL'],
+)
+# # app.include_router(
+# #     flask_cfg.router,
+# #     prefix='/redis',
+# #     tags=['Redis'],
+# # )
+
+app.include_router(
+    account.router,
+    # prefix='/account',
+    tags=['Account'],
+)
+app.include_router(
+    activity_log.router,
+    prefix='/activity_log',
+    tags=['Activity Log'],
+)
+app.include_router(
+    address.router,
+    prefix='/address',
+    tags=['Address'],
+)
+app.include_router(
+    archive.router,
+    # prefix='/archive',
+    tags=['Archive'],
+)
+app.include_router(
+    archive_content.router,
+    prefix='/archive/content',
+    tags=['Archive Content'],
+)
+app.include_router(
+    contact.router,
+    prefix='/contact',
+    tags=['Contact'],
+)
+app.include_router(
+    cont_edu_cert.router,
+    tags=['Cont Edu Cert'],
+)
+app.include_router(
+    cont_edu_cert_person.router,
+    tags=['Cont Edu Cert Person'],
+)
+app.include_router(
+    data_store.router,
+    # prefix='/data_store',
+    tags=['Data Store'],
+)
+app.include_router(
+    event.router,
+    # prefix='/event',
+    tags=['Event'],
+)
+app.include_router(
+    event_abstract.router,
+    tags=['Event Abstract'],
+)
+app.include_router(
+    event_badge.router,
+    tags=['Event Badge'],
+)
+app.include_router(
+    event_badge_importing.router,
+    tags=['Event Badge Importing'],
+)
+app.include_router(
+    event_badge_template.router,
+    # prefix='/event/badge/template',
+    tags=['Event Badge Template'],
+)
+app.include_router(
+    event_device.router,
+    # prefix='/event/device',
+    tags=['Event Device'],
+)
+app.include_router(
+    event_exhibit.router,
+    # prefix='/event/exhibit',
+    tags=['Event Exhibit'],
+)
+app.include_router(
+    event_exhibit_tracking.router,
+    # prefix='/event/exhibit/tracking',
+    tags=['Event Exhibit Tracking'],
+)
+app.include_router(
+    event_file.router,
+    # prefix='/event/file',
+    tags=['Event File'],
+)
+app.include_router(
+    event_importing.router,
+    # prefix='/event/importing',
+    tags=['Event Importing'],
+)
+app.include_router(
+    event_location.router,
+    # prefix='/event/location',
+    tags=['Event Location'],
+)
+app.include_router(
+    event_person.router,
+    # prefix='/event/person',
+    tags=['Event Person'],
+)
+app.include_router(
+    event_person.router,
+    prefix='/event/person/detail',
+    tags=['Event Person Detail'],
+)
+app.include_router(
+    event_person_tracking.router,
+    tags=['Event Person Tracking'],
+)
+app.include_router(
+    event_presentation.router,
+    # prefix='/event/presentation',
+    tags=['Event Presentation'],
+)
+app.include_router(
+    event_presenter.router,
+    prefix='/event/presenter',
+    tags=['Event Presenter'],
+)
+app.include_router(
+    event_registration.router,
+    prefix='/event/registration',
+    tags=['Event Registration'],
+)
+app.include_router(
+    event_session.router,
+    # prefix='/event/session',
+    tags=['Event Session'],
+)
+app.include_router(
+    fundraising.router,
+    tags=['Fundraising'],
+)
+app.include_router(
+    grant.router,
+    tags=['Grant'],
+)
+app.include_router(
+    hosted_file.router,
+    prefix='/hosted_file',
+    tags=['Hosted File'],
+)
+app.include_router(
+    journal.router,
+    prefix='/journal',
+    tags=['Journal'],
+)
+app.include_router(
+    journal_entry.router,
+    # prefix='/journal/entry',
+    tags=['Journal Entry'],
+)
+app.include_router(
+    log_client_viewing.router,
+    # prefix='/log/client_viewing',
+    tags=['Log Client Viewing'],
+)
+app.include_router(
+    lookup.router,
+    prefix='/lu',
+    tags=['Lookup'],
+)
+app.include_router(
+    membership_cfg.router,
+    tags=['Membership Config'],
+)
+app.include_router(
+    membership_group.router,
+    tags=['Membership Group'],
+)
+app.include_router(
+    membership_person_group.router,
+    tags=['Membership Group Person'],
+)
+app.include_router(
+    membership_person_profile.router,
+    tags=['Membership Person Profile'],
+)
+app.include_router(
+    membership_person.router,
+    tags=['Membership Person'],
+)
+app.include_router(
+    membership_type.router,
+    tags=['Membership Type'],
+)
+app.include_router(
+    membership_person_type.router,
+    tags=['Membership Type Person'],
+)
+app.include_router(
+    order.router,
+    # prefix='/order',
+    tags=['Order'],
+)
+app.include_router(
+    order_v3.router,
+    # prefix='/order',
+    tags=['Order v3'],
+)
+app.include_router(
+    order_line.router,
+    # prefix='/order',
+    tags=['Order Line'],
+)
+app.include_router(
+    order_cart.router,
+    prefix='/order/cart',
+    tags=['Order Cart'],
+)
+app.include_router(
+    organization.router,
+    prefix='/organization',
+    tags=['Organization'],
+)
+app.include_router(
+    page.router,
+    prefix='/page',
+    tags=['Page'],
+)
+app.include_router(
+    person.router,
+    tags=['Person'],
+)
+app.include_router(
+    person_user.router,
+    prefix='/person_user',
+    tags=['Person User'],
+)
+app.include_router(
+    post.router,
+    # prefix='/post',
+    tags=['Post'],
+)
+app.include_router(
+    post_comment.router,
+    prefix='/post/comment',
+    tags=['Post Comment'],
+)
+app.include_router(
+    product.router,
+    # prefix='/product',
+    tags=['Product'],
+)
+app.include_router(
+    qr.router,
+    tags=['QR'],
+)
+app.include_router(
+    site.router,
+    # prefix='/site',
+    tags=['Site'],
+)
+app.include_router(
+    site_domain.router,
+    # prefix='/site/domain',
+    tags=['Site Domain'],
+)
+app.include_router(
+    user.router,
+    tags=['User'],
+)
+app.include_router(
+    util_email.router,
+    tags=['Utility: Email'],
+)
+# app.include_router(
+#     websockets.router,
+#     # prefix='/websocket',
+#     tags=['Websockets'],
+#     # dependencies=[Depends(get_token_header)],
+#     # responses={404: {'description': 'Not found'}},
+# )
+app.include_router(
+    websockets_redis.router,
+    tags=['Websockets (Redis)'],
+)
+app.include_router(
+    e_confex.router,
+    prefix='/e/confex',
+    tags=['External Service: Confex'],
+)
+app.include_router(
+    e_cvent.router,
+    prefix='/e/cvent',
+    tags=['External Service: Cvent'],
+)
+app.include_router(
+    e_impexium.router,
+    prefix='/e/impexium',
+    tags=['External Service: Impexium'],
+)
+app.include_router(
+    e_stripe.router,
+    prefix='/e/stripe',
+    tags=['External Service: Stripe'],
+)
+
+app.include_router(
+    c_idaa.router,
+    prefix='/c/idaa',
+    tags=['Client: IDAA'],
+)
+
+
+# BEGIN: CORS
+# NOTE: Eventually this should query the DB for the specific list based on the cfg table and or site_domain table. That way it is dynamic and only allowing those defined in the DB. No wildcards or regex.
+# NOTE: Need to include .localhost for less browser restrictions! Mainly for audio and video.
+app.add_middleware(
+    CORSMiddleware,
+    # allow_origins = origins,
+    allow_origins = config.settings.ORIGINS,
+    allow_origin_regex = config.settings.ORIGINS_REGEX,
+    # allow_origin_regex = 'https://.*\.oneskyit\.com',
+    allow_credentials = True,
+    allow_methods = ['*'],
+    allow_headers = ['*'],
+    #expose_headers = [],
+    #max_age = 600,
+)
+# END: CORS
+
+
+@app.on_event('startup')
+async def startup():
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    log.info('The Aether FastAPI API is starting up...')
+    #await database.connect()
+
+
+@app.on_event('shutdown')
+async def shutdown():
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    log.info('The Aether FastAPI API is shutting down...')
+    #await database.disconnect()
+
+
+#Add the processing time to the response header.
+@app.middleware('http')
+async def add_process_time_header(request: Request, call_next):
+    import time
+    start_time = time.time()
+    response = await call_next(request)
+    process_time = time.time() - start_time
+    response.headers['X-Process-Time'] = str(process_time)
+    return response
+
+
+# ### BEGIN ### API Main ### fastapi_root() ###
+@app.get('/', tags=['Root'], response_class=PlainTextResponse)
+async def fastapi_root(response: Response = Response):
+    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    # log.info(config.settings.APP_NAME)
+    log.info('One Sky IT\'s Aether API root (FastAPI)')
+
+    log.info('***')
+    log.debug('This is debug') # 10 DEBUG
+    log.info('This is info') # 20 INFO
+    log.warning('This is a warning') # 30 WARNING (and WARN)
+    log.error('This is an error') # 40 ERROR
+    log.exception('This is an exception') # 40 ERROR
+    log.critical('This is critical') # 50 CRITICAL
+    log.info('^^^')
+
+    log.warning('Resetting Redis...')
+    reset_redis()
+    log.info('Reset Redis')
+
+    response_data = {}
+    response_data['message'] = 'This is One Sky IT\'s Aether API root (FastAPI).'
+
+
+    current_datetime = datetime.datetime.now()
+    current_datetime_string = current_datetime.isoformat()
+
+    timezone = pytz.timezone("America/New_York")
+    current_datetime_tz = timezone.localize(current_datetime)
+    current_datetime_tz_string = current_datetime_tz.isoformat()
+
+    current_datetime_utc = datetime.datetime.utcnow()
+    current_datetime_utc_string = current_datetime_utc.isoformat()
+
+    current_datetime_utc_localize = pytz.utc.localize(current_datetime_utc)
+    current_datetime_utc_localize_string = current_datetime_utc_localize.isoformat()
+
+    current_datetime_utc_localize_pst = current_datetime_utc_localize.astimezone(pytz.timezone("America/Los_Angeles"))
+    current_datetime_utc_localize_pst_string = current_datetime_utc_localize_pst.isoformat()
+
+    response_data['datetime'] = current_datetime_string
+    response_data['datetime_tz'] = current_datetime_tz_string
+    response_data['datetime_utc'] = current_datetime_utc_string
+    response_data['datetime_utc_localize'] = current_datetime_utc_localize_string
+    response_data['datetime_utc_localize_pst'] = current_datetime_utc_localize_pst_string
+
+    response_data['url_safe_string_4_bytes_1'] = secrets.token_urlsafe(4)
+
+    response_data['url_safe_string_8_bytes_1'] = secrets.token_urlsafe(8)
+    response_data['url_safe_string_8_bytes_2'] = secrets.token_urlsafe(8)
+    response_data['url_safe_string_8_bytes_3'] = secrets.token_urlsafe(8)
+    response_data['url_safe_string_8_bytes_4'] = secrets.token_urlsafe(8)
+    response_data['url_safe_string_8_bytes_5'] = secrets.token_urlsafe(8)
+
+    response_data['url_safe_string_16_bytes_1'] = secrets.token_urlsafe(16)
+    response_data['url_safe_string_16_bytes_2'] = secrets.token_urlsafe(16)
+    response_data['url_safe_string_16_bytes_3'] = secrets.token_urlsafe(16)
+    response_data['url_safe_string_16_bytes_4'] = secrets.token_urlsafe(16)
+    response_data['url_safe_string_16_bytes_5'] = secrets.token_urlsafe(16)
+
+    response_data['hex_string_4_bytes_1'] = secrets.token_hex(4)
+    response_data['hex_string_8_bytes_1'] = secrets.token_hex(8)
+    response_data['hex_string_16_bytes_1'] = secrets.token_hex(16)
+    response_data['hex_string_32_bytes_1'] = secrets.token_hex(32)
+
+    log.debug(json.dumps(response_data, indent=4))
+    return json.dumps(response_data, indent=4) # , sort_keys=True
+# ### END ### API Main ### fastapi_root() ###
+
+
+# ### BEGIN ### API Main ### generate_id_random() ###
+# NOTE: This is just a quick utility function to generate a bunch of random IDs.
+# Updated 2022-03-30
+@app.get('/generate_id_random', tags=['Root'], response_class=PlainTextResponse)
+async def generate_id_random(response: Response = Response):
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    response_data = {}
+
+    html_list = '<ul>'
+    for x in range(50):
+        html_list += f'<li>{secrets.token_urlsafe(8)}</li>'
+    html_list += '</ul>'
+
+    return HTMLResponse(content=html_list, status_code=200)
+# ### END ### API Main ### generate_id_random() ###
+
+
+# ### BEGIN ### API Main ### sql_test() ###
+# ### TEST TEST TEST ### #
+@app.get('/sql_test', tags=['Testing'], response_class=PlainTextResponse)
+async def sql_test(response: Response = Response):
+    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARN, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    return mk_resp(data=False, status_code=501, response=response)
+
+    log.info('Getting all accounts from DB...')
+
+    sql = text(
+        """
+        SELECT id, id_random, name, enable
+        FROM `account`
+        """
+        )
+    try:
+        result = db.execute(sql)
+    except Exception as e:
+        log.error('*** An exception happened. ***')
+        log.error(repr(e))
+        log.error('***')
+        log.error(str(e))
+        log.error('^^^ exception ^^^')
+    else:
+        if result.rowcount:
+            record_li = [dict(record) for record in result.fetchall()]
+            log.debug(record_li)
+        else:
+            log.error('No records found. Something went wrong.')
+
+    log.info('Got the account list')
+
+    response_data = {}
+    response_data['message'] = 'This is the Aether API using FastAPI.'
+    response_data['data'] = record_li
+
+    return json.dumps(response_data, indent=4) # , sort_keys=True
+# ### END ### API Main ### sql_test() ###

app/methods/address_methods.py

@@ -110,7 +110,7 @@ def create_update_address_obj_v4(
         fail_any: bool = False, # Fail if any thing goes wrong for sub objects
         return_outline: bool = False,
         ) -> int|bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     log.info('Checking requirements...')
@@ -172,7 +172,7 @@ def create_update_address_obj_v4(
             address_dict['for_id'] = for_id
         try:
             address_obj = Address_Base(**address_dict)
-            log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+            log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
             log.debug(address_obj)
         except ValidationError as e:
             log.error(e.json())
@@ -231,7 +231,7 @@ def create_address_obj(
         for_id: int|str = None,
         fail_any: bool = False, # Fail if any thing goes wrong for sub objects
         ) -> int|bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     # ### SECTION ### Secondary data validation
@@ -312,7 +312,7 @@ def update_address_obj(
         create_sub_obj: bool = False,
         fail_any: bool = False, # Fail if any thing goes wrong for sub objects
         ) -> bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     # ### SECTION ### Secondary data validation
@@ -381,7 +381,7 @@ def create_update_address_obj(
         process_address: bool = False,
         process_organization: bool = False,
         ) -> bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     if address_id:

app/methods/contact_methods.py

@@ -224,7 +224,7 @@ def create_update_contact_obj_v4(
             contact_dict['for_id'] = for_id
         try:
             contact_obj = Contact_Base(**contact_dict)
-            log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+            log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
             log.debug(contact_obj)
         except ValidationError as e:
             log.error(e.json())
@@ -540,7 +540,7 @@ def create_update_contact_obj(
         process_address: bool = False,
         process_organization: bool = False,
         ) -> bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     if contact_id:

app/methods/data_store_methods.py

@@ -64,9 +64,11 @@ def load_data_store_obj_w_code(
         exclude_unset: bool = True, # NOTE: For now this is ignored
         model_as_dict: bool = False, # NOTE: For now this is ignored
         ) -> Data_Store_Base|dict|bool:
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
+    log.info(f'Getting Data Store record with code: {code} for Account ID: {account_id} and For Type: {for_type} and For ID: {for_id}')
+
     data = {}
     data['account_id'] = account_id
     data['code'] = code
@@ -87,7 +89,6 @@ def load_data_store_obj_w_code(
     sql_limit = sql_limit_offset_part(limit=limit, offset=offset) # Reasonably safe return str
 
     log.debug(data)
-    # log.warning(f'Where are we now??????????? {code}')
 
     sql = f"""
         SELECT *
@@ -106,10 +107,13 @@ def load_data_store_obj_w_code(
     # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(sql)
 
+
     if data_store_rec_li_result := sql_select(data=data, sql=sql, as_list=True):
         data_store_rec_li = data_store_rec_li_result
     else: # [] or False
         data_store_rec_li = data_store_rec_li_result
+        log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        log.info(f'No Data Store records found with code: {code} for Account ID: {account_id} and For Type: {for_type} and For ID: {for_id}')
 
     log.debug(data_store_rec_li_result)
 
@@ -126,14 +130,9 @@ def load_data_store_obj_w_code(
             log.debug(data_store_obj)
     else: pass
 
-    # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.info(f'Found {len(data_store_obj_li)} Data Store records with code: {code} for Account ID: {account_id} and For Type: {for_type} and For ID: {for_id}')
     log.debug(data_store_obj_li)
     return data_store_obj_li
-
-    # if model_as_dict:
-    #     return data_store_obj.dict(by_alias=by_alias, exclude_unset=exclude_unset) # pylint: disable=no-member
-    # else:
-    #     return data_store_obj
 # ### END ### API Data Store Methods ### load_data_store_obj_w_code() ###
 
 

app/methods/e_confex_methods.py

@@ -15,8 +15,8 @@ from app.models.event_file_models import Event_File_Base
 
 
 api = {}
-# api['base_url'] = 'https://aapor.confex.com/aapor/2023/meetingapi.cgi/[object]/[id]'
-api['base_url'] = 'https://aapor.confex.com/aapor/2023/meetingapi.cgi'
+# api['base_url'] = 'https://aapor.confex.com/aapor/20xx/meetingapi.cgi/[object]/[id]'
+api['base_url'] = 'https://aapor.confex.com/aapor/2024/meetingapi.cgi'
 api['headers'] = { 'Content-Type': 'application/json;charset=UTF-8' }
 api['username'] = None
 api['password'] = None
@@ -83,7 +83,7 @@ def get_event_session_list(
 
                 log.warning('Something may have gone wrong during the request.')
 
-                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Impexium on the next request.')
+                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Confex on the next request.')
                 # api['app_user_token_datetime'] = None # Resetting this just in case the App and or User token expired.
 
     return confex_session_list
@@ -152,7 +152,7 @@ def get_event_session_detail(
 
                 log.warning('Something may have gone wrong during the request.')
 
-                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Impexium on the next request.')
+                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Confex on the next request.')
                 # api['app_user_token_datetime'] = None # Resetting this just in case the App and or User token expired.
 
     return confex_session_detail
@@ -223,7 +223,7 @@ def get_event_presentation_detail(
 
                 log.warning('Something may have gone wrong during the request.')
 
-                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Impexium on the next request.')
+                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Confex on the next request.')
                 # api['app_user_token_datetime'] = None # Resetting this just in case the App and or User token expired.
 
     return confex_presentation_detail
@@ -294,7 +294,7 @@ def get_event_presenter_detail(
 
                 log.warning('Something may have gone wrong during the request.')
 
-                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Impexium on the next request.')
+                # log.warning('Something may have gone wrong. Setting the API app_user_token_datetime value to None to re-authenticate with Confex on the next request.')
                 # api['app_user_token_datetime'] = None # Resetting this just in case the App and or User token expired.
 
     return confex_presenter_detail

app/methods/event_abstract_methods.py

@@ -70,7 +70,7 @@ def load_event_abstract_obj(
 
     # Updated 2023-03-20
     if inc_event_file_list:
-        # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        # log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
         log.info('Need to include event file list...')
 
         from app.methods.event_file_methods import get_event_file_rec_list, load_event_file_obj
@@ -123,7 +123,7 @@ def load_event_abstract_obj(
             log.debug(event_person_obj)
             event_abstract_obj.event_person = event_person_obj
         else:
-            log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+            log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
             log.debug(event_person_obj)
             event_abstract_obj.event_person = None
         log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
@@ -227,7 +227,7 @@ def get_event_abstract_rec_list(
         ORDER BY event_abstract.priority DESC, event_abstract.sort DESC, event_abstract.name ASC, `event_abstract`.created_on DESC, `event_abstract`.updated_on DESC
         {sql_limit};
         """
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(sql)
 
     if event_abstract_rec_li_result := sql_select(data=data, sql=sql, as_list=True):
@@ -329,7 +329,7 @@ def create_update_event_abstract_obj_old(
         fail_any: bool = False, # Fail if any thing goes wrong for sub objects
         return_outline: bool = False,
         ) -> int|bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     log.info('Checking requirements...')
@@ -379,7 +379,7 @@ def create_update_event_abstract_obj_old(
             event_abstract_dict['event_person_id'] = event_person_id
         try:
             event_abstract_obj = Event_Abstract_In(**event_abstract_dict)
-            log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+            log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
             log.debug(event_abstract_obj)
         except ValidationError as e:
             log.error(e.json())

app/methods/event_list_methods.py

@@ -77,7 +77,7 @@ def load_event_obj_list(
         """
 
     if event_rec_li_result := sql_select(data=data, sql=sql, as_list=True):
-        log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
         log.debug(event_rec_li_result)
         event_result_li = []
         for event_rec in event_rec_li_result:
@@ -113,7 +113,7 @@ def load_event_obj_list(
                 event_result_li.append(None)
         log.debug(event_result_li)
     else:
-        log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+        log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
         log.debug(event_rec_li_result)
         event_result_li = []
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL

app/methods/event_presentation_methods.py

@@ -165,7 +165,7 @@ def load_event_presentation_obj(
             WHERE `event_presenter`.event_presentation_id = :event_presentation_id
                 {sql_hidden}
                 {sql_enabled}
-            ORDER BY `event_presenter`.priority DESC, -`event_presenter`.sort DESC, `event_presenter`.family_name ASC, `event_presenter`.given_name ASC, `event_presenter`.display_name ASC, `event_presenter`.full_name ASC, `event_presenter`.created_on DESC, `event_presenter`.updated_on DESC;
+            ORDER BY `event_presenter`.priority DESC, -`event_presenter`.sort DESC, `event_presenter`.family_name ASC, `event_presenter`.given_name ASC, `event_presenter`.full_name ASC, `event_presenter`.created_on DESC, `event_presenter`.updated_on DESC;
             """
         # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
         log.debug(sql)

app/methods/event_presenter_methods.py

@@ -262,7 +262,7 @@ def get_event_presenter_rec_list(
             {sql_where_event_presentation_id}
             {sql_hidden}
             {sql_enabled}
-        ORDER BY `event_presenter`.priority DESC, -`event_presenter`.sort DESC, `event_presenter`.family_name ASC, `event_presenter`.given_name ASC, `event_presenter`.display_name ASC, `event_presenter`.full_name ASC, `event_presenter`.created_on DESC, `event_presenter`.updated_on DESC
+        ORDER BY `event_presenter`.priority DESC, -`event_presenter`.sort DESC, `event_presenter`.family_name ASC, `event_presenter`.given_name ASC, `event_presenter`.full_name ASC, `event_presenter`.created_on DESC, `event_presenter`.updated_on DESC
         {sql_limit};
         """
 

app/methods/hosted_file_methods.py

@@ -1,4 +1,4 @@
-import datetime, hashlib, os, pathlib, shutil, time
+import datetime, hashlib, mimetypes, os, pathlib, shutil, time
 
 from fastapi import File, UploadFile
 from typing import Dict, List, Optional, Set, Union
@@ -441,8 +441,7 @@ async def save_file_to_hosted_file(
 
     # There is a difference between Content-Type and MIME type.
     # https://stackoverflow.com/questions/3452381/whats-the-difference-of-contenttype-and-mimetype
-    # file_info['content_type'] = file.content_type # might also include charset or other parameters
-    # file_info['mimetype'] = file.mimetype # This may need to be filled in a different way?
+    file_info['content_type'] = mimetypes.guess_type(filename)[0]
 
     file_obj.seek(0, os.SEEK_END)
     file_size = file_obj.tell()
@@ -670,7 +669,7 @@ def handle_delete_hosted_file(
     # hosted_files_path = '/home/scott/tmp/hosted_files_dev/'
     log.info(f'Hosted Files Path: {hosted_files_path}')
 
-    dir_path = hosted_file_obj.directory_path
+    # dir_path = hosted_file_obj.directory_path
     subdir_path = hosted_file_obj.subdirectory_path
     hash_sha256 = hosted_file_obj.hash_sha256
     hash_filename = hash_sha256+'.file'

app/methods/organization_methods.py

@@ -34,7 +34,7 @@ def load_organization_obj(
     if organization_rec := sql_select(table_name='v_organization', record_id=organization_id): pass
     else: return False
 
-    #log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    #log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(organization_rec)
 
     try:
@@ -114,7 +114,7 @@ def get_organization_rec_list(
         organization_rec_li = organization_rec_li_result
     else:
         organization_rec_li = []
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(organization_rec_li_result)
 
     return organization_rec_li
@@ -154,11 +154,11 @@ def update_organization_obj(
                 account_id = contact_obj_in.account_id,
                 contact_dict_obj=contact_obj_in,
                 ):
-            # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+            # log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
             log.debug(contact_obj_in_result)
             organization_obj_up.contact_id = contact_obj_in_result
         else:
-            # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+            # log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
             log.debug(contact_obj_in_result)
             return False
 
@@ -180,7 +180,7 @@ def create_update_organization_obj(
         organization_obj: Organization_Base,
         process_contact: bool = False,
         ) -> bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     if organization_id:

app/methods/person_methods.py

@@ -1837,8 +1837,8 @@ def handle_email_person_auth_key_url(
     from_email = account_cfg.default_no_reply_email
     from_name = account_cfg.default_no_reply_name
 
-    to_name = person_obj.display_name
-    to_email = person_obj.email
+    to_name = person_obj.full_name
+    to_email = person_obj.email or person_obj.user_email or person_obj.primary_email
 
     bcc_email = account_cfg.confirm_email
     bcc_name = account_cfg.confirm_name
@@ -1856,21 +1856,39 @@ def handle_email_person_auth_key_url(
     # subject = f'{account_short_name}: One Time Use Create Account Link ({new_auth_key})'
 
     body_html = f"""
-        <p>{to_name},</p>
-
-        <p>If you did not request this account creation link, please delete this email. It is suggested that you delete this email after the account creation link has been used or if a new link has been requested.</p>
-
-        <p>The link below can only be used once. If you would like try again using this method, you must <a href="NOT READY YET">request a new account creation link</a>. If you request multiple links, only the newest link will work.</p>
-
-        <p><strong><a href="{person_auth_key_url}" style="appearance: button; display: inline-block; text-align: center; text-decoration: none; padding: .2rem .4rem; border: solid thin gray; border-radius: .2rem; background-color: lightyellow; color: black; font-size: larger;">Click to Finish Account Creation With One Time Use Link</a></strong></p>
-
-        <p>Or copy and paste the link:<br>
-        <strong style="background-color: lightyellow; color: black; font-size: larger;"><a href="{person_auth_key_url}">{person_auth_key_url}</a></strong></p>
-
-        <p>If you have questions about this email or trouble with this one time use link, you can email <a href="mailto:{help_tech_email}">{help_tech_name} ({help_tech_email})</a>.</p>
-
-        <p>Thank you!</p>
-        """
+    <div style="font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; max-width: 600px; margin: 20px auto; padding: 30px; border: 1px solid #ddd; border-radius: 10px; background-color: #ffffff; color: #333;">
+        <div style="text-align: center; margin-bottom: 20px;">
+            <h2 style="color: #444; margin: 0;">{account_short_name}</h2>
+            <div style="height: 2px; background: linear-gradient(to right, #eee, #ccc, #eee); margin: 10px auto; width: 80%;"></div>
+        </div>
+        
+        <p style="font-size: 16px;">Hello <strong>{to_name}</strong>,</p>
+        
+        <p style="line-height: 1.6;">You have requested a one-time use link to complete your account registration. This link will allow you to set up your account securely.</p>
+        
+        <div style="text-align: center; margin: 35px 0;">
+            <a href="{person_auth_key_url}" style="display: inline-block; padding: 14px 28px; background-color: #fdfd96; color: #000; text-decoration: none; border: 1px solid #ccc; border-radius: 6px; font-weight: bold; font-size: 18px; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+                Finish Account Creation
+            </a>
+        </div>
+        
+        <p style="font-size: 14px; color: #666; line-height: 1.6;">
+            <strong>Security Note:</strong> If you did not request this link, please delete this email. The link above can only be used once. If you request multiple links, only the newest one will be active.
+        </p>
+        
+        <div style="margin-top: 40px; padding-top: 20px; border-top: 1px solid #eee; font-size: 12px; color: #888;">
+            <p style="margin-bottom: 5px;">If the button above doesn't work, copy and paste the following URL into your browser:</p>
+            <p style="word-break: break-all; color: #0056b3; background-color: #f9f9f9; padding: 10px; border-radius: 4px; border: 1px dashed #ccc;">
+                {person_auth_key_url}
+            </p>
+            
+            <p style="margin-top: 20px;">
+                Questions or trouble? Contact <a href="mailto:{help_tech_email}" style="color: #0056b3; text-decoration: underline;">{help_tech_name}</a>.
+            </p>
+            <p style="text-align: center; margin-top: 30px; font-weight: bold; color: #aaa;">Thank you!</p>
+        </div>
+    </div>
+    """
 
     if send_email(from_email=from_email, from_name=from_name, to_email=to_email, to_name=to_name, bcc_email=bcc_email, bcc_name=bcc_name, subject=subject, body_text=None, body_html=body_html):
         log.info(f'An email with a one time use sign in link was sent to {to_email}.')

app/methods/site_methods.py

@@ -100,7 +100,7 @@ def update_site_obj(
         site_obj_up: Site_Base,
         create_sub_obj: bool = False,
         ) -> bool:
-    log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     if site_id := redis_lookup_id_random(record_id_random=site_id, table_name='site'): pass

app/methods/user_methods.py

@@ -605,12 +605,13 @@ def get_user_rec_list(
 
 
 # ### BEGIN ### User Methods ### email_user_auth_key_url() ###
-# This emails the actual one time use sign in URL for a user.
-# Updated 2021-12-02
+# This generates a new auth_key token and emails the actual one time use sign in URL to the user's email.
+# Updated 2025-04-08
 def email_user_auth_key_url(
     account_id: int|str,
     user_id: int|str,
     root_url: str,
+    key_param_name: str = 'auth_key',
     ):
     log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
@@ -684,8 +685,13 @@ def email_user_auth_key_url(
     else: enable_to_str = '-- Not Set --'
     auth_key = user_obj.auth_key
 
-    user_login_url = f'{root_url}user/login?username={urllib.parse.quote(username)}&email={urllib.parse.quote(to_email)}'
-    user_login_auth_key_url = f'{root_url}?user_id={urllib.parse.quote(user_id_random)}&auth_key={urllib.parse.quote(new_auth_key)}&valid_email={True}'
+    user_login_url = f'{root_url}?username={urllib.parse.quote(username)}&user_email={urllib.parse.quote(to_email)}'
+    # user_login_url = f'{root_url}user/login?username={urllib.parse.quote(username)}&email={urllib.parse.quote(to_email)}'
+
+    if key_param_name == 'auth_key':
+        user_login_auth_key_url = f'{root_url}?user_id={urllib.parse.quote(user_id_random)}&auth_key={urllib.parse.quote(new_auth_key)}&valid_email={True}'
+    elif key_param_name:
+        user_login_auth_key_url = f'{root_url}?user_id={urllib.parse.quote(user_id_random)}&{key_param_name}={urllib.parse.quote(new_auth_key)}&valid_email={True}'
 
     subject = f'{account_short_name}: One Time Use Sign In Link ({new_auth_key})'
 

app/middleware.py

@@ -0,0 +1,12 @@
+import time
+from fastapi import Request
+
+async def add_process_time_header(request: Request, call_next):
+    """
+    Middleware to add the processing time to the response header.
+    """
+    start_time = time.time()
+    response = await call_next(request)
+    process_time = time.time() - start_time
+    response.headers['X-Process-Time'] = str(process_time)
+    return response

app/models/account_cfg_models.py

@@ -23,6 +23,7 @@ class Account_Cfg_Base(BaseModel):
     id: Optional[int] = Field(
         alias = 'account_cfg_id'
     )
+
     account_id_random: Optional[str]
     account_id: Optional[int]
 

app/models/account_models.py

@@ -38,6 +38,11 @@ class Account_Base(BaseModel):
     short_name: Optional[str]
     description: Optional[str]
 
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+
     enable: Optional[bool]
     enable_from: Optional[datetime.datetime] = None
     enable_to: Optional[datetime.datetime] = None
@@ -95,4 +100,5 @@ class Account_Base(BaseModel):
     class Config:
         underscore_attrs_are_private = True
         fields = base_fields
+        allow_population_by_field_name = True
 # ### END ### API Account Models ### Account_Base() ###

app/models/activity_log_models.py

@@ -67,7 +67,15 @@ class Activity_Log_Base(BaseModel):
     other_json: Optional[str] # When getting the dict version for SQL this should be a string.
     meta_json: Optional[str] # When getting the dict version for SQL this should be a string.
 
+    enable: Optional[bool]
+
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+    
     notes: Optional[str]
+
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
@@ -94,29 +102,23 @@ class Activity_Log_Base(BaseModel):
 
     @validator('account_id', always=True)
     def account_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['account_id_random']:
-            return redis_lookup_id_random(record_id_random=values['account_id_random'], table_name='account')
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('account_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
         return None
 
     @validator('person_id', always=True)
     def person_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['person_id_random']:
-            return redis_lookup_id_random(record_id_random=values['person_id_random'], table_name='person')
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('person_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
         return None
 
     @validator('user_id', always=True)
     def user_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['user_id_random']:
-            return redis_lookup_id_random(record_id_random=values['user_id_random'], table_name='user')
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('user_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='user')
         return None
 
     class Config:

app/models/address_models.py

@@ -1,9 +1,9 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
-from app.db_sql import redis_lookup_id_random
+from app.db_sql import get_id_random, redis_lookup_id_random
 from app.lib_general import log, logging
 
 from app.models.common_field_schema import base_fields, default_num_bytes
@@ -14,23 +14,15 @@ class Address_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['address_id_random'],
-        alias = 'address_id_random',
-        # default_factory = lambda:secrets.token_urlsafe(default_num_bytes),
-    )
-    id: Optional[int] = Field(
-        alias = 'address_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['address_id_random'])
+    address_id: Optional[str] = Field(None, **base_fields['address_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    contact_id: Optional[str] = Field(None, **base_fields['contact_id_random'])
 
     for_type: Optional[str]
     for_id_random: Optional[str]
-    for_id: Optional[int]
-
-    contact_id_random: Optional[str]
-    contact_id: Optional[int]
+    for_id: Optional[int] = Field(None, exclude=True)
 
     #organization: Optional[Organization_Base] = Organization_Base()
 
@@ -60,65 +52,43 @@ class Address_Base(BaseModel):
 
     congressional_district: Optional[str]
 
-    #priority: Optional[int]
-    #sort: Optional[int]
-    #group: Optional[str]
+    enable: Optional[bool]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
 
+    notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('address_id_random', always=True)
-    def address_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def address_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='address')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('account_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
-        return None
-
-    @validator('contact_id', always=True)
-    def contact_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('contact_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='contact')
-        return None
-
-    @validator('for_id', always=True)
-    def for_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['for_id_random'] and values['for_type']:
-            return redis_lookup_id_random(record_id_random=values['for_id_random'], table_name=values['for_type'])
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('address_id_random'):
+            values['id'] = rid
+            values['address_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if c_rid := values.get('contact_id_random'):
+            values['contact_id'] = c_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'account_id', 'contact_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
-# ### END ### API Address Models ### Address_Base() ###
+# ### END ### API Address Models ### Address_Base() ###

app/models/api_crud_models.py

@@ -1,14 +1,37 @@
 import datetime, pytz
 
-from typing import Dict, List, Optional, Set, Union
+from typing import Any, Dict, List, Optional, Set, Union
 from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
 
-from app.db_sql import redis_lookup_id_random
-from app.lib_general import log, logging
+import logging
+log = logging.getLogger(__name__)
 
 from app.models.common_field_schema import base_fields, default_num_bytes
 
 
+# ### BEGIN ### API Search Models ###
+class SearchFilter(BaseModel):
+    """
+    Represents a single filter condition.
+    Example: {"field": "price", "op": "gt", "value": 100}
+    """
+    field: str
+    op: str  # eq, ne, gt, gte, lt, lte, like, in, is_null, is_not_null, contains, startswith, endswith
+    value: Optional[Any] = None
+
+class SearchQuery(BaseModel):
+    """
+    Represents a complex search query with optional logical grouping.
+    """
+    query_string: Optional[str] = Field(None, alias="q")
+    and_filters: Optional[List[Union[SearchFilter, 'SearchQuery']]] = Field(None, alias="and")
+    or_filters: Optional[List[Union[SearchFilter, 'SearchQuery']]] = Field(None, alias="or")
+
+# Support recursive models in Pydantic v1
+SearchQuery.update_forward_refs()
+# ### END ### API Search Models ###
+
+
 # ### BEGIN ### API CRUD Models ### Fundraising_Cfg_Base() ###
 class Api_Crud_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
@@ -16,6 +39,8 @@ class Api_Crud_Base(BaseModel):
 
     super_key: Optional[str] = None # Query(None, min_length=8,  max_length=50),
 
+    jwt: Optional[str] = None
+
     create_key: Optional[str] = None # Query(None, min_length=6,  max_length=50),
     read_key: Optional[str] = None # Query(None, min_length=5,  max_length=50),
     update_key: Optional[str] = None # Query(None, min_length=6,  max_length=50),

app/models/archive_content_models.py

@@ -11,7 +11,7 @@ from app.models.common_field_schema import base_fields, default_num_bytes
 
 # ### BEGIN ### API Archive Content Models ### Archive_Content_Base() ###
 class Archive_Content_Base(BaseModel):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
     # def testing(test_var=None):
@@ -25,8 +25,8 @@ class Archive_Content_Base(BaseModel):
     id: Optional[int] = Field(
         alias = 'archive_content_id'
     )
-    # account_id_random: Optional[str] # Is this field really needed?
-    # account_id: Optional[int] # Is this field really needed?
+    account_id_random: Optional[str]
+    account_id: Optional[int]
 
     archive_id_random: Optional[str]
     archive_id: Optional[int]
@@ -85,6 +85,13 @@ class Archive_Content_Base(BaseModel):
     created_on: Optional[datetime.datetime]
     updated_on: Optional[datetime.datetime]
 
+    # Including convenience data
+    # This is only for convenience. Probably going to keep unless it causes a problem.
+    hosted_file_hash_sha256: Optional[str]
+    hosted_file_subdirectory_path: Optional[str] = Field(None, exclude=True)
+    hosted_file_content_type: Optional[str]
+    hosted_file_size: Optional[str]
+
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
     @validator('id', always=True)

app/models/auth_models.py

@@ -0,0 +1,14 @@
+from typing import Optional
+from pydantic import BaseModel
+
+# Zero-dependency auth models for V3
+# Created 2026-01-07 to resolve circular dependencies in FastAPI startup
+
+class AccountContext(BaseModel):
+    account_id: Optional[int]
+    account_id_random: Optional[str]
+    administrator: bool = False
+    manager: bool = False
+    super: bool = False
+    auth_method: str = 'legacy_header'
+    token_payload: Optional[dict] = None

app/models/common_field_schema.py

@@ -54,6 +54,7 @@ base_fields['event_track_id_random'] = xxx_id_random_field_schema
 base_fields['flask_cfg_id_random'] = xxx_id_random_field_schema
 base_fields['fundraising_id_random'] = xxx_id_random_field_schema
 base_fields['fundraising_cfg_id_random'] = xxx_id_random_field_schema
+base_fields['grant_id_random'] = xxx_id_random_field_schema
 base_fields['hosted_file_id_random'] = xxx_id_random_field_schema
 base_fields['journal_id_random'] = xxx_id_random_field_schema
 base_fields['journal_entry_id_random'] = xxx_id_random_field_schema
@@ -79,6 +80,8 @@ base_fields['post_comment_id_random'] = xxx_id_random_field_schema
 base_fields['product_id_random'] = xxx_id_random_field_schema
 base_fields['site_id_random'] = xxx_id_random_field_schema
 base_fields['site_domain_id_random'] = xxx_id_random_field_schema
+base_fields['sponsorship_cfg_id_random'] = xxx_id_random_field_schema
+base_fields['sponsorship_id_random'] = xxx_id_random_field_schema
 base_fields['user_id_random'] = xxx_id_random_field_schema
 base_fields['user_role_id_random'] = xxx_id_random_field_schema
 

app/models/contact_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import get_id_random, redis_lookup_id_random
 from app.lib_general import log, logging
@@ -16,22 +16,15 @@ class Contact_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['contact_id_random'],
-        alias = 'contact_id_random',
-        # default_factory = lambda:secrets.token_urlsafe(default_num_bytes),
-    )
-    id: Optional[int] = Field(
-        alias = 'contact_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
-
-    address_id_random: Optional[str]
-    address_id: Optional[int]
-
-    linked_address_id_random: Optional[str]
-    linked_address_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['contact_id_random'])
+    contact_id: Optional[str] = Field(None, **base_fields['contact_id_random'])
+    
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    address_id: Optional[str] = Field(None, **base_fields['address_id_random'])
+    
+    # NOTE: Linked Address ID is actually the old contact.address_id (Legacy?)
+    linked_address_id: Optional[str] = Field(None, **base_fields['address_id_random'])
 
     for_type: Optional[str]
     for_id: Optional[int]
@@ -72,10 +65,13 @@ class Contact_Base(BaseModel):
     other_text: Optional[str]
     other_json: Optional[Json]
 
+    enable: Optional[bool]
+    hide: Optional[bool]
     priority: Optional[bool]
     sort: Optional[int]
     group: Optional[str]
 
+    notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
@@ -85,56 +81,30 @@ class Contact_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('contact_id_random', always=True)
-    def contact_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def contact_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='contact')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('account_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
-        return None
-
-    @validator('address_id', always=True)
-    def address_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('address_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='address')
-        return None
-
-    # NOTE: Linked Address ID is actually the old contact.address_id
-    # This should no longer be used...
-    @validator('linked_address_id', always=True)
-    def linked_address_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('linked_address_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='address')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('contact_id_random'):
+            values['id'] = rid
+            values['contact_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if ad_rid := values.get('address_id_random'):
+            values['address_id'] = ad_rid
+        if lad_rid := values.get('linked_address_id_random'):
+            values['linked_address_id'] = lad_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'contact_id', 'account_id', 'address_id', 'linked_address_id']:
+            if k in values and not isinstance(values[k], str) and values[k] is not None:
+                del values[k]
+                
+        return values
 
     @validator('for_id', pre=True, always=True)
     def for_id_lookup(cls, v, values, **kwargs):
@@ -178,6 +148,6 @@ class Contact_Base(BaseModel):
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Contact Models ### Contact_Base() ###

app/models/core_object_models.py

@@ -68,6 +68,10 @@ class Core_Std_Obj_Base(BaseModel):
             # return redis_lookup_id_random(record_id_random=id_random, table_name=otype)
         return None
 
+    class Config:
+        underscore_attrs_are_private = True
+        allow_population_by_field_name = True
+
 
 class Core_Object_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
@@ -148,6 +152,10 @@ class Core_Object_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
+    class Config:
+        underscore_attrs_are_private = True
+        allow_population_by_field_name = True
+
 
 class Example_Object_Base(Core_Object_Base): # Based on Core_Object_Base
     title: Optional[str] = None

app/models/data_store_models.py

@@ -6,7 +6,7 @@ from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationEr
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
 
-from app.models.common_field_schema import base_fields, default_num_bytes
+from app.models.common_field_schema import base_fields
 
 
 # ### BEGIN ### API Data Store Models ### Data_Store_Base() ###
@@ -40,14 +40,25 @@ class Data_Store_Base(BaseModel):
     name: Optional[str]
     description: Optional[str]
 
+    type: Optional[str] # html, json, md, text
+
+    # The JSON fields are case sensitive
     # json: Optional[str] # "json" is reserved; need to change field name? json_str?
     json_str: Optional[Union[Json, None]] = Field(
         alias = 'json',
     )
+
+    # The text fields are case insensitive
     text: Optional[str]
+
     meta_json: Optional[str]
     meta_text: Optional[str]
 
+    access: Optional[str]
+    access_read: Optional[str]
+    access_write: Optional[str]
+    access_delete: Optional[str]
+
     enable: Optional[bool]
 
     hide: Optional[bool]

app/models/error_models.py

@@ -0,0 +1,23 @@
+from typing import Optional, Any, Dict
+from pydantic import BaseModel, Field
+
+class StandardError(BaseModel):
+    """
+    Standardized machine-readable error structure for Aether.
+    Helps the frontend decide how to handle failures.
+    """
+    category: str = Field(..., description="Error category (e.g., 'database', 'validation', 'security')")
+    code: Optional[int] = Field(None, description="Specific error code (e.g., MariaDB error code)")
+    message: str = Field(..., description="Developer-friendly error message")
+    recoverable: bool = Field(False, description="If True, the frontend might want to retry or ask for user input")
+    details: Optional[Any] = Field(None, description="Raw technical details or traceback (if permitted)")
+
+    class Config:
+        schema_extra = {
+            "example": {
+                "category": "database",
+                "code": 1062,
+                "message": "Duplicate entry for key 'id_random'",
+                "recoverable": False
+            }
+        }

app/models/event_abstract_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -23,31 +23,18 @@ class Event_Abstract_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        # **base_fields['event_abstract_id_random'],
-        alias = 'event_abstract_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'event_abstract_id'
-    )
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['event_abstract_id_random'])
+    event_abstract_id: Optional[str] = Field(None, **base_fields['event_abstract_id_random'])
+    
+    event_id: Optional[str] = Field(None, **base_fields['event_id_random'])
+    event_person_id: Optional[str] = Field(None, **base_fields['event_person_id_random'])
+    event_presentation_id: Optional[str] = Field(None, **base_fields['event_presentation_id_random'])
+    event_presenter_id: Optional[str] = Field(None, **base_fields['event_presenter_id_random'])
+    event_session_id: Optional[str] = Field(None, **base_fields['event_session_id_random'])
+    grant_id: Optional[str] = Field(None, **base_fields['grant_id_random'])
 
-    event_id_random: Optional[str]
-    event_id: Optional[int]
-
-    event_person_id_random: Optional[str] # This is the primary person/submitter
-    event_person_id: Optional[int]
-
-    event_presentation_id_random: Optional[str]
-    event_presentation_id: Optional[int]
-
-    event_presenter_id_random: Optional[str]
-    event_presenter_id: Optional[int]
-
-    event_session_id_random: Optional[str]
-    event_session_id: Optional[int]
-
-    # event_track_id_random: Optional[str]
-    # event_track_id: Optional[int]
+    # event_track_id: Optional[str] = Field(None, **base_fields['event_track_id_random'])
 
     # poc_event_person_id_random: Optional[str] # Maybe change this to primary_event_person?
     # poc_event_person_id: Optional[int] # Maybe change this to primary_event_person?
@@ -55,9 +42,6 @@ class Event_Abstract_Base(BaseModel):
     external_id: Optional[str]
     code: Optional[str]
 
-    grant_id_random: Optional[str]
-    grant_id: Optional[int]
-
     grant_code: Optional[str]
     # grant_type_code: Optional[str]
     # grant_json: Optional[Union[Json, None]]
@@ -101,67 +85,40 @@ class Event_Abstract_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('id', always=True)
-    def event_abstract_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_abstract')
-        return None
-
-    @validator('event_id', always=True)
-    def event_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event')
-        return None
-
-    @validator('event_person_id', always=True)
-    def event_person_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_person_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_person')
-        return None
-
-    @validator('event_presentation_id', always=True)
-    def event_presentation_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_presentation_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_presentation')
-        return None
-
-    @validator('event_presenter_id', always=True)
-    def event_presenter_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_presenter_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_presenter')
-        return None
-
-    @validator('event_session_id', always=True)
-    def event_session_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_session_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_session')
-        return None
-
-    @validator('grant_id', always=True)
-    def grant_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('grant_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='grant')
-        return None
-
-    # @validator('poc_event_person_id', always=True)
-    # def poc_event_person_id_lookup(cls, v, values, **kwargs):
-    #     log.setLevel(logging.WARNING)
-    #     log.debug(locals())
-
-    #     if values['poc_event_person_id_random']:
-    #         return redis_lookup_id_random(record_id_random=values['poc_event_person_id_random'], table_name='poc_event_person')
-    #     return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('event_abstract_id_random'):
+            values['id'] = rid
+            values['event_abstract_id'] = rid
+            
+        if e_rid := values.get('event_id_random'):
+            values['event_id'] = e_rid
+        if ep_rid := values.get('event_person_id_random'):
+            values['event_person_id'] = ep_rid
+        if epr_rid := values.get('event_presentation_id_random'):
+            values['event_presentation_id'] = epr_rid
+        if eps_rid := values.get('event_presenter_id_random'):
+            values['event_presenter_id'] = eps_rid
+        if es_rid := values.get('event_session_id_random'):
+            values['event_session_id'] = es_rid
+        if g_rid := values.get('grant_id_random'):
+            values['grant_id'] = g_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'event_abstract_id', 'event_id', 'event_person_id', 'event_presentation_id', 'event_presenter_id', 'event_session_id', 'grant_id']:
+            if k in values and not isinstance(values[k], str) and values[k] is not None:
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Event Abstract Models ### Event_Abstract_Base() ###
 
@@ -175,24 +132,16 @@ class Event_Abstract_Base_New(Core_Std_Obj_Base):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        alias = 'event_abstract_id_random',
-    )
-
-    event_id_random: Optional[str]
-    event_id: Optional[int]
-
-    event_person_id: Optional[int]
-
-    event_session_id: Optional[int]
-
-    event_person_id_random: Optional[str]
-
-    event_presentation_id_random: Optional[str]
-
-    event_presenter_id_random: Optional[str]
-
-    event_session_id_random: Optional[str]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['event_abstract_id_random'])
+    event_abstract_id: Optional[str] = Field(None, **base_fields['event_abstract_id_random'])
+    
+    event_id: Optional[str] = Field(None, **base_fields['event_id_random'])
+    event_person_id: Optional[str] = Field(None, **base_fields['event_person_id_random'])
+    event_presentation_id: Optional[str] = Field(None, **base_fields['event_presentation_id_random'])
+    event_presenter_id: Optional[str] = Field(None, **base_fields['event_presenter_id_random'])
+    event_session_id: Optional[str] = Field(None, **base_fields['event_session_id_random'])
+    grant_id: Optional[str] = Field(None, **base_fields['grant_id_random'])
 
     # event_track_id_random: Optional[str]
 
@@ -209,9 +158,6 @@ class Event_Abstract_Base_New(Core_Std_Obj_Base):
 
     passcode: Optional[str]
 
-    grant_id_random: Optional[str]
-    grant_id: Optional[int]
-
     grant_code: Optional[str]
     grant_type_code: Optional[str]
     grant_json: Optional[Union[Json, None]]
@@ -224,23 +170,40 @@ class Event_Abstract_Base_New(Core_Std_Obj_Base):
     submitter_json: Optional[Union[Json, None]]
     coauthors_json: Optional[Union[Json, None]]
 
-    @validator('event_person_id', always=True)
-    def event_person_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_person_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_person')
-        return None
-
-    # @validator('event_session_id', always=True)
-    # def event_session_id_lookup(cls, v, values, **kwargs):
-    #     if isinstance(v, int) and v > 0: return v
-    #     elif id_random := values.get('event_session_id_random'):
-    #         return redis_lookup_id_random(record_id_random=id_random, table_name='event_session')
-    #     return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('event_abstract_id_random'):
+            values['id'] = rid
+            values['event_abstract_id'] = rid
+            
+        if e_rid := values.get('event_id_random'):
+            values['event_id'] = e_rid
+        if ep_rid := values.get('event_person_id_random'):
+            values['event_person_id'] = ep_rid
+        if epr_rid := values.get('event_presentation_id_random'):
+            values['event_presentation_id'] = epr_rid
+        if eps_rid := values.get('event_presenter_id_random'):
+            values['event_presenter_id'] = eps_rid
+        if es_rid := values.get('event_session_id_random'):
+            values['event_session_id'] = es_rid
+        if g_rid := values.get('grant_id_random'):
+            values['grant_id'] = g_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'event_abstract_id', 'event_id', 'event_person_id', 'event_presentation_id', 'event_presenter_id', 'event_session_id', 'grant_id']:
+            if k in values and not isinstance(values[k], str) and values[k] is not None:
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Event Abstract Models ### Event_Abstract_Base() ###
 
@@ -250,68 +213,11 @@ class Event_Abstract_Base_New(Core_Std_Obj_Base):
 class Event_Abstract_In(Event_Abstract_Base_New):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
+    
+    # Inherits everything from Event_Abstract_Base_New including the Vision ID pattern.
+    # We do NOT redefine 'id' as int here.
 
-    id: Optional[int] = Field(
-        alias = 'event_abstract_id'
-    )
-
-    event_id: Optional[int]
-
-    event_person_id: Optional[int]
-
-    # event_session_id: Optional[int]
-
-    # grant_json: Optional[Union[str, None]]
-
-
-    @validator('id', always=True)
-    def event_abstract_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_abstract')
-        return None
-
-    @validator('event_id', always=True)
-    def event_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event')
-        return None
-
-    @validator('event_person_id', always=True)
-    def event_person_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_person_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_person')
-        return None
-
-    # @validator('event_presentation_id', always=True)
-    # def event_presentation_id_lookup(cls, v, values, **kwargs):
-    #     if isinstance(v, int) and v > 0: return v
-    #     elif id_random := values.get('event_presentation_id_random'):
-    #         return redis_lookup_id_random(record_id_random=id_random, table_name='event_presentation')
-    #     return None
-
-    # @validator('event_presenter_id', always=True)
-    # def event_presenter_id_lookup(cls, v, values, **kwargs):
-    #     if isinstance(v, int) and v > 0: return v
-    #     elif id_random := values.get('event_presenter_id_random'):
-    #         return redis_lookup_id_random(record_id_random=id_random, table_name='event_presenter')
-    #     return None
-
-    # @validator('event_session_id', always=True)
-    # def event_session_id_lookup(cls, v, values, **kwargs):
-    #     if isinstance(v, int) and v > 0: return v
-    #     elif id_random := values.get('event_session_id_random'):
-    #         return redis_lookup_id_random(record_id_random=id_random, table_name='event_session')
-    #     return None
-
-    @validator('grant_id', always=True)
-    def grant_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('grant_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='grant')
-        return None
+    pass
 # ### END ### API Event Abstract Models ### Event_Abstract_In() ###
 
 

app/models/event_badge_models.py

@@ -141,7 +141,11 @@ class Event_Badge_Base(BaseModel):
     # affiliations_font_size: Optional[str] # Not currently used 2023-01-25
     # location_font_size: Optional[str] # Not currently used 2023-01-25
     # css: Optional[str] # Not currently used 2023-01-25
-    # other_json: Optional[str] # Not currently used 2023-01-25
+
+    cfg_json: Optional[Union[Json, None]] # Store per badge config options like font size; Not currently used 2024-06-11
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields! Not currently used 2024-06-11
+
+    default_qry_string: Optional[str] # Default query string used for searching and filtering badges. Updated using SQL triggers and a SQL function
 
     hide: Optional[bool]
     priority: Optional[bool]
@@ -216,6 +220,9 @@ class Event_Badge_Basic_Base(BaseModel):
         **base_fields['event_badge_id_random'],
         alias = 'event_badge_id_random',
     )
+    id: Optional[int] = Field(
+        alias = 'event_badge_id'
+    )
     event_badge_template_id_random: Optional[str]
     # event_badge_template_id: Optional[int]
 
@@ -290,19 +297,19 @@ class Event_Badge_Basic_Base(BaseModel):
     allow_tracking: Optional[bool] # Allow tracking for lead retrieval and other marketing
     # agree_to_tc: Optional[bool] # Agree to terms and conditions
 
-    # print_first_datetime: Optional[datetime.datetime] = None
-    # print_last_datetime: Optional[datetime.datetime] = None
-    # print_count: Optional[int]
+    print_first_datetime: Optional[datetime.datetime] = None
+    print_last_datetime: Optional[datetime.datetime] = None
+    print_count: Optional[int]
 
-    # hide: Optional[bool]
-    # priority: Optional[bool]
-    # sort: Optional[int]
-    # group: Optional[str]
-    # enable: Optional[bool]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+    enable: Optional[bool]
 
-    # notes: Optional[str]
-    # created_on: Optional[datetime.datetime] = None
-    # updated_on: Optional[datetime.datetime] = None
+    notes: Optional[str]
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
 
     # Including other related objects
     # order: Optional[Union[Order_Base, None]]
@@ -311,6 +318,13 @@ class Event_Badge_Basic_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
+    @validator('id', always=True)
+    def event_badge_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='event_badge')
+        return None
+
     class Config:
         underscore_attrs_are_private = True
         allow_population_by_field_name = True

app/models/event_badge_template_models.py

@@ -76,7 +76,10 @@ class Event_Badge_Template_Base(BaseModel):
 
     other_json: Optional[str]
 
+    enable: Optional[bool]
     notes: Optional[str]
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
@@ -110,12 +113,22 @@ class Event_Badge_Template_Base_In(Event_Badge_Template_Base):
 
 
 class Event_Badge_Template_Base_Out(Event_Badge_Template_Base):
+
+
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+
+
     log.debug(locals())
 
+
+
+
+
     # log.info('Using Out template')
 
+
+
+
+
     # badge_type_list: Optional[Json]
 
-    created_on: Optional[datetime.datetime] = None
-    updated_on: Optional[datetime.datetime] = None

app/models/event_device_models.py

@@ -17,7 +17,7 @@ class Event_Device_Base(BaseModel):
     log.debug(locals())
 
     id_random: Optional[str] = Field(
-        **base_fields['event_device_id_random'],
+        # **base_fields['event_device_id_random'],
         alias = 'event_device_id_random',
     )
     id: Optional[int] = Field(
@@ -45,11 +45,11 @@ class Event_Device_Base(BaseModel):
     api_secret_key: Optional[str]
 
     api_base_url: Optional[str]
-    app_server_base_url: Optional[str]
+    app_base_url: Optional[str]
     file_server_base_url: Optional[str]
 
     api_base_url_bak: Optional[str] # Backup URL
-    app_server_base_url_bak: Optional[str] # Backup URL
+    app_base_url_bak: Optional[str] # Backup URL
     file_server_base_url_bak: Optional[str] # Backup URL
 
     trigger_open_filename: Optional[str] # The file hash filename
@@ -83,20 +83,34 @@ class Event_Device_Base(BaseModel):
     check_event_location_loop_period: Optional[int]
     check_event_session_loop_period: Optional[int]
 
+    passcode: Optional[str]
+
     alert: Optional[bool]
     alert_msg: Optional[str]
+    alert_on: Optional[datetime.datetime]
+    status: Optional[str]
+    status_msg: Optional[str]
+    status_msg_on: Optional[datetime.datetime]
+    record_status: Optional[str]
+    record_status_msg: Optional[str]
+    record_status_on: Optional[datetime.datetime]
+
+    heartbeat: Optional[datetime.datetime]
 
     info_hostname: Optional[str]
     info_ip: Optional[str]
     info_ip_list: Optional[str] # string list of IPs separated by ;
     info_os: Optional[str]
 
+    cfg_json: Optional[Union[Json, None]] # Store per device config options like theme, language, etc
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields!
+
     enable: Optional[bool]
 
-    # hide: Optional[bool]
-    # priority: Optional[bool]
-    # sort: Optional[int]
-    # group: Optional[str]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
 
     event_notes: Optional[str]
     notes: Optional[str]

app/models/event_exhibit_models.py

@@ -65,6 +65,11 @@ class Event_Exhibit_Base(BaseModel):
     leads_device_sm_qty: Optional[int] # NOTE: Cell phone sized devices rented by exhibitor. Should this be a separate linked table (event_device)?
     leads_device_lg_qty: Optional[int] # NOTE: Tablet (8 or 9 inch) sized devices rented by exhibitor. Should this be a separate linked table (event_device)?
 
+    data_json: Optional[Union[Json, None]]
+    license_max: Optional[int]
+    license_li_json: Optional[Union[Json, None]]
+    cfg_json: Optional[Union[Json, None]]
+
     enable_organization_name_change: Optional[bool]
     enable_name_change: Optional[bool]
     enable_banner_image: Optional[bool]
@@ -74,10 +79,11 @@ class Event_Exhibit_Base(BaseModel):
 
     # access_key: Optional[str] # Maybe use in the future?
 
-    # enable: Optional[bool] # Maybe use in the future?
+    enable: Optional[bool]
     # enable_from: Optional[datetime.datetime] = None # Maybe use in the future?
     # enable_to: Optional[datetime.datetime] = None # Maybe use in the future?
 
+    hide: Optional[bool]
     priority: Optional[bool]
     sort: Optional[int]
     group: Optional[str]

app/models/event_exhibit_tracking_models.py

@@ -8,7 +8,7 @@ from app.lib_general import log, logging
 
 from app.models.common_field_schema import base_fields, default_num_bytes
 
-from app.models.event_badge_models import Event_Badge_Base
+# from app.models.event_badge_models import Event_Badge_Base
 from app.models.event_person_models import Event_Person_Base
 
 
@@ -37,14 +37,15 @@ class Event_Exhibit_Tracking_Base(BaseModel):
     event_badge_id_random: Optional[str]
     event_badge_id: Optional[int]
 
+    external_person_id: Optional[str] # This is probably an email address
+
     exhibitor_notes: Optional[str]
     responses_json: Optional[Json] # NOTE: Responses to custom questions
-    # responses_json: Json = [{'test': ''}] # NOTE: Responses to custom questions
-    # responses_json: Optional[Json] = Field(
-    #     default_factory = lambda:[{'test': ''}]
-    # )
-    data_json: Optional[Json]
-    # data_json: Optional[str]
+    # Example:
+    # {"5_years": {"response": "I see myself in 5 years doing something."}, "colors": {"response": "green"}}
+    # {"example_text": {"response": "This is an example of an text answer."}, "example_option_list": {"response": "no"}, "the_code": {"response": "yes"}, "question_everything": {"response": "tomorrow"}, "pre_assesment": {"response": "yes"}}
+
+    data_json: Optional[Json] # NOTE: Additional data
 
     enable: Optional[bool]
     hide: Optional[bool]

app/models/event_file_models.py

@@ -81,7 +81,7 @@ class Event_File_Base(BaseModel):
     sort: Optional[int]
     group: Optional[str] # Same or similar as file_purpose?
 
-    # notes: Optional[str]
+    notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
@@ -91,7 +91,8 @@ class Event_File_Base(BaseModel):
         alias = 'hash_sha256'
     )
     hosted_file_subdirectory_path: Optional[str] = Field(  # NOTE: This will frequently only contain numbers, but it still needs to be a string
-        alias = 'subdirectory_path'
+        alias = 'subdirectory_path',
+        exclude = True
     )
     hosted_file_content_type: Optional[str] = Field(
         alias = 'content_type'
@@ -104,6 +105,30 @@ class Event_File_Base(BaseModel):
         alias = 'file_purpose_name'
     )
 
+    event_name: Optional[str]
+    event_code: Optional[str]
+    event_start_datetime: Optional[datetime.datetime]
+    event_end_datetime: Optional[datetime.datetime]
+    event_location_code: Optional[str]
+    event_location_name: Optional[str]
+    event_presentation_code: Optional[str]
+    event_presentation_type_code: Optional[str]
+    event_presentation_name: Optional[str]
+    event_presentation_start_datetime: Optional[datetime.datetime]
+    event_presentation_end_datetime: Optional[datetime.datetime]
+    event_presenter_code: Optional[str]
+    event_presenter_given_name: Optional[str]
+    event_presenter_family_name: Optional[str]
+    event_presenter_full_name: Optional[str]
+    event_presenter_email: Optional[str]
+    event_session_code: Optional[str]
+    event_session_type_code: Optional[str]
+    event_session_name: Optional[str]
+    event_session_start_datetime: Optional[datetime.datetime]
+    event_session_end_datetime: Optional[datetime.datetime]
+    event_track_code: Optional[str]
+    event_track_name: Optional[str]
+
     # Including other related objects
     hosted_file: Optional[Union[Hosted_File_Base, None]]
 
@@ -178,16 +203,16 @@ class Event_File_Base(BaseModel):
             return redis_lookup_id_random(record_id_random=id_random, table_name='event_track')
         return None
 
-    # NOTE: I kind of give up on this. Handeling this outside of Pydantic and before the data is even attempted to be loaded into the Event_File_Base model. -STI 2021-09-10
+    # NOTE: I kind of give up on this. Handling this outside of Pydantic and before the data is even attempted to be loaded into the Event_File_Base model. -STI 2021-09-10
     # NOTE: This validator will try to find and "set" the for_id_random value. However, The value is not really "set" in Pydantic. To get this value, exclude_unset=True when returning a dict from the model.
-    # @validator('for_id_random', always=True)
-    # def for_id_random_lookup(cls, v, values, **kwargs):
-    #     log.setLevel(logging.WARNING)
-    #     log.debug(locals())
-
-    #     if values.get('for_id') and values['for_type']:
-    #         return get_id_random(record_id=values['for_id'], table_name=values['for_type'])
-    #     return None
+    @validator('for_id_random', always=True)
+    def for_id_random_lookup(cls, v, values, **kwargs):
+        log.setLevel(logging.WARNING)
+        log.debug(locals())
+        if isinstance(v, str): return v
+        elif values.get('for_id') and values['for_type']:
+            return get_id_random(record_id=values['for_id'], table_name=values['for_type'])
+        return None
 
     # @validator('for_id', always=True)
     # def for_id_lookup(cls, v, values, **kwargs):

app/models/event_location_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -15,28 +15,21 @@ class Event_Location_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        # **base_fields['event_location_id_random'],
-        alias = 'event_location_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'event_location_id'
-    )
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['event_location_id_random'])
+    event_location_id: Optional[str] = Field(None, **base_fields['event_location_id_random'])
+    
+    event_id: Optional[str] = Field(None, **base_fields['event_id_random'])
+    event_track_id: Optional[str] = Field(None, **base_fields['event_track_id_random'])
 
     code: Optional[str] = Field(
         # alias = 'event_location_code'
     )
 
     external_id: Optional[str] = Field(
-        alias = 'event_location_external_id'
+        # alias = 'event_location_external_id'
     )
 
-    event_id_random: Optional[str]
-    event_id: Optional[int]
-
-    event_track_id_random: Optional[str] # Can a track be assigned to one location?
-    event_track_id: Optional[int] # Can a track be assigned to one location?
-
     lu_location_type_id: Optional[int]
     location_type_code: Optional[str]
     location_type: Optional[str]
@@ -55,8 +48,15 @@ class Event_Location_Base(BaseModel):
     internal_notes_it: Optional[str] # IT and networking
     internal_notes_staff: Optional[str] # staffing and labor
 
+    passcode: Optional[str]
+
+    cfg_json: Optional[Union[Json, None]] # Store per location config options
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields!
+
     file_count: Optional[int]
     internal_use_count: Optional[int] # Should be renamed to "internal_use_file_count"???
+    event_file_id_li_json: Optional[Union[Json, None]] # List of file IDs (actually id_random)
+    file_count_all: Optional[int] # Of all files under a location
 
     alert: Optional[bool]
     alert_msg: Optional[str]
@@ -97,29 +97,31 @@ class Event_Location_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('id', always=True)
-    def event_location_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_location')
-        return None
-
-    @validator('event_id', always=True)
-    def event_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event')
-        return None
-
-    @validator('event_track_id', always=True)
-    def event_track_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_track_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_track')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('event_location_id_random'):
+            values['id'] = rid
+            values['event_location_id'] = rid
+            
+        if e_rid := values.get('event_id_random'):
+            values['event_id'] = e_rid
+        if et_rid := values.get('event_track_id_random'):
+            values['event_track_id'] = et_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'event_location_id', 'event_id', 'event_track_id']:
+            if k in values and not isinstance(values[k], str) and values[k] is not None:
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Event Location Models ### Event_Location_Base() ###

app/models/event_models.py

@@ -70,8 +70,8 @@ class Event_Base(BaseModel):
 
     recurring: Optional[bool]
     recurring_pattern: Optional[str]
-    recurring_start_time: Optional[datetime.time]
-    recurring_end_time: Optional[datetime.time]
+    recurring_start_time: Optional[str]
+    recurring_end_time: Optional[str]
     recurring_text: Optional[str]
 
     weekday_sunday: Optional[bool]
@@ -110,26 +110,42 @@ class Event_Base(BaseModel):
     contact_li_json: Optional[Union[Json, None]] # list of dicts (custom for client); this is SQL FULLTEXT() indexed
 
     attend_url: Optional[str]
+    attend_url_code: Optional[str] # ID, code, nickname
     attend_url_passcode: Optional[str]
     attend_phone: Optional[str]
     attend_phone_passcode: Optional[str]
     attend_text: Optional[str]
-
-    # NOT FINISHED YET
+    attend_json: Optional[Union[Json, None]]
 
     # access_key: Optional[str] # Maybe use in the future?
+    passcode: Optional[str]
 
     file_count: Optional[int]
+    internal_use_count: Optional[int] # Should be renamed to "internal_use_file_count"???
+    event_file_id_li_json: Optional[Union[Json, None]] # List of file IDs (actually id_random)
+    file_count_all: Optional[int] # Of all files under a session
+
+    status: Optional[str]
+    review: Optional[bool]
+    approve: Optional[bool]
+    ready: Optional[bool]
+    ready_on: Optional[datetime.datetime]
+    archive: Optional[bool] # Also in Event_Cfg_Base model
+    archive_on: Optional[datetime.datetime] # Also in Event_Cfg_Base model
+
+    mod_abstracts_json: Optional[Union[Json, None]]
+    mod_badges_json: Optional[Union[Json, None]]
+    mod_exhibits_json: Optional[Union[Json, None]]
+    mod_meetings_json: Optional[Union[Json, None]]
+    mod_pres_mgmt_json: Optional[Union[Json, None]]
+
+    cfg_json: Optional[Union[Json, None]] # Store per event config options; Not currently used 2024-06-11
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields! Not currently used 2024-06-11
 
     enable: Optional[bool] # Also in Event_Cfg_Base model
     enable_from: Optional[datetime.datetime] = None
     enable_to: Optional[datetime.datetime] = None
 
-    archive: Optional[bool] # Also in Event_Cfg_Base model
-    archive_on: Optional[datetime.datetime] # Also in Event_Cfg_Base model
-
-    cfg_json: Optional[Union[Json, None]]
-
     hide: Optional[bool] # Also in Event_Cfg_Base model
     priority: Optional[bool]
     sort: Optional[int]
@@ -265,6 +281,12 @@ class Event_Base(BaseModel):
             return v.astimezone(pytz.UTC).isoformat()
         else: return v
 
+    @validator('recurring_start_time', 'recurring_end_time', pre=True, always=True)
+    def time_to_str(cls, v):
+        if isinstance(v, (datetime.time, datetime.timedelta)):
+            return str(v)
+        return v
+
     class Config:
         underscore_attrs_are_private = True
         allow_population_by_field_name = True
@@ -324,8 +346,8 @@ class Event_Meeting_Flat_Base(BaseModel):
 
     recurring: Optional[bool]
     recurring_pattern: Optional[str]
-    recurring_start_time: Optional[datetime.time]
-    recurring_end_time: Optional[datetime.time]
+    recurring_start_time: Optional[str]
+    recurring_end_time: Optional[str]
     recurring_text: Optional[str]
 
     weekday_sunday: Optional[bool]
@@ -370,20 +392,36 @@ class Event_Meeting_Flat_Base(BaseModel):
     attend_phone: Optional[str]
     attend_phone_passcode: Optional[str]
     attend_text: Optional[str]
-
-    # NOT FINISHED YET
+    attend_json: Optional[Union[Json, None]]
 
     # access_key: Optional[str] # Maybe use in the future?
+    passcode: Optional[str]
 
     file_count: Optional[int]
+    internal_use_count: Optional[int] # Should be renamed to "internal_use_file_count"???
+    event_file_id_li_json: Optional[Union[Json, None]] # List of file IDs (actually id_random)
+    file_count_all: Optional[int] # Of all files under a session
+
+    status: Optional[str]
+    review: Optional[bool]
+    approve: Optional[bool]
+    ready: Optional[bool]
+    ready_on: Optional[datetime.datetime]
+    archive: Optional[bool] # Also in Event_Cfg_Base model
+    archive_on: Optional[datetime.datetime] # Also in Event_Cfg_Base model
+
+    mod_abstracts_json: Optional[Union[Json, None]]
+    mod_badges_json: Optional[Union[Json, None]]
+    mod_exhibits_json: Optional[Union[Json, None]]
+    mod_pres_mgmt_json: Optional[Union[Json, None]]
+
+    cfg_json: Optional[Union[Json, None]] # Store per event config options; Not currently used 2024-06-11
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields! Not currently used 2024-06-11
 
     enable: Optional[bool] # Also in Event_Cfg_Base model
     enable_from: Optional[datetime.datetime] = None
     enable_to: Optional[datetime.datetime] = None
 
-    archive: Optional[bool] # Also in Event_Cfg_Base model
-    archive_on: Optional[datetime.datetime] # Also in Event_Cfg_Base model
-
     hide: Optional[bool] # Also in Event_Cfg_Base model
     priority: Optional[bool]
     sort: Optional[int]

app/models/event_person_models.py

@@ -67,12 +67,18 @@ class Event_Person_Base(BaseModel):
     agree_to_tc: Optional[bool] # Agree to terms and conditions
     allow_tracking: Optional[bool] # Allow tracking for lead retrieval and other marketing
 
+    passcode: Optional[str] # Passcode for accessing the event
+
+    cfg_json: Optional[Union[Json, None]] # Store per person config options like theme, language, etc
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields!
+
     file_count: Optional[int]
 
     priority: Optional[bool]
     sort: Optional[int]
     group: Optional[str]
     enable: Optional[bool]
+    hide: Optional[bool]
 
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
@@ -121,8 +127,9 @@ class Event_Person_Base(BaseModel):
     person_given_name: Optional[str]
     person_middle_name: Optional[str]
     person_family_name: Optional[str]
-    person_display_name: Optional[str]
     person_full_name: Optional[str]
+    person_full_name_override: Optional[str]
+    # person_display_name: Optional[str]
     person_affiliations: Optional[str]
     person_email: Optional[str]
 
@@ -295,7 +302,8 @@ class Event_Person_New_Base(BaseModel):
     person_middle_name: Optional[str]
     person_family_name: Optional[str]
     person_full_name: Optional[str]
-    person_display_name: Optional[str]
+    person_full_name_override: Optional[str]
+    # person_display_name: Optional[str]
 
     # affiliations: Optional[str] # One or more affiliations with organizations, companies, and other groups
 

app/models/event_presentation_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -18,36 +18,21 @@ class Event_Presentation_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['event_presentation_id_random'],
-        alias = 'event_presentation_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'event_presentation_id'
-    )
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['event_presentation_id_random'])
+    event_presentation_id: Optional[str] = Field(None, **base_fields['event_presentation_id_random'])
+    
+    event_id: Optional[str] = Field(None, **base_fields['event_id_random'])
+    event_abstract_id: Optional[str] = Field(None, **base_fields['event_abstract_id_random'])
+    event_location_id: Optional[str] = Field(None, **base_fields['event_location_id_random'])
+    event_session_id: Optional[str] = Field(None, **base_fields['event_session_id_random'])
+    event_track_id: Optional[str] = Field(None, **base_fields['event_track_id_random'])
 
     external_id: Optional[str] = Field(
-        alias = 'event_presentation_external_id'
+        # alias = 'event_presentation_external_id'
     )
 
-    code: Optional[str] = Field(
-        # alias = 'event_presentation_code'
-    )
-
-    event_id_random: Optional[str]
-    event_id: Optional[int]
-
-    event_abstract_id_random: Optional[str]
-    event_abstract_id: Optional[int]
-
-    event_location_id_random: Optional[str]
-    event_location_id: Optional[int]
-
-    event_session_id_random: Optional[str]
-    event_session_id: Optional[int]
-
-    event_track_id_random: Optional[str]
-    event_track_id: Optional[int]
+    code: Optional[str]
 
     poc_event_person: Optional[Event_Person_Base]
     poc_person: Optional[Person_Base]
@@ -55,6 +40,8 @@ class Event_Presentation_Base(BaseModel):
     for_type: Optional[str]
     for_id: Optional[int]
 
+    abstract_code: Optional[str]
+
     # FUTURE: This event_presentation.type_code should override, the type_code of the event_session.type_code.
     type_code: Optional[str] # None, poster (image, video), assume presentation (PPT, Key, PDF, etc)
 
@@ -64,6 +51,8 @@ class Event_Presentation_Base(BaseModel):
     start_datetime: Optional[datetime.datetime]
     end_datetime: Optional[datetime.datetime]
 
+    passcode: Optional[str]
+
     file_count: Optional[int]
 
     enable: Optional[bool]
@@ -109,36 +98,37 @@ class Event_Presentation_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('id', always=True)
-    def event_presentation_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_presentation')
-        return None
-
-    @validator('event_id', always=True)
-    def event_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event')
-        return None
-
-    @validator('event_abstract_id', always=True)
-    def event_abstract_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_abstract_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_abstract')
-        return None
-
-    @validator('event_session_id', always=True)
-    def event_session_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('event_session_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='event_session')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('event_presentation_id_random'):
+            values['id'] = rid
+            values['event_presentation_id'] = rid
+            
+        if e_rid := values.get('event_id_random'):
+            values['event_id'] = e_rid
+        if ea_rid := values.get('event_abstract_id_random'):
+            values['event_abstract_id'] = ea_rid
+        if el_rid := values.get('event_location_id_random'):
+            values['event_location_id'] = el_rid
+        if es_rid := values.get('event_session_id_random'):
+            values['event_session_id'] = es_rid
+        if et_rid := values.get('event_track_id_random'):
+            values['event_track_id'] = et_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'event_presentation_id', 'event_id', 'event_abstract_id', 'event_location_id', 'event_session_id', 'event_track_id']:
+            if k in values and not isinstance(values[k], str) and values[k] is not None:
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Event Presentation Models ### Event_Presentation_Base() ###

app/models/event_presenter_models.py

@@ -20,20 +20,16 @@ class Event_Presenter_Base(BaseModel):
     log.debug(locals())
 
     id_random: Optional[str] = Field(
-        **base_fields['event_presenter_id_random'],
+        # **base_fields['event_presenter_id_random'],
         alias = 'event_presenter_id_random',
     )
     id: Optional[int] = Field(
         alias = 'event_presenter_id'
     )
 
-    external_id: Optional[str] = Field(
-        alias = 'event_presenter_external_id'
-    )
+    external_id: Optional[str]
 
-    code: Optional[str] = Field(
-        # alias = 'event_presenter_code'
-    )
+    code: Optional[str]
 
     event_id_random: Optional[str]
     event_id: Optional[int]
@@ -56,6 +52,9 @@ class Event_Presenter_Base(BaseModel):
     event_track_id_random: Optional[str]
     event_track_id: Optional[int]
 
+    person_id_random: Optional[str]
+    person_id: Optional[int]
+
     for_type: Optional[str]
     for_id: Optional[int]
 
@@ -89,15 +88,36 @@ class Event_Presenter_Base(BaseModel):
     email: Optional[str]
     website_url: Optional[str]
 
+    phone_li_json: Optional[Union[Json, None]]
+
+    # For social media in a JSON object format. The Aether standard field names should be used. Examples: url, url_text, icon, etc.
+    social_li_json: Optional[Union[Json, None]]
+
     tagline: Optional[str]
     biography: Optional[str]
 
-    picture_path: Optional[str]
+    picture_path: Optional[str] # Start using image_li_json instead
     picture_bg_color: Optional[str]
 
+    # For image files only in a JSON object format. The Aether standard field names should be used. Examples: url, url_text, alt_text, width, height, size (in bytes), etc.
+    image_li_json: Optional[Union[Json, None]] # "headshot" is probably the most common
+    # media_li_json: Optional[Union[Json, None]]
+
     role: Optional[str]
 
-    file_count: Optional[int]
+    passcode: Optional[str]
+
+    cfg_json: Optional[Union[Json, None]] # Store per presenter config options like theme, language, etc
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields!
+
+    file_count: Optional[int] # File count for the presenter
+    event_file_id_li_json: Optional[Union[Json, None]] # List of file IDs (actually id_random)
+
+    # General catchall for agreement or consent
+    agree: Optional[bool]
+
+    # Comments from the presenter. This is for internal use only.
+    comments: Optional[str]
 
     enable: Optional[bool]
     enable_from: Optional[datetime.datetime] = None
@@ -136,6 +156,16 @@ class Event_Presenter_Base(BaseModel):
     event_track_code: Optional[str]
     event_track_name: Optional[str]
 
+    person_external_id: Optional[str]
+    person_external_sys_id: Optional[str]
+    person_given_name: Optional[str]
+    person_family_name: Optional[str]
+    person_professional_title: Optional[str]
+    person_full_name: Optional[str]
+    person_affiliations: Optional[str]
+    person_primary_email: Optional[str]
+    person_passcode: Optional[str]
+
     # Including other related objects
     # event: Optional[Event_Base]
     # event_abstract: Optional[Event_Abstract_Base]
@@ -195,6 +225,214 @@ class Event_Presenter_Base(BaseModel):
             return redis_lookup_id_random(record_id_random=id_random, table_name='event_session')
         return None
 
+    @validator('person_id', always=True)
+    def person_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('person_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
+        return None
+
+    class Config:
+        underscore_attrs_are_private = True
+        allow_population_by_field_name = True
+        fields = base_fields
+# ### END ### API Event Presenter Models ### Event_Presenter_Base() ###
+
+
+
+# ### BEGIN ### API Event Presenter Models ### Event_Presenter_Base() ###
+class Event_Presenter_Out_Base(BaseModel):
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    id_random: Optional[str] = Field(
+        **base_fields['event_presenter_id_random'],
+        alias = 'event_presenter_id_random',
+    )
+    id: Optional[int] = Field(
+        alias = 'event_presenter_id'
+    )
+
+    external_id: Optional[str]
+
+    code: Optional[str]
+
+    event_id_random: Optional[str]
+    event_id: Optional[int]
+
+    # event_abstract_id_random: Optional[str]
+    # event_abstract_id: Optional[int]
+
+    # event_location_id_random: Optional[str]
+    # event_location_id: Optional[int]
+
+    # event_person_id_random: Optional[str]
+    # event_person_id: Optional[int]
+
+    event_presentation_id_random: Optional[str]
+    event_presentation_id: Optional[int]
+
+    event_session_id_random: Optional[str]
+    event_session_id: Optional[int]
+
+    # event_track_id_random: Optional[str]
+    # event_track_id: Optional[int]
+
+    person_id_random: Optional[str]
+    person_id: Optional[int]
+
+    # for_type: Optional[str]
+    # for_id: Optional[int]
+
+    pronouns: Optional[str] # Preferred pronouns
+    informal_name: Optional[str] # Informal or nick name they commonly go by
+
+    title_names: Optional[str] # Title for generation, official position, or professional or academic qualification, other honorific, or other name prefix
+    # prefix: Optional[str] # NOTE: Phasing out! Use *title_names* instead.
+    given_name: Optional[str]
+    middle_name: Optional[str]
+    family_name: Optional[str]
+    designations: Optional[str] # Temporary or long-term designations related to family, relationships, person differentiation (Junior/Senior), location, social status, professional qualifications, legal status, or other name suffix
+    # suffix: Optional[str] # NOTE: Phasing out! Use *designations* instead.
+
+    professional_title: Optional[str] # Professional title
+    # title: Optional[str] # NOTE: Phasing out! Use *professional_title* instead.
+
+    # display_name: Optional[str] # NOTE: This will be changed to full_name_override to match event_badge, event_person_profile, and person
+
+    # BEGIN # Auto created name variations
+    full_name: Optional[str] # title_names given_name middle_name family_name designations
+    full_name_override: Optional[str] # Override full_name; Actual name shown for presenter
+
+    # degree: Optional[str] # NOTE: Phasing out! Use *designations* instead.
+    # degrees: Optional[str] # NOTE: Phasing out! Use *designations* instead.
+    # credentials: Optional[str] # NOTE: Phasing out! Use *designations* instead.
+
+    affiliations: Optional[str] # One or more affiliations with organizations, companies, and other groups
+    # affiliation: Optional[str] # NOTE: Phasing out! Use *affiliations* instead.
+
+    email: Optional[str]
+    website_url: Optional[str]
+
+    # phone_li_json: Optional[Union[Json, None]]
+
+    # For social media in a JSON object format. The Aether standard field names should be used. Examples: url, url_text, icon, etc.
+    social_li_json: Optional[Union[Json, None]]
+
+    tagline: Optional[str]
+    biography: Optional[str]
+
+    # picture_path: Optional[str] # Start using image_li_json instead
+    # picture_bg_color: Optional[str]
+
+    # For image files only in a JSON object format. The Aether standard field names should be used. Examples: url, url_text, alt_text, width, height, size (in bytes), etc.
+    image_li_json: Optional[Union[Json, None]] # "headshot" is probably the most common
+    # media_li_json: Optional[Union[Json, None]]
+
+    # role: Optional[str]
+
+    data_json: Optional[Union[Json, None]] # For key value data. Careful with overwriting existing fields!
+    cfg_json: Optional[Union[Json, None]] # Store per presenter config options like theme, language, etc
+
+    # file_count: Optional[int]
+
+    # General catchall for agreement or consent
+    agree: Optional[bool]
+
+    # Comments from the presenter. This is for internal use only.
+    comments: Optional[str]
+
+    enable: Optional[bool]
+    # enable_from: Optional[datetime.datetime] = None
+    # enable_to: Optional[datetime.datetime] = None
+
+    hide: Optional[bool]
+    # public: Optional[bool]
+    # public_hide: Optional[bool]
+    # hide_event_launcher: Optional[bool]
+
+    priority: Optional[bool]
+    sort: Optional[int] # The presenter number if given
+    group: Optional[str]
+
+    notes: Optional[str]
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
+
+    # Including convenience data
+    # This is only for convenience. Probably going to keep unless it causes a problem.
+    # event_name: Optional[str]
+    # event_start_datetime: Optional[datetime.datetime]
+    # event_end_datetime: Optional[datetime.datetime]
+    # event_location_code: Optional[str]
+    # event_location_name: Optional[str]
+    # event_presentation_code: Optional[str]
+    # event_presentation_type_code: Optional[str]
+    # event_presentation_name: Optional[str]
+    # event_presentation_start_datetime: Optional[datetime.datetime]
+    # event_presentation_end_datetime: Optional[datetime.datetime]
+    # event_session_code: Optional[str]
+    # event_session_type_code: Optional[str]
+    # event_session_name: Optional[str]
+    # event_session_start_datetime: Optional[datetime.datetime]
+    # event_session_end_datetime: Optional[datetime.datetime]
+
+    person_external_id: Optional[str]
+    person_external_sys_id: Optional[str]
+    person_given_name: Optional[str]
+    person_family_name: Optional[str]
+    person_professional_title: Optional[str]
+    person_full_name: Optional[str]
+    person_affiliations: Optional[str]
+    person_primary_email: Optional[str]
+    person_passcode: Optional[str]
+
+    # Including other related objects
+
+    _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
+
+    @validator('id', always=True)
+    def event_presenter_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='event_presenter')
+        return None
+
+    @validator('event_id', always=True)
+    def event_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('event_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='event')
+        return None
+
+    # @validator('event_person_id', always=True)
+    # def event_person_id_lookup(cls, v, values, **kwargs):
+    #     if isinstance(v, int) and v > 0: return v
+    #     elif id_random := values.get('event_person_id_random'):
+    #         return redis_lookup_id_random(record_id_random=id_random, table_name='event_person')
+    #     return None
+
+    @validator('event_presentation_id', always=True)
+    def event_presentation_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('event_presentation_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='event_presentation')
+        return None
+
+    @validator('event_session_id', always=True)
+    def event_session_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('event_session_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='event_session')
+        return None
+
+    @validator('person_id', always=True)
+    def person_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('person_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
+        return None
+
     class Config:
         underscore_attrs_are_private = True
         allow_population_by_field_name = True

app/models/event_session_models.py

@@ -28,7 +28,7 @@ class Event_Session_Base(BaseModel):
     )
 
     external_id: Optional[str] = Field(
-        alias = 'event_session_external_id'
+        # alias = 'event_session_external_id'
     )
 
     code: Optional[str] = Field(
@@ -47,33 +47,44 @@ class Event_Session_Base(BaseModel):
     poc_event_person_id_random: Optional[str]
     poc_event_person_id: Optional[int]
 
-    # poc_person_id_random: Optional[str] # Not used or needed?
-    # poc_person_id: Optional[int] # Not used or needed?
+    poc_person_id_random: Optional[str]
+    poc_person_id: Optional[int]
+
+    # General catchall for agreement or consent
+    poc_agree: Optional[bool]
+
+    poc_kv_json: Optional[Union[Json, None]]
 
     # type_id_random: Optional[str] # Not used or needed?
     # type_id: Optional[int] # Not used or needed?
-    type_code: Optional[str] # None, poster (image, video), assume presentation (PPT, Key, PDF, etc)
+    type_code: Optional[str] # From client; max 25 characters for now; This is a bug with MariaDB?
 
     name: Optional[str]
     description: Optional[str]
 
+    proposal_json: Optional[Union[Json, None]] # Is this still used or needed? 2024-09-12
+
     start_datetime: Optional[datetime.datetime]
     end_datetime: Optional[datetime.datetime]
 
-    attend_url: Optional[str]
-    attend_url_text: Optional[str]
-    attend_url_passcode: Optional[str]
-    attend_phone: Optional[str]
-    attend_phone_passcode: Optional[str]
-    attend_text: Optional[str]
+    # Need to redo this using a JSON field
+    # attend_url: Optional[str]
+    # attend_url_text: Optional[str]
+    # attend_url_passcode: Optional[str]
+    # attend_phone: Optional[str]
+    # attend_phone_passcode: Optional[str]
+    # attend_text: Optional[str]
+    attend_json: Optional[Union[Json, None]]
 
-    rehearsal_start_datetime: Optional[datetime.datetime]
-    rehearsal_end_datetime: Optional[datetime.datetime]
-    rehearsal_url: Optional[str]
-    rehearsal_url_passcode: Optional[str]
-    rehearsal_phone: Optional[str]
-    rehearsal_phone_passcode: Optional[str]
-    rehearsal_text: Optional[str]
+    # Need to redo this using a JSON field
+    # rehearsal_start_datetime: Optional[datetime.datetime]
+    # rehearsal_end_datetime: Optional[datetime.datetime]
+    # rehearsal_url: Optional[str]
+    # rehearsal_url_passcode: Optional[str]
+    # rehearsal_phone: Optional[str]
+    # rehearsal_phone_passcode: Optional[str]
+    # rehearsal_text: Optional[str]
+    rehearsal_json: Optional[Union[Json, None]]
 
     image_path: Optional[str] # Not currently in use. For a banner or logo
     # presentation_file_path: Optional[str] # No longer used 2022-09-15
@@ -94,8 +105,12 @@ class Event_Session_Base(BaseModel):
     internal_notes_it: Optional[str] # IT and networking
     internal_notes_staff: Optional[str] # staffing and labor
 
-    file_count: Optional[int]
+    passcode: Optional[str]
+
+    file_count: Optional[int] # Only files directly under the session
     internal_use_count: Optional[int] # Should be renamed to "internal_use_file_count"???
+    event_file_id_li_json: Optional[Union[Json, None]] # List of file IDs (actually id_random)
+    file_count_all: Optional[int] # Of all files under a session
 
     status: Optional[int]
     review: Optional[bool]
@@ -105,6 +120,11 @@ class Event_Session_Base(BaseModel):
     alert: Optional[bool]
     alert_msg: Optional[str]
 
+    # Options: 'colloquium', 'lecture', 'panel', 'poster', 'symposium', 'workshop'
+    # This is mainly reflected in the Launcher.
+    ux_mode: Optional[str]
+    # Other options??? None, poster (image, video), assume presentation (PPT, Key, PDF, etc)
+
     enable: Optional[bool]
     enable_from: Optional[datetime.datetime] = None
     enable_to: Optional[datetime.datetime] = None
@@ -153,9 +173,16 @@ class Event_Session_Base(BaseModel):
     event_presentation_list: Optional[list[Event_Presentation_Base]] # Optional[Event_Presentation_Base]
     event_presenter_list: Optional[list] # Optional[Event_Presenter_Base]
     event_track: Optional[Event_Track_Base]
+
     poc_event_person: Optional[Event_Person_Base] # NOTE: Using thi will probably create an import loop
 
-    proposal_json: Optional[Union[Json, None]]
+    poc_person: Optional[Person_Base]
+    poc_person_external_id: Optional[str]
+    poc_person_given_name: Optional[str]
+    poc_person_family_name: Optional[str]
+    poc_person_full_name: Optional[str]
+    poc_person_primary_email: Optional[str]
+    poc_person_passcode: Optional[str]
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
@@ -187,6 +214,13 @@ class Event_Session_Base(BaseModel):
             return redis_lookup_id_random(record_id_random=id_random, table_name='event_track')
         return None
 
+    @validator('poc_person_id', always=True)
+    def poc_person_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('poc_person_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
+        return None
+
     class Config:
         underscore_attrs_are_private = True
         allow_population_by_field_name = True

app/models/hosted_file_link_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
-from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from typing import Dict, List, Optional, Set, Union, ClassVar
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -14,22 +14,13 @@ class Hosted_File_Link_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    # id_random: Optional[str] = Field(
-    #     **base_fields['hosted_file_link_id_random'],
-    #     alias = 'hosted_file_link_id_random',
-    # )
-    id: Optional[int] = Field(
-        #alias = 'hosted_file_link_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
+    id: Optional[Union[int, str]] = Field(None)
+    account_id: Optional[Union[int, str]] = Field(None, **base_fields['account_id_random'])
 
-    hosted_file_id_random: Optional[str]
-    hosted_file_id: Optional[int]
+    hosted_file_id: Optional[Union[int, str]] = Field(None, **base_fields['hosted_file_id_random'])
 
     link_to_type: Optional[str] # Should this be renamed to "link_to_obj_type" for clarity?
-    link_to_id_random: Optional[str] # Should this be renamed to "link_to_obj_id_random" for clarity?
-    link_to_id: Optional[int] # Should this be renamed to "link_to_obj_id" for clarity?
+    link_to_id: Optional[Union[int, str]] = Field(None) # Random string or integer
 
     # notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
@@ -40,21 +31,33 @@ class Hosted_File_Link_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('account_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map account_id
+        if a_rid := values.get('account_id_random'):
+            if not isinstance(values.get('account_id'), int):
+                values['account_id'] = a_rid
+                
+        # 2. Map hosted_file_id
+        if f_rid := values.get('hosted_file_id_random'):
+            if not isinstance(values.get('hosted_file_id'), int):
+                values['hosted_file_id'] = f_rid
+                
+        # 3. Map link_to_id
+        if l_rid := values.get('link_to_id_random'):
+            if not isinstance(values.get('link_to_id'), int):
+                values['link_to_id'] = l_rid
+                
+        return values
 
-    @validator('link_to_id', always=True)
-    def link_to_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['link_to_id_random'] and values['link_to_type']:
-            return redis_lookup_id_random(record_id_random=values['link_to_id_random'], table_name=values['link_to_type'])
-        return None
+    # Fields that are part of the model (for reading) but should not be saved to the DB table
+    fields_to_exclude_from_db: ClassVar[list] = [
+        'link_to'
+    ]
 
     class Config:
         underscore_attrs_are_private = True

app/models/hosted_file_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -14,17 +14,10 @@ class Hosted_File_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['hosted_file_id_random'],
-        alias = 'hosted_file_id_random',
-        # default_factory = lambda:secrets.token_urlsafe(default_num_bytes),
-    )
-    id: Optional[int] = Field(
-        alias = 'hosted_file_id'
-    )
-
-    account_id_random: Optional[str]
-    account_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[Union[int, str]] = Field(None, **base_fields['hosted_file_id_random'])
+    hosted_file_id: Optional[Union[int, str]] = Field(None, **base_fields['hosted_file_id_random'])
+    account_id: Optional[Union[int, str]] = Field(None, **base_fields['account_id_random'])
 
     hash_sha256: Optional[str]
     title: Optional[str]
@@ -32,30 +25,23 @@ class Hosted_File_Base(BaseModel):
 
     version: Optional[int]
 
-    directory_path: Optional[str]
-    subdirectory_path: Optional[str] # NOTE: This will frequently only contain numbers, but it still needs to be a string
+    subdirectory_path: Optional[str] = Field(None, exclude=True) # NOTE: This will frequently only contain numbers, but it still needs to be a string
     filename: Optional[str]
     extension: Optional[str]
     content_type: Optional[str]
     mimetype: Optional[str]
     size: Optional[int] # In bytes
 
-    cloud_storage: Optional[str]
-    owner_user_id: Optional[int]
-    group_user_id: Optional[str]
-
-    package_name: Optional[str]
-
     already_exists: Optional[str] # This will probably only be populated on upload results
     copy_timer: Optional[str] # This will probably only be populated on upload results
     saved: Optional[str] # This will probably only be populated on upload results
 
-    # metadata: Optional[str]
+    enable: Optional[bool]
 
-    # hide: Optional[bool]
-    # priority: Optional[bool]
-    # sort: Optional[int]
-    # group: Optional[str]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
 
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
@@ -75,28 +61,29 @@ class Hosted_File_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('hosted_file_id_random', always=True)
-    def hosted_file_id_random_copy(cls, v, values, **kwargs):
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def hosted_file_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='hosted_file')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('account_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Capture the random ID string
+        rid = values.get('id_random') or values.get('hosted_file_id_random')
+            
+        # 2. Map Random Strings to Clean Names for the Frontend
+        # We always want the string version in 'id' and 'hosted_file_id' for the API response
+        if rid:
+            values['id'] = rid
+            values['hosted_file_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            # If we have a random account ID string, use it for the Vision API
+            values['account_id'] = a_rid
+            
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Hosted File Models ### Hosted_File_Base() ###

app/models/journal_entry_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
-from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from typing import Dict, List, Optional, Set, Union, ClassVar
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -14,34 +14,84 @@ class Journal_Entry_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['journal_entry_id_random'],
-        alias = 'journal_entry_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'journal_entry_id'
-    )
-
-    journal_id_random: Optional[str]
-    journal_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['journal_entry_id_random'])
+    journal_entry_id: Optional[str] = Field(None, **base_fields['journal_entry_id_random'])
+    journal_id: Optional[str] = Field(None, **base_fields['journal_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
 
+    external_id: Optional[str] # ID generated by or for external systems (should be stable and not change)
+    import_id: Optional[str] # Used for import purposes to track the source of the data
     code: Optional[str]
 
+    for_type: Optional[str] # 'person', 'user', 'account', etc
+    for_id: Optional[int] = Field(None, exclude=True)
+    for_id_random: Optional[str]
+
     name: Optional[str]
+    short_name: Optional[str]
     summary: Optional[str]
+    outline: Optional[str]
+
     content: Optional[str]
+    content_html: Optional[str]
+    content_json: Optional[Union[Json, None]]
+    content_encrypted: Optional[str]
+
+    history: Optional[str] # Used to store the history of the journal entry content
+    history_encrypted: Optional[str]
+
+    passcode_hash: Optional[str] # Used to store the hash of the passcode for looking up the passcode
+
+    template: Optional[bool] = False # If this is a template entry, it can be used to create new entries based on this template
+
+    type_code: Optional[str] # 'log', 'tracking', 'personal', 'professional', etc
+    topic_code: Optional[str]
+    category_code: Optional[str]
+    # keywords: Optional[str]
+    tags: Optional[str]
+
+    start_datetime: Optional[datetime.datetime]
+    end_datetime: Optional[datetime.datetime]
+    timezone: Optional[str] # = 'UTC' # Default to UTC
+    seconds: Optional[int]
+
+    location: Optional[str]
+    latitude: Optional[float]
+    longitude: Optional[float]
+
+    billable: Optional[bool]
+    bill_to: Optional[str]
+
+    alert: Optional[bool] = False
+    alert_msg: Optional[str] = None
 
     private: Optional[bool] = True
     public: Optional[bool] = False
     personal: Optional[bool] = True
     professional: Optional[bool] = False
 
-    keywords: Optional[str]
+    parent_id: Optional[str] = Field(None, **base_fields['journal_entry_id_random'])
+    # parent_id_random: Optional[str]
+    
+    related_entry_id_random: Optional[List[str]]
+    related_entry_id_li: Optional[List[int]] = Field(None, exclude=True)
 
+    due_datetime: Optional[datetime.datetime]
+    due_alert: Optional[bool]
     archive_on: Optional[datetime.datetime]
     archive: Optional[bool]
 
-    data_json: Optional[Json]
+    passcode: Optional[str] # Used to read and write to the journal entry
+    passcode_timeout: Optional[int] # Number of seconds before asking for the passcode again
+    passcode_read: Optional[str] # Used to read the journal entry
+    passcode_read_expire: Optional[int] # Number of seconds to expire the read passcode
+    # passcode_write: Optional[str] # Used to write to the journal entry
+    # passcode_write_expire: Optional[int] # Number of seconds to expire the write passcode
+
+    url_kv_json: Optional[Union[Json, None]]
+    data_json: Optional[Union[Json, None]] # Used to store additional data for the journal
+    meta_json: Optional[Union[Json, None]] # Used to store additional data about the journal entry
 
     enable: Optional[bool]
     hide: Optional[bool]
@@ -53,29 +103,44 @@ class Journal_Entry_Base(BaseModel):
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
+    # Including other related objects
+    # This is only for convenience. Probably going to keep unless it causes a problem.
+    file_count: Optional[int]
+
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('id', always=True)
-    def journal_entry_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('journal_entry_id_random'):
+            values['id'] = rid
+            values['journal_entry_id'] = rid
+            
+        if j_rid := values.get('journal_id_random'):
+            values['journal_id'] = j_rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+            
+        if p_rid := values.get('parent_id_random'):
+            values['parent_id'] = p_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'journal_entry_id', 'journal_id', 'account_id', 'parent_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='journal_entry')
-        return None
-
-    @validator('journal_id', always=True)
-    def journal_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['journal_id_random']:
-            return redis_lookup_id_random(record_id_random=values['journal_id_random'], table_name='journal')
-        return None
+    # Fields that are part of the model (for reading) but should not be saved to the DB table
+    fields_to_exclude_from_db: ClassVar[list] = ['file_count']
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Journal Entry Models ### Journal_Entry_Base() ###

app/models/journal_models.py

@@ -1,13 +1,14 @@
 import datetime, pytz
 
-from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from typing import Dict, List, Optional, Set, Union, ClassVar
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
 
 from app.models.common_field_schema import base_fields, default_num_bytes
 from app.models.journal_entry_models import Journal_Entry_Base
+# from app.models.person_models import Person_Base
 
 
 # ### BEGIN ### API Journal Models ### Journal_Base() ###
@@ -15,34 +16,86 @@ class Journal_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['journal_id_random'],
-        alias = 'journal_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'journal_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
-
-    person_id_random: Optional[str]
-    person_id: Optional[int]
-
-    user_id_random: Optional[str]
-    user_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['journal_id_random'])
+    journal_id: Optional[str] = Field(None, **base_fields['journal_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
 
+    external_id: Optional[str] # ID generated by or for external systems (should be stable and not change)
+    import_id: Optional[str] # Used for import purposes to track the source of the data
     code: Optional[str]
 
+    for_type: Optional[str] # 'person', 'user', 'account', etc
+    for_id: Optional[int] = Field(None, exclude=True)
+    for_id_random: Optional[str]
+
+    name: Optional[str]
+    short_name: Optional[str]
+    summary: Optional[str]
+    outline: Optional[str]
+
+    description: Optional[str]
+    description_html: Optional[str]
+    description_json: Optional[str]
+
+    type_code: Optional[str] # 'log', 'tracking', 'personal', 'professional', etc
+    tags: Optional[str]
+
+    start_datetime: Optional[datetime.datetime]
+    end_datetime: Optional[datetime.datetime]
+    timezone: Optional[str] # = 'UTC' # Default to UTC
+    seconds: Optional[int]
+
+    location: Optional[str]
+    latitude: Optional[float]
+    longitude: Optional[float]
+
+    billable: Optional[bool]
+    bill_to: Optional[str]
+
+    alert: Optional[bool] = False
+    alert_msg: Optional[str] = None
+
+    private: Optional[bool] = True
+    public: Optional[bool] = False
+    personal: Optional[bool] = True
+    professional: Optional[bool] = False
+
+    # Default the Journal Entries to private, public, personal, and professional
     default_private: Optional[bool]
     default_public: Optional[bool]
     default_personal: Optional[bool]
     default_professional: Optional[bool]
 
-    private_passcode: Optional[str]
-    public_passcode: Optional[str]
+    due_datetime: Optional[datetime.datetime]
+    due_alert: Optional[bool]
+    archive_on: Optional[datetime.datetime]
+    archive: Optional[bool]
 
-    name: Optional[str]
-    summary: Optional[str]
+    allow_auth: Optional[bool]
+    auth_key: Optional[str]
+
+    passcode: Optional[str] # Used to read and write to the journal entry
+    passcode_timeout: Optional[int] # Number of seconds before asking for the passcode again
+    # private_passcode: Optional[str]
+    # public_passcode: Optional[str]
+
+    passcode_read: Optional[str] # Used to read the journal
+    passcode_read_expire: Optional[int] # Number of seconds to expire the read passcode
+    # passcode_write: Optional[str] # Used to write to the journal
+    # passcode_write_expire: Optional[int] # Number of seconds to expire the write passcode
+
+    private_passcode: Optional[str] # Used with the passcode to encrypt and decrypt the Journal Entries
+    public_passcode: Optional[str] # Used to allow external people to view a Journal Entry
+
+    sort_by: Optional[str]
+    sort_by_desc: Optional[bool]
+
+    cfg_json: Optional[Union[Json, None]]
+    data_json: Optional[Union[Json, None]] # Used to store additional data for the journal
+    meta_json: Optional[Union[Json, None]] # Used to store additional data for about the journal
 
     enable: Optional[bool]
     hide: Optional[bool]
@@ -55,40 +108,60 @@ class Journal_Base(BaseModel):
     updated_on: Optional[datetime.datetime] = None
 
     # Including other related objects
+    journal_entry_count: Optional[int] # Number of journal entries in the journal
     journal_entry_list: Optional[list[Journal_Entry_Base]] # Journal_Entry_Base()
+    # This is only for convenience. Probably going to keep unless it causes a problem.
+    file_count: Optional[int] # Only files directly under the journal
+    file_count_all: Optional[int] # All files under a journal and entries
+
+    # This person is essentially the owner of the journal
+    # person: Optional[Person_Base]
+    person_external_id: Optional[str]
+    person_given_name: Optional[str] = None
+    person_family_name: Optional[str] = None
+    person_full_name: Optional[str] = None
+    person_primary_email: Optional[str] = None
+    person_passcode: Optional[str] = None
+
+    # person: Optional[Person_Base]
+    # user: Optional[User_Base]
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('id', always=True)
-    def journal_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('journal_id_random'):
+            values['id'] = rid
+            values['journal_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if p_rid := values.get('person_id_random'):
+            values['person_id'] = p_rid
+        if u_rid := values.get('user_id_random'):
+            values['user_id'] = u_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'journal_id', 'account_id', 'person_id', 'user_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='journal')
-        return None
-
-    @validator('person_id', always=True)
-    def person_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['person_id_random']:
-            return redis_lookup_id_random(record_id_random=values['person_id_random'], table_name='person')
-        return None
-
-    @validator('user_id', always=True)
-    def user_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['user_id_random']:
-            return redis_lookup_id_random(record_id_random=values['user_id_random'], table_name='user')
-        return None
+    # Fields that are part of the model (for reading) but should not be saved to the DB table
+    fields_to_exclude_from_db: ClassVar[list] = [
+        'person_external_id', 'person_given_name', 'person_family_name', 
+        'person_full_name', 'person_primary_email', 'person_passcode',
+        'journal_entry_count', 'file_count', 'file_count_all'
+    ]
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Journal Models ### Journal_Base() ###

app/models/membership_group_models.py

@@ -42,6 +42,15 @@ class Membership_Group_Base(BaseModel):
 
     expire_in_days: Optional[int]
 
+    enable: Optional[bool]
+    enable_from: Optional[datetime.datetime] = None
+    enable_to: Optional[datetime.datetime] = None
+
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
@@ -104,5 +113,6 @@ class Membership_Group_Base(BaseModel):
     class Config:
         underscore_attrs_are_private = True
         fields = base_fields
+        allow_population_by_field_name = True
 
 # Membership_Group_Base.update_forward_refs()

app/models/membership_person_group_models.py

@@ -55,6 +55,12 @@ class Membership_Person_Group_Base(BaseModel):
     flag: Optional[bool]
     flag_message: Optional[str]
 
+    enable: Optional[bool]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+
     notes: Optional[str]
 
     created_on: Optional[datetime.datetime] = None
@@ -117,5 +123,6 @@ class Membership_Person_Group_Base(BaseModel):
     class Config:
         underscore_attrs_are_private = True
         fields = base_fields
+        allow_population_by_field_name = True
 
 # Membership_Base.update_forward_refs()

app/models/membership_type_models.py

@@ -58,6 +58,7 @@ class Membership_Type_Base(BaseModel):
     enable_from: Optional[datetime.datetime] = None
     enable_to: Optional[datetime.datetime] = None
 
+    hide: Optional[bool]
     priority: Optional[bool]
     sort: Optional[int] = Field(0, ge=0, lt=100) # Essentially the membership level. Should there be a range limit?
     group: Optional[str]
@@ -106,4 +107,9 @@ class Membership_Type_Base(BaseModel):
     #         return redis_lookup_id_random(record_id_random=values['account_id_random'], table_name='account')
     #     return None
 
+    class Config:
+        underscore_attrs_are_private = True
+        fields = base_fields
+        allow_population_by_field_name = True
+
 # Membership_Type_Base.update_forward_refs()

app/models/order_cfg_models.py

@@ -34,6 +34,16 @@ class Order_Cfg_Base(BaseModel):
     stripe_publishable_key: Optional[str] # Publish/Sharable
     stripe_account_id: Optional[str] # Connected Stripe Account ID
 
+    enable: Optional[bool]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+
+    notes: Optional[str]
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
+
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
     @validator('account_id', always=True)
@@ -44,3 +54,8 @@ class Order_Cfg_Base(BaseModel):
         if values['account_id_random']:
             return redis_lookup_id_random(record_id_random=values['account_id_random'], table_name='account')
         return None
+
+    class Config:
+        underscore_attrs_are_private = True
+        fields = base_fields
+        allow_population_by_field_name = True

app/models/order_line_models.py

@@ -70,8 +70,9 @@ class Order_Line_Base(BaseModel):
     for_person_id_random: Optional[str]
     for_person_given_name: Optional[str] # Dynamic from v_order_line
     for_person_family_name: Optional[str] # Dynamic from v_order_line
-    for_person_display_name: Optional[str] # Dynamic from v_order_line
     for_person_full_name: Optional[str] # Dynamic from v_order_line
+    for_person_full_name_override: Optional[str] # Dynamic from v_order_line
+    # for_person_display_name: Optional[str] # Dynamic from v_order_line
 
     name: Optional[str] # Should be the same as product_name above
     quantity: int = Field(0, ge=0, lt=150)

app/models/order_line_models_v3.py

@@ -71,8 +71,9 @@ class Order_Line_Base(BaseModel):
     for_person_id_random: Optional[str]
     for_person_given_name: Optional[str] # Dynamic from v_order_line
     for_person_family_name: Optional[str] # Dynamic from v_order_line
-    for_person_display_name: Optional[str] # Dynamic from v_order_line
     for_person_full_name: Optional[str] # Dynamic from v_order_line
+    for_person_full_name_override: Optional[str] # Dynamic from v_order_line
+    # for_person_display_name: Optional[str] # Dynamic from v_order_line
 
     name: Optional[str] # Should be the same as product_name above
     quantity: int = Field(0, ge=0, lt=150)
@@ -248,8 +249,9 @@ class Order_Line_Full_Detail_Base(Order_Line_Base):
     person_id_random: Optional[str]
     person_given_name: Optional[str]
     person_family_name: Optional[str]
-    person_display_name: Optional[str]
     person_full_name: Optional[str]
+    person_full_name_override: Optional[str]
+    # person_display_name: Optional[str]
 
     person_contact_id: Optional[int]
     person_contact_id_random: Optional[str]

app/models/organization_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -18,21 +18,13 @@ class Organization_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['organization_id_random'],
-        alias = 'organization_id_random',
-    )
-    id: Optional[int] = Field(
-        #alias = 'organization_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
-    contact_id_random: Optional[str]
-    contact_id: Optional[int]
-    person_id_random: Optional[str]
-    person_id: Optional[int]
-    user_id_random: Optional[str]
-    user_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['organization_id_random'])
+    organization_id: Optional[str] = Field(None, **base_fields['organization_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    contact_id: Optional[str] = Field(None, **base_fields['contact_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
 
     name: Optional[str]
     tagline: Optional[str]
@@ -49,6 +41,8 @@ class Organization_Base(BaseModel):
     thumbnail_path: Optional[str]
     thumbnail_bg_color: Optional[str]
 
+    enable: Optional[bool]
+    hide: Optional[bool]
     priority: Optional[int]
     sort: Optional[int]
     group: Optional[str]
@@ -65,63 +59,35 @@ class Organization_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('organization_id_random', always=True)
-    def organization_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def organization_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='organization')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['account_id_random']:
-            return redis_lookup_id_random(record_id_random=values['account_id_random'], table_name='account')
-        return None
-
-    @validator('contact_id', always=True)
-    def contact_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['contact_id_random']:
-            return redis_lookup_id_random(record_id_random=values['contact_id_random'], table_name='contact')
-        return None
-
-    @validator('person_id', always=True)
-    def person_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['person_id_random']:
-            return redis_lookup_id_random(record_id_random=values['person_id_random'], table_name='person')
-        return None
-
-    @validator('user_id', always=True)
-    def user_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['user_id_random']:
-            return redis_lookup_id_random(record_id_random=values['user_id_random'], table_name='user')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('organization_id_random'):
+            values['id'] = rid
+            values['organization_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if c_rid := values.get('contact_id_random'):
+            values['contact_id'] = c_rid
+        if p_rid := values.get('person_id_random'):
+            values['person_id'] = p_rid
+        if u_rid := values.get('user_id_random'):
+            values['user_id'] = u_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'account_id', 'contact_id', 'person_id', 'user_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Organization Models ### Organization_Base() ###

app/models/page_models.py

@@ -1,10 +1,10 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
-from app.lib_general import *
+from app.lib_general import log, logging
 
 from app.models.common_field_schema import base_fields, default_num_bytes
 
@@ -14,29 +14,50 @@ class Page_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['page_id_random'],
-        alias = 'page_id_random',
-    )
-    id: Optional[int] = Field(
-        #alias = 'page_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['page_id_random'])
+    page_id: Optional[str] = Field(None, **base_fields['page_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    site_id: Optional[str] = Field(None, **base_fields['site_id_random'])
 
-    alias: Optional[str]
+    # page_id_random: Optional[str] = Field(
+    #     **base_fields['page_id_random'],
+    #     alias = 'page_id_random',
+    # )
+    # id: Optional[int] = Field(
+    #     alias = 'page_id'
+    # )
+
+    code: Optional[str]
     name: Optional[str]
+    title: Optional[str]
+    description: Optional[str]
+    summary: Optional[str]
+    outline: Optional[str]
+
+    head_html: Optional[str]
+    body_html: Optional[str]
+    footer_html: Optional[str]
+
+    content_html: Optional[str]
+    content_json: Optional[Union[Json, None]]
+
+    # keywords: Optional[str]
+    tags: Optional[str]
+
+    start_datetime: Optional[datetime.datetime]
+    end_datetime: Optional[datetime.datetime]
+    timezone: Optional[str] # = 'UTC' # Default to UTC
+
+    cfg_json: Optional[Union[Json, None]]
+    data_json: Optional[Union[Json, None]] # Used to store additional data for the page
+    meta_json: Optional[Union[Json, None]] # Used to store additional data for about the page
 
     enable: Optional[bool]
-    enable_from: Optional[datetime.datetime] = None
-    enable_to: Optional[datetime.datetime] = None
-
-    title: Optional[str]
-    body: Optional[str]
-    style_href: Optional[str]
-    script_src: Optional[str]
-
-    authentication_required: Optional[bool]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
 
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
@@ -44,27 +65,31 @@ class Page_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('page_id_random', always=True)
-    def page_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def page_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='page')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('page_id_random'):
+            values['id'] = rid
+            values['page_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if s_rid := values.get('site_id_random'):
+            values['site_id'] = s_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'account_id', 'site_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Page Models ### Page_Base() ###

app/models/person_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging
@@ -23,31 +23,17 @@ class Person_Base(BaseModel):
     log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['person_id_random'],
-        alias = 'person_id_random',
-        # default_factory = lambda:secrets.token_urlsafe(default_num_bytes),
-    )
-    id: Optional[int] = Field(
-        alias = 'person_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['person_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    contact_id: Optional[str] = Field(None, **base_fields['contact_id_random'])
+    organization_id: Optional[str] = Field(None, **base_fields['organization_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
+    membership_person_id: Optional[str] = Field(None, **base_fields['membership_person_id_random'])
 
-    contact_id_random: Optional[str]
-    contact_id: Optional[int]
-
-    organization_id_random: Optional[str]
-    organization_id: Optional[int]
-
-    user_id_random: Optional[str]
-    user_id: Optional[int]
-
-    membership_person_id_random: Optional[str] # Linked from membership_person using the v_person view
-    membership_person_id: Optional[int] # Linked from membership_person using the v_person view
-
-    pronouns: Optional[str] # Preferred pronouns
-    informal_name: Optional[str] # Informal or nick name they commonly go by
+    # pronouns: Optional[str] # MISSING in physical table
+    # informal_name: Optional[str] # MISSING in physical table
 
     title_names: Optional[str] # Title for generation, official position, or professional or academic qualification, other honorific, or other name prefix
     prefix: Optional[str] # NOTE: Phasing out! Use *title_names* instead.
@@ -55,17 +41,11 @@ class Person_Base(BaseModel):
     middle_name: Optional[str]
     family_name: Optional[str]
     designations: Optional[str] # Temporary or long-term designations related to family, relationships, person differentiation (Junior/Senior), location, social status, professional qualifications, legal status, or other name suffix
-    designation: Optional[str] # NOTE: Phasing out! Use *designations* instead.
     suffix: Optional[str] # NOTE: Phasing out! Use *designations* instead.
 
     professional_title: Optional[str] # Professional title
-    # title: Optional[str] # NOTE: Phasing out! Use *professional_title* instead.
 
-    display_name: Optional[str] # NOTE: This will be changed to full_name_override to match event_badge, event_presenter, and event_profile
-    informal_display_name: Optional[str] # Custom what they want for informal public display
-    professional_display_name: Optional[str] # Custom what they want for professional public display. This should include professional title.
-
-    preferred_display_name: Optional[str] # '', 'informal', 'professional'
+    # preferred_display_name: Optional[str] # MISSING in physical table
 
     # BEGIN # Auto created name variations
     first_last_name: Optional[str] # With SQL view?
@@ -79,8 +59,8 @@ class Person_Base(BaseModel):
     # END # Auto created name variations
 
     affiliations: Optional[str] # One or more affiliations with organizations, companies, and other groups
-    # affiliation: Optional[str] # NOTE: Phasing out! Use *affiliations* instead.
-    # organization_name: Optional[str] # NOTE: Phasing out! Use *affiliations* instead.
+
+    primary_email: Optional[str]
 
     tagline: Optional[Union[None, str]]
 
@@ -91,24 +71,28 @@ class Person_Base(BaseModel):
     email_allowed: Optional[bool]
     paper_mail_allowed: Optional[bool]
 
+    source_code: Optional[str]
+
     external_id: Optional[str] # Generated internally or externally. Needs to be stable. It should not change.
     external_sys_id: Optional[str] # Generated by external system (should be stable and not change)
 
     stripe_customer_id: Optional[str]
 
     allow_auth_key: Optional[bool]
-    auth_key: Optional[str]
+    auth_key: Optional[str] # Intended for one time use; more complex
+    passcode: Optional[str] # For basic quick access; 8 characters or more; can change as needed
 
     status: Optional[str]
     # status_id: Optional[int] # From a lookup
     # status_name: Optional[str] # Status name from the lookup
 
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
     enable: Optional[bool]
 
-    group: Optional[str]
-
     notes: Optional[str]
-
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
@@ -120,10 +104,19 @@ class Person_Base(BaseModel):
     cc_email: Optional[str]
     # Maybe add timezone in the future?
 
+    username: Optional[str]
+    user_name: Optional[str]
+    user_email: Optional[str]
+    user_allow_auth_key: Optional[bool]
+    user_super: Optional[bool]
+    user_manager: Optional[bool]
+    user_administrator: Optional[bool]
+    user_public: Optional[bool]
+
     # Including JSON data
-    data_json: Optional[Json]
-    other_json: Optional[Json]
-    meta_json: Optional[Json]
+    data_json: Optional[Union[Json, None]]
+    other_json: Optional[Union[Json, None]]
+    meta_json: Optional[Union[Json, None]]
 
     # Including other related objects
     # archive_list: Optional[list] # Archive_Base()
@@ -150,67 +143,49 @@ class Person_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('person_id_random', always=True)
-    def person_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('person_id_random'):
+            values['id'] = rid
+            values['person_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if c_rid := values.get('contact_id_random'):
+            values['contact_id'] = c_rid
+        if o_rid := values.get('organization_id_random'):
+            values['organization_id'] = o_rid
+        if u_rid := values.get('user_id_random'):
+            values['user_id'] = u_rid
+        if mp_rid := values.get('membership_person_id_random'):
+            values['membership_person_id'] = mp_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'account_id', 'contact_id', 'organization_id', 'user_id', 'membership_person_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
-        if values['id_random']:
-            return values['id_random']
-        return None
+    @validator('given_name', always=True)
+    def given_name_validator(cls, v):
+        if v is None:
+            return ""
+        return v
 
-    @validator('id', always=True)
-    def person_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('account_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
-        return None
-
-    @validator('contact_id', always=True)
-    def contact_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('contact_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='contact')
-        return None
-
-    @validator('organization_id', always=True)
-    def organization_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('organization_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='organization')
-        return None
-
-    @validator('user_id', always=True)
-    def user_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('user_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='user')
-        return None
+    @validator('allow_auth_key', always=True)
+    def allow_auth_key_validator(cls, v):
+        if v is None:
+            return True
+        return v
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Person Models ### Person_Base() ###

app/models/post_comment_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging, secure_hash_string
@@ -12,41 +12,40 @@ from app.models.user_models import User_Base
 
 
 # ### BEGIN ### API Post Comment Models ### Post_Comment_Base() ###
+# Updated 2024-11-13
 class Post_Comment_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['post_comment_id_random'],
-        alias = 'post_comment_id_random',
-    )
-    id: Optional[int] = Field(
-        #alias = 'post_comment_id'
-    )
-
-    post_id_random: Optional[str]
-    post_id: Optional[int]
-
-    person_id_random: Optional[str]
-    person_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['post_comment_id_random'])
+    post_comment_id: Optional[str] = Field(None, **base_fields['post_comment_id_random'])
+    post_id: Optional[str] = Field(None, **base_fields['post_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
 
     external_person_id: Optional[str] # Person ID generated by external system (should be stable and not change)
 
-    user_id_random: Optional[str]
-    user_id: Optional[int]
-
     title: Optional[str]
     content: Optional[str]
 
     anonymous: Optional[bool]
     full_name: Optional[str]
     email: Optional[str]
-    #timezone: Optional[str]
+    notify: Optional[bool]
+    # timezone: Optional[str]
 
+    linked_li_json: Optional[Union[Json, None]]
+    # cfg_json: Optional[Union[Json, None]]
+
+    enable: Optional[bool]
+
+    hide: Optional[bool]
     priority: Optional[bool]
     sort: Optional[int]
     group: Optional[str]
 
+    notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
@@ -56,54 +55,33 @@ class Post_Comment_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('post_comment_id_random', always=True)
-    def post_comment_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def post_comment_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='post_comment')
-        return None
-
-    @validator('post_id', always=True)
-    def post_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['post_id_random']:
-            return redis_lookup_id_random(record_id_random=values['post_id_random'], table_name='post')
-        return None
-
-    @validator('person_id', always=True)
-    def person_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values.get('person_id_random'):
-            return redis_lookup_id_random(record_id_random=values['person_id_random'], table_name='person')
-        return None
-
-    @validator('user_id', always=True)
-    def user_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values.get('user_id_random'):
-            return redis_lookup_id_random(record_id_random=values['user_id_random'], table_name='user')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('post_comment_id_random'):
+            values['id'] = rid
+            values['post_comment_id'] = rid
+            
+        if p_rid := values.get('post_id_random'):
+            values['post_id'] = p_rid
+        if per_rid := values.get('person_id_random'):
+            values['person_id'] = per_rid
+        if u_rid := values.get('user_id_random'):
+            values['user_id'] = u_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'post_comment_id', 'post_id', 'person_id', 'user_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        allow_population_by_field_name = True
+        allow_population_by_field_name = False
         fields = base_fields
 # ### END ### API Post Comment Models ### Post_Comment_Base() ###

app/models/post_models.py

@@ -1,7 +1,7 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
 from app.db_sql import redis_lookup_id_random
 from app.lib_general import log, logging, secure_hash_string
@@ -16,29 +16,23 @@ class Post_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        # **base_fields['post_id_random'],
-        alias = 'post_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'post_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
-
-    person_id_random: Optional[str]
-    person_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['post_id_random'])
+    post_id: Optional[str] = Field(None, **base_fields['post_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
 
     external_person_id: Optional[str] # Person ID generated by external system (should be stable and not change)
 
-    user_id_random: Optional[str]
-    user_id: Optional[int]
+    # user_id_random: Optional[str]
+    # user_id: Optional[int]
 
-    type_id_random: Optional[str]
-    type_id: Optional[int]
+    # type_id_random: Optional[str]
+    # type_id: Optional[int]
 
-    topic_id_random: Optional[str]
-    topic_id: Optional[int]
+    # topic_id_random: Optional[str]
+    # topic_id: Optional[int]
 
     type: Optional[str]
 
@@ -50,17 +44,17 @@ class Post_Base(BaseModel):
     anonymous: Optional[bool]
     full_name: Optional[str]
     email: Optional[str]
-    timezone: Optional[str]
+    notify: Optional[bool]
+    # timezone: Optional[str]
 
     post_comment_count: Optional[int] # post comment count using view
 
-    enable: Optional[bool]
     enable_from: Optional[datetime.datetime] = None
     enable_to: Optional[datetime.datetime] = None
 
     enable_comments: Optional[bool]
     unauthenticated_access: Optional[bool]
-    hide: Optional[bool]
+
     status: Optional[int]
     review: Optional[bool]
     approve: Optional[bool]
@@ -69,6 +63,12 @@ class Post_Base(BaseModel):
     archive_on: Optional[datetime.datetime] = None
     archive: Optional[bool]
 
+    linked_li_json: Optional[Union[Json, None]]
+    cfg_json: Optional[Union[Json, None]]
+
+    enable: Optional[bool]
+
+    hide: Optional[bool]
     priority: Optional[bool]
     sort: Optional[int]
     group: Optional[str]
@@ -89,47 +89,33 @@ class Post_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('post_id_random', always=True)
-    def post_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def post_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='post')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('account_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
-        return None
-
-    @validator('person_id', always=True)
-    def person_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('person_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
-        return None
-
-    @validator('user_id', always=True)
-    def user_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('user_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='user')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('post_id_random'):
+            values['id'] = rid
+            values['post_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if p_rid := values.get('person_id_random'):
+            values['person_id'] = p_rid
+        if u_rid := values.get('user_id_random'):
+            values['user_id'] = u_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'post_id', 'account_id', 'person_id', 'user_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
     class Config:
         underscore_attrs_are_private = True
         fields = base_fields
+        allow_population_by_field_name = False
 # ### END ### API Post Models ### Post_Base() ###

app/models/product_models.py

@@ -103,4 +103,5 @@ class Product_Base(BaseModel):
     class Config:
         underscore_attrs_are_private = True
         fields = base_fields
+        allow_population_by_field_name = True
 # ### END ### API Product Models ### Product_Base() ###

app/models/response_models.py

@@ -4,7 +4,10 @@ from typing import Dict, List, Optional, Set, Union
 from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
 
 from app.db_sql import redis_lookup_id_random
-from app.lib_general import log, logging, Response, status
+from app.lib_general import Response, status
+
+import logging
+log = logging.getLogger(__name__)
 
 from app.config import settings
 
@@ -22,7 +25,7 @@ class Resp_Body_Base(BaseModel):
     #     alias = 'test_prop_alias'
     # )
 
-    data: Union[list, dict]
+    data: Union[None, list, dict]
     meta: Optional[dict]
 # ### END ### API Response Model ### Resp_Body_Base() ###
 
@@ -38,7 +41,7 @@ def mk_resp(
         status_message: str = '',
         status_name: str = '',
         success: bool = True,
-        details: str = '',
+        details: Union[None, str, dict, list] = '',
         include: dict = None,
         exclude: dict = None,
         by_alias: bool = True,

app/models/response_models.py.snapshot

@@ -0,0 +1,134 @@
+from app.lib_general import log, logging, Response, status
+
+# ### BEGIN ### API Response Model ### Resp_Body_Base() ###
+# The pydantic BaseModel to help make consistent REST responses.
+# Updated 2021-03-05
+class Resp_Body_Base(BaseModel):
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    # test_prop: Optional[str] = Field(
+    #     alias = 'test_prop_alias'
+    # )
+
+    data: Union[None, list, dict]
+    meta: Optional[dict]
+# ### END ### API Response Model ### Resp_Body_Base() ###
+
+
+# ### BEGIN ### API Response Model ### mk_resp() ###
+# The method for making responses for REST. Returns a dict, but currently uses Resp_Body_Base inside the function.
+# Update 2021-08-23
+def mk_resp(
+        data: None|bool|dict|list,
+        tmp_file_path: None|str = None,
+        dict_to_json: bool = False,
+        status_code: int = 200,
+        status_message: str = '',
+        status_name: str = '',
+        success: bool = True,
+        details: str = '',
+        include: dict = None,
+        exclude: dict = None,
+        by_alias: bool = True,
+        exclude_unset: bool = False,
+        response: Response = None
+        ) -> dict:
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    if data is None: data_out = { 'result': data }
+    elif data == False: data_out = { 'result': data }
+    elif data == True: data_out = { 'result': data }
+    elif isinstance(data, dict):
+        log.info('Data type is a dict')
+        data_out = data
+    elif isinstance(data, list):
+        log.info('Data type is a list')
+        data_out = data
+    elif isinstance(data, int):
+        log.info('Data type is an int')
+        data_out = { 'result': data }
+    elif isinstance(data, str):
+        log.info('Data type is a str')
+        data_out = { 'result': data }
+    else: # Assuming it is still and object. This should be improved. Example model type: "<class 'app.models.account_models.Account_Base'>"
+        log.info('Data type is other')
+        data_out = data.dict(include=include, exclude=exclude, by_alias=by_alias, exclude_unset=exclude_unset)
+    # log.debug(data_out)
+
+    resp_body = {}
+    resp_body['data'] = data_out
+    resp_body['meta'] = {}
+    resp_body['meta']['details'] = details
+    resp_body['meta']['status_code'] = status_code
+    if status_message:
+        resp_body['meta']['status_message'] = status_message
+    else:
+        resp_body['meta']['status_message'] = http_status_li[status_code]['message']
+    resp_body['meta']['status_name'] = http_status_li[status_code]['name']
+    resp_body['meta']['success'] = success
+    resp_body['meta']['tmp_file_path'] = tmp_file_path
+
+    if isinstance(data, bool):
+        resp_body['meta']['data_type'] = 'bool'
+    elif isinstance(data, int):
+        resp_body['meta']['data_type'] = 'int'
+    elif isinstance(data, str):
+        resp_body['meta']['data_type'] = 'str'
+    elif isinstance(data, dict):
+        resp_body['meta']['data_type'] = 'dict'
+    elif isinstance(data, list):
+        resp_body['meta']['data_type'] = 'list'
+        resp_body['meta']['data_list_count'] = len(data)
+
+    if response:
+        log.debug(response)
+        if status_code == 400:
+            log.warning('Likely bad request')
+            response.status_code = status.HTTP_400_BAD_REQUEST
+        elif status_code == 401: response.status_code = status.HTTP_401_UNAUTHORIZED
+        # elif status_code == 402: response.status_code = status.HTTP_402_X
+        elif status_code == 403: response.status_code = status.HTTP_403_FORBIDDEN
+        elif status_code == 404:
+            log.info('No results')
+            response.status_code = status.HTTP_404_NOT_FOUND
+        elif status_code == 408: response.status_code = status.HTTP_408_REQUEST_TIMEOUT
+        elif status_code == 429: response.status_code = status.HTTP_429_TOO_MANY_REQUESTS
+        elif status_code == 500: response.status_code = status.HTTP_500_INTERNAL_SERVER_ERROR
+        elif status_code == 501: response.status_code = status.HTTP_501_NOT_IMPLEMENTED
+        elif status_code == 502: response.status_code = status.HTTP_502_BAD_GATEWAY
+        elif status_code == 503: response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
+        elif status_code == 504: response.status_code = status.HTTP_504_GATEWAY_TIMEOUT
+
+    # log.setLevel(logging.DEBUG) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    # log.debug(resp_body)
+    # log.debug(type(resp_body['data']))
+
+    # import json
+    # with open('data.txt', 'w') as outfile:
+    #     json.dump(resp_body['data'], outfile)
+
+    resp_body_obj = Resp_Body_Base(**resp_body)
+    log.debug(resp_body_obj)
+    resp_body_dict = resp_body_obj.dict(by_alias=by_alias, exclude_unset=exclude_unset)
+    log.debug(resp_body_dict)
+
+    return resp_body_dict
+# ### END ### API Response Model ### mk_resp() ###
+
+
+http_status_li = {}
+http_status_li[200] = { 'name': 'OK', 'message': 'The request has succeeded.' }
+http_status_li[400] = { 'name': 'Bad Request', 'message': 'The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.' }
+http_status_li[401] = { 'name': 'Unauthorized', 'message': 'The server could not verify that you are authorized to access the URL requested. You either supplied the wrong credentials (e.g. a bad password), or your browser does not understand how to supply the credentials required.' }
+http_status_li[402] = { 'name': '?Request Failed?', 'message': '??The parameters were valid but the request failed.??' }
+http_status_li[403] = { 'name': 'Forbidden', 'message': 'The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.' }
+http_status_li[404] = { 'name': 'Not Found', 'message': 'The requested resource does not exist.' }
+http_status_li[409] = { 'name': 'Conflict', 'message': 'The request conflicts with another request (perhaps due to using the same idempotent key).' }
+http_status_li[429] = { 'name': 'Too Many Requests', 'message': 'Too many requests hit the API too quickly. We recommend an exponential backoff of your requests.' }
+http_status_li[500] = { 'name': 'Internal Server Error', 'message': 'The server encountered an unexpected condition which prevented it from fulfilling the request.' }
+http_status_li[501] = { 'name': 'Not Implemented', 'message': 'The server does not support the functionality required to fulfill the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource.' }
+http_status_li[502] = { 'name': 'Bad Gateway', 'message': 'The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request.' }
+http_status_li[503] = { 'name': 'Service Unavailable', 'message': 'The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.' }
+http_status_li[504] = { 'name': 'Gateway Timeout', 'message': 'The server, while acting as a gateway or proxy, did not receive a timely response from the upstream server specified by the URI (e.g. HTTP, FTP, LDAP) or some other auxiliary server (e.g. DNS) it needed to access in attempting to complete the request.' }

app/models/site_domain_models.py

@@ -1,9 +1,9 @@
 import datetime, pytz
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
-from app.db_sql import redis_lookup_id_random
+from app.db_sql import get_id_random, redis_lookup_id_random
 from app.lib_general import log, logging
 
 from app.models.common_field_schema import base_fields, default_num_bytes
@@ -14,16 +14,11 @@ class Site_Domain_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['site_domain_id_random'],
-        alias = 'site_domain_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'site_domain_id'
-    )
-
-    site_id_random: Optional[str]
-    site_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['site_domain_id_random'])
+    site_domain_id: Optional[str] = Field(None, **base_fields['site_domain_id_random'])
+    site_id: Optional[str] = Field(None, **base_fields['site_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
 
     fqdn: Optional[str]
 
@@ -31,39 +26,120 @@ class Site_Domain_Base(BaseModel):
     access_key: Optional[str]
     required_referrer: Optional[str]
 
+    access_code_kv_json: Optional[Union[Json, None]]
+
     valid_for: Optional[int] # number of hours
     enable: Optional[bool]
 
+    cfg_json: Optional[Union[Json, None]] # In use 2024-03-04
+
+    hide: Optional[bool] = None # Field missing in physical table but common in views
+    # priority: Optional[bool] # MISSING in physical table
+    # sort: Optional[int] # MISSING in physical table
+    # group: Optional[str] # MISSING in physical table
+
+    notes: Optional[str] = None # MISSING in physical table
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
+
+    _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
+
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB-centric keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        # We prioritize the random strings to ensure the Vision is string-based.
+        if rid := values.get('id_random') or values.get('site_domain_id_random'):
+            values['id'] = rid
+            values['site_domain_id'] = rid
+            
+        if s_rid := values.get('site_id_random'):
+            values['site_id'] = s_rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'site_id', 'account_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
+
+    class Config:
+        underscore_attrs_are_private = True
+        allow_population_by_field_name = False
+# ### END ### API Site Domain Models ### Site_Domain_Base() ###
+
+
+# ### BEGIN ### API Site Domain Models ### Site_Domain_FQDN_ID_Base() ###
+class Site_Domain_FQDN_ID_Base(BaseModel):
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['site_domain_id_random'])
+    site_domain_id: Optional[str] = Field(None, **base_fields['site_domain_id_random'])
+    site_id: Optional[str] = Field(None, **base_fields['site_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+
+    fqdn: Optional[str]
+
+    # restrict_access: Optional[bool]
+    access_key: Optional[str]
+    required_referrer: Optional[str]
+
+    access_code_kv_json: Optional[Union[Json, None]]
+
+    valid_for: Optional[int] # number of hours
+    enable: Optional[bool]
+
+    hide: Optional[bool] = None
+    
+    notes: Optional[str] = None
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
 
     # Including convenience data
-    # This is only for convenience. Probably going to keep unless it causes a problem.
-    account_id: Optional[int]
-    account_id_random: Optional[str]
+    account_code: Optional[str] # Useful for export file naming
+    account_name: Optional[str] # Generally useful for display
+    account_enable: Optional[bool]
+    account_enable_from: Optional[datetime.datetime]
+    account_enable_to: Optional[datetime.datetime]
+
+    site_enable_from: Optional[datetime.datetime]
+    site_enable_to: Optional[datetime.datetime]
+    site_domain_access_key: Optional[str]
+
+    logo_path: Optional[str]
+    style_href: Optional[str]
+    script_src: Optional[str]
+
+    google_tracking_id: Optional[str]
+
+    cfg_json: Optional[Union[Json, None]] # In use 2024-03-04
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    @validator('id', always=True)
-    def site_domain_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='site_domain')
-        return None
-
-    @validator('site_id', always=True)
-    def site_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['site_id_random']:
-            return redis_lookup_id_random(record_id_random=values['site_id_random'], table_name='site')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        if rid := values.get('id_random') or values.get('site_domain_id_random'):
+            values['id'] = rid
+            values['site_domain_id'] = rid
+        if s_rid := values.get('site_id_random'):
+            values['site_id'] = s_rid
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+            
+        for k in ['id', 'site_id', 'account_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+        return values
 
     class Config:
         underscore_attrs_are_private = True
-        fields = base_fields
-# ### END ### API Site Domain Models ### Site_Domain_Base() ###
+        allow_population_by_field_name = False
+# ### END ### API Site Domain Models ### Site_Domain_FQDN_ID_Base() ###

app/models/site_models.py

@@ -34,6 +34,8 @@ class Site_Base(BaseModel):
     restrict_access: Optional[bool]
     access_key: Optional[str]
 
+    access_code_kv_json: Optional[Union[Json, None]]
+
     enable: Optional[bool]
     enable_from: Optional[datetime.datetime] = None
     enable_to: Optional[datetime.datetime] = None
@@ -83,6 +85,13 @@ class Site_Base(BaseModel):
 
     google_tracking_id: Optional[str] # In use 2022-07-19
 
+    cfg_json: Optional[Union[Json, None]] # In use 2024-03-04
+
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+
     notes: Optional[str]
     created_on: Optional[datetime.datetime] = None
     updated_on: Optional[datetime.datetime] = None
@@ -107,8 +116,16 @@ class Site_Base(BaseModel):
         log.setLevel(logging.WARNING)
         log.debug(locals())
 
-        if values['account_id_random']:
-            return redis_lookup_id_random(record_id_random=values['account_id_random'], table_name='account')
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('account_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
+        return None
+
+    @validator('account_id_random', always=True)
+    def account_id_random_lookup(cls, v, values, **kwargs):
+        if isinstance(v, str) and len(v) >= 11: return v
+        elif account_id := values.get('account_id'):
+            return get_id_random(record_id=account_id, table_name='account')
         return None
 
     class Config:

app/models/sponsorship_cfg_models.py

@@ -0,0 +1,132 @@
+import datetime, pytz
+
+from typing import Dict, List, Optional, Set, Union
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+
+from app.db_sql import get_id_random, redis_lookup_id_random
+from app.lib_general import log, logging
+
+from app.models.common_field_schema import base_fields, default_num_bytes
+
+
+# ### BEGIN ### API Sponsorship Cfg Models ### Sponsorship_Cfg_Base() ###
+class Sponsorship_Cfg_Base(BaseModel):
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    id_random: Optional[str] = Field(
+        **base_fields['sponsorship_cfg_id_random'],
+        alias = 'sponsorship_cfg_id_random',
+    )
+    id: Optional[int] = Field(
+        alias = 'sponsorship_cfg_id'
+    )
+
+    account_id_random: Optional[str]
+    account_id: Optional[int]
+
+    code: Optional[str]
+    name: Optional[str]
+    description: Optional[str]
+
+    for_type: Optional[str]
+    for_id: Optional[int]
+    for_id_random: Optional[str] # This should be after for_id if we want for_id_random filled in.
+
+    # For levels in a JSON object list format. A level option should contain: num, str, name, desc. Example: {"num": 1, "code": "platinum", "name": "Platinum", "desc": "Platinum Sponsorship"}
+    level_li_json: Optional[Union[Json, None]]
+
+    # For options in a JSON object list format. An option should contain: id, code, name, desc, note. Example: {"id": 1, "code": "option_1", "name": "Option 1", "desc": "Option 1 Description", "note": "Option 1 Note"}
+    option_li_json: Optional[Union[Json, None]]
+
+    # These are the common dates and deadlines used. They can be overridden by the deadline_li_json.
+    start_datetime: Optional[datetime.datetime] = None
+    end_datetime: Optional[datetime.datetime] = None
+    start_deadline: Optional[datetime.datetime] = None
+    end_deadline: Optional[datetime.datetime] = None
+    payment_deadline: Optional[datetime.datetime] = None
+    rsvp_deadline: Optional[datetime.datetime] = None
+
+    # For additional dates and deadlines in a JSON object list format. Example: {"early_bird": "2025-01-01", "regular": "2025-02-01", "late": "2025-03-01"}
+    schedule_datetime_li_json: Optional[Union[Json, None]]
+
+    default_no_reply_email: Optional[str]
+    default_no_reply_name: Optional[str]
+    default_reply_to_email: Optional[str]
+    default_reply_to_name: Optional[str]
+
+    # This is for a confirmation email to be sent to a staff email address
+    confirm_email: Optional[str]
+    confirm_name: Optional[str]
+
+    # For help options in a JSON object list format. Options for who to contact for help or support in a list format. A help option should contain: purpose, name, email, subject. Example: {"purpose": "sponsorship", "name": "John Doe", "email": ", "subject": "Sponsorship Help"}
+    help_li_json: Optional[Union[Json, None]]
+
+    # For additional configuration options in a JSON object format.
+    cfg_json: Optional[Union[Json, None]]
+
+    # The standard fields:
+    enable: Optional[bool]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+    notes: Optional[str]
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
+
+    # Including other related objects
+    # example_cfg: Optional[Example_Cfg_Base]
+
+    _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
+
+    @validator('id', always=True)
+    def sponsorship_cfg_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='sponsorship_cfg')
+        return None
+
+    @validator('account_id', always=True)
+    def account_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('account_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
+        return None
+
+    @validator('for_id', always=True)
+    def for_id_lookup(cls, v, values, **kwargs):
+        log.setLevel(logging.WARNING)
+        log.debug(locals())
+        if isinstance(v, int) and v > 0: return v
+        elif values.get('for_id_random') and values.get('for_type'):
+            for_id_random = values.get('for_id_random')
+            for_type = values.get('for_type')
+            return redis_lookup_id_random(record_id_random=for_id_random, table_name=for_type)
+        return None
+
+    @validator('for_id_random', always=True)
+    def for_id_random_lookup(cls, v, values, **kwargs):
+        log.setLevel(logging.DEBUG)
+        log.debug(locals())
+
+        for_type = values.get('for_type')
+        for_id = values.get('for_id')
+        for_id_random = v
+
+        if for_id_random:
+            log.info(f'Got For ID Random: {for_id_random}')
+            return for_id_random
+        elif for_id and for_type:
+            log.info(f'Got For ID: {for_id}; For Type: {for_type}')
+            for_id_random = get_id_random(for_id, table_name=for_type)
+            log.info(f'Got ID Random: {for_id_random}')
+            return for_id_random
+        log.info(f'Got nothing? For ID: {for_id}; For ID Random: {for_id_random}; For Type: {for_type}')
+        return None
+
+    class Config:
+        underscore_attrs_are_private = True
+        allow_population_by_field_name = True
+        fields = base_fields
+# ### END ### API Sponsorship Cfg Models ### Sponsorship_Cfg_Base() ###

app/models/sponsorship_models.py

@@ -0,0 +1,135 @@
+import datetime, pytz
+
+from typing import Dict, List, Optional, Set, Union
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+
+from app.db_sql import redis_lookup_id_random
+from app.lib_general import log, logging
+
+from app.models.common_field_schema import base_fields, default_num_bytes
+# from app.models.sponsorship_cfg_models import Sponsorship_Cfg_Base
+
+
+# ### BEGIN ### API Sponsorship Models ### Sponsorship_Base() ###
+class Sponsorship_Base(BaseModel):
+    log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    id_random: Optional[str] = Field(
+        **base_fields['sponsorship_id_random'],
+        alias = 'sponsorship_id_random',
+    )
+    id: Optional[int] = Field(
+        alias = 'sponsorship_id'
+    )
+
+    account_id_random: Optional[str]
+    account_id: Optional[int]
+
+    sponsorship_cfg_id_random: Optional[str]
+    sponsorship_cfg_id: Optional[int]
+
+    name: Optional[str]
+    description: Optional[str]
+
+    # This should be required for the confirmation email to be sent to the sponsor. The person's name and email address in the "To" address line.
+    poc_email_name: Optional[str]
+    poc_email: Optional[str]
+
+    # Store this here and under social_li_json. However, website_url should be the primary source for the website URL.
+    website_url: Optional[str]
+
+    # For the sponsoring organization, person, and point of contact in a JSON object format. The Aether standard field names should be used. Examples: name, given_name, family_name, full_name, full_name_override, email, phone, address_line_1, city, state_province, postal_code, country, etc.
+    # Example poc_json: {"given_name": "John", "family_name": "Doe", "full_name": "John Doe", "full_name_override": "John Doe", "email": "john.doe@example.com"}
+    organization_json: Optional[Union[Json, None]]
+    person_json: Optional[Union[Json, None]]
+    poc_json: Optional[Union[Json, None]]
+
+    # For the address in a JSON object format. The address types are expected to be: mailing, billing, home, work, etc. The Aether standard field names should be used. Examples: address_line_1, address_line_2, city, state_province, postal_code, country, etc.
+    address_li_json: Optional[Union[Json, None]]
+
+    # For additional contacts in a JSON object list (array) format. A contact person should contain: given_name, family_name, full_name, email, phone, etc.
+    contact_li_json: Optional[Union[Json, None]]
+
+    # For the logo and image in a JSON object format. The Aether standard field names should be used. Examples: url, url_text, alt_text, width, height, etc.
+    logo_li_json: Optional[Union[Json, None]]
+
+    # For media that have different predefined purposes in a JSON object list format. The Aether standard field names should be used. Examples: purpose, (file) type, (file) extension, (file) name, url, url_text, alt_text, width, height, size (in bytes), etc.
+    media_li_json: Optional[Union[Json, None]]
+
+    # For simple question answers in a JSON object list format. A question should contain: id, code, name, desc, note, answer, etc.
+    questions_li_json: Optional[Union[Json, None]]
+
+    # For social media in a JSON object format. The Aether standard field names should be used. Examples: url, url_text, icon, etc.
+    social_li_json: Optional[Union[Json, None]]
+
+    # For a (simple and short) guest list in a JSON object list (array) format. A guest person should contain: given_name, family_name, full_name, title, affiliations, email, phone, assistance, dietry, etc.
+    # Example: [{"given_name": "John", "family_name": "Doe", "full_name": "John Doe", "email": "john.doe@example.com"}, {"given_name": "Jane", "family_name": "Doe", "full_name": "Jane Doe", "email": "jane.doe@example.com"}]
+    # Example 2: [{"full_name": "Albert Einstein", "email": "albert.einstein@example.com"}, {"full_name": "Marie Curie", "email": "marie.curie@example.com"}]
+    guest_li_json: Optional[Union[Json, None]]
+
+    level_num: Optional[int]
+    level_str: Optional[str]
+
+    # For their selected sponsorship level in a JSON object format. A level option should contain: num, code, name, desc. Example: {"num": 1, "code": "platinum", "name": "Platinum", "desc": "Platinum Sponsorship"}
+    slct_level_json: Optional[Union[Json, None]]
+
+    # For their selected options in a JSON object list format. An option should contain: id, code, name, desc, note. Example: {"id": 1, "code": "option_1", "name": "Option 1", "desc": "Option 1 Description", "note": "Option 1 Note"}
+    slct_option_li_json: Optional[Union[Json, None]]
+
+    # Amount as an integer in cents. Example: 1000 = $10.00
+    amount: Optional[int]
+    paid: Optional[bool]
+
+    access_key: Optional[str] # This is for a unique access key or passcode to be used for a sponsorship page edit access.
+
+    # General catchall for agreement or consent
+    agree: Optional[bool]
+
+    # Comments from the sponsor. Assumed to be the POC. This is for internal use only.
+    comments: Optional[str]
+
+    cfg_json: Optional[Union[Json, None]]
+    meta_data: Optional[str]
+
+    # The standard fields:
+    enable: Optional[bool]
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+    notes: Optional[str]
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
+
+    # Including other related objects
+    # example_cfg: Optional[Example_Cfg_Base]
+
+    _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
+
+    @validator('id', always=True)
+    def account_cfg_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='account_cfg')
+        return None
+
+    @validator('account_id', always=True)
+    def account_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('account_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
+        return None
+
+    @validator('sponsorship_cfg_id', always=True)
+    def sponsorship_cfg_id_lookup(cls, v, values, **kwargs):
+        if isinstance(v, int) and v > 0: return v
+        elif id_random := values.get('sponsorship_cfg_id_random'):
+            return redis_lookup_id_random(record_id_random=id_random, table_name='sponsorship_cfg')
+        return None
+
+    class Config:
+        underscore_attrs_are_private = True
+        allow_population_by_field_name = True
+        fields = base_fields
+# ### END ### API Sponsorship Models ### Sponsorship_Base() ###

app/models/user_models.py

@@ -1,10 +1,9 @@
-import datetime, pytz, secrets
-# import datetime, hashlib, logging, os, pytz, redis, secrets
+import datetime, hashlib, logging, os, pytz, redis, secrets
 
 from typing import Dict, List, Optional, Set, Union
-from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator
+from pydantic import BaseModel, EmailStr, Field, Json, PrivateAttr, ValidationError, validator, root_validator
 
-from app.db_sql import redis_lookup_id_random
+from app.db_sql import get_id_random, redis_lookup_id_random
 from app.lib_general import log, logging, secure_hash_string
 
 from app.models.common_field_schema import base_fields, default_num_bytes
@@ -14,32 +13,131 @@ from app.models.organization_models import Organization_Base
 # from app.models.user_role_models import User_Role_Base
 
 
+# ### BEGIN ### API User Models ### User_Base() ###
+class User_Base(BaseModel):
+    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
+    log.debug(locals())
+
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['user_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    contact_id: Optional[str] = Field(None, **base_fields['contact_id_random'])
+    organization_id: Optional[str] = Field(None, **base_fields['organization_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
+
+    account_name: Optional[str]
+
+    username: Optional[str]
+    name: Optional[str]
+    email: Optional[str]
+    email_verified: Optional[bool]
+    password: Optional[str]
+    current_password: Optional[str]
+    new_password: Optional[str]
+
+    allow_auth_key: Optional[int]
+    auth_key: Optional[str]
+
+    enable: Optional[bool]
+    enable_from: Optional[datetime.datetime] = None
+    enable_to: Optional[datetime.datetime] = None
+
+    super: Optional[bool]
+    manager: Optional[bool]
+    administrator: Optional[bool]
+    public: Optional[bool]
+    verified: Optional[bool]
+    status_id: Optional[int]
+    status_name: Optional[str]
+
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
+    group: Optional[str]
+
+    notes: Optional[str]
+    created_on: Optional[datetime.datetime] = None
+    updated_on: Optional[datetime.datetime] = None
+
+    # Including other related objects
+    # from app.models.person_models import Person_Base # Causes circular import
+    # archive_list: Optional[list] # Archive_Base()
+    # contact: Optional[Contact_Base]
+    event_list: Optional[list] # Event_Base() # Priority l1
+    hosted_file_list: Optional[list] # Hosted_File_Base() # Priority l2
+    journal_list: Optional[list] # Journal_Base() # Priority l3
+    order_list: Optional[list] # Order_Base() # Priority l2
+    order_cart_list: Optional[list] # Order_Base() # Priority l2
+    organization: Optional[Union[Organization_Base, None]] # Organization_Base() # Priority l3
+    person: Optional[dict] # Person_Base() # Priority l2
+    # person: Optional[Union[Person_Base, None]]
+    post_list: Optional[list] # Post_Base() # Priority l1
+    user_role_list: Optional[list] = Field(
+        alias = 'role_list'
+    ) # User_Role_Base()
+    # role_list: Optional[list] = [] # User_Role_Base() # NOTE <- This is a duplicate of above!
+
+    _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
+
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('user_id_random'):
+            values['id'] = rid
+            values['user_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if c_rid := values.get('contact_id_random'):
+            values['contact_id'] = c_rid
+        if o_rid := values.get('organization_id_random'):
+            values['organization_id'] = o_rid
+        if p_rid := values.get('person_id_random'):
+            values['person_id'] = p_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'user_id', 'account_id', 'contact_id', 'organization_id', 'person_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
+
+    @validator('password', always=True)
+    def hash_new_password(cls, v, values, **kwargs):
+        log.setLevel(logging.WARNING)
+        log.debug(locals())
+
+        if values.get('new_password'):
+            return secure_hash_string(string=values['new_password'])
+        return None
+
+    class Config:
+        underscore_attrs_are_private = True
+        allow_population_by_field_name = True
+        fields = base_fields
+# ### END ### API User Models ### User_Base() ###
+
+
 # ### BEGIN ### API User Models ### User_New_Base() ###
 class User_New_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['user_id_random'],
-        alias = 'user_id_random',
-        # default_factory = lambda:secrets.token_urlsafe(default_num_bytes),
-    )
-    id: Optional[int] = Field(
-        alias = 'user_id'
-    )
-    account_id_random: Optional[str]
-    account_id: Optional[int]
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['user_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    contact_id: Optional[str] = Field(None, **base_fields['contact_id_random'])
+    organization_id: Optional[str] = Field(None, **base_fields['organization_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
+    
     account_name: Optional[str]
 
-    contact_id_random: Optional[str]
-    contact_id: Optional[int]
-
-    organization_id_random: Optional[str]
-    organization_id: Optional[int]
-
-    person_id_random: Optional[str]
-    person_id: Optional[int]
-
     username: str
     name: str
     email: str
@@ -61,6 +159,9 @@ class User_New_Base(BaseModel):
     public: bool = False
     verified: bool = False
 
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
     group: Optional[str]
 
     notes: Optional[str]
@@ -70,52 +171,32 @@ class User_New_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
-    #@validator('user_id_random', always=True)
-    def user_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def user_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='user')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('account_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='account')
-        return None
-
-    @validator('contact_id', always=True)
-    def contact_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('contact_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='contact')
-        return None
-
-    @validator('organization_id', always=True)
-    def organization_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('organization_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='organization')
-        return None
-
-    @validator('person_id', always=True)
-    def person_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('person_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
-        return None
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('user_id_random'):
+            values['id'] = rid
+            values['user_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if c_rid := values.get('contact_id_random'):
+            values['contact_id'] = c_rid
+        if o_rid := values.get('organization_id_random'):
+            values['organization_id'] = o_rid
+        if p_rid := values.get('person_id_random'):
+            values['person_id'] = p_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'user_id', 'account_id', 'contact_id', 'organization_id', 'person_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
 
     @validator('password', always=True)
     def hash_new_password(cls, v, values, **kwargs):
@@ -138,27 +219,16 @@ class User_Out_Base(BaseModel):
     log.setLevel(logging.WARNING) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
     log.debug(locals())
 
-    id_random: Optional[str] = Field(
-        **base_fields['user_id_random'],
-        alias = 'user_id_random',
-    )
-    id: Optional[int] = Field(
-        alias = 'user_id'
-    )
+    # --- Standardized Vision IDs (Strings) ---
+    id: Optional[str] = Field(None, **base_fields['user_id_random'])
+    user_id: Optional[str] = Field(None, **base_fields['user_id_random'])
+    account_id: Optional[str] = Field(None, **base_fields['account_id_random'])
+    contact_id: Optional[str] = Field(None, **base_fields['contact_id_random'])
+    organization_id: Optional[str] = Field(None, **base_fields['organization_id_random'])
+    person_id: Optional[str] = Field(None, **base_fields['person_id_random'])
 
-    account_id_random: Optional[str]
-    #account_id: Optional[int]
     account_name: Optional[str]
 
-    contact_id_random: Optional[str]
-    #contact_id: Optional[int]
-
-    organization_id_random: Optional[str]
-    #organization_id: Optional[int]
-
-    person_id_random: Optional[str]
-    #person_id: Optional[int]
-
     username: Optional[str]
     name: Optional[str]
     email: Optional[str]
@@ -186,6 +256,9 @@ class User_Out_Base(BaseModel):
     logged_in_on: Optional[datetime.datetime]
     last_activity_on: Optional[datetime.datetime]
 
+    hide: Optional[bool]
+    priority: Optional[bool]
+    sort: Optional[int]
     group: Optional[str]
 
     notes: Optional[str]
@@ -196,8 +269,8 @@ class User_Out_Base(BaseModel):
     # from app.models.person_models import Person_Base # Causes circular import
     # archive_list: Optional[list] # Archive_Base()
     # contact: Optional[Contact_Base]
-    event_list: Optional[list] # Event_Base() # Priority complete
-    hosted_file_list: Optional[list] # Hosted_File_Base() # Priority l3
+    event_list: Optional[list] # Event_Base() # Priority l1
+    hosted_file_list: Optional[list] # Hosted_File_Base() # Priority l2
     journal_list: Optional[list] # Journal_Base() # Priority l3
     # membership_person: Optional[Membership_Person_Base] # Priority l2
     # membership_person_list: Optional[list] # Membership_Base() ???
@@ -213,158 +286,35 @@ class User_Out_Base(BaseModel):
 
     _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
 
+    @root_validator(pre=True)
+    def map_v3_ids(cls, values):
+        """
+        Vision Transformer:
+        Map DB keys to clean API keys and strip internal integers.
+        """
+        # 1. Map Random Strings to Clean Names
+        if rid := values.get('id_random') or values.get('user_id_random'):
+            values['id'] = rid
+            values['user_id'] = rid
+            
+        if a_rid := values.get('account_id_random'):
+            values['account_id'] = a_rid
+        if c_rid := values.get('contact_id_random'):
+            values['contact_id'] = c_rid
+        if o_rid := values.get('organization_id_random'):
+            values['organization_id'] = o_rid
+        if p_rid := values.get('person_id_random'):
+            values['person_id'] = p_rid
+            
+        # 2. Prevent "Collision Population"
+        for k in ['id', 'user_id', 'account_id', 'contact_id', 'organization_id', 'person_id']:
+            if k in values and not isinstance(values[k], str):
+                del values[k]
+                
+        return values
+
     class Config:
         underscore_attrs_are_private = True
         allow_population_by_field_name = True
         fields = base_fields
 # ### END ### API User Models ### User_Out_Base() ###
-
-
-# ### BEGIN ### API User Models ### User_Base() ###
-class User_Base(BaseModel):
-    log.setLevel(logging.INFO) # DEBUG, INFO, WARNING, ERROR, EXCEPTION, CRITICAL
-    log.debug(locals())
-
-    id_random: Optional[str] = Field(
-        **base_fields['user_id_random'],
-        alias = 'user_id_random',
-        # default_factory = lambda:secrets.token_urlsafe(default_num_bytes),
-    )
-    id: Optional[int] = Field(
-        alias = 'user_id'
-    )
-
-    account_id_random: Optional[str]
-    account_id: Optional[int]
-
-    account_name: Optional[str]
-
-    # contact_id_random: Optional[str]
-    # contact_id: Optional[int]
-
-    organization_id_random: Optional[str]
-    organization_id: Optional[int]
-
-    person_id_random: Optional[str]
-    person_id: Optional[int]
-
-    username: Optional[str]
-    name: Optional[str]
-    email: Optional[str]
-    email_verified: Optional[bool]
-    password: Optional[str]
-    current_password: Optional[str]
-    new_password: Optional[str]
-
-    allow_auth_key: Optional[int]
-    auth_key: Optional[str]
-
-    enable: Optional[bool]
-    enable_from: Optional[datetime.datetime] = None
-    enable_to: Optional[datetime.datetime] = None
-
-    super: Optional[bool]
-    manager: Optional[bool]
-    administrator: Optional[bool]
-    public: Optional[bool]
-    verified: Optional[bool]
-    status_id: Optional[int]
-    status_name: Optional[str]
-
-    password_set_on: Optional[datetime.datetime] = None
-    password_reset_token: Optional[str] = None
-    password_reset_expire_on: Optional[datetime.datetime] = None
-    logged_in_on: Optional[datetime.datetime] = None
-    last_activity_on: Optional[datetime.datetime] = None
-
-    group: Optional[str]
-
-    notes: Optional[str]
-    created_on: Optional[datetime.datetime] = None
-    updated_on: Optional[datetime.datetime] = None
-
-    # Including other related objects
-    # from app.models.person_models import Person_Base # Causes circular import
-    # archive_list: Optional[list] # Archive_Base()
-    # contact: Optional[Contact_Base]
-    event_list: Optional[list] # Event_Base() # Priority l1
-    hosted_file_list: Optional[list] # Hosted_File_Base() # Priority l2
-    journal_list: Optional[list] # Journal_Base() # Priority l3
-    order_list: Optional[list] # Order_Base() # Priority l2
-    order_cart_list: Optional[list] # Order_Base() # Priority l2
-    organization: Optional[Union[Organization_Base, None]] # Organization_Base() # Priority l3
-    person: Optional[dict] # Person_Base() # Priority l2
-    # person: Optional[Union[Person_Base, None]]
-    post_list: Optional[list] # Post_Base() # Priority l1
-    user_role_list: Optional[list] = Field(
-        alias = 'role_list'
-    ) # User_Role_Base()
-    # role_list: Optional[list] = [] # User_Role_Base() # NOTE <- This is a duplicate of above!
-
-    _processed_at: datetime.datetime = PrivateAttr(default_factory=datetime.datetime.now)
-
-    #@validator('user_id_random', always=True)
-    def user_id_random_copy(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            return values['id_random']
-        return None
-
-    @validator('id', always=True)
-    def user_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['id_random']:
-            log.debug(values['id_random'])
-            return redis_lookup_id_random(record_id_random=values['id_random'], table_name='user')
-        return None
-
-    @validator('account_id', always=True)
-    def account_id_lookup(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values['account_id_random']:
-            return redis_lookup_id_random(record_id_random=values['account_id_random'], table_name='account')
-        return None
-
-    # @validator('contact_id', always=True)
-    # def contact_id_lookup(cls, v, values, **kwargs):
-    #     log.setLevel(logging.WARNING)
-    #     log.debug(locals())
-
-    #     if values['contact_id_random']:
-    #         return redis_lookup_id_random(record_id_random=values['contact_id_random'], table_name='contact')
-    #     return None
-
-    @validator('organization_id', always=True)
-    def organization_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('organization_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='organization')
-        return None
-
-    @validator('person_id', always=True)
-    def person_id_lookup(cls, v, values, **kwargs):
-        if isinstance(v, int) and v > 0: return v
-        elif id_random := values.get('person_id_random'):
-            return redis_lookup_id_random(record_id_random=id_random, table_name='person')
-        return None
-
-    @validator('password', always=True)
-    def hash_new_password(cls, v, values, **kwargs):
-        log.setLevel(logging.WARNING)
-        log.debug(locals())
-
-        if values.get('new_password'):
-            return secure_hash_string(string=values['new_password'])
-        return None
-
-    class Config:
-        underscore_attrs_are_private = True
-        allow_population_by_field_name = True
-        fields = base_fields
-# ### END ### API User Models ### User_Base() ###

app/object_definitions/cms.py

@@ -0,0 +1,131 @@
+from app.models.page_models import *
+from app.models.post_models import *
+from app.models.post_comment_models import *
+from app.models.site_models import *
+from app.models.site_domain_models import *
+
+cms_obj_li = {
+    'page': {
+        'tbl': 'page',
+        'tbl_default': 'page',
+        'tbl_update': 'page',
+        'mdl': Page_Base,
+        'mdl_default': Page_Base,
+        'mdl_in': Page_Base,
+        'mdl_out': Page_Base,
+        # Legacy V2 keys:
+        'table_name': 'page',
+        'tbl_name_update': 'page',
+        'base_name': Page_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'site_id',
+            'page_id_random', 'account_id_random', 'site_id_random',
+            'code', 'name', 'title', 'description', 'content_html', 
+            'enable', 'hide', 'priority', 'sort', 'group', 'notes', 
+            'created_on', 'updated_on'
+        ],
+    },
+    'post': {
+        'tbl': 'post',
+        'tbl_default': 'v_post',
+        'tbl_alt': 'v_post_detail',
+        'tbl_update': 'post',
+        'mdl': Post_Base,
+        'mdl_default': Post_Base,
+        'mdl_in': Post_Base,
+        'mdl_out': Post_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_post',
+        'table_name_alt': 'v_post_detail',
+        'tbl_name_update': 'post',
+        'base_name': Post_Base,
+        'public_read': True,
+        'exp_default': [
+            'post_id_random',
+            'account_id_random',
+            'title', 'content',
+            'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on',
+            ],
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'person_id', 'user_id',
+            'post_id_random', 'account_id_random', 'organization_id_random',
+            'person_id_random', 'user_id_random', 'external_person_id', 'title', 'content',
+            'type_code', 'topic_code', 'category_code', 'tags', 'location',
+            'enable', 'hide', 'priority', 'sort', 'group', 'notes',
+            'archive_on', 'created_on', 'updated_on'
+        ],
+    },
+    'post_comment': {
+        'tbl': 'post_comment',
+        'tbl_default': 'v_post_comment',
+        'tbl_alt': 'v_post_comment_detail',
+        'tbl_update': 'post_comment',
+        'mdl': Post_Comment_Base,
+        'mdl_default': Post_Comment_Base,
+        'mdl_in': Post_Comment_Base,
+        'mdl_out': Post_Comment_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_post_comment',
+        'table_name_alt': 'v_post_comment_detail',
+        'tbl_name_update': 'post_comment',
+        'base_name': Post_Comment_Base,
+        'public_read': True,
+        'exp_default': [
+            'post_comment_id_random',
+            'account_id_random', 'post_id_random',
+            'content',
+            'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on',
+            ],
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'post_id', 'person_id', 'user_id', 'account_id',
+            'post_comment_id_random', 'account_id_random', 'post_id_random',
+            'person_id_random', 'user_id_random', 'content', 'enable', 'hide',
+            'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'site': {
+        'tbl': 'site',
+        'tbl_default': 'site',
+        'tbl_update': 'site',
+        'mdl': Site_Base,
+        'mdl_default': Site_Base,
+        'mdl_in': Site_Base,
+        'mdl_out': Site_Base,
+        # Legacy V2 keys:
+        'table_name': 'site',
+        'tbl_name_update': 'site',
+        'base_name': Site_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'site_id_random', 'account_id_random', 'code', 'name', 'tagline',
+            'description', 'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'site_domain': {
+        'tbl': 'site_domain',
+        'tbl_default': 'v_site_domain',
+        'tbl_alt': 'v_site_domain_fqdn_id',
+        'tbl_update': 'site_domain',
+        'mdl': Site_Domain_Base,
+        'mdl_default': Site_Domain_Base,
+        'mdl_alt': Site_Domain_FQDN_ID_Base,
+        'mdl_in': Site_Domain_Base,
+        'mdl_out': Site_Domain_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_site_domain',
+        'table_name_alt': 'v_site_domain_fqdn_id',
+        'tbl_name_update': 'site_domain',
+        'base_name': Site_Domain_Base,
+        'base_name_alt': Site_Domain_FQDN_ID_Base,
+        'public_read': True,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'site_id',
+            'id_random', 'account_id_random', 'site_id_random',
+            'fqdn', 'enable', 'created_on', 'updated_on'
+        ],
+    },
+}

app/object_definitions/core.py

@@ -0,0 +1,245 @@
+from app.models.account_models import *
+from app.models.account_cfg_models import *
+from app.models.activity_log_models import *
+from app.models.address_models import *
+from app.models.contact_models import *
+from app.models.data_store_models import *
+from app.models.organization_models import *
+from app.models.person_models import *
+from app.models.user_models import *
+from app.models.user_role_models import *
+from app.models.log_client_viewing_models import Log_Client_Viewing_Base
+
+core_obj_li = {
+    'activity_log': {
+        'tbl': 'activity_log',
+        'tbl_default': 'v_activity_log',
+        'tbl_update': 'activity_log',
+        'mdl': Activity_Log_Base,
+        'mdl_default': Activity_Log_Base,
+        'mdl_in': Activity_Log_Base,
+        'mdl_out': Activity_Log_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_activity_log',
+        'tbl_name_update': 'activity_log',
+        'base_name': Activity_Log_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'activity_log_id_random', 'account_id_random', 'person_id_random',
+            'user_id_random', 'external_client_id', 'name', 'description',
+            'source', 'url_root', 'url_full_path', 'object_type',
+            'object_id_random', 'action', 'action_with', 'action_on_type',
+            'action_on_id_random', 'action_on_code', 'code', 'type_name',
+            'details', 'enable', 'hide', 'priority', 'group', 'created_on', 'updated_on'
+        ],
+    },
+    'account': {
+        'tbl': 'account',
+        'tbl_default': 'account',
+        'tbl_update': 'account',
+        'mdl': Account_Base,
+        'mdl_default': Account_Base,
+        'mdl_in': Account_Base,
+        'mdl_out': Account_Base,
+        # Legacy V2 keys:
+        'table_name': 'account',
+        'tbl_name_update': 'account',
+        'base_name': Account_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'id_random', 'account_id_random', 
+            'code', 'name', 'short_name', 'description',
+            'enable', 'hide', 'priority', 'sort', 'group', 'created_on', 'updated_on'
+        ],
+    },
+    'account_cfg': {
+        'tbl': 'account_cfg',
+        'tbl_default': 'v_account_cfg',
+        'tbl_update': 'account_cfg',
+        'mdl': Account_Cfg_Base,
+        'mdl_default': Account_Cfg_Base,
+        'mdl_in': Account_Cfg_Base,
+        'mdl_out': Account_Cfg_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_account_cfg',
+        'tbl_name_update': 'account_cfg',
+        'base_name': Account_Cfg_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'account_cfg_id_random', 'account_id_random', 'account_code',
+            'account_name', 'account_short_name', 'default_no_reply_email',
+            'default_no_reply_name', 'confirm_email', 'help_event_email',
+            'help_general_email', 'help_tech_email', 'stripe_account_id',
+            'created_on', 'updated_on'
+        ],
+    },
+    'address': {
+        'tbl': 'address',
+        'tbl_default': 'v_address',
+        'tbl_update': 'address',
+        'mdl': Address_Base,
+        'mdl_default': Address_Base,
+        'mdl_in': Address_Base,
+        'mdl_out': Address_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_address',
+        'tbl_name_update': 'address',
+        'base_name': Address_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'contact_id', 'address_id_random', 'account_id_random', 
+            'for_type', 'for_id_random', 'contact_id_random', 'name', 'attention_to', 
+            'organization_name', 'line_1', 'line_2', 'line_3', 'city', 'country_subdivision_code',
+            'country_subdivision_name', 'state_province', 'postal_code',
+            'country_alpha_2_code', 'country_name', 'timezone',
+            'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'contact': {
+        'tbl': 'contact',
+        'tbl_default': 'v_contact',
+        'tbl_update': 'contact',
+        'mdl': Contact_Base,
+        'mdl_default': Contact_Base,
+        'mdl_in': Contact_Base,
+        'mdl_out': Contact_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_contact',
+        'tbl_name_update': 'contact',
+        'base_name': Contact_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'contact_id_random', 'account_id_random', 'for_type', 'for_id_random',
+            'name', 'title', 'tagline', 'description', 'timezone_name',
+            'email', 'email_status', 'phone_mobile', 'phone_office',
+            'website_url', 'website_name', 'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'data_store': {
+        'tbl': 'data_store',
+        'tbl_default': 'v_data_store',
+        'tbl_update': 'data_store',
+        'mdl': Data_Store_Base,
+        'mdl_default': Data_Store_Base,
+        'mdl_in': Data_Store_Base,
+        'mdl_out': Data_Store_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_data_store',
+        'tbl_name_update': 'data_store',
+        'base_name': Data_Store_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'person_id', 'user_id',
+            'data_store_id_random', 'account_id_random', 'for_type', 'for_id_random',
+            'person_id_random', 'user_id_random', 'code', 'name', 'description',
+            'type', 'text', 'meta_text', 'access', 'enable', 'hide', 'priority',
+            'sort', 'group', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'organization': {
+        'tbl': 'organization',
+        'tbl_default': 'v_organization',
+        'tbl_update': 'organization',
+        'mdl': Organization_Base,
+        'mdl_default': Organization_Base,
+        'mdl_in': Organization_Base,
+        'mdl_out': Organization_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_organization',
+        'tbl_name_update': 'organization',
+        'base_name': Organization_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'contact_id', 'person_id', 'user_id',
+            'organization_id_random', 'account_id_random', 'contact_id_random',
+            'person_id_random', 'user_id_random', 'name', 'tagline', 'description',
+            'company', 'nonprofit', 'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'person': {
+        'tbl': 'v_person',
+        'tbl_default': 'v_person',
+        'tbl_alt': 'v_person',
+        'tbl_update': 'person',
+        'mdl': Person_Base,
+        'mdl_default': Person_Base,
+        'mdl_in': Person_Base,
+        'mdl_out': Person_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_person',
+        'tbl_name_update': 'person',
+        'base_name': Person_Base,
+        'exp_default': [
+            'person_id_random',
+            'given_name', 'middle_name', 'family_name', 'full_name',
+            'primary_email',
+            'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on',
+        ],
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'contact_id', 'organization_id', 'user_id', 'membership_person_id',
+            'person_id_random', 'account_id_random', 'contact_id_random',
+            'organization_id_random', 'user_id_random', 'membership_person_id_random',
+            'title_names', 'given_name', 'middle_name',
+            'family_name', 'designations', 'professional_title', 'full_name',
+            'informal_full_name', 'affiliations',
+            'primary_email', 'tagline', 'source_code',
+            'external_id', 'status', 'hide', 'priority', 'sort', 'group', 'enable', 'notes',
+            'created_on', 'updated_on', 'username', 'user_name'
+        ],
+    },
+    'user': {
+        'tbl': 'v_user',
+        'tbl_default': 'v_user',
+        'tbl_alt': 'v_user',
+        'tbl_update': 'user',
+        'mdl': User_Base,
+        'mdl_default': User_Base,
+        'mdl_in': User_New_Base,
+        'mdl_out': User_Out_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_user',
+        'tbl_name_update': 'user',
+        'base_name': User_Base,
+        'exp_default': [
+            'user_id_random',
+            'account_id_random',
+            'username', 'name', 'email',
+            'enable', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on',
+        ],
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'contact_id', 'organization_id', 'person_id',
+            'user_id_random', 'account_id_random', 'contact_id_random',
+            'organization_id_random', 'person_id_random', 'username', 'name',
+            'email', 'enable', 'super', 'manager', 'administrator', 'public',
+            'verified', 'status_name', 'hide', 'priority', 'sort', 'group', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'user_role': {
+        'mdl': User_Role_Base,
+        # Legacy V2 keys:
+        'table_name': 'v_user_role',
+        'tbl_name_update': 'user_role',
+        'base_name': User_Role_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'user_id', 'user_id_random', 'for_type', 'for_id_random', 'code', 'name',
+            'description', 'enable', 'notes', 'created_on', 'updated_on'
+        ],
+    },
+    'log_client_viewing': {
+        'mdl': Log_Client_Viewing_Base,
+        # Legacy V2 keys:
+        'table_name': 'log_client_viewing',
+        'tbl_name_update': 'log_client_viewing',
+        'base_name': Log_Client_Viewing_Base,
+        # V3 Search Security:
+        'searchable_fields': [
+            'id', 'account_id', 'person_id', 'user_id',
+            'log_client_viewing_id_random', 'account_id_random', 'person_id_random',
+            'user_id_random', 'external_client_id', 'name', 'source', 'url_root',
+            'url_full_path', 'object_type', 'object_id', 'created_on', 'updated_on'
+        ],
+    },
+}

app/object_definitions/events.py

@@ -0,0 +1,11 @@
+from .events_general import events_general_obj_li
+from .events_presentation import events_presentation_obj_li
+from .events_registration import events_registration_obj_li
+from .events_exhibits import events_exhibits_obj_li
+
+event_obj_li = {
+    **events_general_obj_li,
+    **events_presentation_obj_li,
+    **events_registration_obj_li,
+    **events_exhibits_obj_li,
+}