OSIT-AE-API-FastAPI/documentation/BOOTSTRAP__AI_Agent_Quickstart.md

Aether API — AI Agent Bootstrap / Quickstart

Read this first. This doc is the fast path to being productive on this project. It covers the rules, patterns, and gotchas that matter most. Deep dives are in the linked docs at the bottom.

---

1. What This Project Is

Aether is an event management platform built by One Sky IT (Scott Idem). This repo is the backend: FastAPI + MariaDB, running inside Docker.

The frontend (aether_app_sveltekit/) talks to this API exclusively via the V3 REST API. There is no standalone dev server — the API runs only in Docker.

Key clients:

• Conference organizers — Presentation management, badges, sessions
• Exhibitors — Leads capture
• IDAA — International Doctors in Alcoholics Anonymous (strictly private medical/recovery community)

Stack at a glance:

Layer	Technology
Framework	FastAPI + Pydantic V1 (upgrade deferred)
Database	MariaDB via SQLAlchemy 1.4 (upgrade deferred)
Cache	Redis
Auth	Custom headers: x-aether-api-key + x-account-id
Container	Docker / Gunicorn — source is volume-mounted (no rebuild for Python changes, but restart required)

---

2. Critical Rules — Read Before Touching Any Code

Privacy (Sev-1 class failures if violated)

• IDAA content is ALWAYS private. All routes under /idaa/ require authentication. A previous agent accidentally exposed IDAA bulletin board data publicly. This is the single most serious class of mistake on this project. When in doubt — it's private. Always verify with Depends(get_account_context).
• Journals are private personal data. Always authenticated.

File Safety

• Never use rm to delete files. Move to ~/tmp/gemini_trash instead.
• Never commit .env files, API keys, or passwords of any kind.
• Third-party credentials (Novi API key, Mailman credentials) live in the MariaDB site.cfg_json column, not in .env or code.

Before Every Commit

1. python3 -m py_compile <changed_file> — syntax check
2. Restart Docker and verify the API starts clean: docker compose restart ae_api
3. Check logs: docker compose logs -f ae_api (look for startup errors or import failures)
4. Run the relevant E2E or unit test suite

Before Starting Any Task

• Read documentation/TODO__Agents.md — active tasks, known bugs, and what was recently changed and why.
• Check tests/README.md — which test suite covers the area you're about to touch.

Docker Restart is Mandatory After Python Changes

The API source is volume-mounted, so file edits appear instantly inside the container — but Gunicorn does NOT hot-reload. You MUST run:

docker compose restart ae_api

after any Python file change before testing. This is the #1 cause of "my change didn't take effect."

---

3. Environment & Commands Cheat Sheet

# Start full stack
cd ~/OSIT_dev/aether_container_env && docker compose up -d

# Restart API after any Python change
docker compose restart ae_api

# Follow API logs
docker compose logs -f ae_api

# Shell into the container
docker compose exec ae_api bash

# Run unit tests (from project root)
./environment/bin/python3 -m pytest tests/unit/ -v

# Run a single E2E test
./environment/bin/python3 tests/e2e/test_e2e_v3_search_engine.py

# Check a specific Python file compiles
./environment/bin/python3 -m py_compile app/methods/my_new_methods.py

Development API: https://dev-api.oneskyit.com Dev API secret key: nT0jPeiCfxSifkiDZur9jA Standard test Agent API key: PMM4n50teUCaOMMTN8qOJA

Dev DB via phpMyAdmin: http://localhost:8081

---

4. V3 Action Router Pattern

When adding a new action endpoint (not CRUD), follow this structure:

File layout

app/methods/my_feature_methods.py   ← business logic
app/routers/api_v3_actions_my_feature.py  ← route handler (thin)

Then register in app/routers/registry.py:

from app.routers import api_v3_actions_my_feature
# ...
app.include_router(api_v3_actions_my_feature.router, prefix='/v3/action/my_feature', tags=['My Feature (V3 Actions)'])

Standard route handler pattern

from fastapi import APIRouter, Depends
from app.lib_general_v3 import AccountContext, get_account_context, DelayParams
from app.models.response_models import Resp_Body_Base, mk_resp
from app.methods.my_feature_methods import my_business_logic

router = APIRouter()

@router.get('/my_endpoint/{obj_id}', response_model=Resp_Body_Base)
async def get_my_endpoint(
        obj_id: str,
        account: AccountContext = Depends(get_account_context),
        delay: DelayParams = Depends(),
        ):
    if delay.sleep_time_s > 0:
        await asyncio.sleep(delay.sleep_time_s)

    result = my_business_logic(obj_id)
    status = result.get('status', 503)

    if status == 200:
        return mk_resp(data=result['data'])

    return mk_resp(data=False, status_code=status, status_message=result.get('reason', 'Error.'))

Auth dependency

Depends(get_account_context) is the standard V3 gate:

• Requires a valid x-aether-api-key
• Requires x-account-id OR a valid JWT session OR bypass mode
• Raises 403 if auth_method == 'guest' or no account context can be resolved
• For IDAA/private routes: this is the minimum gate. Never relax it.

---

5. Loading Site-Based Credentials (the _load_idaa_cfg Pattern)

Third-party credentials (Novi API key, Mailman credentials, etc.) are stored in MariaDB in the site.cfg_json column for the relevant site record. Do NOT store them in .env.

The IDAA site record is:

• id_random = '58_gJESdlUh' (site id=17)
• Fields: novi_api_root_url, novi_idaa_api_key, mailman_base_url, mailman_username, mailman_password, novi_mailman_sync

Pattern to load them (use deferred import to avoid circular deps):

IDAA_SITE_ID_RANDOM = '58_gJESdlUh'

def _load_idaa_cfg() -> dict:
    from app.db_sql import load_site_obj
    site = load_site_obj(IDAA_SITE_ID_RANDOM)
    if not site:
        raise RuntimeError('IDAA site record not found')
    cfg = site.get('cfg_json') or {}
    if isinstance(cfg, str):
        import json
        cfg = json.loads(cfg)
    return cfg

---

6. Redis Cache Pattern

import json
import datetime
from app.lib_redis_helpers import redis_client

_CACHE_TTL = datetime.timedelta(hours=4)

def _cache_key(uuid: str) -> str:
    return f'idaa:novi_member:{uuid}'

# Reading
raw = redis_client.get(_cache_key(uuid))
if raw:
    return json.loads(raw)

# Writing (only on success — never cache error states)
redis_client.setex(_cache_key(uuid), _CACHE_TTL, json.dumps(result))

Key naming convention: {module}:{object_type}:{identifier} — e.g. idaa:novi_member:{uuid}

Never cache:

• 404 responses — the member may have just joined; next call should hit the source
• 429 / 503 errors — transient failures should not poison future callers

Cache key scoping: If the underlying data source is the same regardless of caller (e.g. Novi credentials are hardcoded to the IDAA site — same UUID always returns same data), drop account_id from the key. Per-caller scoping wastes Redis space and halves hit rate.

---

7. The @logger_reset Decorator — Unit Test Gotcha

Business logic methods use @logger_reset from app.lib_general:

from app.lib_general import logger_reset

@logger_reset
def my_method(arg):
    ...

In unit tests, this decorator MUST be mocked as a passthrough. If app.lib_general is replaced with a plain MagicMock(), the decorator becomes a MagicMock, which when applied to a function replaces it with MagicMock()() — the decorated function is gone and every call returns garbage.

# WRONG — logger_reset becomes a MagicMock and swallows the function:
sys.modules['app.lib_general'] = MagicMock()

# CORRECT — make it a passthrough decorator:
mock_lib_general = MagicMock()
mock_lib_general.logger_reset = lambda f: f
sys.modules['app.lib_general'] = mock_lib_general

This is the #1 cause of unit tests returning <MagicMock name='...'> instead of real dicts.

---

8. Field Evolution Checklist

When a table or view gains, loses, or renames fields — do all of these in order:

1. Update the Pydantic model in app/models/
2. Update the SQL view or table projection so GET/SEARCH return the field
3. Update searchable_fields in app/object_definitions/ (only for searchable fields)
4. Add virtual/view-only fields to fields_to_exclude_from_db if they must not be persisted
5. Run the relevant schema/search E2E tests
6. Restart Docker: docker compose restart ae_api

Alt-view fields (in tbl_alt only)

Some objects have a richer alternate SQL view triggered by ?view=alt. These fields must still be declared in the Pydantic model and searchable_fields even if they only appear in the alt view — Pydantic strips undeclared fields silently.

---

9. Deferred Imports in Library Modules

To avoid circular dependency traps, never import from app.db_sql or other app modules at the top level of a library module. Use deferred imports inside functions:

# WRONG — causes circular import at startup:
from app.db_sql import load_site_obj

def my_func():
    return load_site_obj('abc')

# CORRECT:
def my_func():
    from app.db_sql import load_site_obj
    return load_site_obj('abc')

---

10. Pydantic / SQLAlchemy Version Pins — Do Not Remove

Current intentional pins:

• pydantic==1.*
• SQLAlchemy==1.4.52

A Pydantic V2 and SQLAlchemy 2.0 migration is planned but not started. Until then, do not upgrade these packages — V2 touches every model definition and the migration is a dedicated project.

---

11. Mistakes Agents Have Made on This Project

These are real incidents — know them before you start.

1. IDAA data exposed publicly — an agent removed an auth guard from the bulletin board router. Consequence: private IDAA recovery community data was publicly accessible. Always verify Depends(get_account_context) is present on every IDAA route.

2. "My code change has no effect" — Python file was edited but Docker was not restarted. The API runs Gunicorn inside Docker with no hot-reload. docker compose restart ae_api is required after every Python change.

3. @logger_reset mock swallows functions in unit tests — see Section 7 above. Symptom: assert result['status'] == 200 fails with TypeError: 'MagicMock' is not subscriptable.

4. pytest / pytest-asyncio not installed after venv rebuild — these are dev-only dependencies not in requirements.txt. After any OS Python update (e.g., Arch Linux upgrading to a new Python minor), rebuild the venv and reinstall:

   ./environment/bin/pip install pytest pytest-asyncio
   

5. Global db connection used instead of context manager — lib_sql_core.py has a global db = engine.connect() that is a fragile single connection, not a pool. For new methods, prefer engine.connect() as a context manager. See TODO__Agents.md → "[P3 full]" task for the planned migration.

6. bypass mode hardcodes account_id=1 — x-no-account-id: bypass resolves to account_id=1 (One Sky IT Demo). Lookup overrides from the Demo account can leak into bypass sessions. Do not expand bypass usage without documenting the allowlist case.

7. Caching error states in Redis — caching 404 or 5xx responses poisons the cache. A member who just joined Novi would be denied for 4 hours if their 404 was cached. Only cache verified success (200) results.

8. Not running docker compose restart ae_api between model changes and E2E tests — the E2E suite hits the live API, which is still running the old code until restarted. Tests will pass or fail against stale behavior and the results are meaningless.

---

12. Test Patterns

Unit tests (fast, no DB/network)

./environment/bin/python3 -m pytest tests/unit/ -v

• Mock all DB/Redis/HTTP at the top of the file before importing the module under test
• @logger_reset must be mocked as a passthrough (see Section 7)
• Always run from the project root — scripts use sys.path.append(os.getcwd())

E2E tests (live API at dev-api.oneskyit.com)

./environment/bin/python3 tests/e2e/test_e2e_v3_search_engine.py

• Require the Docker stack to be running with a working DB connection
• Use standard output format: [✅ PASS] / [❌ FAIL]
• Verify IDs in responses are strings (not integers) — ID Vision compliance

Which tests to run

Change type	Required suites
Model / ID Vision changes	test_e2e_v3_demo_parity.py, vision parity tests
Search / filter changes	test_e2e_v3_search_engine.py
Auth / account context changes	test_e2e_v3_security_audit.py, test_e2e_v3_auth_security.py
Any router or registry change	test_e2e_v3_security_audit.py
Novi-Mailman bridge changes	test_e2e_v3_action_novi_mailman.py
IDAA Novi verify changes	tests/unit/test_unit_idaa_novi_verify.py

---

13. Source Layout (Quick Reference)

app/
  main.py                    — FastAPI app, lifespan, CORS
  config.py                  — Pydantic Settings (from .env via Docker)
  routers/
    registry.py              — All router registrations live here
    api_crud_v3.py           — Generic V3 CRUD (flat)
    api_crud_v3_nested.py    — Generic V3 CRUD (nested/parent-owned)
    api_v3_actions_*.py      — Action endpoints (one file per domain)
    dependencies_v3.py       — Shared FastAPI Depends() helpers
  models/                    — Pydantic V1 models (one file per domain)
  object_definitions/        — Per-object searchable_fields, field metadata
  ae_obj_types_def.py        — Object type registry (all Aether types defined here)
  methods/                   — Business logic (one file per feature/domain)
  lib_api_crud_v3.py         — Generic CRUD handler (all object types share this)
  lib_schema_v3.py           — Dynamic schema/field resolution per object type
  lib_general_v3.py          — AccountContext, get_account_context, DelayParams
  lib_general.py             — logger_reset and general utilities
  db_sql.py                  — Import facade (always import from here)
  lib_sql_core.py            — SQLAlchemy engine, global db connection (Source of Truth)
  lib_sql_crud.py            — sql_insert, sql_select, sql_update, etc.
  lib_redis_helpers.py       — redis_client global instance

tests/
  unit/                      — Isolated logic tests (mock everything, no DB)
  integration/               — Requires local MariaDB/Redis
  e2e/                       — Network tests against live dev API
  tools/                     — Admin utilities (stress test, registry generator)
  mock_config_helper.py      — Mock app.config.settings — use in all unit tests
  README.md                  — Which tests cover what; when to run them

documentation/
  TODO__Agents.md            — Active tasks + session notes ← always read first
  ARCH__V3_DEVELOPMENT_STANDARDS.md — Master V3 standards doc
  ARCH__V3_CORE.md           — Module architecture (lifespan, DB layers, logging)
  GUIDE__AE_API_V3_for_Frontend.md  — Frontend integration guide (keep current)
  GUIDE__DEVELOPMENT.md      — Commit SOP, verification checklist

---

14. Reading Order for Deeper Dives

What you need	Read
Active tasks + known bugs	documentation/TODO__Agents.md ← always first
V3 standards and strategy	documentation/ARCH__V3_DEVELOPMENT_STANDARDS.md
Module architecture	documentation/ARCH__V3_CORE.md
Frontend API integration guide	documentation/GUIDE__AE_API_V3_for_Frontend.md
WebSocket integration	documentation/GUIDE__AE_API_V3_for_Frontend_websockets.md
Commit / verification SOP	documentation/GUIDE__DEVELOPMENT.md
Which tests to run	tests/README.md
Object type registry	app/ae_obj_types_def.py
Shared agent docs	~/agents_sync/aether/docs/UNIFIED_AGENT_ARCH.md