OSIT-AE-API-FastAPI/documentation/TODO__Agents.md

Backend Agent Task List

Use this file to track steps for complex features or bug fixes. Status: 🔵 DEPLOYMENT READY - Unified Docker Orchestration Complete.

🚀 Recent Infrastructure Wins

• Self-Contained Build: Dockerfile and requirements.txt moved to project root.
• Dependency Pruning: Removed 6 redundant/unused Python packages.
• Unified Orchestration: API now builds as part of the aether_container_env stack.

📋 Operational Hardening (Next Steps)

• Healthcheck: Implement /health route to verify DB/Redis status for Docker orchestration.
• Config Refactor: Switch app/config.py to pydantic-settings to use direct Env Vars (Stop mounting config files).
• Locking: Generate a requirements.lock for bit-identical builds.

📋 Feature Tasks

• Core Isolation: Harden apply_forced_account_filter to Fail-Closed.
• IDAA Baseline: Remove public_read from Event, CMS, and Archive objects.
• Detailed Feedback: Implement descriptive 403 Forbidden reasons.
• Polymorphic For_ID Patterns: Add ID Vision to Address, Contact, and DataStore objects.
• Event File Hash_SHA256 Fix: Populate hosted_file_hash_sha256 correctly.
• Step 1: ID Vision Parity Audit
  • Audit Core Event Models (Badge, Session, Presentation).
  • Audit File/Exhibit Models (File, Template, Tracking).
  • Whitelist account_id in all Event search definitions.
  • Audit Relational "Low-Priority" Models (Address, Contact, DataStore).
  • V3 Uniform Lookup System: Phase 1 & 2 Complete.
  • Verify SQL Views join in all required _random IDs for performance.
• Step 2: Coordination (Verify Frontend uses x-account-id instead of token).
• Step 3: Frontend V3 WebSocket integration test — queued after IDAA-specific work. Backend is ready (auth wired, heartbeat presence refresh confirmed, unit tests passing). Frontend guide updated at GUIDE__AE_API_V3_for_Frontend_websockets.md.

🛡️ Security & Privacy Baseline (IDAA)

• Status: ENFORCED.
• Maintenance: Run tests/e2e/test_e2e_v3_security_audit.py after ANY router or registry change.

🔑 Credentials / Access Maintenance

• Bitbucket API Token Migration: Bitbucket is deprecating app passwords — all existing ones become inactive 2026-06-09. SSH migration complete; Gitea remote also configured. Ref: https://support.atlassian.com/bitbucket-cloud/docs/api-tokens/

🚧 Strategic Goals (V3.5+)

• Pydantic V2 / SQLAlchemy 2.0: Major framework upgrade for performance and type safety.
  • SQLAlchemy 2.0 is likely the easier migration (additive, legacy mode available).
  • Pydantic v2 touches every model definition — do this second.
  • Current pins: pydantic==1.*, SQLAlchemy==1.4.52 — intentional, do not remove until migration is done.
• Novi-Mailman Bridge: Cron-based mirror sync between Novi AMS and Mailman 3 — POC complete 2026-03-17.
  • Files: app/methods/e_novi_mailman_methods.py, app/routers/api_v3_actions_e_novi_mailman.py
  • Registered at /v3/action/e_novi_mailman/
  • Confirmed Novi API shape: No flat member list. Fetch via /groups/{guid}/members → UUIDs, then /customers/{uuid} for full record. Fields: Email, FirstName, LastName, Active (bool), UnsubscribeFromEmails (bool). Emails may contain spaces instead of + — sanitized with .replace(' ', '+').
  • Credentials: All in IDAA site cfg_json (id_random='58_gJESdlUh', site id=17). Keys: novi_api_root_url, novi_idaa_api_key, mailman_base_url, mailman_username, mailman_password, novi_mailman_sync (array).
  • Mailman 3 REST API: https://lists.idaa.org/mailman-api (Nginx proxy → 127.0.0.1:8008 → Docker container). Roster: /3.1/lists/{list_id_dot}/roster/member.
  • Sync logic: Full mirror — subscribe Novi-only addresses, unsubscribe Mailman-only addresses. Respects Active=false and UnsubscribeFromEmails=true.
  • Cron target: POST /v3/action/e_novi_mailman/sync — runs all novi_mailman_sync mappings.
  • Webhook approach abandoned — cron is simpler; Novi webhook payload format is unknown and Novi hasn't been configured to send webhooks.
  • Remaining: Set production group→list mappings in cfg_json, configure cron schedule, rotate Mailman restadmin password.
• Lookup System Batch 2: Migration of post_topic, user_status, file_purpose.
• Zoom Events Integration: Implement cron synchronization for OAuth2 ticket retrieval.

📝 Session Notes (March 11, 2026)

• Media Methods Hardened: clip_video_method and convert_file_method in app/methods/lib_media.py updated with improved error logging, PDF validation, and guaranteed temp-file cleanup.
• V3 Action Migration: clip_video endpoint promoted from legacy hosted_file router to V3 action (/v3/action/hosted_file/{id}/clip_video). Legacy route now issues a 307 redirect for backward compatibility.
• Background Scheduling: clip_video V3 action supports ?background=true (returns 202 Accepted), enabling async clipping for large files.
• Robust Deletion: delete_file_action unlink wrapped in try/except OSError — filesystem errors are now logged and non-fatal.
• Unit Tests Added: tests/unit/test_unit_media_methods.py covers clip_video_method and convert_file_method with full async mocking.
• Docs Renamed & Updated: GUIDE__V3_FRONTEND_API.md → GUIDE__AE_API_V3_for_Frontend.md; GUIDE__V3_FRONTEND_WEBSOCKETS.md → GUIDE__AE_API_V3_for_Frontend_websockets.md. Frontend guide updated with V3 action paths, background scheduling notes, and correct example URLs.

📝 Session Notes (March 10, 2026)

• Unified Stack: Merged API orchestration into the master environment.
• Root Assets: Docker assets are now part of this git repo again.
• Pruning: Successfully reduced dependency bloat in requirements.txt.
• Operational Hardening complete: Healthcheck, config refactor (pydantic-settings), requirements.lock all done.
• BuildKit pip cache: Dockerfile now uses --mount=type=cache — rebuild with docker compose up -d --build ae_api.
• Novi-Mailman Bridge scaffolded: Auth pattern and field names confirmed from existing IDAA Jitsi frontend code.