Commit Graph

39 Commits

Author SHA1 Message Date
Scott Idem
687472f4e3 feat(user): V3 action endpoints + auth bug fixes (19/19 + 22/22 tests)
New router: /v3/action/user/ (api_v3_actions_user.py)
  - POST /authenticate  — credentials in body (not query params; security fix)
  - POST /verify_password
  - POST /{user_id}/change_password  — optional current-password verification
  - GET  /{user_id}/new_auth_key
  - GET  /{user_id}/email_auth_key_url
  Registered in registry.py under /v3/action/user with V3 AccountContext auth.

Bug fixes (from audit in previous session):
  - user.py: fix broken @router.get decorator (authenticate was unreachable)
  - user.py + user_methods.py: fix AttributeError id_random → id (Vision ID)
  - user_models.py: add fields_to_exclude_from_db to User_New_Base; narrow
    collision prevention to self-reference IDs only
  - user_models.py: pre-inject hashed password in root_validator(pre=True) so
    exclude_unset=True in CRUD POST handler includes it (was writing NULL)
  - api_crud_v3.py: move sanitize_payload + account_id injection to after
    model validation (fixes FK integer collision with Vision ID constraints)

Docs: GUIDE__AE_API_V3_for_Frontend.md — new Section 7 with full migration
  table (legacy → V3), request/response docs for all 5 action endpoints,
  and V3 CRUD search equivalents for the 3 lookup routes.

Tests: tests/e2e/test_e2e_v3_user_action_routes.py — 19 tests, 19/19 pass.
  Legacy tests/e2e/test_e2e_v3_user_auth_routes.py — 22/22 still pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 21:54:09 -04:00
Scott Idem
91434968f7 docs+site_domain: Add guidance for restoring access_key validation in site_domain lookup; stage recent user/auth changes and frontend guide updates 2026-03-25 19:33:53 -04:00
Scott Idem
c9ec3d7ea1 revert(lookup): restore PARTITION BY group; tests now track data fix
Reverts the PARTITION BY name change — group is the correct dedup key.
Partitioning by name broke country deduplication (two US records both
survived, causing Svelte each_key_duplicate on alpha_2_code='US').

Root cause is bad seed data in lu_v3_time_zone: group='United States'
for 13 US/* zones and group='Europe' for 63 Europe/* zones instead of
group=name. A separate DB UPDATE is required to fix those rows.

Tests updated to assert:
- No duplicate alpha_2_code in country list (PARTITION BY group regression)
- All 13 US/* and Europe/* spot-check zones present (pending DB data fix)
- priority-only timezone count == 72 (pending DB data fix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 17:31:30 -04:00
Scott Idem
ccf2f30e11 fix(lookup): partition dedup by name instead of group
ROW_NUMBER() was partitioning by `group`, collapsing all 12 US/* timezones
(which share group="United States") down to a single record. Partitioning
by `name` correctly deduplicates by timezone identity while still preserving
the object > account > global override hierarchy.

Priority-only list now returns the expected 72 entries. Adds a regression
test asserting all 12 US/* timezones are present in the full list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 16:46:47 -04:00
Scott Idem
6b25cf9c6d feat: add Novi AMS → Mailman 3 cron-based mirror sync bridge (IDAA)
Implements a full proof-of-concept for syncing IDAA's Novi AMS membership
groups to Mailman 3 mailing lists via a cron-triggered reconciliation approach.

Key changes:
- methods: rewrote sync engine around confirmed Novi API shape — group-based
  member fetch (/groups/{guid}/members + /customers/{uuid}), respects
  Active=false and UnsubscribeFromEmails=true flags
- methods: mirror_novi_group_to_mailman_list() diffs Novi group against
  Mailman roster and subscribes/unsubscribes accordingly (full mirror)
- methods: mirror_all_configured_mappings() iterates novi_mailman_sync
  config array in IDAA site cfg_json — this is the cron target
- router: replaced old /sync endpoint with POST /sync (all mappings) and
  POST /sync/group/{guid} (single mapping); removed webhook endpoint
  (sync is cron-based, not event-driven)
- router: added GET/POST/DELETE endpoints for list member inspection
  and manual subscribe/unsubscribe
- tests: two new e2e scripts covering connection checks and full member
  lifecycle; old webhook integration test archived

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 16:36:32 -04:00
Scott Idem
29579fd9f1 feat: add V3 action endpoint for event exhibit tracking export
- New router: app/routers/api_v3_actions_event_exhibit.py
  - GET /v3/action/event_exhibit/{exhibit_id}/tracking_export
  - Full V3 auth (x-aether-api-key + account context)
  - Multi-tenant ownership check via check_account_access
  - Permission gate: leads_api_access flag OR manager-level access
  - Returns CSV or XLSX file attachment (return_file=false for JSON)
  - Flattens responses_json custom Q&A columns; strips HTML from exhibitor_notes
  - Exports all records regardless of hidden/enabled state

- Registered in registry.py under prefix /v3/action/event_exhibit

- New E2E test: tests/e2e/test_e2e_v3_action_event_exhibit_tracking_export.py
  - 7/7 tests passing against dev-api.oneskyit.com

- Docs: GUIDE__AE_API_V3_for_Frontend.md — new Section 7 covering endpoint
  usage, columns, leads_api_access dual-purpose (3rd-party API + UI export gate)

- Docs: tests/README.md — added test to table and when-to-run matrix
2026-03-16 16:50:32 -04:00
Scott Idem
eaa18a1d45 fix(nested-crud): re-inject parent FK after model serialization to prevent 1364 errors
Root cause: child model root_validators (Vision ID anti-leakage guard) strip
integer IDs before they can be serialized into the INSERT dict, causing MariaDB
to reject the INSERT with 'Field does not have a default value' (1364).

Fix: re-inject resolved_parent_id into data_to_insert after validated_obj.dict()
in post_child_obj(). This is safe — the integer was already verified against the
DB before model validation.

Affected (were all broken since ~2026-01-27):
  - journal/{id}/journal_entry/
  - event/{id}/event_session/
  - event/{id}/event_person/
  - event/{id}/event_registration/
  - event/{id}/event_presenter/
  - event/{id}/event_presentation/
  - event/{id}/event_location/
  - event/{id}/event_track/
  - event/{id}/event_device/
  - event/{id}/event_abstract/
  - event/{id}/event_badge/ (different symptom: NULL FK)

Tests: add nested create lifecycle regression tests to test_e2e_v3_demo_parity.py
  - POST + Vision check + DELETE for journal/journal_entry and event/event_session
  - All 9 checks passing (7s)

Docs: update tests/README.md with accurate demo_parity description and
  a 'When to Run Tests' matrix to prevent future gaps in coverage.
2026-03-16 12:39:45 -04:00
Scott Idem
a20c436013 Migrate clip/convert to V3 actions; add background clip support, redirect legacy route; update frontend guide 2026-03-11 14:51:08 -04:00
Scott Idem
bc78ac4c2e test: Add E2E test for V3 model refactoring verification
This commit introduces a new end-to-end test script  to validate the recent model refactoring changes.

The test suite performs two primary checks for a list of target objects:
- **ID Vision Compliance:** Verifies that all primary and foreign key fields are returned as string IDs, ensuring adherence to the V3 ID Vision standard.
- **Excluded Fields Stripping:** Attempts a PATCH operation with fields explicitly listed in  and verifies that these fields are not updated in the database, confirming the  mechanism functions as intended.

This test is essential for ensuring the stability and correctness of the API's interaction with the refactored models.
2026-02-24 16:22:19 -05:00
Scott Idem
48fc97cf46 feat: add priority filtering and sort stability to V3 Lookup System 2026-02-20 17:18:21 -05:00
Scott Idem
6bfbff309a feat: implement V3 Uniform Lookup System with hierarchical overrides and site-based whitelisting 2026-02-20 14:48:50 -05:00
Scott Idem
17a627a981 feat: Implement Event File Hosted Data Fix and API Guide Update
Address critical data visibility issues for Event Files and enhance frontend documentation.

This commit resolves the persistent problem where top-level hosted file convenience fields
(e.g., , , ) were
returning as  in V3 Event File API responses, even when .

Key changes include:
- Refactored  Pydantic model:
    - Removed redundant  definitions from top-level hosted file convenience fields,
      allowing direct mapping from SQL view columns.
    - Simplified  to focus solely on conditionally loading the nested
       object, as top-level fields are now populated directly by Pydantic
      from the  view.
    - Added comprehensive comments to clarify data flow, Pydantic's behavior, and the
      expected origin of these convenience fields from SQL views.
- Updated :
    - Introduced a new section detailing how to retrieve Event File data, including the
      use of  to get both top-level convenience fields and a nested
       object.
    - Clarified all ID references as random string IDs.
    - Renumbered the troubleshooting section.
- Copied updated guide to .
- Continued ID Vision compliance audit, ensuring consistent handling of random string IDs
  across various core and event models (Account, Address, Contact, DataStore, Event Badge Template).
- Consolidated ID Vision E2E tests and updated related documentation.
- Minor updates to  and
  to support Event File data retrieval with .
2026-02-19 15:22:17 -05:00
Scott Idem
2266f149f7 security(v3): harden multi-tenant isolation and enhance failure feedback 2026-02-13 18:45:20 -05:00
Scott Idem
3e6ea108cf feat(redis): implement bidirectional ID caching and extensive E2E benchmarks 2026-02-10 18:08:04 -05:00
Scott Idem
68e883ba98 feat(v3-data-store): harden search security and standardize test suite 2026-02-09 19:03:04 -05:00
Scott Idem
2a7c27ba80 fix(v3-vision): implement fallback resolution for relational IDs
1. Added fallback mechanism to Event_File_Base to resolve string IDs from integers when views return partial data.\n2. Added 'a2pPIT_W28o' as a permanent regression test target.\n3. Hardened lu_file_purpose_id stripping.
2026-02-06 18:13:40 -05:00
Scott Idem
d5e685dee8 fix(v3-vision): strict integer stripping and demo parity verification
1. Hardened all demo models to set non-string ID fields to None, ensuring full Vision Standard compliance.\n2. Added status_id_random to common field schema.\n3. Verified account_id availability in exhibit tracking.\n4. Added comprehensive E2E parity test suite for demo objects.\n5. Fixed NameError by importing root_validator.
2026-02-06 16:16:08 -05:00
Scott Idem
b10b5839c7 fix(config): harden bootstrap logic and add email E2E test
Harden bootstrap_db_config to prioritize .env settings for core infrastructure (DB/SMTP) and only use DB values if placeholders are detected or values have explicitly changed. Added test_e2e_email_send.py for functional SMTP verification.
2026-02-06 12:56:13 -05:00
Scott Idem
03a1569eba test(v3-cms): add CMS Vision Parity E2E test 2026-02-05 19:28:03 -05:00
Scott Idem
2fe783784c feat(integration): initial Zoom Events backend integration
- Implemented Server-to-Server OAuth2 logic in e_zoom_methods.py.
- Created V3 Action router for Zoom (test_connection, fetch tickets, atomic sync).
- Added sync_zoom_attendees_to_event with mapping to Aether person/badge models.
- Registered /v3/action/e_zoom router.
- Added E2E connection test script.
2026-02-03 18:23:07 -05:00
Scott Idem
69622dbea6 refactor(core): modularize monolithic routers and methods
- Reduced api_crud.py (1843 -> 143 lines) by extracting V1 registry and logic.
- Reduced hosted_file.py (1596 -> 361 lines) by moving storage and media logic to methods.
- Created lib_media.py for specialized video/image processing.
- Created api_crud_methods.py for legacy template handlers.
- Created legacy_v1.py for the legacy object registry.
- Fixed subdirectory_path bug in Hosted File creation.
- Verified full File Lifecycle via consolidated E2E suite.
2026-02-03 17:53:14 -05:00
Scott Idem
37c84de57b chore(tests): consolidate E2E test suite into standardized primary scripts
- Combined 10+ one-off tests into 4 primary functional suites (Search, Auth, Lifecycle, Vision).
- Archived original scripts to tests/archive/.
- Updated README with the new standardized inventory.
- Applied clean output formatting across the new suite.
2026-02-03 16:50:18 -05:00
Scott Idem
29f6cf258f chore(tests): reorganize test suite and archive redundant scripts
- Moved legacy/redundant tests to tests/archive/.
- Relocated root-level debug scripts to tests/integration/.
- Updated tests/README.md with final organized inventory.
- Cleaned up root directory from one-off reproduction scripts.
2026-02-03 16:18:57 -05:00
Scott Idem
d43474ea4b feat(registry): standardize searchable_fields and add developer reminders
- Added id_random, account_id_random, created_on, and updated_on to searchable whitelists.
- Standardized field coverage for Core and Other (Archive/HostedFile) modules.
- Added Developer Handshake comments to prevent future whitelist/model desync.
- Verified via new E2E registry test suite.
2026-02-03 15:51:05 -05:00
Scott Idem
54d6bd8864 test(api-v3): update E2E tests for ID Vision, hash-downloads, and data_store parity 2026-02-03 15:34:12 -05:00
Scott Idem
9362938ffe feat(api-v3): add advanced search and view support to nested CRUD router 2026-02-03 14:10:41 -05:00
Scott Idem
6f7fde7b87 test(hosted-file): add debug and E2E tests for hash-based download 2026-02-03 13:44:28 -05:00
Scott Idem
faa6de866d test(event-file): add E2E test for V3 event_file upload action 2026-02-03 12:54:41 -05:00
Scott Idem
a02abbbe4f feat(models): implement Vision ID pattern for event_device and event_session
- Migrated event_device and event_session models to the V3 Vision ID pattern (string-based public IDs).
- Added root_validator for automatic id_random mapping and integer stripping.
- Implemented fields_to_exclude_from_db to protect database updates from convenience/view fields.
- Fixed description_json type in Journal_Base for correct JSON parsing.
- Added E2E verification tests for event_device and event_session V3 endpoints.
2026-01-30 12:38:16 -05:00
Scott Idem
0606cecb61 feat(data_store): finalize V3 cascading lookup with limit override
- Update GET /v3/data_store/code/{code} to support 'limit' query parameter.
- Refactor return logic: returns single object if limit=1, otherwise returns a list.
- Clean up formatting in GUIDE__V3_FRONTEND_API.md and sync to agents_sync.
- Finalize unified E2E test script: tests/e2e/test_e2e_v3_data_store_lookup.py.
2026-01-28 17:01:34 -05:00
Scott Idem
fdcc859017 feat(data_store): implement V3 cascading lookup and ID Vision standardization
- Added GET /v3/data_store/code/{code} with hierarchical context-aware fallback.
- Implemented ID Vision standard in Data_Store_Base (string IDs, internal int exclusion).
- Enhanced Data_Store_Base robustness to handle stringified 'NULL' values from the database.
- Fixed legacy router bugs by removing undefined parameters (inc_event_cfg, inc_event_location).
- Corrected type hints and resolved UnboundLocalError in data_store methods.
- Updated Frontend Integration Guide with Section 8: Data Store V3.
- Added unified E2E test script: tests/e2e/test_e2e_v3_data_store_lookup.py.
2026-01-28 16:51:48 -05:00
Scott Idem
2e4fbfc8ab Saving more test scripts 2026-01-22 18:59:01 -05:00
Scott Idem
60345dd21e V3 Migration Phase 2-4: Implementation of specialized Binary Actions (Upload, Stream, Delete) and Orphan management logic. Full E2E coverage. 2026-01-22 18:51:26 -05:00
Scott Idem
df0ce7f910 Saving fixes to the hosted file delete function. 2026-01-22 17:31:29 -05:00
Scott Idem
1e6b9d1c18 Bug fixes for uploading the files. I though the changes being made where not supposed to break legacy endpoints. Not sure what happened. Either way things are almost back to normal. 2026-01-22 16:49:03 -05:00
Scott Idem
6ca79e9a02 chore(api): stabilize SQL core and enhance searchability
- Refactor SQL CRUD to use engine.connect() context managers for thread safety
- Optimize connection pooling in lib_sql_core
- Clean up app/routers/api.py to fix duplicate definitions and OpenAPI KeyError
- Add 'default_qry_str' to searchable_fields for Event, Session, Presentation, Presenter, Badge, and Journal
- Add 'event_location_name' to searchable_fields for Event Session
- Verified 20/20 E2E success via repro_intermittent_errors.py
2026-01-21 15:23:04 -05:00
Scott Idem
d4e46a4a97 feat(auth): implement site-based passcode-to-JWT endpoint
- Add POST /api/authenticate_passcode to verify site access codes
- Refactor sign_jwt to support arbitrary role flags (super, admin, etc.)
- Update dependencies_v3 to extract role flags from JWT payloads
- Add E2E test for passcode auth verification
2026-01-20 17:51:54 -05:00
Scott Idem
dc7732ab5f feat(security): implement safe guest auth flow and harden request_jwt
- Patched request_jwt to strip privileged IDs when signing with public keys
- Updated AccountContext and V3 dependencies to preserve JWT payloads for guests
- Whitelisted Archive, Post, Event, and other core objects for public read access
- Added 'default_qry_str' to Event searchable fields
- Added test_e2e_jwt_guest_auth.py for security verification
2026-01-20 14:56:56 -05:00
Scott Idem
b2384f2869 Tests: Reorganize test suite into functional subdirectories
- Categorized scripts into tests/unit/, tests/integration/, tests/e2e/, and tests/tools/.
- Adopted consistent naming prefixes (test_unit_*, test_int_*, test_e2e_*, tool_*).
- Renamed conftest_mock.py to mock_config_helper.py for clarity.
- Updated test_int_boot_diagnosis.py with sys.path setup for root-level execution.
2026-01-16 10:46:19 -05:00