Commit Graph

1241 Commits

Author SHA1 Message Date
Scott Idem
89e12b9f97 fix: Resolve ID Vision conflicts and validation errors in Event Exhibit Tracking
- Modified 'sanitize_payload' to ignore 'external_person_id', preventing incorrect lookup attempts for email/passcode fields.
- Refined 'Event_Exhibit_Tracking_Base' to allow 'Union[int, str]' for relational IDs, bypassing string-length validation for internal integers.
- Adjusted root validator to preserve relational integers during POST/PUT operations while still stripping primary/account IDs for Vision-compliant READ views.
- Aligned model configuration with other V3 objects for consistency.
2026-03-03 17:08:34 -05:00
Scott Idem
403b543ed2 Added the duplex field and some other minor clean up 2026-03-02 19:49:19 -05:00
Scott Idem
bc78ac4c2e test: Add E2E test for V3 model refactoring verification
This commit introduces a new end-to-end test script  to validate the recent model refactoring changes.

The test suite performs two primary checks for a list of target objects:
- **ID Vision Compliance:** Verifies that all primary and foreign key fields are returned as string IDs, ensuring adherence to the V3 ID Vision standard.
- **Excluded Fields Stripping:** Attempts a PATCH operation with fields explicitly listed in  and verifies that these fields are not updated in the database, confirming the  mechanism functions as intended.

This test is essential for ensuring the stability and correctness of the API's interaction with the refactored models.
2026-02-24 16:22:19 -05:00
Scott Idem
0f4b4d2f51 feat: Implement V3 ID Vision and fields_to_exclude_from_db across core models
This commit refactors numerous Pydantic models to align with the V3 ID Vision standard, ensuring that primary and foreign key fields are represented as clean string IDs in the API. It also introduces and populates the  ClassVar in each model to prevent view-only fields and linked objects from being inadvertently written to the database during PATCH/POST operations.

Specifically, this includes:
- Adding  to exclude view-derived or joined fields such as , , nested objects (e.g., , ), and convenience fields (e.g., ).
- Adjusting root validators to correctly map string IDs and strip internal integer IDs for API responses.
- Resolving a KeyError by adding  to .

These changes are crucial for maintaining data integrity and consistency with the V3 API architecture.
2026-02-24 16:21:27 -05:00
Scott Idem
9d89d4c8e4 fix: exclude account_id and virtual fields from archive_content DB writes
- Adds fields_to_exclude_from_db to Archive_Content_Base to prevent SQL errors on non-existent columns.
- Updates documentation for V3 Create/Update patterns and the x-ae-ignore-extra-fields header.
- Propagates account_id_random to hosted file and media processing methods.
2026-02-24 11:30:17 -05:00
Scott Idem
719ca5240b Work on CORS again. Chromium now enforces the new "Local Network Access" permission and prompting. This was a problem because pfSense was set to resolve dev-api.oneskyit.com back to 192.168.32.7 instead of the normal external IP. Turned this off. Hopefully it won't break other things. 2026-02-23 17:57:13 -05:00
Scott Idem
f518d7a433 Saving notes 2026-02-20 19:46:03 -05:00
Scott Idem
48fc97cf46 feat: add priority filtering and sort stability to V3 Lookup System 2026-02-20 17:18:21 -05:00
Scott Idem
6bfbff309a feat: implement V3 Uniform Lookup System with hierarchical overrides and site-based whitelisting 2026-02-20 14:48:50 -05:00
Scott Idem
2b2a2bc00f Saving notes about new lookup tables. 2026-02-19 19:19:59 -05:00
Scott Idem
6a023a82f5 Saving notes about reviewing the SQL VIEWs. 2026-02-19 17:56:08 -05:00
Scott Idem
1db71f85a5 Saving updates to notes. Less reference to the _random. 2026-02-19 16:20:25 -05:00
Scott Idem
17a627a981 feat: Implement Event File Hosted Data Fix and API Guide Update
Address critical data visibility issues for Event Files and enhance frontend documentation.

This commit resolves the persistent problem where top-level hosted file convenience fields
(e.g., , , ) were
returning as  in V3 Event File API responses, even when .

Key changes include:
- Refactored  Pydantic model:
    - Removed redundant  definitions from top-level hosted file convenience fields,
      allowing direct mapping from SQL view columns.
    - Simplified  to focus solely on conditionally loading the nested
       object, as top-level fields are now populated directly by Pydantic
      from the  view.
    - Added comprehensive comments to clarify data flow, Pydantic's behavior, and the
      expected origin of these convenience fields from SQL views.
- Updated :
    - Introduced a new section detailing how to retrieve Event File data, including the
      use of  to get both top-level convenience fields and a nested
       object.
    - Clarified all ID references as random string IDs.
    - Renumbered the troubleshooting section.
- Copied updated guide to .
- Continued ID Vision compliance audit, ensuring consistent handling of random string IDs
  across various core and event models (Account, Address, Contact, DataStore, Event Badge Template).
- Consolidated ID Vision E2E tests and updated related documentation.
- Minor updates to  and
  to support Event File data retrieval with .
2026-02-19 15:22:17 -05:00
Scott Idem
577d784fb8 Serious notes about security updates. 2026-02-13 19:22:33 -05:00
Scott Idem
aca15aab91 security(v3): implement IDAA-baseline maximum lockdown 2026-02-13 19:19:53 -05:00
Scott Idem
2266f149f7 security(v3): harden multi-tenant isolation and enhance failure feedback 2026-02-13 18:45:20 -05:00
Scott Idem
61e17f1efa Updating the documentation for things. Hopefully improvements to Gemini CLI. 2026-02-11 17:59:15 -05:00
Scott Idem
3e6ea108cf feat(redis): implement bidirectional ID caching and extensive E2E benchmarks 2026-02-10 18:08:04 -05:00
Scott Idem
17ae70992f refactor(redis): consolidate ID resolution and remove probabilistic refresh hack 2026-02-10 17:45:31 -05:00
Scott Idem
6d5633dc86 fix(v3-vision): prevent process hang on lu_ tables missing id_random 2026-02-10 17:30:38 -05:00
Scott Idem
68e883ba98 feat(v3-data-store): harden search security and standardize test suite 2026-02-09 19:03:04 -05:00
Scott Idem
9715d28bd6 feat(v3-vision): implement resilient "Heal-on-Read" ID resolution
1. Hardened Event_Exhibit and Event_Exhibit_Tracking models with automatic Redis/DB fallback for missing string IDs.
2. Fully modernized Event_Person_Tracking_Base to the Vision Standard (Union IDs + Root Validator).
3. Enabled account-based search for event_person_tracking.
4. Verified all changes via e2e demo parity suite.
2026-02-07 19:27:44 -05:00
Scott Idem
e8f9472c5c feat(v3-api): whitelist lead detail fields for exhibit tracking search
Added 'event_badge_full_name', 'event_badge_affiliations', 'event_badge_email', and 'event_badge_location' to the searchable_fields for event_exhibit_tracking.
2026-02-07 18:51:15 -05:00
Scott Idem
fe368e2f64 feat(v3-api): enable account-based search for exhibit tracking
1. Added 'account_id' and 'account_id_random' to searchable_fields for event_exhibit_tracking.
2. Updated tests/README.md with descriptions for the latest E2E test scripts (Demo Parity, Event Actions, Zoom).
2026-02-07 18:09:45 -05:00
Scott Idem
7084dd3472 Saving notes 2026-02-06 18:15:12 -05:00
Scott Idem
42bea571e9 test: preserve archived debug scripts 2026-02-06 18:13:48 -05:00
Scott Idem
2a7c27ba80 fix(v3-vision): implement fallback resolution for relational IDs
1. Added fallback mechanism to Event_File_Base to resolve string IDs from integers when views return partial data.\n2. Added 'a2pPIT_W28o' as a permanent regression test target.\n3. Hardened lu_file_purpose_id stripping.
2026-02-06 18:13:40 -05:00
Scott Idem
8270f7ff7a fix(v3-actions): implement from_hosted_file and harden vision IDs
1. Implemented specialized 'from_hosted_file' action for Event Files.\n2. Fixed ValueError in Pydantic models by removing default/default_factory conflict.\n3. Hardened integer stripping to strictly enforce Vision Standards.\n4. Updated documentation for the new action route.
2026-02-06 16:23:18 -05:00
Scott Idem
64d73c4d5c fix(registry): finalize event_badge model mapping 2026-02-06 16:16:18 -05:00
Scott Idem
d5e685dee8 fix(v3-vision): strict integer stripping and demo parity verification
1. Hardened all demo models to set non-string ID fields to None, ensuring full Vision Standard compliance.\n2. Added status_id_random to common field schema.\n3. Verified account_id availability in exhibit tracking.\n4. Added comprehensive E2E parity test suite for demo objects.\n5. Fixed NameError by importing root_validator.
2026-02-06 16:16:08 -05:00
Scott Idem
f3662f9462 fix(v3-vision): final hardening of demo models
Standardized Badge, Exhibit, and Tracking models to ID Vision standards. Included account_id support for exhibit tracking and removed legacy validators to ensure stable CRUD operations for the Tuesday demo.
2026-02-06 15:46:05 -05:00
Scott Idem
4aadb4ec1c Update related to object aliases. 2026-02-06 14:20:19 -05:00
Scott Idem
b63131e3fa fix(v3-nested): support aliases in nested CRUD routes
1. Added 'entry' alias for 'journal_entry' in object definitions.\n2. Updated nested router to resolve physical table names from the registry before ID resolution.\n3. Updated ID resolution helpers to recognize 'entry' prefix.\nThis resolves 404 errors when using shorter aliases in nested paths (e.g., /journal/{id}/entry/).
2026-02-06 14:13:22 -05:00
Scott Idem
b9c00e423c Less debug! 2026-02-06 13:56:40 -05:00
Scott Idem
b10b5839c7 fix(config): harden bootstrap logic and add email E2E test
Harden bootstrap_db_config to prioritize .env settings for core infrastructure (DB/SMTP) and only use DB values if placeholders are detected or values have explicitly changed. Added test_e2e_email_send.py for functional SMTP verification.
2026-02-06 12:56:13 -05:00
Scott Idem
1053d8a81b fix(sql): handle record_id=0 correctly in CRUD utilities
Updated sql_select, sql_update, and sql_delete to use explicit 'is not None' checks for record_id. This prevents falsy ID values (like 0) from triggering generic table scans or failing to filter, which was causing the config bootstrap to accidentally load record ID 1 when ID 0 was requested.
2026-02-06 12:53:59 -05:00
Scott Idem
37a43babb9 fix(v3-actions): fix UnicodeEncodeError in download filenames
Hardened StreamingResponse headers to use RFC 6266 filename* parameter with UTF-8 encoding. This prevents the latin-1 codec crash when filenames contain non-ASCII characters like smart quotes.
2026-02-06 10:44:22 -05:00
Scott Idem
1492b01dad Saving notes 2026-02-05 20:42:43 -05:00
Scott Idem
a7c82615ab fix(v3-vision): platform-wide hardening of ID Vision models
Hardened root_validators in Journal, Post, Page, and Hosted File models to use Union[int, str] for Vision ID fields. This prevents resolved integer IDs from being deleted during CREATE/UPDATE operations, resolving a critical regression found during Post Comment bug fixing.
2026-02-05 20:38:19 -05:00
Scott Idem
78f04bca50 fix(v3-vision): allow resolved integers to pass model validation during creation
Hardened root_validators in Event and Post Comment models to use Union[int, str] for Vision ID fields. This ensures that integer IDs resolved by sanitize_payload reach the database during POST/PATCH operations while maintaining clean string outputs for clients.
2026-02-05 20:30:44 -05:00
Scott Idem
03a1569eba test(v3-cms): add CMS Vision Parity E2E test 2026-02-05 19:28:03 -05:00
Scott Idem
1cfbf9ebad feat(v3-cms): standardize Post Comment IDs and enhance CMS searchability 2026-02-05 19:20:18 -05:00
Scott Idem
12d725f468 feat(v3-standardization): implement ID Vision for Event models and update docs 2026-02-05 17:37:52 -05:00
Scott Idem
907ff9a2f8 feat(v3-events): update events_general object definition with virtual/physical and external ID fields 2026-02-05 16:37:49 -05:00
Scott Idem
ac516c4d77 Saving notes 2026-02-03 19:10:31 -05:00
Scott Idem
2fe783784c feat(integration): initial Zoom Events backend integration
- Implemented Server-to-Server OAuth2 logic in e_zoom_methods.py.
- Created V3 Action router for Zoom (test_connection, fetch tickets, atomic sync).
- Added sync_zoom_attendees_to_event with mapping to Aether person/badge models.
- Registered /v3/action/e_zoom router.
- Added E2E connection test script.
2026-02-03 18:23:07 -05:00
Scott Idem
cc5af1c2e2 feat(api-v3): implement temporary ?key= access pattern and update guide
- Added ?key= query param support for unauthenticated direct downloads.
- Fixed site table column bug (auth_key -> access_key).
- Updated GUIDE__V3_FRONTEND_API.md with temporary auth documentation.
- Ensured valid keys bypass the 403 Machine Auth requirement.
2026-02-03 18:03:03 -05:00
Scott Idem
e29ff23f32 fix(api-v3): ensure subdirectory_path persistence and reload in file actions 2026-02-03 17:53:27 -05:00
Scott Idem
69622dbea6 refactor(core): modularize monolithic routers and methods
- Reduced api_crud.py (1843 -> 143 lines) by extracting V1 registry and logic.
- Reduced hosted_file.py (1596 -> 361 lines) by moving storage and media logic to methods.
- Created lib_media.py for specialized video/image processing.
- Created api_crud_methods.py for legacy template handlers.
- Created legacy_v1.py for the legacy object registry.
- Fixed subdirectory_path bug in Hosted File creation.
- Verified full File Lifecycle via consolidated E2E suite.
2026-02-03 17:53:14 -05:00
Scott Idem
37c84de57b chore(tests): consolidate E2E test suite into standardized primary scripts
- Combined 10+ one-off tests into 4 primary functional suites (Search, Auth, Lifecycle, Vision).
- Archived original scripts to tests/archive/.
- Updated README with the new standardized inventory.
- Applied clean output formatting across the new suite.
2026-02-03 16:50:18 -05:00