Commit Graph

66 Commits

Author SHA1 Message Date
Scott Idem
fc3277086f feat: Add BuildKit pip cache, unlock fastapi pin, mark Locking TODO complete
- Dockerfile: enable BuildKit syntax, use --mount=type=cache for pip to speed up rebuilds
- requirements.txt: relax fastapi==0.115.5 → fastapi>=0.115.5
- TODO: mark Locking task as complete

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 18:53:54 -04:00
Scott Idem
32560d2257 feat: Operational hardening — healthcheck, config refactor, requirements lock
- Add GET /health route (DB + Redis ping, 200/503) with Dockerfile HEALTHCHECK directive
- Replace config.py stub with real pydantic BaseSettings reading directly from env vars;
  remove external config file mount from docker-compose
- Add requirements.lock (pip freeze snapshot for bit-identical builds)
- Untrack config.py globally but allow app/config.py via .gitignore negation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-10 18:44:58 -04:00
Scott Idem
d35f374a45 Renamed the todo file for agents. 2026-03-10 16:26:51 -04:00
Scott Idem
9d89d4c8e4 fix: exclude account_id and virtual fields from archive_content DB writes
- Adds fields_to_exclude_from_db to Archive_Content_Base to prevent SQL errors on non-existent columns.
- Updates documentation for V3 Create/Update patterns and the x-ae-ignore-extra-fields header.
- Propagates account_id_random to hosted file and media processing methods.
2026-02-24 11:30:17 -05:00
Scott Idem
f518d7a433 Saving notes 2026-02-20 19:46:03 -05:00
Scott Idem
48fc97cf46 feat: add priority filtering and sort stability to V3 Lookup System 2026-02-20 17:18:21 -05:00
Scott Idem
6bfbff309a feat: implement V3 Uniform Lookup System with hierarchical overrides and site-based whitelisting 2026-02-20 14:48:50 -05:00
Scott Idem
2b2a2bc00f Saving notes about new lookup tables. 2026-02-19 19:19:59 -05:00
Scott Idem
6a023a82f5 Saving notes about reviewing the SQL VIEWs. 2026-02-19 17:56:08 -05:00
Scott Idem
1db71f85a5 Saving updates to notes. Less reference to the _random. 2026-02-19 16:20:25 -05:00
Scott Idem
17a627a981 feat: Implement Event File Hosted Data Fix and API Guide Update
Address critical data visibility issues for Event Files and enhance frontend documentation.

This commit resolves the persistent problem where top-level hosted file convenience fields
(e.g., , , ) were
returning as  in V3 Event File API responses, even when .

Key changes include:
- Refactored  Pydantic model:
    - Removed redundant  definitions from top-level hosted file convenience fields,
      allowing direct mapping from SQL view columns.
    - Simplified  to focus solely on conditionally loading the nested
       object, as top-level fields are now populated directly by Pydantic
      from the  view.
    - Added comprehensive comments to clarify data flow, Pydantic's behavior, and the
      expected origin of these convenience fields from SQL views.
- Updated :
    - Introduced a new section detailing how to retrieve Event File data, including the
      use of  to get both top-level convenience fields and a nested
       object.
    - Clarified all ID references as random string IDs.
    - Renumbered the troubleshooting section.
- Copied updated guide to .
- Continued ID Vision compliance audit, ensuring consistent handling of random string IDs
  across various core and event models (Account, Address, Contact, DataStore, Event Badge Template).
- Consolidated ID Vision E2E tests and updated related documentation.
- Minor updates to  and
  to support Event File data retrieval with .
2026-02-19 15:22:17 -05:00
Scott Idem
577d784fb8 Serious notes about security updates. 2026-02-13 19:22:33 -05:00
Scott Idem
2266f149f7 security(v3): harden multi-tenant isolation and enhance failure feedback 2026-02-13 18:45:20 -05:00
Scott Idem
61e17f1efa Updating the documentation for things. Hopefully improvements to Gemini CLI. 2026-02-11 17:59:15 -05:00
Scott Idem
8270f7ff7a fix(v3-actions): implement from_hosted_file and harden vision IDs
1. Implemented specialized 'from_hosted_file' action for Event Files.\n2. Fixed ValueError in Pydantic models by removing default/default_factory conflict.\n3. Hardened integer stripping to strictly enforce Vision Standards.\n4. Updated documentation for the new action route.
2026-02-06 16:23:18 -05:00
Scott Idem
4aadb4ec1c Update related to object aliases. 2026-02-06 14:20:19 -05:00
Scott Idem
12d725f468 feat(v3-standardization): implement ID Vision for Event models and update docs 2026-02-05 17:37:52 -05:00
Scott Idem
cc5af1c2e2 feat(api-v3): implement temporary ?key= access pattern and update guide
- Added ?key= query param support for unauthenticated direct downloads.
- Fixed site table column bug (auth_key -> access_key).
- Updated GUIDE__V3_FRONTEND_API.md with temporary auth documentation.
- Ensured valid keys bypass the 403 Machine Auth requirement.
2026-02-03 18:03:03 -05:00
Scott Idem
07609bae9a feat(hosted-file): implement hash-based download action and flexible auth
- Adds GET /v3/action/hosted_file/hash/{sha256}/download for direct content-addressable storage access.
- Updates V3 authentication dependencies to support 'api_key' in the query parameter (alias 'api_key').
- Implements auth_method: 'api_key' for machine-to-machine requests that provide a valid key but no user/account context.
- Updates GUIDE__V3_FRONTEND_API.md with the new endpoint and auth options.
2026-02-03 13:44:07 -05:00
Scott Idem
bcd466edc7 feat(event-file): implement atomic V3 upload action for event files
- Creates api_v3_actions_event_file.py with a specialized /upload endpoint.
- Handles physical storage (hosted_file), generic linking, and event association (event_file) in one request.
- Implements intelligent ID resolution to prevent duplicate event associations for the same physical file.
- Updates documentation in GUIDE__V3_FRONTEND_API.md.
2026-02-03 12:54:17 -05:00
Scott Idem
ea117bf268 feat(hosted-file): implement intelligent ID resolution for V3 download action
- Updates download_file_action to automatically resolve container IDs (like event_file) to the underlying hosted_file.
- Updates GUIDE__V3_FRONTEND_API.md to document the 'ID Vision' standard for downloads.
- Resolves 404 errors observed when frontend passed event_file IDs to the hosted_file download endpoint.
2026-02-03 12:05:04 -05:00
Scott Idem
b89264fe19 docs(websockets): add infrastructure requirements and troubleshooting to guides 2026-01-30 18:11:23 -05:00
Scott Idem
48c3ce76f0 feat(websockets): implement WebSockets V3 with granular Redis Pub/Sub
- Introduced WS_Message_V3 standardized Pydantic model and WS_Manager_V3.
- Implemented /v3/ws/ endpoint with granular Redis routing to solve "noisy neighbor" scaling issues.
- Added presence tracking using Redis Sets for group coordination.
- Comprehensive test suite added (unit and integration) covering models, manager, and routing logic.
- Documentation: Created V3 Frontend WebSocket Guide and Project design spec.
- Updated main Frontend API guide and tests README with new standards.
2026-01-30 14:44:02 -05:00
Scott Idem
5d91c05925 feat(api): standardize V3 error status codes and documentation
- Updated V3 CRUD routers to return 400 Bad Request for database schema
  errors (unknown columns) across all list and search endpoints.
- Fixed serialization issue in nested patch endpoint.
- Overhauled Section 7 of Frontend Integration Guide to document HTTP
  status code mappings for common error categories.
2026-01-29 18:18:04 -05:00
Scott Idem
9dd941eb36 docs: overhaul Data Store cascading guide for frontend agents
- Replaced Section 8 with 'The Hierarchy of Truth' examples.
- Added explicit rules for Vision IDs and automatic JSON parsing.
- Clarified dynamic return behavior based on the 'limit' parameter.
- Cleaned up formatting and synced to agents_sync documentation path.
2026-01-28 17:36:33 -05:00
Scott Idem
0606cecb61 feat(data_store): finalize V3 cascading lookup with limit override
- Update GET /v3/data_store/code/{code} to support 'limit' query parameter.
- Refactor return logic: returns single object if limit=1, otherwise returns a list.
- Clean up formatting in GUIDE__V3_FRONTEND_API.md and sync to agents_sync.
- Finalize unified E2E test script: tests/e2e/test_e2e_v3_data_store_lookup.py.
2026-01-28 17:01:34 -05:00
Scott Idem
fdcc859017 feat(data_store): implement V3 cascading lookup and ID Vision standardization
- Added GET /v3/data_store/code/{code} with hierarchical context-aware fallback.
- Implemented ID Vision standard in Data_Store_Base (string IDs, internal int exclusion).
- Enhanced Data_Store_Base robustness to handle stringified 'NULL' values from the database.
- Fixed legacy router bugs by removing undefined parameters (inc_event_cfg, inc_event_location).
- Corrected type hints and resolved UnboundLocalError in data_store methods.
- Updated Frontend Integration Guide with Section 8: Data Store V3.
- Added unified E2E test script: tests/e2e/test_e2e_v3_data_store_lookup.py.
2026-01-28 16:51:48 -05:00
Scott Idem
9c0aae9a6d docs: formalize Aether V4 Architecture Standards
- Create ARCH__V4_CORE_STANDARDS.md to document V4 identity, lifecycle, and search patterns.
- Standardize default_qry_str in Event_Badge_Base.
- Clean up obsolete .snapshot files.
- Update README baseline to v3.0.99.
2026-01-28 14:36:41 -05:00
Scott Idem
0f8c5dc825 Documentation Archive: Retire static SQL snapshots and legacy logs
- Archived 'aether_sql_tables' and 'legacy_router_logs' to documentation/archive/.
- Static schema documentation is now superseded by the 'ae_obj_info' MCP tool and shared agents_sync metadata.
2026-01-28 12:33:20 -05:00
Scott Idem
860cf80a4e Documentation Standardisation & Unit Test Stabilization
- Overhauled README.md to serve as a unified system index and WIP tracker.
- Standardized documentation filenames (ARCH__, GUIDE__, PLAN__) for better discoverability.
- Archived completed project plans and scopes.
- Fixed regressions in unit tests (errors, models, email) caused by V3 architectural updates.
- Ensured unit tests remain non-destructive and environment-independent via mocking.
2026-01-28 12:15:01 -05:00
Scott Idem
a754525a59 Some quick documentation for old legacy routes. 2026-01-26 17:36:32 -05:00
Scott Idem
061c153061 Saving updated notes. 2026-01-22 19:05:22 -05:00
Scott Idem
1837b442cf V3 Migration Phase 1: Stabilize Hosted File models, IDs, and whitelisting. Added comprehensive verification tests. 2026-01-22 18:30:34 -05:00
Scott Idem
43ac62b561 feat(auth): consolidate and secure V3 authentication flow
- Re-apply safe guest auth and passcode-to-JWT endpoint
- Consolidate AccountContext with token_payload and role flags
- Restore documentation for new guest flows and public read whitelists
- Fix 403 error in get_obj_li by allowing optional account context
2026-01-20 18:42:43 -05:00
Scott Idem
579772977b Docs: Final formatting and cleanup of V3 Frontend Guide 2026-01-19 17:08:48 -05:00
Scott Idem
ede4cfabf0 Docs: Document Structured Error Handling (Rich Bubbling) in V3 Guide 2026-01-19 17:04:32 -05:00
Scott Idem
4d439e63a9 Docs: Update V3 Frontend Guide to reflect ID Vision and Permissive Mode 2026-01-19 16:02:18 -05:00
Scott Idem
cad0d2e867 Security: Enforce mandatory API Keys for V3, fix search logic, and update frontend guide 2026-01-19 14:11:13 -05:00
Scott Idem
9e0f94964e Bug fix for trying to use the wrong hosted file and tmp paths or src. Also saving documentation for the new MCP AE DB field manager. 2026-01-16 14:40:12 -05:00
Scott Idem
31fd384704 Docs: Consolidate admin documentation and migrate reference data
- Created LOCAL_DEVELOPMENT_GUIDE.md and DEPLOYMENT_GUIDE_MANUAL.md from legacy txt files.
- Migrated country/time_zone data and requirements.txt to documentation/reference_data/.
- Removed redundant admin/documentation/ and admin/data_files/ directories.
- Enhanced app/lib_schema_v3.py to explicitly capture 'required' fields from DB 'NOT NULL' constraint.
- Added verification tests for schema logic and standalone DB connectivity.
2026-01-16 10:06:51 -05:00
Scott Idem
68862e4545 Docs: Consolidate V3 standards and cleanup documentation directory 2026-01-15 17:53:06 -05:00
Scott Idem
28d5843d52 Saving current notes. About to reorganize the documentation directory. 2026-01-15 17:48:44 -05:00
Scott Idem
ed3dda6cf5 Bug fixes for SQL testing 2026-01-13 13:57:18 -05:00
Scott Idem
5ce193d474 Saving more note updates 2026-01-09 17:10:25 -05:00
Scott Idem
3885cc6aba Refactor V3 CRUD: Extract schema introspection logic.
- Created app/lib_schema_v3.py to isolate database and Pydantic model introspection.
- Updated app/routers/api_crud_v3.py to use get_object_schema_info(), completing the modularization.
- Finalized refactoring plan documentation in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:29:10 -05:00
Scott Idem
812181acb5 Refactor V3 CRUD: Extract nested child routes into separate router.
- Created app/routers/api_crud_v3_nested.py to handle all parent-child relational routes.
- Updated app/routers/api_crud_v3.py to include the nested router, significantly reducing file size.
- Documented Phase 2 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:23:14 -05:00
Scott Idem
8459b57e1b Refactor V3 CRUD: Extract helper functions and unify sanitization logic.
- Created app/lib_api_crud_v3.py to house core security, filtering, and sanitization logic.
- Implemented reusable sanitize_payload() to generically strip virtual lookup fields (*_id_random) and view-only fields (fields_to_exclude_from_db).
- Updated app/routers/api_crud_v3.py to use the new library and consolidated sanitization across all Create/Update endpoints.
- Documented Phase 1 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:16:44 -05:00
Scott Idem
59d5b81da0 Arch: Finalize V3 Auth modularization and Unified Agent spec.
- Integrated zero-dependency Auth models and dependencies_v3.py.
- Successfully resolved circular dependency boot loops.
- Verified site_domain search exception via verify_v3_exceptions.py.
- Refined Unified Agent Architecture with Storage Layer and API-driven access details.
- Updated project roadmap and milestones in GEMINI.md.
2026-01-07 19:07:21 -05:00
Scott Idem
90c6b914fa Docs: Update Unified Agent Architecture and Platform Roadmap. 2026-01-07 18:53:04 -05:00
Scott Idem
734576817c Refactor: Modularize lib_general.py by extracting core functionalities.
- Extracted Email functions to app/lib_email.py.
- Extracted Export functions to app/lib_export.py.
- Extracted JWT utilities to app/lib_jwt.py.
- Extracted Hash utilities to app/lib_hash.py.
- Updated app/lib_general.py to import from these new modules for backward compatibility.
- Updated V3 Frontend API Guide with latest security and site_domain exception details.
2026-01-07 17:41:04 -05:00