Commit Graph

1087 Commits

Author SHA1 Message Date
Scott Idem
4b86432381 Enhance V3 CRUD: Implement Error Bubbling and Dry-Run Validation.
- Updated app/db_sql.py to capture SQL exceptions in thread-local storage for later retrieval.
- Implemented format_db_error() in app/lib_api_crud_v3.py to clean up raw MariaDB error strings.
- Added POST /v3/crud/{obj_type}/validate endpoint for dry-run payload validation.
- Updated main and nested routers to bubble up validation and database errors into the response 'meta.details' field.
- Added tests/test_v3_error_bubbling.py to verify formatting logic.
2026-01-09 16:57:54 -05:00
Scott Idem
3885cc6aba Refactor V3 CRUD: Extract schema introspection logic.
- Created app/lib_schema_v3.py to isolate database and Pydantic model introspection.
- Updated app/routers/api_crud_v3.py to use get_object_schema_info(), completing the modularization.
- Finalized refactoring plan documentation in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:29:10 -05:00
Scott Idem
812181acb5 Refactor V3 CRUD: Extract nested child routes into separate router.
- Created app/routers/api_crud_v3_nested.py to handle all parent-child relational routes.
- Updated app/routers/api_crud_v3.py to include the nested router, significantly reducing file size.
- Documented Phase 2 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:23:14 -05:00
Scott Idem
8459b57e1b Refactor V3 CRUD: Extract helper functions and unify sanitization logic.
- Created app/lib_api_crud_v3.py to house core security, filtering, and sanitization logic.
- Implemented reusable sanitize_payload() to generically strip virtual lookup fields (*_id_random) and view-only fields (fields_to_exclude_from_db).
- Updated app/routers/api_crud_v3.py to use the new library and consolidated sanitization across all Create/Update endpoints.
- Documented Phase 1 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:16:44 -05:00
Scott Idem
2ff211f2c2 Update API documentation and finalize model validators/mappings.
- Added comprehensive docstrings to api_crud_v3.py explaining multi-tenancy, sanitization, and soft-delete logic.
- Finalized Address and Contact models/mappings with correct validators and field maps.
- Consolidated test suite in tests/ directory.
2026-01-09 15:52:00 -05:00
Scott Idem
8dc37f274f Just saving changes 2026-01-09 15:40:46 -05:00
Scott Idem
4c83e02c4a Update V3 CRUD router and object definitions.
- Added 'external_person_id' to Post searchable fields.
- Updated api_crud_v3.py to respect 'fields_to_exclude_from_db' model attribute.
- Cleaned up old verification scripts (moved to tests/).
2026-01-09 15:36:50 -05:00
Scott Idem
1c0922ace2 Enhance API robustness: Add model validators, view-field filtering, and test suite.
- Added validators to Person_Base, Journal_Base, Journal_Entry_Base, Address_Base, and Contact_Base to handle null values and unsafe lookups.
- Implemented 'fields_to_exclude_from_db' ClassVar in Journal models to prevent view-only fields from causing DB errors.
- Updated Contact object map to align with DB schema.
- Added comprehensive test suite in 'tests/' directory (model validation, filtering logic).
- Updated GEMINI.md with progress.
2026-01-09 15:36:28 -05:00
Scott Idem
29b4d5ae4b Fix Person creation issues and enhance V3 CRUD robustness.
- Added Pydantic validators to Person_Base to handle null values for given_name and allow_auth_key, ensuring database NOT NULL constraints are met.
- Updated api_crud_v3.py (POST and PATCH) to filter out virtual *_id_random fields from data payloads before database operations to prevent "Unknown column" errors.
- Updated GEMINI.md with session progress.
2026-01-09 14:30:45 -05:00
Scott Idem
d32304c50a Saving notes 2026-01-08 19:27:09 -05:00
Scott Idem
46a3998fe0 Saving Gemini notes 2026-01-08 15:40:55 -05:00
Scott Idem
765949fdfd Cleanup: Remove temporary scripts from local root. 2026-01-08 12:56:34 -05:00
Scott Idem
1bcc6dae3f Cleanup: Remove ae_object_info.py from local root after relocation. 2026-01-08 12:56:01 -05:00
Scott Idem
9ee2ed444b Docs: Update GEMINI.md with MCP tool registration issue. 2026-01-08 12:55:55 -05:00
Scott Idem
802c75bad9 V3: Standardize Primary AE Objects and Synchronize Search Whitelists.
- Synchronized searchable_fields (V3 whitelists) across all Primary and Active AE objects (Identity, People, Events, Journals, Posts, Archives, Business).
- Standardized Pydantic models for core objects to include the 10 common fields (id, id_random, enable, hide, priority, sort, group, notes, created_on, updated_on).
- Fixed field aliases and uncommented valid database columns in User_Base and Organization_Base.
- Pruned non-existent fields from searchable lists in legacy or config-specific definitions (account_cfg, user_role, log_client_viewing).
- Added system discovery and validation tools:
    - ae_object_info.py: AE object status and metadata browser.
    - export_all_interfaces.py: E2E TypeScript interface generator.
    - Verification scripts for searchable field consistency.
- Updated Jan 8 milestone progress and platform roadmap in GEMINI.md.
2026-01-08 12:24:34 -05:00
Scott Idem
59d5b81da0 Arch: Finalize V3 Auth modularization and Unified Agent spec.
- Integrated zero-dependency Auth models and dependencies_v3.py.
- Successfully resolved circular dependency boot loops.
- Verified site_domain search exception via verify_v3_exceptions.py.
- Refined Unified Agent Architecture with Storage Layer and API-driven access details.
- Updated project roadmap and milestones in GEMINI.md.
2026-01-07 19:07:21 -05:00
Scott Idem
90c6b914fa Docs: Update Unified Agent Architecture and Platform Roadmap. 2026-01-07 18:53:04 -05:00
Scott Idem
d4805ebb09 Cleanup: Remove unused imports in lib_general.py after extraction. 2026-01-07 17:43:43 -05:00
Scott Idem
734576817c Refactor: Modularize lib_general.py by extracting core functionalities.
- Extracted Email functions to app/lib_email.py.
- Extracted Export functions to app/lib_export.py.
- Extracted JWT utilities to app/lib_jwt.py.
- Extracted Hash utilities to app/lib_hash.py.
- Updated app/lib_general.py to import from these new modules for backward compatibility.
- Updated V3 Frontend API Guide with latest security and site_domain exception details.
2026-01-07 17:41:04 -05:00
Scott Idem
d61dd0f00e Restored V3 search and implemented site_domain exception via dependency isolation.
- Implemented 'Isolation Mode' in api_crud_v3.py to bypass circular dependencies.
- Locally defined AccountContext and auth dependencies to ensure stable boot.
- Added site_domain lookup exception for guest users in search endpoint.
- Maintained agent_bridge disablement in main.py for stability.
2026-01-07 17:08:52 -05:00
Scott Idem
6937f9dca4 Saving these changes in a working state. Just in case. 2026-01-07 16:54:56 -05:00
Scott Idem
caf2868d02 Saving things while they work again!!! Still working on adding a special exception or something for site domain search. 2026-01-07 16:25:18 -05:00
Scott Idem
cf96d93246 fix: import SearchFilter in api_crud_v3.py to resolve NameError 2026-01-07 14:25:58 -05:00
Scott Idem
6d13b952c4 Implement V3 API security hardening and multi-tenant data isolation
- Enhanced AuthContext with role-aware fields (administrator, manager, super).
- Implemented deferred database lookups for user roles in get_v3_auth_context.
- Added global account isolation in api_crud_v3.py using check_account_access and apply_forced_account_filter.
- Hardened all V3 CRUD endpoints (GET, POST, PATCH, DELETE) and nested routes with ownership verification.
- Enforced forced account filtering at the SQL level for Listing and Searching.
- Updated documentation with details on the new security and data isolation architecture.
2026-01-07 13:34:38 -05:00
Scott Idem
270712f905 Another quick snapshot in case something breaks again. There are issues with this new agent bridge and the psutil and Gemini CLI. Not sure, but it causes problems. 2026-01-07 13:10:16 -05:00
Scott Idem
7fb2f00846 Things are currently working. At least 90% sure they are. 2026-01-07 12:24:52 -05:00
Scott Idem
c47ae47a2f Add agent_bridge.py administrative endpoints and mcp_docker_explorer.py script
- Implemented /status, /system/usage, /logs, /logs/list, /processes, and /container/metadata in agent_bridge.py.
- Added mcp_docker_explorer.py for Docker MCP integration testing.
- Enhanced administrative access checks in agent_bridge.py.
2026-01-07 12:01:48 -05:00
Scott Idem
75b771f87c feat: add 'archive_on' to searchable_fields for posts and update agent bridge auth logic 2026-01-07 11:07:45 -05:00
Scott Idem
ec4656eca9 Bug fix to make lookup_id_random_pop work again. 2026-01-07 10:02:57 -05:00
Scott Idem
13620a63d0 Save notes and documentation updates 2026-01-06 19:36:58 -05:00
Scott Idem
836ed97d07 feat(agent): implement Agent Bridge for secure diagnostics 2026-01-06 19:00:03 -05:00
Scott Idem
6470af0a01 feat(v3): populate searchable_fields for all remaining object definitions 2026-01-06 18:18:39 -05:00
Scott Idem
c33ae332e9 refactor(sql): clean up db_sql.py by removing commented-out code and consolidating logic 2026-01-06 18:12:51 -05:00
Scott Idem
a6ec6d1b2b Saving our work just in case. 2026-01-06 18:06:45 -05:00
Scott Idem
55033d0749 feat(events): add conference to searchable fields and update progress 2026-01-06 18:03:57 -05:00
Scott Idem
868a0060dc refactor(sql): complete modularization of search builders and ID resolution 2026-01-06 17:58:34 -05:00
Scott Idem
56fe7ed953 refactor(sql): modularize Redis and ID lookup functions 2026-01-06 17:32:22 -05:00
Scott Idem
a6a5162385 refactor(sql): modularize status and where query builders 2026-01-06 17:24:47 -05:00
Scott Idem
b5e874bd99 refactor(sql): modularize basic search query builders 2026-01-06 17:22:10 -05:00
Scott Idem
d584457997 fix(legacy): resolve 422 error on site domain lookup and enhance V3 filtering 2026-01-06 16:29:09 -05:00
Scott Idem
459bd89198 feat(v3): implement schema discovery endpoint 2026-01-06 16:03:54 -05:00
Scott Idem
45f6303219 feat(v3): robust search wildcards, smart status filtering, and fixed ID population 2026-01-06 15:54:31 -05:00
Scott Idem
a42f32acf4 Added more documentation. Improved CRUD V3 endpoints and better responses. 2026-01-06 13:52:05 -05:00
Scott Idem
9c06b07665 Saving changes now that most things have been migrated to CRUD V3 and appear to be working. This still needs testing though. 2026-01-06 13:11:03 -05:00
Scott Idem
552ca31603 Adding more searchable_fields for security. Broke up large files to make them easier to manage. 2026-01-06 11:14:37 -05:00
Scott Idem
b8a417a5d7 Key Accomplishments:
1. Badge Model Updates: Added print_count, print_first_datetime, and print_last_datetime to the
      Event_Badge_Basic_Base model.
   2. Soft Delete in V3 CRUD: Implemented a method query parameter (delete, hide, disable) for the DELETE
      endpoints in api_crud_v3.py.
   3. Security Hardening: Populated searchable_fields allowlists for all objects across the V3 CRUD
      definitions (core.py, events.py, orders.py, cms.py, lookups.py, membership.py, other.py).
   4. Shared Documentation: Created /home/scott/agents_sync/Aether/api_v3.md to coordinate these V3 API
      features with the Svelte agent and other tools.
2026-01-05 20:05:33 -05:00
Scott Idem
314a031dd1 Now with some soft delete options for safer operations. 2026-01-05 19:49:28 -05:00
Scott Idem
3790983b5e Quick update to include some more fields. 2026-01-05 19:24:17 -05:00
Scott Idem
872279de0b One last round of testing and documentation updates. 2026-01-02 21:03:32 -05:00
Scott Idem
f5ab2118ad Fix: Enhance V3 Search with 'contains', 'startswith', 'endswith' operators and improve error reporting. 2026-01-02 20:42:19 -05:00