Commit Graph

742 Commits

Author SHA1 Message Date
Scott Idem
9362938ffe feat(api-v3): add advanced search and view support to nested CRUD router 2026-02-03 14:10:41 -05:00
Scott Idem
07609bae9a feat(hosted-file): implement hash-based download action and flexible auth
- Adds GET /v3/action/hosted_file/hash/{sha256}/download for direct content-addressable storage access.
- Updates V3 authentication dependencies to support 'api_key' in the query parameter (alias 'api_key').
- Implements auth_method: 'api_key' for machine-to-machine requests that provide a valid key but no user/account context.
- Updates GUIDE__V3_FRONTEND_API.md with the new endpoint and auth options.
2026-02-03 13:44:07 -05:00
Scott Idem
bcd466edc7 feat(event-file): implement atomic V3 upload action for event files
- Creates api_v3_actions_event_file.py with a specialized /upload endpoint.
- Handles physical storage (hosted_file), generic linking, and event association (event_file) in one request.
- Implements intelligent ID resolution to prevent duplicate event associations for the same physical file.
- Updates documentation in GUIDE__V3_FRONTEND_API.md.
2026-02-03 12:54:17 -05:00
Scott Idem
ea117bf268 feat(hosted-file): implement intelligent ID resolution for V3 download action
- Updates download_file_action to automatically resolve container IDs (like event_file) to the underlying hosted_file.
- Updates GUIDE__V3_FRONTEND_API.md to document the 'ID Vision' standard for downloads.
- Resolves 404 errors observed when frontend passed event_file IDs to the hosted_file download endpoint.
2026-02-03 12:05:04 -05:00
Scott Idem
48c3ce76f0 feat(websockets): implement WebSockets V3 with granular Redis Pub/Sub
- Introduced WS_Message_V3 standardized Pydantic model and WS_Manager_V3.
- Implemented /v3/ws/ endpoint with granular Redis routing to solve "noisy neighbor" scaling issues.
- Added presence tracking using Redis Sets for group coordination.
- Comprehensive test suite added (unit and integration) covering models, manager, and routing logic.
- Documentation: Created V3 Frontend WebSocket Guide and Project design spec.
- Updated main Frontend API guide and tests README with new standards.
2026-01-30 14:44:02 -05:00
Scott Idem
cd19c738f1 Commented out deprecated routes. Things seem to be working... 2026-01-29 18:29:00 -05:00
Scott Idem
5d91c05925 feat(api): standardize V3 error status codes and documentation
- Updated V3 CRUD routers to return 400 Bad Request for database schema
  errors (unknown columns) across all list and search endpoints.
- Fixed serialization issue in nested patch endpoint.
- Overhauled Section 7 of Frontend Integration Guide to document HTTP
  status code mappings for common error categories.
2026-01-29 18:18:04 -05:00
Scott Idem
b862d59e65 feat(api): return 400 for database schema errors in V3 search
- Updated api_crud_v3 and api_crud_v3_nested to detect 'database_schema'
  errors (like Unknown Column) and return a 400 Bad Request instead of
  a generic 500 Internal Server Error.
- Added missing error handling for sql_select failure in get_child_obj_li.
2026-01-29 18:08:03 -05:00
Scott Idem
0606cecb61 feat(data_store): finalize V3 cascading lookup with limit override
- Update GET /v3/data_store/code/{code} to support 'limit' query parameter.
- Refactor return logic: returns single object if limit=1, otherwise returns a list.
- Clean up formatting in GUIDE__V3_FRONTEND_API.md and sync to agents_sync.
- Finalize unified E2E test script: tests/e2e/test_e2e_v3_data_store_lookup.py.
2026-01-28 17:01:34 -05:00
Scott Idem
fdcc859017 feat(data_store): implement V3 cascading lookup and ID Vision standardization
- Added GET /v3/data_store/code/{code} with hierarchical context-aware fallback.
- Implemented ID Vision standard in Data_Store_Base (string IDs, internal int exclusion).
- Enhanced Data_Store_Base robustness to handle stringified 'NULL' values from the database.
- Fixed legacy router bugs by removing undefined parameters (inc_event_cfg, inc_event_location).
- Corrected type hints and resolved UnboundLocalError in data_store methods.
- Updated Frontend Integration Guide with Section 8: Data Store V3.
- Added unified E2E test script: tests/e2e/test_e2e_v3_data_store_lookup.py.
2026-01-28 16:51:48 -05:00
Scott Idem
3eaf176b05 Router Registry Cleanup: Archive unreferenced legacy routes
- Moved 29 unreferenced legacy router files to app/routers/archive/
- Restored and registered 'websockets' router in registry.py (exempted from deprecation)
- Cleaned up registry imports and verified application compilation.
2026-01-28 11:47:18 -05:00
Scott Idem
a97e80baab Implement Deprecation Warning System and Router Registry Cleanup
- Added DeprecationParams dependency to log warnings when legacy routes are accessed.
- Updated setup_routers to apply deprecation warnings to non-V3 legacy endpoints.
- Exempted core infrastructure, special routers, and routers currently in use from deprecation warnings.
- Cleaned up 24 unused router imports from the registry.
2026-01-28 10:55:03 -05:00
Scott Idem
60345dd21e V3 Migration Phase 2-4: Implementation of specialized Binary Actions (Upload, Stream, Delete) and Orphan management logic. Full E2E coverage. 2026-01-22 18:51:26 -05:00
Scott Idem
df0ce7f910 Saving fixes to the hosted file delete function. 2026-01-22 17:31:29 -05:00
Scott Idem
1e6b9d1c18 Bug fixes for uploading the files. I though the changes being made where not supposed to break legacy endpoints. Not sure what happened. Either way things are almost back to normal. 2026-01-22 16:49:03 -05:00
Scott Idem
48d9e38c39 Bug fixes related to file uploads. Fixing id_random int vs str confusion. For account and for hosted_file. 2026-01-22 16:01:23 -05:00
Scott Idem
e8322b4b1a fix(db): prevent silent failures in sql_select
- Modify sql_select to return False on database exceptions instead of an empty result
- Update V3 Search endpoint to catch False results and return a 500 Internal Server Error
- Enhances error visibility for frontend developers and prevents misleading 200 OK responses during SQL errors
2026-01-21 19:49:58 -05:00
Scott Idem
6ca79e9a02 chore(api): stabilize SQL core and enhance searchability
- Refactor SQL CRUD to use engine.connect() context managers for thread safety
- Optimize connection pooling in lib_sql_core
- Clean up app/routers/api.py to fix duplicate definitions and OpenAPI KeyError
- Add 'default_qry_str' to searchable_fields for Event, Session, Presentation, Presenter, Badge, and Journal
- Add 'event_location_name' to searchable_fields for Event Session
- Verified 20/20 E2E success via repro_intermittent_errors.py
2026-01-21 15:23:04 -05:00
Scott Idem
45ca81a3e3 Removing debugging. Changing it to INFO in many locations. 2026-01-20 19:23:30 -05:00
Scott Idem
c795f42290 fix(auth): handle list response from sql_select in dependencies_v3
- Check if api_key_results is a list before calling .get()
- Prevents 500 AttributeError on machine auth verification
2026-01-20 18:52:59 -05:00
Scott Idem
43ac62b561 feat(auth): consolidate and secure V3 authentication flow
- Re-apply safe guest auth and passcode-to-JWT endpoint
- Consolidate AccountContext with token_payload and role flags
- Restore documentation for new guest flows and public read whitelists
- Fix 403 error in get_obj_li by allowing optional account context
2026-01-20 18:42:43 -05:00
Scott Idem
d4e46a4a97 feat(auth): implement site-based passcode-to-JWT endpoint
- Add POST /api/authenticate_passcode to verify site access codes
- Refactor sign_jwt to support arbitrary role flags (super, admin, etc.)
- Update dependencies_v3 to extract role flags from JWT payloads
- Add E2E test for passcode auth verification
2026-01-20 17:51:54 -05:00
Scott Idem
dc7732ab5f feat(security): implement safe guest auth flow and harden request_jwt
- Patched request_jwt to strip privileged IDs when signing with public keys
- Updated AccountContext and V3 dependencies to preserve JWT payloads for guests
- Whitelisted Archive, Post, Event, and other core objects for public read access
- Added 'default_qry_str' to Event searchable fields
- Added test_e2e_jwt_guest_auth.py for security verification
2026-01-20 14:56:56 -05:00
Scott Idem
ab8afb72d2 Fix: Make forced account filtering schema-aware to prevent crashes on specialized views 2026-01-19 17:17:34 -05:00
Scott Idem
eeb19647f5 Error Bubbling: Implement machine-readable rich error objects for CRUD operations 2026-01-19 17:01:58 -05:00
Scott Idem
19e64135ca Permissive Update: Implement x-ae-ignore-extra-fields header support for nested routes 2026-01-19 16:48:48 -05:00
Scott Idem
2dbf47d874 Security: Implement JWT verification in V3 and prevent numeric ID signing 2026-01-19 14:41:20 -05:00
Scott Idem
cad0d2e867 Security: Enforce mandatory API Keys for V3, fix search logic, and update frontend guide 2026-01-19 14:11:13 -05:00
Scott Idem
2227432970 Refactor: Modularize middleware and router registration in app/main.py 2026-01-15 16:36:19 -05:00
Scott Idem
34a752d455 feat(api-v3): implement permissive updates, automatic ID resolution, and structured error reporting
- Added 'x-ae-ignore-extra-fields' header to support stripping unknown fields in POST/PATCH.
- Added automatic resolution of '*_id_random' strings to integer IDs in 'sanitize_payload'.
- Refactored 'post_obj' to return structured (field -> message) validation errors in 'meta.details'.
- Updated 'mk_resp' to support non-string 'details' in response metadata.
- Added 'tests/verify_feedback_fixes.py' to validate logic changes.

Ref: V3 API Refinement Feedback from mcp_agent.
2026-01-14 19:11:56 -05:00
Scott Idem
19a9890dd9 Better bug fix for working SQL test. 2026-01-13 14:21:56 -05:00
Scott Idem
f9a51e243f More clean up of old routes 2026-01-13 14:15:17 -05:00
Scott Idem
6346d4ccd6 Commenting out a bunch of old routes. Hopefully none of them are used by anything still out there.... 2026-01-13 14:12:07 -05:00
Scott Idem
ed3dda6cf5 Bug fixes for SQL testing 2026-01-13 13:57:18 -05:00
Scott Idem
4b86432381 Enhance V3 CRUD: Implement Error Bubbling and Dry-Run Validation.
- Updated app/db_sql.py to capture SQL exceptions in thread-local storage for later retrieval.
- Implemented format_db_error() in app/lib_api_crud_v3.py to clean up raw MariaDB error strings.
- Added POST /v3/crud/{obj_type}/validate endpoint for dry-run payload validation.
- Updated main and nested routers to bubble up validation and database errors into the response 'meta.details' field.
- Added tests/test_v3_error_bubbling.py to verify formatting logic.
2026-01-09 16:57:54 -05:00
Scott Idem
3885cc6aba Refactor V3 CRUD: Extract schema introspection logic.
- Created app/lib_schema_v3.py to isolate database and Pydantic model introspection.
- Updated app/routers/api_crud_v3.py to use get_object_schema_info(), completing the modularization.
- Finalized refactoring plan documentation in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:29:10 -05:00
Scott Idem
812181acb5 Refactor V3 CRUD: Extract nested child routes into separate router.
- Created app/routers/api_crud_v3_nested.py to handle all parent-child relational routes.
- Updated app/routers/api_crud_v3.py to include the nested router, significantly reducing file size.
- Documented Phase 2 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:23:14 -05:00
Scott Idem
8459b57e1b Refactor V3 CRUD: Extract helper functions and unify sanitization logic.
- Created app/lib_api_crud_v3.py to house core security, filtering, and sanitization logic.
- Implemented reusable sanitize_payload() to generically strip virtual lookup fields (*_id_random) and view-only fields (fields_to_exclude_from_db).
- Updated app/routers/api_crud_v3.py to use the new library and consolidated sanitization across all Create/Update endpoints.
- Documented Phase 1 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:16:44 -05:00
Scott Idem
2ff211f2c2 Update API documentation and finalize model validators/mappings.
- Added comprehensive docstrings to api_crud_v3.py explaining multi-tenancy, sanitization, and soft-delete logic.
- Finalized Address and Contact models/mappings with correct validators and field maps.
- Consolidated test suite in tests/ directory.
2026-01-09 15:52:00 -05:00
Scott Idem
4c83e02c4a Update V3 CRUD router and object definitions.
- Added 'external_person_id' to Post searchable fields.
- Updated api_crud_v3.py to respect 'fields_to_exclude_from_db' model attribute.
- Cleaned up old verification scripts (moved to tests/).
2026-01-09 15:36:50 -05:00
Scott Idem
29b4d5ae4b Fix Person creation issues and enhance V3 CRUD robustness.
- Added Pydantic validators to Person_Base to handle null values for given_name and allow_auth_key, ensuring database NOT NULL constraints are met.
- Updated api_crud_v3.py (POST and PATCH) to filter out virtual *_id_random fields from data payloads before database operations to prevent "Unknown column" errors.
- Updated GEMINI.md with session progress.
2026-01-09 14:30:45 -05:00
Scott Idem
59d5b81da0 Arch: Finalize V3 Auth modularization and Unified Agent spec.
- Integrated zero-dependency Auth models and dependencies_v3.py.
- Successfully resolved circular dependency boot loops.
- Verified site_domain search exception via verify_v3_exceptions.py.
- Refined Unified Agent Architecture with Storage Layer and API-driven access details.
- Updated project roadmap and milestones in GEMINI.md.
2026-01-07 19:07:21 -05:00
Scott Idem
d61dd0f00e Restored V3 search and implemented site_domain exception via dependency isolation.
- Implemented 'Isolation Mode' in api_crud_v3.py to bypass circular dependencies.
- Locally defined AccountContext and auth dependencies to ensure stable boot.
- Added site_domain lookup exception for guest users in search endpoint.
- Maintained agent_bridge disablement in main.py for stability.
2026-01-07 17:08:52 -05:00
Scott Idem
6937f9dca4 Saving these changes in a working state. Just in case. 2026-01-07 16:54:56 -05:00
Scott Idem
caf2868d02 Saving things while they work again!!! Still working on adding a special exception or something for site domain search. 2026-01-07 16:25:18 -05:00
Scott Idem
cf96d93246 fix: import SearchFilter in api_crud_v3.py to resolve NameError 2026-01-07 14:25:58 -05:00
Scott Idem
6d13b952c4 Implement V3 API security hardening and multi-tenant data isolation
- Enhanced AuthContext with role-aware fields (administrator, manager, super).
- Implemented deferred database lookups for user roles in get_v3_auth_context.
- Added global account isolation in api_crud_v3.py using check_account_access and apply_forced_account_filter.
- Hardened all V3 CRUD endpoints (GET, POST, PATCH, DELETE) and nested routes with ownership verification.
- Enforced forced account filtering at the SQL level for Listing and Searching.
- Updated documentation with details on the new security and data isolation architecture.
2026-01-07 13:34:38 -05:00
Scott Idem
270712f905 Another quick snapshot in case something breaks again. There are issues with this new agent bridge and the psutil and Gemini CLI. Not sure, but it causes problems. 2026-01-07 13:10:16 -05:00
Scott Idem
c47ae47a2f Add agent_bridge.py administrative endpoints and mcp_docker_explorer.py script
- Implemented /status, /system/usage, /logs, /logs/list, /processes, and /container/metadata in agent_bridge.py.
- Added mcp_docker_explorer.py for Docker MCP integration testing.
- Enhanced administrative access checks in agent_bridge.py.
2026-01-07 12:01:48 -05:00
Scott Idem
75b771f87c feat: add 'archive_on' to searchable_fields for posts and update agent bridge auth logic 2026-01-07 11:07:45 -05:00