Commit Graph

1098 Commits

Author SHA1 Message Date
Scott Idem
f2420b958d Bug fix for Event Device related fields. the ID needed to be searchable.
Quick removal of the password from the log output
2026-01-26 11:56:31 -05:00
Scott Idem
60345dd21e V3 Migration Phase 2-4: Implementation of specialized Binary Actions (Upload, Stream, Delete) and Orphan management logic. Full E2E coverage. 2026-01-22 18:51:26 -05:00
Scott Idem
1837b442cf V3 Migration Phase 1: Stabilize Hosted File models, IDs, and whitelisting. Added comprehensive verification tests. 2026-01-22 18:30:34 -05:00
Scott Idem
df0ce7f910 Saving fixes to the hosted file delete function. 2026-01-22 17:31:29 -05:00
Scott Idem
1e6b9d1c18 Bug fixes for uploading the files. I though the changes being made where not supposed to break legacy endpoints. Not sure what happened. Either way things are almost back to normal. 2026-01-22 16:49:03 -05:00
Scott Idem
48d9e38c39 Bug fixes related to file uploads. Fixing id_random int vs str confusion. For account and for hosted_file. 2026-01-22 16:01:23 -05:00
Scott Idem
329ea51487 Added a sort of alias for the enabled options. Really disabled should be the alias for "not_enabled". "not_enabled" is what is used on the frontend and has been. 2026-01-21 20:25:11 -05:00
Scott Idem
e8322b4b1a fix(db): prevent silent failures in sql_select
- Modify sql_select to return False on database exceptions instead of an empty result
- Update V3 Search endpoint to catch False results and return a 500 Internal Server Error
- Enhances error visibility for frontend developers and prevents misleading 200 OK responses during SQL errors
2026-01-21 19:49:58 -05:00
Scott Idem
bdd1bd2ba2 feat(search): enhance V3 ID Vision mapping and searchable fields
- Update lib_sql_search.py to include comprehensive 'vision_fields' mapping for most core objects
- Ensure Vision Mapping only triggers for non-integer values to support backend filters
- Add clean ID names (e.g., 'event_id', 'account_id') to searchable_fields whitelists in Events, Badges, and Journal object definitions
- Resolve Concatenation typo in vision_fields list
- Improve searchability for Journal Entries by adding 'default_qry_str'
2026-01-21 19:21:52 -05:00
Scott Idem
6ca79e9a02 chore(api): stabilize SQL core and enhance searchability
- Refactor SQL CRUD to use engine.connect() context managers for thread safety
- Optimize connection pooling in lib_sql_core
- Clean up app/routers/api.py to fix duplicate definitions and OpenAPI KeyError
- Add 'default_qry_str' to searchable_fields for Event, Session, Presentation, Presenter, Badge, and Journal
- Add 'event_location_name' to searchable_fields for Event Session
- Verified 20/20 E2E success via repro_intermittent_errors.py
2026-01-21 15:23:04 -05:00
Scott Idem
89bf87cb62 fix(db): stabilize connection refreshing and prevent ResourceClosedError
- Update sql_connect to refresh global db object via reconnect_db
- Add returns_rows check and safe fetch block in sql_select
- Prevents 500 errors during transient database connection issues
2026-01-21 12:49:47 -05:00
Scott Idem
b2ee1f2760 Less debug. Also why was this using the print() function? It should have been using the normal log.info() or whatever. 2026-01-20 19:27:16 -05:00
Scott Idem
45ca81a3e3 Removing debugging. Changing it to INFO in many locations. 2026-01-20 19:23:30 -05:00
Scott Idem
c795f42290 fix(auth): handle list response from sql_select in dependencies_v3
- Check if api_key_results is a list before calling .get()
- Prevents 500 AttributeError on machine auth verification
2026-01-20 18:52:59 -05:00
Scott Idem
43ac62b561 feat(auth): consolidate and secure V3 authentication flow
- Re-apply safe guest auth and passcode-to-JWT endpoint
- Consolidate AccountContext with token_payload and role flags
- Restore documentation for new guest flows and public read whitelists
- Fix 403 error in get_obj_li by allowing optional account context
2026-01-20 18:42:43 -05:00
Scott Idem
d4e46a4a97 feat(auth): implement site-based passcode-to-JWT endpoint
- Add POST /api/authenticate_passcode to verify site access codes
- Refactor sign_jwt to support arbitrary role flags (super, admin, etc.)
- Update dependencies_v3 to extract role flags from JWT payloads
- Add E2E test for passcode auth verification
2026-01-20 17:51:54 -05:00
Scott Idem
e16fbaa34b fix(api): resolve SQL unpacking crash and Event serialization errors
- Refactor SQL helpers in lib_sql_search to return empty tuples instead of False
- Add Pydantic pre-validators to Event_Base to coerce time objects to strings
- Improves API stability for Event searches and filtered lists
2026-01-20 15:49:13 -05:00
Scott Idem
dc7732ab5f feat(security): implement safe guest auth flow and harden request_jwt
- Patched request_jwt to strip privileged IDs when signing with public keys
- Updated AccountContext and V3 dependencies to preserve JWT payloads for guests
- Whitelisted Archive, Post, Event, and other core objects for public read access
- Added 'default_qry_str' to Event searchable fields
- Added test_e2e_jwt_guest_auth.py for security verification
2026-01-20 14:56:56 -05:00
Scott Idem
8a22ac324c Fix: Refactor sql_select to reliably handle result counts and prevent return-type mismatch 2026-01-19 18:17:41 -05:00
Scott Idem
817bb80f87 ID Vision Phase 2: Standardize Page, Post, Person, Organization, and Hosted File objects 2026-01-19 18:04:17 -05:00
Scott Idem
ab8afb72d2 Fix: Make forced account filtering schema-aware to prevent crashes on specialized views 2026-01-19 17:17:34 -05:00
Scott Idem
eeb19647f5 Error Bubbling: Implement machine-readable rich error objects for CRUD operations 2026-01-19 17:01:58 -05:00
Scott Idem
19e64135ca Permissive Update: Implement x-ae-ignore-extra-fields header support for nested routes 2026-01-19 16:48:48 -05:00
Scott Idem
7db937f8af Vision ID: Standardize Site Domain and Journal objects with string-only IDs and searchable mapping 2026-01-19 15:57:00 -05:00
Scott Idem
2dbf47d874 Security: Implement JWT verification in V3 and prevent numeric ID signing 2026-01-19 14:41:20 -05:00
Scott Idem
cad0d2e867 Security: Enforce mandatory API Keys for V3, fix search logic, and update frontend guide 2026-01-19 14:11:13 -05:00
Scott Idem
9e0f94964e Bug fix for trying to use the wrong hosted file and tmp paths or src. Also saving documentation for the new MCP AE DB field manager. 2026-01-16 14:40:12 -05:00
Scott Idem
31fd384704 Docs: Consolidate admin documentation and migrate reference data
- Created LOCAL_DEVELOPMENT_GUIDE.md and DEPLOYMENT_GUIDE_MANUAL.md from legacy txt files.
- Migrated country/time_zone data and requirements.txt to documentation/reference_data/.
- Removed redundant admin/documentation/ and admin/data_files/ directories.
- Enhanced app/lib_schema_v3.py to explicitly capture 'required' fields from DB 'NOT NULL' constraint.
- Added verification tests for schema logic and standalone DB connectivity.
2026-01-16 10:06:51 -05:00
Scott Idem
acd770962b Refactor: Modularize logging and finalize lifespan integration 2026-01-15 17:31:32 -05:00
Scott Idem
eccd71f450 Refactor: Modularize database logic and extract core CRUD operations 2026-01-15 17:16:48 -05:00
Scott Idem
5ece1d34e3 Refactor: Relocate bootstrap and validation logic into lifespan context manager 2026-01-15 17:10:42 -05:00
Scott Idem
3f276a42e1 Refactor: Modularize configuration and implement robust DB bootstrap 2026-01-15 16:59:18 -05:00
Scott Idem
16c79aca39 Cleanup: Finalize modularization of app/main.py 2026-01-15 16:45:10 -05:00
Scott Idem
2227432970 Refactor: Modularize middleware and router registration in app/main.py 2026-01-15 16:36:19 -05:00
Scott Idem
d321b94395 chore(tests): organize test scripts and beautify account creation email
- Moved scattered Python test scripts from root and 'admin/development/' to 'tests/'.
- Beautified the HTML email body for account creation links in 'app/methods/person_methods.py' with a modern responsive design.
2026-01-15 14:38:00 -05:00
Scott Idem
f0711f27b4 fix(email): resolve SMTP authentication failure and improve configuration resilience
- Fixed a bug where missing 'id=0' in the 'cfg' table caused SMTP authentication to fail by defaulting to placeholder credentials.
- Updated 'app/lib_email.py' to explicitly validate SMTP server and port settings before connecting, preventing crashes with 'please run connect() first'.
- Added email fallback logic in 'app/methods/person_methods.py' to use 'user_email' or 'primary_email' if the primary contact email is missing.
- Aligned 'app/config.py.default' with the production structure, explicitly re-adding 'SMTP' and 'FILES_PATH' dictionaries.
- Added comprehensive unit tests in 'tests/test_email_configuration.py' to verify configuration handling.
2026-01-15 13:19:58 -05:00
Scott Idem
34a752d455 feat(api-v3): implement permissive updates, automatic ID resolution, and structured error reporting
- Added 'x-ae-ignore-extra-fields' header to support stripping unknown fields in POST/PATCH.
- Added automatic resolution of '*_id_random' strings to integer IDs in 'sanitize_payload'.
- Refactored 'post_obj' to return structured (field -> message) validation errors in 'meta.details'.
- Updated 'mk_resp' to support non-string 'details' in response metadata.
- Added 'tests/verify_feedback_fixes.py' to validate logic changes.

Ref: V3 API Refinement Feedback from mcp_agent.
2026-01-14 19:11:56 -05:00
Scott Idem
19a9890dd9 Better bug fix for working SQL test. 2026-01-13 14:21:56 -05:00
Scott Idem
f9a51e243f More clean up of old routes 2026-01-13 14:15:17 -05:00
Scott Idem
6346d4ccd6 Commenting out a bunch of old routes. Hopefully none of them are used by anything still out there.... 2026-01-13 14:12:07 -05:00
Scott Idem
ed3dda6cf5 Bug fixes for SQL testing 2026-01-13 13:57:18 -05:00
Scott Idem
8927f07bcf Added some extra print debugs for now. 2026-01-12 20:30:45 -05:00
Scott Idem
4b86432381 Enhance V3 CRUD: Implement Error Bubbling and Dry-Run Validation.
- Updated app/db_sql.py to capture SQL exceptions in thread-local storage for later retrieval.
- Implemented format_db_error() in app/lib_api_crud_v3.py to clean up raw MariaDB error strings.
- Added POST /v3/crud/{obj_type}/validate endpoint for dry-run payload validation.
- Updated main and nested routers to bubble up validation and database errors into the response 'meta.details' field.
- Added tests/test_v3_error_bubbling.py to verify formatting logic.
2026-01-09 16:57:54 -05:00
Scott Idem
3885cc6aba Refactor V3 CRUD: Extract schema introspection logic.
- Created app/lib_schema_v3.py to isolate database and Pydantic model introspection.
- Updated app/routers/api_crud_v3.py to use get_object_schema_info(), completing the modularization.
- Finalized refactoring plan documentation in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:29:10 -05:00
Scott Idem
812181acb5 Refactor V3 CRUD: Extract nested child routes into separate router.
- Created app/routers/api_crud_v3_nested.py to handle all parent-child relational routes.
- Updated app/routers/api_crud_v3.py to include the nested router, significantly reducing file size.
- Documented Phase 2 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:23:14 -05:00
Scott Idem
8459b57e1b Refactor V3 CRUD: Extract helper functions and unify sanitization logic.
- Created app/lib_api_crud_v3.py to house core security, filtering, and sanitization logic.
- Implemented reusable sanitize_payload() to generically strip virtual lookup fields (*_id_random) and view-only fields (fields_to_exclude_from_db).
- Updated app/routers/api_crud_v3.py to use the new library and consolidated sanitization across all Create/Update endpoints.
- Documented Phase 1 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:16:44 -05:00
Scott Idem
2ff211f2c2 Update API documentation and finalize model validators/mappings.
- Added comprehensive docstrings to api_crud_v3.py explaining multi-tenancy, sanitization, and soft-delete logic.
- Finalized Address and Contact models/mappings with correct validators and field maps.
- Consolidated test suite in tests/ directory.
2026-01-09 15:52:00 -05:00
Scott Idem
4c83e02c4a Update V3 CRUD router and object definitions.
- Added 'external_person_id' to Post searchable fields.
- Updated api_crud_v3.py to respect 'fields_to_exclude_from_db' model attribute.
- Cleaned up old verification scripts (moved to tests/).
2026-01-09 15:36:50 -05:00
Scott Idem
1c0922ace2 Enhance API robustness: Add model validators, view-field filtering, and test suite.
- Added validators to Person_Base, Journal_Base, Journal_Entry_Base, Address_Base, and Contact_Base to handle null values and unsafe lookups.
- Implemented 'fields_to_exclude_from_db' ClassVar in Journal models to prevent view-only fields from causing DB errors.
- Updated Contact object map to align with DB schema.
- Added comprehensive test suite in 'tests/' directory (model validation, filtering logic).
- Updated GEMINI.md with progress.
2026-01-09 15:36:28 -05:00
Scott Idem
29b4d5ae4b Fix Person creation issues and enhance V3 CRUD robustness.
- Added Pydantic validators to Person_Base to handle null values for given_name and allow_auth_key, ensuring database NOT NULL constraints are met.
- Updated api_crud_v3.py (POST and PATCH) to filter out virtual *_id_random fields from data payloads before database operations to prevent "Unknown column" errors.
- Updated GEMINI.md with session progress.
2026-01-09 14:30:45 -05:00