Commit Graph

1315 Commits

Author SHA1 Message Date
Scott Idem
4d439e63a9 Docs: Update V3 Frontend Guide to reflect ID Vision and Permissive Mode 2026-01-19 16:02:18 -05:00
Scott Idem
7db937f8af Vision ID: Standardize Site Domain and Journal objects with string-only IDs and searchable mapping 2026-01-19 15:57:00 -05:00
Scott Idem
2dbf47d874 Security: Implement JWT verification in V3 and prevent numeric ID signing 2026-01-19 14:41:20 -05:00
Scott Idem
cad0d2e867 Security: Enforce mandatory API Keys for V3, fix search logic, and update frontend guide 2026-01-19 14:11:13 -05:00
Scott Idem
d8b0c3b0a4 Saving notes and data 2026-01-16 17:27:27 -05:00
Scott Idem
9e0f94964e Bug fix for trying to use the wrong hosted file and tmp paths or src. Also saving documentation for the new MCP AE DB field manager. 2026-01-16 14:40:12 -05:00
Scott Idem
1bbe5cc31f Tests: Add README and fix diagnosis script paths 2026-01-16 11:12:17 -05:00
Scott Idem
b2384f2869 Tests: Reorganize test suite into functional subdirectories
- Categorized scripts into tests/unit/, tests/integration/, tests/e2e/, and tests/tools/.
- Adopted consistent naming prefixes (test_unit_*, test_int_*, test_e2e_*, tool_*).
- Renamed conftest_mock.py to mock_config_helper.py for clarity.
- Updated test_int_boot_diagnosis.py with sys.path setup for root-level execution.
2026-01-16 10:46:19 -05:00
Scott Idem
31fd384704 Docs: Consolidate admin documentation and migrate reference data
- Created LOCAL_DEVELOPMENT_GUIDE.md and DEPLOYMENT_GUIDE_MANUAL.md from legacy txt files.
- Migrated country/time_zone data and requirements.txt to documentation/reference_data/.
- Removed redundant admin/documentation/ and admin/data_files/ directories.
- Enhanced app/lib_schema_v3.py to explicitly capture 'required' fields from DB 'NOT NULL' constraint.
- Added verification tests for schema logic and standalone DB connectivity.
2026-01-16 10:06:51 -05:00
Scott Idem
db5cf2502a Session: Infrastructure and Documentation Finalization 2026-01-15 18:09:38 -05:00
Scott Idem
68862e4545 Docs: Consolidate V3 standards and cleanup documentation directory 2026-01-15 17:53:06 -05:00
Scott Idem
28d5843d52 Saving current notes. About to reorganize the documentation directory. 2026-01-15 17:48:44 -05:00
Scott Idem
acd770962b Refactor: Modularize logging and finalize lifespan integration 2026-01-15 17:31:32 -05:00
Scott Idem
eccd71f450 Refactor: Modularize database logic and extract core CRUD operations 2026-01-15 17:16:48 -05:00
Scott Idem
5ece1d34e3 Refactor: Relocate bootstrap and validation logic into lifespan context manager 2026-01-15 17:10:42 -05:00
Scott Idem
3f276a42e1 Refactor: Modularize configuration and implement robust DB bootstrap 2026-01-15 16:59:18 -05:00
Scott Idem
16c79aca39 Cleanup: Finalize modularization of app/main.py 2026-01-15 16:45:10 -05:00
Scott Idem
2227432970 Refactor: Modularize middleware and router registration in app/main.py 2026-01-15 16:36:19 -05:00
Scott Idem
d321b94395 chore(tests): organize test scripts and beautify account creation email
- Moved scattered Python test scripts from root and 'admin/development/' to 'tests/'.
- Beautified the HTML email body for account creation links in 'app/methods/person_methods.py' with a modern responsive design.
2026-01-15 14:38:00 -05:00
Scott Idem
f0711f27b4 fix(email): resolve SMTP authentication failure and improve configuration resilience
- Fixed a bug where missing 'id=0' in the 'cfg' table caused SMTP authentication to fail by defaulting to placeholder credentials.
- Updated 'app/lib_email.py' to explicitly validate SMTP server and port settings before connecting, preventing crashes with 'please run connect() first'.
- Added email fallback logic in 'app/methods/person_methods.py' to use 'user_email' or 'primary_email' if the primary contact email is missing.
- Aligned 'app/config.py.default' with the production structure, explicitly re-adding 'SMTP' and 'FILES_PATH' dictionaries.
- Added comprehensive unit tests in 'tests/test_email_configuration.py' to verify configuration handling.
2026-01-15 13:19:58 -05:00
Scott Idem
34a752d455 feat(api-v3): implement permissive updates, automatic ID resolution, and structured error reporting
- Added 'x-ae-ignore-extra-fields' header to support stripping unknown fields in POST/PATCH.
- Added automatic resolution of '*_id_random' strings to integer IDs in 'sanitize_payload'.
- Refactored 'post_obj' to return structured (field -> message) validation errors in 'meta.details'.
- Updated 'mk_resp' to support non-string 'details' in response metadata.
- Added 'tests/verify_feedback_fixes.py' to validate logic changes.

Ref: V3 API Refinement Feedback from mcp_agent.
2026-01-14 19:11:56 -05:00
Scott Idem
722409de0b Saving notes 2026-01-13 14:35:10 -05:00
Scott Idem
19a9890dd9 Better bug fix for working SQL test. 2026-01-13 14:21:56 -05:00
Scott Idem
f9a51e243f More clean up of old routes 2026-01-13 14:15:17 -05:00
Scott Idem
6346d4ccd6 Commenting out a bunch of old routes. Hopefully none of them are used by anything still out there.... 2026-01-13 14:12:07 -05:00
Scott Idem
ed3dda6cf5 Bug fixes for SQL testing 2026-01-13 13:57:18 -05:00
Scott Idem
8927f07bcf Added some extra print debugs for now. 2026-01-12 20:30:45 -05:00
Scott Idem
5ce193d474 Saving more note updates 2026-01-09 17:10:25 -05:00
Scott Idem
4b86432381 Enhance V3 CRUD: Implement Error Bubbling and Dry-Run Validation.
- Updated app/db_sql.py to capture SQL exceptions in thread-local storage for later retrieval.
- Implemented format_db_error() in app/lib_api_crud_v3.py to clean up raw MariaDB error strings.
- Added POST /v3/crud/{obj_type}/validate endpoint for dry-run payload validation.
- Updated main and nested routers to bubble up validation and database errors into the response 'meta.details' field.
- Added tests/test_v3_error_bubbling.py to verify formatting logic.
2026-01-09 16:57:54 -05:00
Scott Idem
3885cc6aba Refactor V3 CRUD: Extract schema introspection logic.
- Created app/lib_schema_v3.py to isolate database and Pydantic model introspection.
- Updated app/routers/api_crud_v3.py to use get_object_schema_info(), completing the modularization.
- Finalized refactoring plan documentation in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:29:10 -05:00
Scott Idem
812181acb5 Refactor V3 CRUD: Extract nested child routes into separate router.
- Created app/routers/api_crud_v3_nested.py to handle all parent-child relational routes.
- Updated app/routers/api_crud_v3.py to include the nested router, significantly reducing file size.
- Documented Phase 2 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:23:14 -05:00
Scott Idem
8459b57e1b Refactor V3 CRUD: Extract helper functions and unify sanitization logic.
- Created app/lib_api_crud_v3.py to house core security, filtering, and sanitization logic.
- Implemented reusable sanitize_payload() to generically strip virtual lookup fields (*_id_random) and view-only fields (fields_to_exclude_from_db).
- Updated app/routers/api_crud_v3.py to use the new library and consolidated sanitization across all Create/Update endpoints.
- Documented Phase 1 completion in documentation/REFACTOR_API_CRUD_V3.md.
2026-01-09 16:16:44 -05:00
Scott Idem
2ff211f2c2 Update API documentation and finalize model validators/mappings.
- Added comprehensive docstrings to api_crud_v3.py explaining multi-tenancy, sanitization, and soft-delete logic.
- Finalized Address and Contact models/mappings with correct validators and field maps.
- Consolidated test suite in tests/ directory.
2026-01-09 15:52:00 -05:00
Scott Idem
8dc37f274f Just saving changes 2026-01-09 15:40:46 -05:00
Scott Idem
4c83e02c4a Update V3 CRUD router and object definitions.
- Added 'external_person_id' to Post searchable fields.
- Updated api_crud_v3.py to respect 'fields_to_exclude_from_db' model attribute.
- Cleaned up old verification scripts (moved to tests/).
2026-01-09 15:36:50 -05:00
Scott Idem
1c0922ace2 Enhance API robustness: Add model validators, view-field filtering, and test suite.
- Added validators to Person_Base, Journal_Base, Journal_Entry_Base, Address_Base, and Contact_Base to handle null values and unsafe lookups.
- Implemented 'fields_to_exclude_from_db' ClassVar in Journal models to prevent view-only fields from causing DB errors.
- Updated Contact object map to align with DB schema.
- Added comprehensive test suite in 'tests/' directory (model validation, filtering logic).
- Updated GEMINI.md with progress.
2026-01-09 15:36:28 -05:00
Scott Idem
29b4d5ae4b Fix Person creation issues and enhance V3 CRUD robustness.
- Added Pydantic validators to Person_Base to handle null values for given_name and allow_auth_key, ensuring database NOT NULL constraints are met.
- Updated api_crud_v3.py (POST and PATCH) to filter out virtual *_id_random fields from data payloads before database operations to prevent "Unknown column" errors.
- Updated GEMINI.md with session progress.
2026-01-09 14:30:45 -05:00
Scott Idem
d32304c50a Saving notes 2026-01-08 19:27:09 -05:00
Scott Idem
46a3998fe0 Saving Gemini notes 2026-01-08 15:40:55 -05:00
Scott Idem
765949fdfd Cleanup: Remove temporary scripts from local root. 2026-01-08 12:56:34 -05:00
Scott Idem
1bcc6dae3f Cleanup: Remove ae_object_info.py from local root after relocation. 2026-01-08 12:56:01 -05:00
Scott Idem
9ee2ed444b Docs: Update GEMINI.md with MCP tool registration issue. 2026-01-08 12:55:55 -05:00
Scott Idem
802c75bad9 V3: Standardize Primary AE Objects and Synchronize Search Whitelists.
- Synchronized searchable_fields (V3 whitelists) across all Primary and Active AE objects (Identity, People, Events, Journals, Posts, Archives, Business).
- Standardized Pydantic models for core objects to include the 10 common fields (id, id_random, enable, hide, priority, sort, group, notes, created_on, updated_on).
- Fixed field aliases and uncommented valid database columns in User_Base and Organization_Base.
- Pruned non-existent fields from searchable lists in legacy or config-specific definitions (account_cfg, user_role, log_client_viewing).
- Added system discovery and validation tools:
    - ae_object_info.py: AE object status and metadata browser.
    - export_all_interfaces.py: E2E TypeScript interface generator.
    - Verification scripts for searchable field consistency.
- Updated Jan 8 milestone progress and platform roadmap in GEMINI.md.
2026-01-08 12:24:34 -05:00
Scott Idem
59d5b81da0 Arch: Finalize V3 Auth modularization and Unified Agent spec.
- Integrated zero-dependency Auth models and dependencies_v3.py.
- Successfully resolved circular dependency boot loops.
- Verified site_domain search exception via verify_v3_exceptions.py.
- Refined Unified Agent Architecture with Storage Layer and API-driven access details.
- Updated project roadmap and milestones in GEMINI.md.
2026-01-07 19:07:21 -05:00
Scott Idem
90c6b914fa Docs: Update Unified Agent Architecture and Platform Roadmap. 2026-01-07 18:53:04 -05:00
Scott Idem
d4805ebb09 Cleanup: Remove unused imports in lib_general.py after extraction. 2026-01-07 17:43:43 -05:00
Scott Idem
734576817c Refactor: Modularize lib_general.py by extracting core functionalities.
- Extracted Email functions to app/lib_email.py.
- Extracted Export functions to app/lib_export.py.
- Extracted JWT utilities to app/lib_jwt.py.
- Extracted Hash utilities to app/lib_hash.py.
- Updated app/lib_general.py to import from these new modules for backward compatibility.
- Updated V3 Frontend API Guide with latest security and site_domain exception details.
2026-01-07 17:41:04 -05:00
Scott Idem
d61dd0f00e Restored V3 search and implemented site_domain exception via dependency isolation.
- Implemented 'Isolation Mode' in api_crud_v3.py to bypass circular dependencies.
- Locally defined AccountContext and auth dependencies to ensure stable boot.
- Added site_domain lookup exception for guest users in search endpoint.
- Maintained agent_bridge disablement in main.py for stability.
2026-01-07 17:08:52 -05:00
Scott Idem
6937f9dca4 Saving these changes in a working state. Just in case. 2026-01-07 16:54:56 -05:00
Scott Idem
caf2868d02 Saving things while they work again!!! Still working on adding a special exception or something for site domain search. 2026-01-07 16:25:18 -05:00