Added more documentation. Improved CRUD V3 endpoints and better responses.

This commit is contained in:
Scott Idem
2026-01-06 13:52:05 -05:00
parent 9c06b07665
commit a42f32acf4
20 changed files with 1768 additions and 65 deletions

View File

@@ -44,27 +44,28 @@ I am an interactive CLI agent assisting with software engineering tasks for One
- **Security Hardening:** Implemented a 5-level recursion depth limit and a field allowlist (`searchable_fields`) for the Search API.
- **Non-blocking Concurrency:** Standardized on `asyncio.sleep()` for delay simulation to prevent Gunicorn worker hangs.
## Session Learnings & Progress (Jan 2-5, 2026)
## Session Learnings & Progress (Jan 2-6, 2026)
### V3 CRUD Infrastructure & Security
- **Modular Object Definitions**: Successfully refactored the monolithic `ae_obj_types_def.py` into a domain-driven structure under `app/object_definitions/`. This improved maintainability while keeping legacy V2 keys for backward compatibility.
- **Advanced Search (POST)**: Implemented a robust `/search` endpoint supporting recursive AND/OR logic and standardized full-text search via the `q` property.
- **Soft Delete Implementation**: Updated `DELETE /v3/crud/{obj}/{id}` and its child equivalent to support a `method` query parameter (`delete`, `hide`, `disable`). This allows for soft deletion by setting `hide=True` or `enable=False`, while preserving the default hard delete behavior.
- **Badge Model Updates**: Added `print_count`, `print_first_datetime`, and `print_last_datetime` to `Event_Badge_Basic_Base` to ensure these fields are returned in basic badge queries.
- **Security Hardening**: Enforced a 5-level recursion depth limit and a field allowlist (`searchable_fields`) per object to prevent unauthorized data leaks.
- **JWT Authentication**: Implemented modern JWT validation for V3, supporting both the `Authorization` header and a `jwt` query parameter (enabling secure, header-free file downloads).
- **Frontend Integration**: Created a dedicated `V3_FRONTEND_API_GUIDE.md` to help the Svelte Gemini agent and developers migrate to the new endpoints.
### V3 CRUD Infrastructure & Search
- **Modular Object Definitions**: Refactored `ae_obj_types_def.py` into modular domain files in `app/object_definitions/`.
- **Advanced Search Fixes**:
- Resolved account listing and search issues by implementing `get_supported_filters` in `api_crud_v3.py`. This helper automatically adjusts `status_filter` (enabled/hidden) based on model field support (e.g., handles tables missing the `hide` column).
- Improved standardized full-text search (`q` parameter) in `sql_search_qry_part` with a dry-run check for the `default_qry_str` column. It now falls back to a `LIKE` search across all searchable fields if the column is missing.
- Added support for the `%` wildcard in `q`, allowing it to bypass filtering and return all records.
- **Data Integrity & Aliasing**: Fixed a critical issue where aliased fields (like `account_id_random`) were returned as `null` by adding `allow_population_by_field_name = True` to Pydantic models (updated `Account_Base`, `Site_Domain_Base`, etc.).
- **Consolidated V3 Router**: Systematically cleaned up `api_crud_v3.py`, removing duplicate endpoint definitions and standardizing logic across all CRUD and Search routes.
- **Robust Error Handling**: Updated V3 routers to return 500 status codes on database failures instead of masking errors with empty result sets.
### Technical & Environment Stability
- **Robust Logging**: Standardized on module-level loggers and wrapped logging configuration in `try/except` to prevent Docker startup crashes.
- **Circular Dependency Resolution**: Identified and resolved a major circular dependency loop between `lib_general_v3`, `response_models`, and `db_sql`.
- **FastAPI Standards**: Fixed `Response` injection and parameter ordering issues that were causing "Worker failed to boot" errors.
### Database Schema Insights
- Verified schema for core tables (`account`, `event`, `person`, `user`, `data_store`, `site`, `site_domain`).
- Noted that `site_domain` lacks a `hide` column, and `hosted_file_link` lacks both `hide` and `enable`.
- Most major tables now have a standardized `hide` column (added to `account` on Jan 6, 2026).
## Current To-Do List
1. **Docker Environment Insight Improvements (Priority: High)**: Implement methods/endpoints to give the agent more insight into the actual Docker runtime environment (environment variables, container status, log accessibility) to better diagnose recurring startup and configuration issues.
1. **Docker Environment Insight Improvements (Priority: High)**: Implement methods/endpoints to give the agent more insight into the actual Docker runtime environment.
2. **Security - Field Allowlists (Priority: High)**: Finish populating `searchable_fields` for all remaining object definitions.
3. **Security - Authentication (Priority: High)**: Continue refining and enforcing JWT-based authentication across all V3 endpoints.
4. **Specialized Endpoints (Priority: Medium)**: Identify and plan the modernization of custom logic (e.g., importing, websockets) to match V3 patterns.
5. **Directory Cleanup (Priority: Low)**: Long-term plan to archive old projects and standardize directory naming in `OSIT_dev`.
6. **Unused Route Cleanup**: Successfully commented out `cont_edu_cert` routers in `main.py`.
3. **Refactoring - Modularize `db_sql.py` (Priority: Medium)**: Given the file's size (>2200 lines), plan to split it into functional modules (Search Builder, Core CRUD, Redis Helpers) to improve maintainability and agent reliability.
4. **Specialized Endpoints (Priority: Medium)**: Plan modernization of custom logic (importing, websockets) to match V3 patterns.
5. **Security - Authentication (Priority: High)**: Continue refining and enforcing JWT-based authentication across all V3 endpoints.