Fix: Enhance V3 Search with 'contains', 'startswith', 'endswith' operators and improve error reporting.

This commit is contained in:
Scott Idem
2026-01-02 20:42:19 -05:00
parent f865b1cfb7
commit f5ab2118ad
5 changed files with 56 additions and 11 deletions

View File

@@ -46,10 +46,17 @@ I am an interactive CLI agent assisting with software engineering tasks for One
## Session Learnings & Progress (Jan 2-3, 2026) ## Session Learnings & Progress (Jan 2-3, 2026)
- **Logging Robustness:** All core modules and routers now use module-level loggers (`logging.getLogger(__name__)`). `app/log.py` includes robust `dictConfig` initialization with error handling. ### V3 CRUD Infrastructure & Security
- **Backward Compatibility:** Hybrid object definitions ensure that `/v2/crud` continues to work by including both modern (`tbl`, `mdl`) and legacy (`table_name`, `base_name`) keys. - **Modular Object Definitions**: Successfully refactored the monolithic `ae_obj_types_def.py` into a domain-driven structure under `app/object_definitions/`. This improved maintainability while keeping legacy V2 keys for backward compatibility.
- **FastAPI Best Practices:** Standardized `Response` injection via `response: Response` type hints instead of `Depends(Response)`. - **Advanced Search (POST)**: Implemented a robust `/search` endpoint supporting recursive AND/OR logic and standardized full-text search via the `q` property.
- **Documentation:** Created `V3_FRONTEND_API_GUIDE.md` for Svelte/TypeScript integration and `V3_CRUD_ARCHITECTURE_AND_LEARNINGS.md` for backend maintenance. - **Security Hardening**: Enforced a 5-level recursion depth limit and a field allowlist (`searchable_fields`) per object to prevent unauthorized data leaks.
- **JWT Authentication**: Implemented modern JWT validation for V3, supporting both the `Authorization` header and a `jwt` query parameter (enabling secure, header-free file downloads).
- **Frontend Integration**: Created a dedicated `V3_FRONTEND_API_GUIDE.md` to help the Svelte Gemini agent and developers migrate to the new endpoints.
### Technical & Environment Stability
- **Robust Logging**: Standardized on module-level loggers and wrapped logging configuration in `try/except` to prevent Docker startup crashes.
- **Circular Dependency Resolution**: Identified and resolved a major circular dependency loop between `lib_general_v3`, `response_models`, and `db_sql`.
- **FastAPI Standards**: Fixed `Response` injection and parameter ordering issues that were causing "Worker failed to boot" errors.
## Current To-Do List ## Current To-Do List

View File

@@ -2140,7 +2140,13 @@ def sql_search_qry_part(
"like": "LIKE", "like": "LIKE",
"in": "IN", "in": "IN",
"is_null": "IS NULL", "is_null": "IS NULL",
"is_not_null": "IS NOT NULL" "is_not_null": "IS NOT NULL",
"contains": "LIKE",
"icontains": "LIKE",
"startswith": "LIKE",
"istartswith": "LIKE",
"endswith": "LIKE",
"iendswith": "LIKE"
} }
def process_node(query_node, current_depth: int) -> str: def process_node(query_node, current_depth: int) -> str:
@@ -2188,17 +2194,30 @@ def sql_search_qry_part(
if searchable_fields is not None and f.field not in searchable_fields: if searchable_fields is not None and f.field not in searchable_fields:
raise HTTPException(status_code=400, detail=f"Searching on field '{f.field}' is not permitted.") raise HTTPException(status_code=400, detail=f"Searching on field '{f.field}' is not permitted.")
sql_op = operator_map.get(f.op.lower()) op_lower = f.op.lower()
sql_op = operator_map.get(op_lower)
if not sql_op: if not sql_op:
raise ValueError(f"Unsupported search operator: {f.op}") raise HTTPException(status_code=400, detail=f"Unsupported search operator: {f.op}")
filter_data = {} filter_data = {}
if f.op.lower() in ['is_null', 'is_not_null']: if op_lower in ['is_null', 'is_not_null']:
clause = f"`{f.field}` {sql_op}" clause = f"`{f.field}` {sql_op}"
elif f.op.lower() == 'in': elif op_lower == 'in':
p_name = get_param_name() p_name = get_param_name()
clause = f"`{f.field}` IN (:{p_name})" clause = f"`{f.field}` IN (:{p_name})"
filter_data[p_name] = f.value filter_data[p_name] = f.value
elif op_lower in ['contains', 'icontains']:
p_name = get_param_name()
clause = f"`{f.field}` LIKE :{p_name}"
filter_data[p_name] = f"%{f.value}%"
elif op_lower in ['startswith', 'istartswith']:
p_name = get_param_name()
clause = f"`{f.field}` LIKE :{p_name}"
filter_data[p_name] = f"{f.value}%"
elif op_lower in ['endswith', 'iendswith']:
p_name = get_param_name()
clause = f"`{f.field}` LIKE :{p_name}"
filter_data[p_name] = f"%{f.value}"
else: else:
p_name = get_param_name() p_name = get_param_name()
clause = f"`{f.field}` {sql_op} :{p_name}" clause = f"`{f.field}` {sql_op} :{p_name}"

View File

@@ -16,7 +16,7 @@ class SearchFilter(BaseModel):
Example: {"field": "price", "op": "gt", "value": 100} Example: {"field": "price", "op": "gt", "value": 100}
""" """
field: str field: str
op: str # eq, ne, gt, gte, lt, lte, like, in, is_null, is_not_null op: str # eq, ne, gt, gte, lt, lte, like, in, is_null, is_not_null, contains, startswith, endswith
value: Optional[Any] = None value: Optional[Any] = None
class SearchQuery(BaseModel): class SearchQuery(BaseModel):

View File

@@ -18,7 +18,7 @@ The V3 CRUD API (`/v3/crud/`) is designed to run in parallel with legacy V1 and
- **Data-Driven Configuration**: Uses the modern format in `app/ae_obj_types_def.py` to map objects to tables and models. - **Data-Driven Configuration**: Uses the modern format in `app/ae_obj_types_def.py` to map objects to tables and models.
- **Advanced Search (POST)**: Supports complex, nested filtering via `POST /v3/crud/{obj_type}/search`. - **Advanced Search (POST)**: Supports complex, nested filtering via `POST /v3/crud/{obj_type}/search`.
- Recursive AND/OR logic. - Recursive AND/OR logic.
- Full operator support: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `like`, `in`, `is_null`, `is_not_null`. - Full operator support: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `is_null`, `is_not_null`, `like`, `contains`, `startswith`, `endswith`.
- Safe parameterization using unique generated names (e.g., `:sp_1`) to prevent collisions. - Safe parameterization using unique generated names (e.g., `:sp_1`) to prevent collisions.
## 2. Backward Compatibility Strategy ## 2. Backward Compatibility Strategy

View File

@@ -73,6 +73,25 @@ Use the `q` property in your search body for a general keyword search across ind
} }
``` ```
### D. Supported Search Operators
The `op` property in a `SearchFilter` supports the following values:
| Operator | SQL equivalent | Description |
| --- | --- | --- |
| `eq` | `=` | Equal to |
| `ne` | `!=` | Not equal to |
| `gt` | `>` | Greater than |
| `gte` | `>=` | Greater than or equal to |
| `lt` | `<` | Less than |
| `lte` | `<=` | Less than or equal to |
| `in` | `IN (...)` | Matches any value in a provided list |
| `is_null` | `IS NULL` | Field is null (ignores `value`) |
| `is_not_null` | `IS NOT NULL` | Field is not null (ignores `value`) |
| `like` | `LIKE` | Standard SQL LIKE (requires manual `%` in `value`) |
| `contains` | `LIKE %val%` | Wraps value in `%` automatically |
| `startswith` | `LIKE val%` | Appends `%` to value automatically |
| `endswith` | `LIKE %val` | Prepends `%` to value automatically |
--- ---
## 4. Authentication in V3 ## 4. Authentication in V3