Saving things while they work again!!! Still working on adding a special exception or something for site domain search.
This commit is contained in:
Scott Idem
28
GEMINI.md
28
GEMINI.md
@@ -41,7 +41,7 @@ I am an interactive CLI agent assisting with software engineering tasks for One
|
||||
|
||||
### V3 Architectural Progress (Jan 2026)
|
||||
|
||||
- **Modular Object Definitions:** Monolithic `ae_obj_types_def.py` refactored into domain-specific files in `app/object_definitions/` (core, events, journals, orders, cms, lookups, membership, other).
|
||||
- **Modular Object Definitions:** Monolithic `ae_obj_types_def.py` refactored into domain-specific files in `app/object_definitions/`.
|
||||
- **Granular Dependencies:** Monolithic `Common_Route_Params` replaced with specialized dependencies in `app/lib_general_v3.py` (AccountContext, Pagination, StatusFilter, Serialization, Delay).
|
||||
- **Advanced Search (POST):** Implemented `POST /v3/crud/{obj}/search` supporting recursive AND/OR grouping and standardized full-text search via the `q` property.
|
||||
- **Security Hardening:** Implemented a 5-level recursion depth limit and a field allowlist (`searchable_fields`) for the Search API.
|
||||
@@ -57,9 +57,8 @@ I am an interactive CLI agent assisting with software engineering tasks for One
|
||||
|
||||
### Agent Bridge & Docker Integration
|
||||
- **Agent Bridge Implementation**: Developed `app/routers/agent_bridge.py` for environment diagnostics.
|
||||
- **Dependency Management**: Noted `psutil` is missing from the container, causing system usage metrics to fail.
|
||||
- **Decision**: Holding off on adding `psutil` to minimize extra dependencies; the code fails gracefully for now.
|
||||
- **MCP Docker Explorer**: Created `mcp_docker_explorer.py` to test MCP server integration.
|
||||
- **MCP Docker Explorer**: Attempted to run `mcp_docker_explorer.py`, but failed with `ModuleNotFoundError: No module named 'mcp'`.
|
||||
- **Lesson**: The system python (`/usr/bin/python3`) does not have the `mcp` package installed. We must use the specific virtual environment `env_mcp` (e.g., `./env_mcp/bin/python`) or ensure the package is installed in the active environment.
|
||||
|
||||
### V3 CRUD Infrastructure & Search
|
||||
- **Modular Object Definitions**: Refactored `ae_obj_types_def.py` into modular domain files in `app/object_definitions/`.
|
||||
@@ -68,18 +67,21 @@ I am an interactive CLI agent assisting with software engineering tasks for One
|
||||
- Improved standardized full-text search (`q` parameter) with fallback logic for missing columns.
|
||||
- **Data Integrity & Aliasing**: Fixed aliased field population by enabling `allow_population_by_field_name` in Pydantic models.
|
||||
|
||||
### Tool Stability & Recovery Note (Jan 7, 2026)
|
||||
- **Frequent Hangs:** The agent experienced multiple hangs (18+) when using shell commands or `list_directory`.
|
||||
- **Recovery Action:** A `NameError: name 'SearchFilter' is not defined` in `app/routers/api_crud_v3.py` was fixed by adding the missing import. This error was triggered by the new account isolation logic in the search endpoint.
|
||||
- **Verification:** The fix was applied, but verification was interrupted by a tool hang. Verification resumes in the next session.
|
||||
### Startup Failure Resolution (Jan 7, 2026)
|
||||
- **Root Cause Identified**: The `app/routers/agent_bridge.py` module was preventing the FastAPI worker from booting, likely due to a missing or incompatible dependency (suspected `psutil` in the Docker environment) or a top-level import issue.
|
||||
- **Resolution**: Commented out the `agent_bridge` router inclusion in `app/main.py`.
|
||||
- **Status**: The API server has successfully started.
|
||||
- **Retrospective**: The previous circular dependency refactoring in `lib_general_v3` and `api_crud_v3` might have been unnecessary or at least wasn't the *primary* blocker, though deferring imports is good practice.
|
||||
|
||||
## Current To-Do List
|
||||
|
||||
1. **Frontend Integration (Priority: High)**: Coordinate with the frontend agent to ensure they adopt the mandatory JWT authentication pattern.
|
||||
2. **Routing - Nginx (Priority: High)**: Resolve 404 errors on `/v3/` and `/agent/` routes (Port 8888) by updating the Nginx configuration.
|
||||
3. **Docker MCP Integration (Priority: Medium)**: Proceed with integrating the Docker MCP server into the Gemini CLI environment.
|
||||
4. **Specialized Endpoints (Priority: Medium)**: Plan modernization of custom logic (importing, websockets) to match V3 patterns.
|
||||
5. **Account ID Handling (Priority: Low)**: Address the `x_no_account_id` usage with a more permanent architecture.
|
||||
1. **Frontend Integration (Priority: Urgent)**: Re-implement the `site_domain` lookup exception.
|
||||
- *Constraint*: Must allow searching `site_domain` without an `account_id` or JWT.
|
||||
- *Approach*: Re-apply the `optional` authentication dependency logic to `api_crud_v3.py` and `lib_general_v3.py`, now that the server is stable.
|
||||
2. **Docker MCP Integration (Priority: High)**: Re-attempt running the MCP explorer using the correct virtual environment path (`./env_mcp/bin/python`) once the API is stable.
|
||||
3. **Routing - Nginx (Priority: Medium)**: Resolve 404 errors on `/v3/` and `/agent/` routes.
|
||||
4. **Specialized Endpoints (Priority: Medium)**: Plan modernization of custom logic.
|
||||
5. **Agent Bridge Repair (Priority: Low)**: Investigate why `agent_bridge.py` crashes the server (check `psutil` availability).
|
||||
|
||||
### Workflow & Collaboration
|
||||
- **`GEMINI.md` Strategy:** The user is creating `GEMINI.md` files in key project directories. Their understanding is that context flows from the current directory up the tree, with `~/.gemini/GEMINI.md` serving as a global catch-all for general memories.
|
||||
|
||||
@@ -7,7 +7,6 @@ that are relevant to the v3 API, while removing unused or outdated functionaliti
|
||||
# Standard library imports
|
||||
import time
|
||||
import logging
|
||||
import jwt
|
||||
from typing import (
|
||||
Any,
|
||||
Dict,
|
||||
@@ -31,146 +30,70 @@ from pydantic import (
|
||||
BaseModel,
|
||||
Field,
|
||||
ValidationError,
|
||||
computed_field,
|
||||
model_validator,
|
||||
)
|
||||
|
||||
# Internal imports (from this project)
|
||||
from app.config import settings
|
||||
from app.db_sql import redis_lookup_id_random
|
||||
from app.log import get_logger
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def decode_jwt(
|
||||
secret_key: str,
|
||||
token: str,
|
||||
) -> dict:
|
||||
"""
|
||||
Decodes and validates a JWT token.
|
||||
Ported from lib_general.py to break circular dependencies.
|
||||
"""
|
||||
algorithm = 'HS256'
|
||||
try:
|
||||
decoded_token = jwt.decode(token, secret_key, algorithms=[algorithm])
|
||||
if decoded_token['eat'] >= time.time():
|
||||
return decoded_token
|
||||
else:
|
||||
return False
|
||||
except Exception:
|
||||
return None
|
||||
# --- Pydantic Model for Account Context ---
|
||||
class AccountContext(BaseModel):
|
||||
account_id: Optional[int]
|
||||
account_id_random: Optional[str]
|
||||
|
||||
|
||||
# --- Pydantic Model for Authentication Context ---
|
||||
class AuthContext(BaseModel):
|
||||
account_id: Optional[int] = None
|
||||
account_id_random: Optional[str] = None
|
||||
user_id: Optional[int] = None
|
||||
person_id: Optional[int] = None
|
||||
administrator: bool = False
|
||||
manager: bool = False
|
||||
super: bool = False
|
||||
auth_method: str = 'none' # 'jwt_header', 'jwt_query', 'legacy_header', 'bypass'
|
||||
|
||||
# Alias for backward compatibility with initial V3 implementation
|
||||
AccountContext = AuthContext
|
||||
|
||||
|
||||
# --- Dependency Function for V3 Authentication ---
|
||||
def get_v3_auth_context(
|
||||
request: Request,
|
||||
authorization: Optional[str] = Header(None, description="Bearer <jwt_token>"),
|
||||
jwt_query: Optional[str] = Query(None, alias="jwt", description="JWT token for URL-based auth (e.g., file downloads)"),
|
||||
x_account_id: Optional[str] = Header(None, min_length=11, max_length=22, description="Legacy X-Account-ID header"),
|
||||
x_no_account_id: Optional[str] = Header(None, min_length=3, max_length=100, description="Bypass account context header"),
|
||||
) -> AuthContext:
|
||||
"""
|
||||
Standardized V3 Authentication Dependency.
|
||||
Supports JWT in Authorization header (Bearer) OR 'jwt' query parameter.
|
||||
Falls back to legacy headers for backward compatibility.
|
||||
"""
|
||||
# Defer imports to break circular dependency
|
||||
from app.db_sql import redis_lookup_id_random
|
||||
from app.methods.user_methods import load_user_obj
|
||||
|
||||
# 1. Check for JWT (Header preferred, then Query for downloads)
|
||||
token = None
|
||||
method = 'none'
|
||||
|
||||
if authorization and authorization.startswith("Bearer "):
|
||||
token = authorization.split(" ")[1]
|
||||
method = 'jwt_header'
|
||||
elif jwt_query:
|
||||
token = jwt_query
|
||||
method = 'jwt_query'
|
||||
|
||||
if token:
|
||||
payload = decode_jwt(settings.JWT_KEY, token)
|
||||
if payload:
|
||||
logger.info(f"JWT Validated ({method}). User: {payload.get('user_id')}, Account: {payload.get('account_id')}")
|
||||
|
||||
# Initialize AuthContext
|
||||
ctx = AuthContext(
|
||||
account_id=payload.get('account_id'),
|
||||
account_id_random=payload.get('public_key'), # existing sign_jwt uses public_key for id_random
|
||||
user_id=payload.get('user_id'),
|
||||
person_id=payload.get('person_id'),
|
||||
auth_method=method
|
||||
)
|
||||
|
||||
# Populate roles if user_id is present
|
||||
if ctx.user_id:
|
||||
try:
|
||||
user = load_user_obj(user_id=ctx.user_id)
|
||||
if user:
|
||||
ctx.administrator = getattr(user, 'administrator', False)
|
||||
ctx.manager = getattr(user, 'manager', False)
|
||||
ctx.super = getattr(user, 'super', False)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to load user roles for user {ctx.user_id}: {e}")
|
||||
|
||||
return ctx
|
||||
else:
|
||||
logger.warning(f"Invalid or expired JWT provided via {method}")
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid or expired authentication token.")
|
||||
|
||||
# 2. Legacy / Testing Fallback: x_account_id
|
||||
if x_account_id:
|
||||
if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_account_id):
|
||||
logger.info(f"Authenticated via legacy header: {looked_up_id}")
|
||||
return AuthContext(
|
||||
account_id=looked_up_id,
|
||||
account_id_random=x_account_id,
|
||||
auth_method='legacy_header'
|
||||
)
|
||||
else:
|
||||
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Invalid X-Account-ID header.")
|
||||
|
||||
# 3. Bypass Fallback
|
||||
if x_no_account_id:
|
||||
logger.info("Authentication bypassed via X-No-Account-ID")
|
||||
return AuthContext(
|
||||
account_id_random='--- NO ACCOUNT ---',
|
||||
auth_method='bypass',
|
||||
administrator=True, # Bypass usually implies admin for dev/utility
|
||||
manager=True,
|
||||
super=True
|
||||
)
|
||||
|
||||
# 4. No Auth Found
|
||||
logger.warning("No authentication provided for V3 endpoint.")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail="Authentication required. Provide Authorization header or 'jwt' query parameter."
|
||||
)
|
||||
|
||||
|
||||
# --- Legacy wrapper to avoid breaking current V3 code ---
|
||||
# --- Dependency Function for Account Context ---
|
||||
def get_account_context(
|
||||
auth: AuthContext = Depends(get_v3_auth_context)
|
||||
) -> AuthContext:
|
||||
x_account_id: Optional[str] = Header(None, min_length=11, max_length=22),
|
||||
x_no_account_id: Optional[str] = Header(None, min_length=3, max_length=100), # Assuming 'bypass' or similar string
|
||||
x_no_account_id_token: Optional[str] = Query(None, min_length=11, max_length=22),
|
||||
) -> AccountContext:
|
||||
"""
|
||||
Alias for the new auth dependency to maintain compatibility
|
||||
with existing V3 routes.
|
||||
Resolves the account context from headers/query parameters with defined precedence.
|
||||
Precedence: x_account_id (header) > x_no_account_id_token (query) > x_no_account_id (header flag)
|
||||
Raises HTTPException 403 if no valid account is found and no bypass is indicated.
|
||||
"""
|
||||
return auth
|
||||
logger.setLevel(logging.DEBUG) # Adjust as needed
|
||||
logger.debug(locals())
|
||||
|
||||
resolved_account_id = None
|
||||
resolved_account_id_random = None
|
||||
|
||||
if x_account_id:
|
||||
# Primary check: x_account_id header
|
||||
resolved_account_id_random = x_account_id
|
||||
if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_account_id):
|
||||
resolved_account_id = looked_up_id
|
||||
logger.info(f'Found account from x_account_id header: {resolved_account_id}')
|
||||
else:
|
||||
logger.warning(f'Invalid x_account_id header provided: {x_account_id}')
|
||||
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Invalid X-Account-ID header.')
|
||||
elif x_no_account_id_token:
|
||||
# Secondary check: x_no_account_id_token query parameter
|
||||
resolved_account_id_random = x_no_account_id_token
|
||||
if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_no_account_id_token):
|
||||
resolved_account_id = looked_up_id
|
||||
logger.info(f'Found account from x_no_account_id_token query: {resolved_account_id}')
|
||||
else:
|
||||
logger.warning(f'Invalid x_no_account_id_token query provided: {x_no_account_id_token}')
|
||||
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Invalid X-No-Account-ID-Token query parameter.')
|
||||
elif x_no_account_id:
|
||||
# Tertiary check: x_no_account_id header for bypass
|
||||
# For now, just presence indicates bypass. Can add a specific value check later if needed.
|
||||
logger.info(f'X-No-Account-ID header found: {x_no_account_id}. Proceeding without specific account context.')
|
||||
resolved_account_id = None # Explicitly None for "no specific account"
|
||||
resolved_account_id_random = '--- NO ACCOUNT ---'
|
||||
else:
|
||||
logger.warning('No valid account context provided via X-Account-ID, X-No-Account-ID-Token, or X-No-Account-ID.')
|
||||
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Account context required. Please provide X-Account-ID, X-No-Account-ID-Token, or X-No-Account-ID.')
|
||||
|
||||
return AccountContext(account_id=resolved_account_id, account_id_random=resolved_account_id_random)
|
||||
|
||||
|
||||
# --- Pydantic Model for Pagination ---
|
||||
|
||||
12
app/main.py
12
app/main.py
@@ -123,12 +123,12 @@ app.include_router(
|
||||
)
|
||||
|
||||
# Deferred import to avoid circular dependencies and ensure environment is ready
|
||||
from app.routers import agent_bridge
|
||||
app.include_router(
|
||||
agent_bridge.router,
|
||||
prefix='/agent',
|
||||
tags=['Agent Bridge'],
|
||||
)
|
||||
# from app.routers import agent_bridge
|
||||
# app.include_router(
|
||||
# agent_bridge.router,
|
||||
# prefix='/agent',
|
||||
# tags=['Agent Bridge'],
|
||||
# )
|
||||
|
||||
app.include_router(
|
||||
api.router,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
36
verify_imports.py
Normal file
36
verify_imports.py
Normal file
@@ -0,0 +1,36 @@
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Add current directory to path
|
||||
sys.path.append(os.getcwd())
|
||||
|
||||
print("Attempting to import app.lib_general_v3...")
|
||||
try:
|
||||
import app.lib_general_v3
|
||||
print("Success: app.lib_general_v3")
|
||||
except Exception as e:
|
||||
print(f"Failed: app.lib_general_v3 - {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
print("-" * 20)
|
||||
|
||||
print("Attempting to import app.routers.api_crud_v3...")
|
||||
try:
|
||||
import app.routers.api_crud_v3
|
||||
print("Success: app.routers.api_crud_v3")
|
||||
except Exception as e:
|
||||
print(f"Failed: app.routers.api_crud_v3 - {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
print("-" * 20)
|
||||
|
||||
print("Attempting to import app.routers.agent_bridge...")
|
||||
try:
|
||||
import app.routers.agent_bridge
|
||||
print("Success: app.routers.agent_bridge")
|
||||
except Exception as e:
|
||||
print(f"Failed: app.routers.agent_bridge - {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
Reference in New Issue
Block a user