Saving things while they work again!!! Still working on adding a special exception or something for site domain search.

GEMINI.md

@@ -41,7 +41,7 @@ I am an interactive CLI agent assisting with software engineering tasks for One
 
 ### V3 Architectural Progress (Jan 2026)
 
--   **Modular Object Definitions:** Monolithic `ae_obj_types_def.py` refactored into domain-specific files in `app/object_definitions/` (core, events, journals, orders, cms, lookups, membership, other).
+-   **Modular Object Definitions:** Monolithic `ae_obj_types_def.py` refactored into domain-specific files in `app/object_definitions/`.
 -   **Granular Dependencies:** Monolithic `Common_Route_Params` replaced with specialized dependencies in `app/lib_general_v3.py` (AccountContext, Pagination, StatusFilter, Serialization, Delay).
 -   **Advanced Search (POST):** Implemented `POST /v3/crud/{obj}/search` supporting recursive AND/OR grouping and standardized full-text search via the `q` property.
 -   **Security Hardening:** Implemented a 5-level recursion depth limit and a field allowlist (`searchable_fields`) for the Search API.
@@ -57,9 +57,8 @@ I am an interactive CLI agent assisting with software engineering tasks for One
 
 ### Agent Bridge & Docker Integration
 - **Agent Bridge Implementation**: Developed `app/routers/agent_bridge.py` for environment diagnostics.
-- **Dependency Management**: Noted `psutil` is missing from the container, causing system usage metrics to fail.
+- **MCP Docker Explorer**: Attempted to run `mcp_docker_explorer.py`, but failed with `ModuleNotFoundError: No module named 'mcp'`.
-    - **Decision**: Holding off on adding `psutil` to minimize extra dependencies; the code fails gracefully for now.
+    - **Lesson**: The system python (`/usr/bin/python3`) does not have the `mcp` package installed. We must use the specific virtual environment `env_mcp` (e.g., `./env_mcp/bin/python`) or ensure the package is installed in the active environment.
-- **MCP Docker Explorer**: Created `mcp_docker_explorer.py` to test MCP server integration.
 
 ### V3 CRUD Infrastructure & Search
 - **Modular Object Definitions**: Refactored `ae_obj_types_def.py` into modular domain files in `app/object_definitions/`.
@@ -68,18 +67,21 @@ I am an interactive CLI agent assisting with software engineering tasks for One
     - Improved standardized full-text search (`q` parameter) with fallback logic for missing columns.
 - **Data Integrity & Aliasing**: Fixed aliased field population by enabling `allow_population_by_field_name` in Pydantic models.
 
-### Tool Stability & Recovery Note (Jan 7, 2026)
+### Startup Failure Resolution (Jan 7, 2026)
-- **Frequent Hangs:** The agent experienced multiple hangs (18+) when using shell commands or `list_directory`.
+- **Root Cause Identified**: The `app/routers/agent_bridge.py` module was preventing the FastAPI worker from booting, likely due to a missing or incompatible dependency (suspected `psutil` in the Docker environment) or a top-level import issue.
-- **Recovery Action:** A `NameError: name 'SearchFilter' is not defined` in `app/routers/api_crud_v3.py` was fixed by adding the missing import. This error was triggered by the new account isolation logic in the search endpoint.
+- **Resolution**: Commented out the `agent_bridge` router inclusion in `app/main.py`.
-- **Verification:** The fix was applied, but verification was interrupted by a tool hang. Verification resumes in the next session.
+- **Status**: The API server has successfully started.
+- **Retrospective**: The previous circular dependency refactoring in `lib_general_v3` and `api_crud_v3` might have been unnecessary or at least wasn't the *primary* blocker, though deferring imports is good practice.
 
 ## Current To-Do List
 
-1. **Frontend Integration (Priority: High)**: Coordinate with the frontend agent to ensure they adopt the mandatory JWT authentication pattern.
+1.  **Frontend Integration (Priority: Urgent)**: Re-implement the `site_domain` lookup exception.
-2. **Routing - Nginx (Priority: High)**: Resolve 404 errors on `/v3/` and `/agent/` routes (Port 8888) by updating the Nginx configuration.
+    -   *Constraint*: Must allow searching `site_domain` without an `account_id` or JWT.
-3. **Docker MCP Integration (Priority: Medium)**: Proceed with integrating the Docker MCP server into the Gemini CLI environment.
+    -   *Approach*: Re-apply the `optional` authentication dependency logic to `api_crud_v3.py` and `lib_general_v3.py`, now that the server is stable.
-4. **Specialized Endpoints (Priority: Medium)**: Plan modernization of custom logic (importing, websockets) to match V3 patterns.
+2.  **Docker MCP Integration (Priority: High)**: Re-attempt running the MCP explorer using the correct virtual environment path (`./env_mcp/bin/python`) once the API is stable.
-5. **Account ID Handling (Priority: Low)**: Address the `x_no_account_id` usage with a more permanent architecture.
+3.  **Routing - Nginx (Priority: Medium)**: Resolve 404 errors on `/v3/` and `/agent/` routes.
+4.  **Specialized Endpoints (Priority: Medium)**: Plan modernization of custom logic.
+5.  **Agent Bridge Repair (Priority: Low)**: Investigate why `agent_bridge.py` crashes the server (check `psutil` availability).
 
 ### Workflow & Collaboration
 - **`GEMINI.md` Strategy:** The user is creating `GEMINI.md` files in key project directories. Their understanding is that context flows from the current directory up the tree, with `~/.gemini/GEMINI.md` serving as a global catch-all for general memories.

app/lib_general_v3.py

@@ -7,7 +7,6 @@ that are relevant to the v3 API, while removing unused or outdated functionaliti
 # Standard library imports
 import time
 import logging
-import jwt
 from typing import (
     Any,
     Dict,
@@ -31,146 +30,70 @@ from pydantic import (
     BaseModel,
     Field,
     ValidationError,
+    computed_field,
+    model_validator,
 )
 
 # Internal imports (from this project)
 from app.config import settings
+from app.db_sql import redis_lookup_id_random
+from app.log import get_logger
 
-logger = logging.getLogger(__name__)
+logger = get_logger(__name__)
 
 
-def decode_jwt(
+# --- Pydantic Model for Account Context ---
-        secret_key: str,
+class AccountContext(BaseModel):
-        token: str,
+    account_id: Optional[int]
-        ) -> dict:
+    account_id_random: Optional[str]
-    """
-    Decodes and validates a JWT token.
-    Ported from lib_general.py to break circular dependencies.
-    """
-    algorithm = 'HS256'
-    try:
-        decoded_token = jwt.decode(token, secret_key, algorithms=[algorithm])
-        if decoded_token['eat'] >= time.time():
-            return decoded_token
-        else:
-            return False
-    except Exception:
-        return None
 
 
-# --- Pydantic Model for Authentication Context ---
+# --- Dependency Function for Account Context ---
-class AuthContext(BaseModel):
-    account_id: Optional[int] = None
-    account_id_random: Optional[str] = None
-    user_id: Optional[int] = None
-    person_id: Optional[int] = None
-    administrator: bool = False
-    manager: bool = False
-    super: bool = False
-    auth_method: str = 'none' # 'jwt_header', 'jwt_query', 'legacy_header', 'bypass'
-
-# Alias for backward compatibility with initial V3 implementation
-AccountContext = AuthContext
-
-
-# --- Dependency Function for V3 Authentication ---
-def get_v3_auth_context(
-    request: Request,
-    authorization: Optional[str] = Header(None, description="Bearer <jwt_token>"),
-    jwt_query: Optional[str] = Query(None, alias="jwt", description="JWT token for URL-based auth (e.g., file downloads)"),
-    x_account_id: Optional[str] = Header(None, min_length=11, max_length=22, description="Legacy X-Account-ID header"),
-    x_no_account_id: Optional[str] = Header(None, min_length=3, max_length=100, description="Bypass account context header"),
-) -> AuthContext:
-    """
-    Standardized V3 Authentication Dependency.
-    Supports JWT in Authorization header (Bearer) OR 'jwt' query parameter.
-    Falls back to legacy headers for backward compatibility.
-    """
-    # Defer imports to break circular dependency
-    from app.db_sql import redis_lookup_id_random
-    from app.methods.user_methods import load_user_obj
-
-    # 1. Check for JWT (Header preferred, then Query for downloads)
-    token = None
-    method = 'none'
-
-    if authorization and authorization.startswith("Bearer "):
-        token = authorization.split(" ")[1]
-        method = 'jwt_header'
-    elif jwt_query:
-        token = jwt_query
-        method = 'jwt_query'
-
-    if token:
-        payload = decode_jwt(settings.JWT_KEY, token)
-        if payload:
-            logger.info(f"JWT Validated ({method}). User: {payload.get('user_id')}, Account: {payload.get('account_id')}")
-            
-            # Initialize AuthContext
-            ctx = AuthContext(
-                account_id=payload.get('account_id'),
-                account_id_random=payload.get('public_key'), # existing sign_jwt uses public_key for id_random
-                user_id=payload.get('user_id'),
-                person_id=payload.get('person_id'),
-                auth_method=method
-            )
-
-            # Populate roles if user_id is present
-            if ctx.user_id:
-                try:
-                    user = load_user_obj(user_id=ctx.user_id)
-                    if user:
-                        ctx.administrator = getattr(user, 'administrator', False)
-                        ctx.manager = getattr(user, 'manager', False)
-                        ctx.super = getattr(user, 'super', False)
-                except Exception as e:
-                    logger.warning(f"Failed to load user roles for user {ctx.user_id}: {e}")
-
-            return ctx
-        else:
-            logger.warning(f"Invalid or expired JWT provided via {method}")
-            raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid or expired authentication token.")
-
-    # 2. Legacy / Testing Fallback: x_account_id
-    if x_account_id:
-        if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_account_id):
-            logger.info(f"Authenticated via legacy header: {looked_up_id}")
-            return AuthContext(
-                account_id=looked_up_id,
-                account_id_random=x_account_id,
-                auth_method='legacy_header'
-            )
-        else:
-            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Invalid X-Account-ID header.")
-
-    # 3. Bypass Fallback
-    if x_no_account_id:
-        logger.info("Authentication bypassed via X-No-Account-ID")
-        return AuthContext(
-            account_id_random='--- NO ACCOUNT ---',
-            auth_method='bypass',
-            administrator=True, # Bypass usually implies admin for dev/utility
-            manager=True,
-            super=True
-        )
-
-    # 4. No Auth Found
-    logger.warning("No authentication provided for V3 endpoint.")
-    raise HTTPException(
-        status_code=status.HTTP_403_FORBIDDEN, 
-        detail="Authentication required. Provide Authorization header or 'jwt' query parameter."
-    )
-
-
-# --- Legacy wrapper to avoid breaking current V3 code ---
 def get_account_context(
-    auth: AuthContext = Depends(get_v3_auth_context)
+    x_account_id: Optional[str] = Header(None, min_length=11, max_length=22),
-) -> AuthContext:
+    x_no_account_id: Optional[str] = Header(None, min_length=3, max_length=100), # Assuming 'bypass' or similar string
+    x_no_account_id_token: Optional[str] = Query(None, min_length=11, max_length=22),
+) -> AccountContext:
     """
-    Alias for the new auth dependency to maintain compatibility 
+    Resolves the account context from headers/query parameters with defined precedence.
-    with existing V3 routes.
+    Precedence: x_account_id (header) > x_no_account_id_token (query) > x_no_account_id (header flag)
+    Raises HTTPException 403 if no valid account is found and no bypass is indicated.
     """
-    return auth
+    logger.setLevel(logging.DEBUG) # Adjust as needed
+    logger.debug(locals())
+
+    resolved_account_id = None
+    resolved_account_id_random = None
+
+    if x_account_id:
+        # Primary check: x_account_id header
+        resolved_account_id_random = x_account_id
+        if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_account_id):
+            resolved_account_id = looked_up_id
+            logger.info(f'Found account from x_account_id header: {resolved_account_id}')
+        else:
+            logger.warning(f'Invalid x_account_id header provided: {x_account_id}')
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Invalid X-Account-ID header.')
+    elif x_no_account_id_token:
+        # Secondary check: x_no_account_id_token query parameter
+        resolved_account_id_random = x_no_account_id_token
+        if looked_up_id := redis_lookup_id_random(table_name='account', record_id_random=x_no_account_id_token):
+            resolved_account_id = looked_up_id
+            logger.info(f'Found account from x_no_account_id_token query: {resolved_account_id}')
+        else:
+            logger.warning(f'Invalid x_no_account_id_token query provided: {x_no_account_id_token}')
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Invalid X-No-Account-ID-Token query parameter.')
+    elif x_no_account_id:
+        # Tertiary check: x_no_account_id header for bypass
+        # For now, just presence indicates bypass. Can add a specific value check later if needed.
+        logger.info(f'X-No-Account-ID header found: {x_no_account_id}. Proceeding without specific account context.')
+        resolved_account_id = None # Explicitly None for "no specific account"
+        resolved_account_id_random = '--- NO ACCOUNT ---'
+    else:
+        logger.warning('No valid account context provided via X-Account-ID, X-No-Account-ID-Token, or X-No-Account-ID.')
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail='Account context required. Please provide X-Account-ID, X-No-Account-ID-Token, or X-No-Account-ID.')
+
+    return AccountContext(account_id=resolved_account_id, account_id_random=resolved_account_id_random)
 
 
 # --- Pydantic Model for Pagination ---

app/main.py

@@ -123,12 +123,12 @@ app.include_router(
 )
 
 # Deferred import to avoid circular dependencies and ensure environment is ready
-from app.routers import agent_bridge
+# from app.routers import agent_bridge
-app.include_router(
+# app.include_router(
-    agent_bridge.router,
+#     agent_bridge.router,
-    prefix='/agent',
+#     prefix='/agent',
-    tags=['Agent Bridge'],
+#     tags=['Agent Bridge'],
-)
+# )
 
 app.include_router(
     api.router,

verify_imports.py

@@ -0,0 +1,36 @@
+import sys
+import os
+
+# Add current directory to path
+sys.path.append(os.getcwd())
+
+print("Attempting to import app.lib_general_v3...")
+try:
+    import app.lib_general_v3
+    print("Success: app.lib_general_v3")
+except Exception as e:
+    print(f"Failed: app.lib_general_v3 - {e}")
+    import traceback
+    traceback.print_exc()
+
+print("-" * 20)
+
+print("Attempting to import app.routers.api_crud_v3...")
+try:
+    import app.routers.api_crud_v3
+    print("Success: app.routers.api_crud_v3")
+except Exception as e:
+    print(f"Failed: app.routers.api_crud_v3 - {e}")
+    import traceback
+    traceback.print_exc()
+
+print("-" * 20)
+
+print("Attempting to import app.routers.agent_bridge...")
+try:
+    import app.routers.agent_bridge
+    print("Success: app.routers.agent_bridge")
+except Exception as e:
+    print(f"Failed: app.routers.agent_bridge - {e}")
+    import traceback
+    traceback.print_exc()