OSIT-AE-App-Svelte/GEMINI.md

# Aether (AE) SvelteKit Application

This project is a Svelte and SvelteKit based application, part of the Aether (AE) system. It uses Tailwind CSS and Skeleton for styling and some elements. This is the frontend UI/UX. The backend API uses Python FastAPI.

Core Aether modules

- accounts - client account, not user account
- hosted_files
- people
- users
- sites and site_domains

Additional Aether modules

- events
    - presentation management - import the program data (events, session, presentations, presenters, event files, locations/rooms, devices)
    - launcher - Technically this is used with presentation management. It is part of the native app that uses Electron. One of the libraries is for functions that only work when the site is opened in an Electron app. For example the regular browser can not move files around on the local computer or run local commands.
    - badge printing
    - lead retrieval - attendee tracking; QR codes
- journals - journal, documentation, notes, diary, blog, etc
- idaa - One of my clients

## Documentation

- TODO.md
- Svelte - Introducing runes - https://svelte.dev/blog/runes
- Svelte - Breaking changes in runes mode - https://svelte.dev/docs/svelte/v5-migration-guide#Breaking-changes-in-runes-mode
- Dexie.js - Getting Started - https://dexie.org/docs/Tutorial/Getting-started
- Dexie.js - API Quick Reference - https://dexie.org/docs/API-Reference#quick-reference

## Ignored Directories

The following directories are ignored for various operations (e.g., search, file listing) to focus on relevant source code:

- `build`
- `node_modules`
- `tests`

## Development Guidelines

### Svelte v5 and SvelteKit

This project uses Svelte v5 with runes enabled. This introduces significant differences from Svelte v4. It is critical to adhere to v5 conventions to avoid bugs.

- **Reactivity:** State is managed with `$state` and `$derived`. Props can be made two-way bindable with `$bindable`. Avoid direct mutation of props.
- **Event Handling (Updated 2025-11-20):**
    - **DOM Events:** Use the lowercase `onevent` attribute (e.g., `onclick`, `oninput`). Event modifiers like `|preventDefault` may not work as expected; handle prevention logic inside the function (e.g., `event.preventDefault()`).
    - **Component Events:** Continue to use the `on:eventname` directive for events dispatched from child components.
- **Stores and `liveQuery`:**
    - To access the value of a store in Svelte v5, you must use the `$store_name` syntax (e.g., `$ae_api`).
    - Dexie `liveQuery` returns an observable. To use it in a component, you must subscribe to it within `onMount` to avoid SSR errors. The value from the subscription should then be assigned to a `$state` variable.
- **Migration Guide:** For a comprehensive overview of the changes, refer to the official [Svelte 5 Migration Guide](https://svelte.dev/docs/svelte/v5-migration-guide).

### ID Convention: `id` vs. `id_random`

- **Always use `id_random`:** The API returns both a numeric `id` and a string-based `[obj_type]_id_random`. For all frontend operations (routing, data fetching, local storage), you **must** use the `_id_random` string.
- **Local Storage:** When saving an object to the local IndexedDB (Dexie), the `id_random` value should be aliased to both `id` (as the primary key) and `[obj_type]_id`. This ensures consistency with Dexie's expectations and the rest of the application's data access patterns.

---

## Refactoring Notes

### UI Library Updates (2025-12-08)

**Context:** All project packages, including `@skeletonlabs/skeleton` (v4.7.4) and `flowbite-svelte` (v4.0.1), have been successfully updated to their current versions. The `npm run dev` and `build` commands are now working as expected.

**Key Insight:** The previous issues and blocking build errors related to UI library compatibility were largely resolved by addressing specific conflicts, particularly those associated with the `ae_comp__badge_template_form.svelte` file. This indicates that direct integration and careful handling of component-specific dependencies were crucial to achieving library compatibility.

**Current State:**
*   The project successfully builds and runs with the latest versions of SkeletonLabs and Flowbite-Svelte.
*   The codebase can now move forward with development, leveraging the updated UI libraries.

### Data Fetching & Processing Pattern (2025-11-20)

A standard pattern for fetching, processing, and caching data has been established to ensure consistency and separation of concerns.

1.  **API Function (`load_*`, `search_*`, etc.):** This function is responsible for interacting with the API. It takes parameters needed for the API call (e.g., `event_id`, search strings).
2.  **Data Processor (`process_ae_obj__*_props`):** The API function's results are immediately passed to this dedicated processor function for the specific object type.
    - The API function MUST pass any necessary contextual data (like a parent `event_id`) to the processor.
    - The processor is responsible for all data shaping and enrichment before the data is cached.
3.  **Handling API Inconsistencies:** The processor is the designated location for handling any inconsistencies in the data returned by the API.
    - **Example (`event_badge`):** The `search__event_badge` API endpoint does not include the `event_id` in its results. The `process_ae_obj__event_badge_props` function now accepts the `event_id` as a parameter and injects it into each badge object before it's saved. This centralizes the fix and keeps the API-calling function clean.
4.  **Database Caching (`db_save_ae_obj_li__ae_obj`):** After processing, the clean and consistent data is passed to the generic `db_save` function to be cached in IndexedDB.

This pattern isolates API logic from data shaping logic, making the code more modular, predictable, and easier to debug.

### Session Learnings (2025-12-12)

**Context:** Prepared for the upcoming IDAA meeting by working on the new Jitsi video conferencing page and its corresponding Novi iframe template.

**Key Learnings & Actions:**
*   **Suppressing Svelte Warnings:** Added an `onwarn` handler to `svelte.config.js` to suppress `state_referenced_locally`, `non_reactive_update`, and `css_unused_selector` warnings for the time being. This helps to focus on critical errors during development.
*   **Configuration Management:**
    - Initial approach of using `.env` files for Novi/Jitsi configuration was incorrect for this project.
    - **Correct Approach:** Configuration values (`jitsi_token_endpoint`, `novi_idaa_api_key`, `novi_idaa_group_guid_li`) are stored in the database and accessed via the `$ae_loc.site_cfg_json` Svelte store.
    - The `.../video_conferences/+page.svelte` component was refactored to use the `$ae_loc` store instead of `import.meta.env`.
    - The logic for fetching moderators was updated to iterate through a list of group GUIDs (`novi_idaa_group_guid_li`) from the store.
*   **Static Iframe Bridges:**
    - The `static/idaa_novi_iframe_jitsi_meeting.html` file acts as a bridge, taking Novi-templated user variables (`<%=Novi.User.*%>`) and passing them as URL parameters to the Svelte application (`/idaa/video_conferences`).
    - **User Feedback:** An attempt to simplify this bridge by removing a client-side `fetch` call for user details was reverted by the user. This indicates that the original, more complex logic (which constructs `full_name` from `FirstName` and `LastName`) is necessary. This is an important lesson in not over-simplifying third-party integration code without full context.
    - The bridge was updated to include an interactive UI for setting Jitsi parameters (e.g., sound settings, moderator settings) and dynamically rebuilding the iframe `src`.

---
## Jitsi "God Mode" Live Stats Architecture (2025-12-15)

### Goal
For the IDAA client, create an administrative dashboard ("god mode") to monitor live Jitsi meetings—including meeting duration, participant lists, and moderator roles—without needing to join the meetings directly. This is primarily for ensuring meetings are running correctly and for data collection.

### Architectural Decision
A purely client-side approach using the Jitsi External API is not feasible, as it requires being a participant in the meeting. A server-side solution is necessary. After considering a log-parsing approach, the following hybrid architecture was chosen as the most robust and scalable solution.

1.  **Server-Side XMPP Bot (Stats Collector):**
    *   A new, lightweight backend service will be created (likely in Python or Node.js).
    *   This service will act as a bot, using special credentials to connect to the Jitsi instance's main XMPP server (Prosody).
    *   The bot will automatically and invisibly join the Multi-User Chat (MUC) for each active conference. It will not process any audio or video, only presence and metadata.
    *   By listening to real-time XMPP events, it will track participant joins, leaves, and role changes.

2.  **Database Integration:**
    *   As the XMPP bot receives events, it will write structured data directly into the Aether system's database (e.g., a `jitsi_events` table).
    *   This provides two key benefits: a source for the live dashboard and a clean, persistent historical record for future analysis, avoiding the fragility of parsing raw text logs.

3.  **API Endpoint:**
    *   A new, secure endpoint (e.g., `/api/jitsi/live-stats`) will be added to the FastAPI backend.
    *   This endpoint will query the database to get the current state of all active meetings.

4.  **Frontend Admin Dashboard:**
    *   A new, access-controlled page (e.g., `/admin/jitsi-stats`) will be built in the Svelte application.
    *   This page will periodically fetch data from the new API endpoint to display a live overview of all ongoing meetings.

### Rationale
This approach was chosen over log-parsing because the XMPP protocol is a stable, official API for Jitsi, making the solution less likely to break on future Jitsi updates. It provides true real-time data for the live dashboard while simultaneously creating a valuable, structured dataset for historical reporting.

---
## Jitsi Stats Reporting Debugging (End of Day 2025-12-15)

### Objective
As a pragmatic first step towards the "God Mode" view, implement a "moderator-pushed" stats reporting system. A moderator's browser will periodically send the meeting stats to an existing Aether activity log endpoint.

### Progress
1.  **Client-Side Logic:** The Svelte component now collects live meeting stats (duration, participants, roles) via the Jitsi External API.
2.  **Reporting Function:** A `report_meeting_stats` function has been created to bundle this data and call the `create_ae_obj__activity_log` API function.
3.  **Trigger:** A 30-second timer (`setInterval`) is implemented to call the reporting function automatically for users where `is_moderator` is true.

### Current Problem
The feature is not working. The browser's console logs show that the `report_meeting_stats` function is being called and the data payload is being correctly prepared. However, the API call fails silently:
*   No success or error messages are logged in the console from the `try...catch` block around the API call.
*   The API call never reaches the FastAPI server (no entry in the server's access logs).

### Investigation & Last Actions
*   My initial hypothesis was a hanging network request due to a missing timeout.
*   **Action Taken:** I modified the underlying `post_object` function in `src/lib/ae_api/api_post_object.ts` to add a 20-second timeout to the `fetch` call.
*   **Result:** This had no effect. The request still disappears without triggering the success, error, or timeout conditions. This suggests the issue is preventing the `fetch` call from executing correctly at a fundamental level.

### Next Step
The crucial next step is to use the **Network** tab in the browser's developer tools. We need to observe what happens, if anything, at the exact moment the "Stats payload being sent" message appears in the console. This will tell us if a request is being blocked by the browser itself (e.g., CORS pre-flight failure) or if it's failing to even be dispatched.

---
## Jitsi Stats Reporting Debugging (Final Resolution - 2025-12-16)

**Context:** Continued debugging the silent failure of the `create_ae_obj__activity_log` API call within the Jitsi video conferencing page.

**Root Cause Identification:**
1.  **Hidden Console Errors:** The primary reason for the prolonged debugging was that Firefox Developer Tools had console errors hidden. This meant that crucial `TypeError` messages, which would have immediately pointed to the issue, were not visible.
2.  **Incorrect Parameter Passing:** The `api_cfg` parameter, expected by `create_ae_obj__activity_log`, was not correctly resolved in the calling context. Specifically, the `$ae_api` Svelte store, which provides the necessary `api_cfg` object, was not being imported into the component. Accessing properties on an `undefined` `$ae_api` (or a malformed `api_cfg` extracted from it) led to an uncaught `TypeError` that was being suppressed by the console settings.

**Solution:**
*   Ensured console errors were visible in Firefox Developer Tools.
*   Added `import { ae_api } from '$lib/stores/ae_stores';` to the relevant Svelte component (`src/routes/idaa/(idaa)/video_conferences/+page.svelte`).
*   Ensured `api_cfg` was correctly passed as `$ae_api` to the `create_ae_obj__activity_log` function.

**Key Learnings:**
*   **Always Verify Console Settings:** Before deep-diving into complex debugging, ensure that browser developer tools are configured to show all error types. A simple UI misclick can hide critical information.
*   **Parameter Resolution is Key:** Even with `async/await` and `try...catch` blocks, errors related to `undefined` or improperly resolved parameters passed to functions can cause silent failures if the error occurs before the `try` block can effectively catch it, or if console reporting is suppressed.
*   **Initial Error Messages are Paramount:** The first error message is often the most accurate indicator of the root cause, even if subsequent attempts to fix it seem to fail. Had the `TypeError` been visible, the debugging process would have been significantly shorter.
*   **The "Silent Failure" often has a hidden error message:** A function that appears to "disappear" or fail silently often has an underlying error being suppressed or unhandled in a way that prevents it from being logged.

**Outcome:**
The activity logging functionality is now working as expected. While the original hypothesis of a circular dependency was a plausible architectural issue, the immediate problem was a more fundamental runtime error exacerbated by hidden console output. The temporary isolation of the activity log function (`src/lib/ae_idaa/idaa_activity_log.ts`) is no longer needed.
---
## Session Learnings (2025-12-16)

### Client-Side Reporting Pattern

**Context:** Implemented a new reports page for Jitsi meetings (`/idaa/jitsi_reports`) without requiring a new, dedicated API endpoint.

**Strategy:**
1.  **New Reports Module:** Created a dedicated file for client-side reporting logic at `src/lib/ae_reports/reports_functions.ts`.
2.  **Fetch Flat Data:** The `load_jitsi_report` function uses the generic `api.get_ae_obj_li_for_obj_id_crud_v2` function to fetch a flat list of all relevant records (e.g., all `activity_log` entries related to Jitsi meetings).
3.  **Process on Client:** The flat list is then processed in the browser. A JavaScript `Map` is used to group the individual log entries by `external_client_id` (the unique meeting ID), transforming the flat data into a hierarchical structure suitable for a report (one parent object per meeting, with an array of discrete events inside).
4.  **Render:** The SvelteKit `load` function in `+page.ts` calls the processing function, and the `+page.svelte` component receives the structured data and renders it.

**Key Insight:** This pattern is effective for creating new views and reports quickly when the underlying data is accessible via existing generic API functions, avoiding the need for immediate backend development. However, it can be less performant for very large datasets, as the filtering and aggregation happen on the client.

### API Helper Completeness & SvelteKit URL Handling

**Context:** Encountered two subsequent errors while building the reports page.

**Learnings:**
1.  **`Unknown object type` Error:** The generic `get_ae_obj_li_for_obj_id_crud_v2` function failed with this error for `activity_log`.
    *   **Root Cause:** The function relies on an internal lookup map (`objTypeToEndpointMap`) to resolve object types to API endpoints. This map was missing an entry for `activity_log`.
    *   **Solution:** The fix was to edit `src/lib/ae_api/api_get__crud_obj_li_v2.ts` and add `'activity_log': '/crud/activity_log/list'` to the map. This is a reminder that generic helpers need to be explicitly updated to support new object types.
2.  **`data.url is undefined` TypeError:** This error occurred inside an asynchronous Jitsi event handler when trying to access `data.url.origin`.
    *   **Root Cause:** The `data` prop, passed from SvelteKit's `load` function, is not reliably scoped or available inside asynchronous callbacks fired by external, third-party libraries.
    *   **Solution:** The robust solution is to use SvelteKit's dedicated `$page` store. By importing `import { page } from '$app/stores';` and using `$page.url`, we can reliably access the current URL information from anywhere in the component, including async callbacks.

### Session Learnings (2026-01-02)

**Context:** Migrated the Journals and Events - Badges modules to the new Aether API V3 CRUD endpoints and resolved several Svelte 5 / Dexie integration issues.

**Key Accomplishments:**
- **Aether API V3 Frontend Implementation:** Created robust TypeScript wrappers for V3: `get_ae_obj_v3`, `get_ae_obj_li_v3`, `get_nested_obj_li_v3`, and `search_ae_obj_v3`. These support the new nested URL structure, hybrid filtering (query params + POST body), and global keyword search (`q` property).
- **Journals Module Migration:** Partially migrated `ae_journals` to V3 (List and Search).
    - **Bug Fix (Missing Parent ID):** Discovered that V3 nested routes often omit the parent ID in response objects. Implemented automatic parent ID injection in the data processor (`process_ae_obj__journal_entry_props`) to maintain local Dexie relationships.
    - **Bug Fix (New Entry Button):** Fixed a runtime crash on the "New Entry" button in `+layout.svelte` by removing illegal `$effect` calls from the `onclick` handler and fixing incorrect `$derived` variable access.
- **Events - Badges Module Migration:** Partially migrated `ae_events__event_badge` to V3 (List and Search).
    - **Search Optimization:** Refined the `search_query` structure to utilize the backend's new `q` property for global text search and standard `and` filters for specific fields like `badge_type_code` and `print_count`.
    - **Filter Fixes:** Resolved issues with "print status" and "affiliations" filters by reverting to the standard `like` operator (as `ilike` and `contains` were unsupported by the backend) and ensuring `print_count` is included in the persistent schema.
- **Documentation & Planning:** Updated `TODO.md` with a comprehensive V3 integration roadmap and documented actionable feedback for the FastAPI backend.

**Key Learnings:**
- **Svelte 5 & Dexie LiveQuery:** Remember that while Svelte 5 runes (`$state`, `$derived`) don't use the `$` prefix, **Dexie `liveQuery` results are observables** and still require the `$` prefix (e.g., `$lq__obj`) to access their reactive value in the template and script.
- **API V3 Implicit Context:** When using nested API routes (e.g., `/v3/crud/parent/{id}/child/`), the child objects returned may not contain the `parent_id`. The frontend must proactively inject this ID during processing if it's required for local database indexing or filtering.
- **Search Logic Construction:** When building complex V3 `search_query` objects, avoid including empty `and` or `or` arrays, as some backend parsers may strictly validate their presence or content. Only attach these properties if they contain at least one filter.
- **Backend Operator Support:** Always verify supported operators (`like`, `eq`, `gt`, etc.) in the backend FastAPI implementation. Using unsupported operators like `ilike` or `contains` will cause immediate backend `ValueError` crashes.

### Session Learnings (2026-01-05)

**Context:** Finalized core Aether API V3 wrappers and completed the full migration of the Journals module.

**Key Accomplishments:**
- **Completed V3 CRUD Wrappers:** Implemented and verified `create_ae_obj_v3`, `create_nested_obj_v3`, `update_ae_obj_v3`, `update_nested_obj_v3`, `delete_ae_obj_v3`, and `delete_nested_ae_obj_v3`.
- **Soft Delete Pattern:** Standardized the `method` parameter in deletion wrappers to support `disable` (sets `enable=false`) and `hide` (sets `hide=true`), matching the system's "safe removal" requirements.
- **Journals Module Migration:** Successfully moved all Journal and Journal Entry CRUD operations to V3. Refactored functions to follow the standard "API -> Processor -> DB Save" pattern for maximum reactivity and local caching.
- **Data Integrity Fixes:** Resolved an issue where `print_count` and print timestamps were missing from Dexie by changing the default API view to `base` (fetching `Event_Badge_Base`) and aligning the `Badge` interface/schema.
- **Sorting Logic Restoration:** Fixed a bug where `db.bulkGet` would return items in ID-order, overriding the API's sort order. Implemented a `Map`-based re-sorting logic in display components to ensure items are rendered exactly as sorted by the backend.

**Next Steps:**
- **Events - Badges Migration:** Complete the V3 migration for the Badges module (Create, Update, Delete).
- **Core Module Migration:** Begin migrating high-level modules (Accounts, Sites, People) to V3.
- **Cleanup:** Audit remaining legacy `api_get__crud_obj_id` calls and transition them to `get_ae_obj_v3`.